I have a dir which has such structure:
├── aaa.py
├── src
│ └── subsrc
│ ├── else.py
│ └── util.py (there is a "foo" function")
└── tests
├── __init__.py
└── unittests
├── __init__.py
└── test_aaa.py
so "aaa.py", "tests" dir and "src" dir are in project root. and in "test_aaa.py", I use mock to mock function in "util.py":
from src.subsrc.util import foo
import pytest
from unittest import mock
#mock.patch("src.subsrc.util.foo")
def test_foo(mock):
mock.return_value = 111
and then I run python3.7 -m pytest inside "unittests" dir, it worked. This makes sense to me since pytest will find the first dir without __init__.py and then add it to PATH(in this case project root dir will be added) so it could find "src.subsrc.util.foo".
But then I made a small change to "test_aaa.py", in its "mock.patch", I added "aaa" at the beginning:
from src.subsrc.util import foo
import pytest
from unittest import mock
#mock.patch("aaa.src.subsrc.util.foo")
def test_foo(mock):
mock.return_value = 111
it still worked, "aaa.py" is an executable, in "aaa.py":
#!python3.7
from src.subsrc.else import other
if __name__ = "__main__":
# ...
pass
I am very confused why #mock.patch("aaa.src.subsrc.util.foo") also worked, is Python so smart that it could ignore 'aaa' then go "src.subsrc.." to find what it needs? Thanks!
update:
I suspect if because "aaa.py"'s name is special so I changed it to different names, but it still worked. Like I change it to "bbb.py", then in mock.patch, "aaa.src..." does not work but "bbb.src..." still worked. So I am sure "mock.patch" find this executable first.
update:
I guess it could be related to how "mock.patch()" works?
Your example seems to be a bit too stripped-down, but I'll try to expand it in order to explain. When reading about mocks in Python, you will often encounter the phrase "mock it where it's used", which isn't really helpful if you are new to the topic (but here's an excellent article on this concept).
In your test_aaa.py you will probably want to test some functionality of your aaa.py module, which may call some function from src/subsrc/util.py. After importing your foo() function in the aaa.py module, that's the exact location where you should point #mock.patch to: #mock.patch("aaa.foo"). By doing this, your mock will have access to all invocations of foo() in the functions you are about to test, namely aaa.do_something(). I've expanded your example as follows:
# aaa.py
from src.subsrc.util import foo
def do_something():
return foo()
if __name__ == "__main__":
value = do_something()
print(f"value is {value}")
# src/subsrc/util.py
def foo():
return 222
# tests/unittests/test_aaa.py
from unittest import mock
from aaa import do_something
#mock.patch("aaa.foo")
def test_foo(foo_mocked):
foo_mocked.return_value = 111
value = do_something()
assert value == 111
When executing this like python aaa.py, I get the output as expected (value is 222) while the test passes with its assert value == 111.
In your example, #mock.patch("src.subsrc.util.foo") obviously worked, but probably didn't do what you intended. From your example code, I cannot see how #mock.patch("aaa.src.subsrc.util.foo") shouldn't have returned a ModuleNotFoundError.
Related
Have a module in a subdirectory and when I try to import it, I get a NameError: namefoois not defined. When I put the class code directly into the __main__.py file it works fine. __init__.py files are empty.
I've tried the following all with other errors:
MyProject/
├── __init__.py
├── __main__.py
├── foo/
│ ├── bar.py
│ ├── __init__.py
bar.py
class Bar:
def __init__(self):
print( 'am here' )
pass
__main__.py
from MyProject import foo
#from MyProject import bar # errors with cannot import bar from MyProject
#from foo import bar # errors with No module named foo
if __name__ == '__main__':
w = Bar()
Is there perhaps a better way to organise this?
The Bar class is in the file bar.py, so I think you'd need to do
from MyProject.foo import bar
w = bar.Bar()
or
from MyProject.foo.bar import Bar
w = Bar()
You didn't share your foo/__init__.py, but you could fix the situation by adding something like this to it:
from .bar import Bar
That adds Bar to the namespace of foo and causes Python to find Bar when you just import foo.
However, you would probably do well to look at a few standard examples for writing a package. For one, you probably shouldn't name your project MyProject, as that name signals it's a class (with TitleCase). Naming it 'project' further confuses the issue, as it appears you're writing a package, so my_package sounds about right.
If you dont know from what direcrory file will be runed then use . to show where package is
from .foo.bar import Bar
or
from .foo import bar
w = bar.Bar()
or
import .foo
w = foo.bar.Bar()
. before package name means that package lockated in same directory with current file
For my Python 3 project, I have the following directory structure:
├── myapp
│ ├── compute
│ │ └── compute.py
│ └── mymain.py
├── setup.py
└── tests
└── compute
└── compute_test.py
My goal is to be able to run the code here in three ways:
Unit tests. I've randomly chosen pytest for these but whichever framework should be fine;
python myapp/mymain.py <arguments> for when I want to do a quick "manual test";
Something like pip install and/or a Docker image for a proper deployment.
Now, the first and third of these seem to be no problem, but I'm having trouble with the middle one.
Here are the contents of the files:
compute.py:
import math
class MyComputation:
# performs an extremely difficult and relevant computation
#staticmethod
def compute(i: int) -> float:
return math.sqrt(abs(i))
compute_test.py:
import pytest
from myapp.compute.compute import MyComputation
def test_computation_normal_case():
ins = [-4, 9, -36, 121]
outs = list(map(lambda i: MyComputation.compute(i), ins))
expected = [2.0, 3.0, 6.0, 11.0]
assert outs == expected
mymain.py:
import random
from myapp.compute.compute import MyComputation
class MyApp:
#staticmethod
def main():
print("Loading data...")
i = random.randint(1, 100000)
print("Input: {}".format(i))
print("Computing...")
f = MyComputation.compute(i)
print("Output: {}".format(f))
print("Done!")
if __name__ == "__main__":
MyApp.main()
When I run, say, pytest from the command line, it works fine: finds the test, runs it, test passes.
However, when I try to run the main class:
$ python myapp/mymain.py
Traceback (most recent call last):
File "myapp/mymain.py", line 8, in <module>
from myapp.compute.compute import MyComputation
ImportError: No module named myapp.compute.compute
It makes no difference whether I add __init__.py files inside the directories or not.
But if I add the following to mymain.py, it can then be run from the command line as expected:
import os
import sys
root_path = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '../'))
sys.path.insert(0, root_path)
So, questions:
1) What is the correct, Pythonic, idiomatic way to do the main class? What I want is essentially "run this code here, in-place, as is". Where do I put my main class? Do I need to pip install my stuff locally first? Do I need to do the imports differently?
2) Surely the sys.path.insert() stuff cannot be the "official" way of accomplishing what I want to do here? There must be a less ridiculous way... right?
3) Why do the unit tests work just fine while the main class doesn't? Does the unit test framework do something similar to the sys.path.insert() stuff under the covers? Or is there a better way of handling the imports?
I am trying to do a test in a project, and I am having an weird error.
I reproduced the very similar situation with the toy example below:
This is the file structure:
.
├── some_package
│ ├── __init__.py
│ └── some_file.py
└── test_mock_patch.py
"""some_package/some_file.py"""
# when I import here, the test fails
from math import floor
def some_func(a, b):
# if I import here, the test passes
# from math import floor
return floor(a + b)
"""test_mock_patch.py"""
import pytest
from unittest import mock
from some_package.some_file import some_func
#pytest.fixture
def mock_floor():
with mock.patch('math.floor', autospec=True) as m:
yield m
def test_some_func(mock_floor):
some_func(1.1, 1)
assert mock_floor.call_count == 1
Command used: pytest -v -s test_mock_patch.py
The error:
Why when I import inside the function the test_some_func passes and when I import at the top the test fails?
Thank you in advance for any help to explain this behaviour of mock.patch
Versions:
Python 3.7.3
pytest 4.4.1
Here is a minimal example how to achieve the desired result by changing your test_mock_patch.py file.
import pytest
from some_package.some_file import some_func
def test_some_func(monkeypatch):
with monkeypatch.context() as mc:
mc.setattr('some_package.some_file.floor', lambda x: 'foo')
res = some_func(1.1, 1)
assert res == 'foo'
Like I mentioned in the comments, you need to patch the function where it is being imported.
I have code that connects to a MongoDB Client and I'm trying to test it. For testing, I don't want to connect to the actual client, so I'm trying to figure out make a fake one for testing purposes. The basic flow of the code is I have a function somewhere the creates a pymongo client, then queries that and makes a dict that is used elsewhere.
I want to write some tests using pytest that will test different functions and classes that will call get_stuff. My problem is that get_stuff calls mongo() which is what actually makes the connection to the database. I was trying to just use pytest.fixture(autouse=True) and mongomock.MongoClient() to replace mongo().
But this isn't replacing the mongo_stuff.mongo(). Is there some way I can tell pytest to replace a function so my fixture is called instead of the actual function? I thought making the fixture would put my testing mongo() higher priority in the namespace than the function in the actual module.
Here is an example file structure with my example:
.
├── project
│ ├── __init__.py
│ ├── mongo_stuff
│ │ ├── __init__.py
│ │ └── mongo_stuff.py
│ └── working_class
│ ├── __init__.py
│ └── somewhere_else.py
└── testing
├── __init__.py
└── test_stuff.py
mongo_stuff.py
import pymongo
def mongo():
return pymongo.MongoClient(connection_params)
def get_stuff():
db = mongo() # Makes the connection using another function
stuff = query_function(db) # Does the query and makes a dict
return result
somewhere_else.py
from project.mongo_stuff import mongo_stuff
mongo_dict = mongo_stuff.get_stuff()
test_stuff.py
import pytest
import mongomock
#pytest.fixture(autouse=True)
def patch_mongo(monkeypatch):
db = mongomock.MongoClient()
def fake_mongo():
return db
monkeypatch.setattr('project.mongo_stuff.mongo', fake_mongo)
from poject.working_class import working_class # This starts by calling project.mongo_stuff.mongo_stuff.get_stuff()
And this will currently give me a connection error since the connection params in mongo_stuff.py are only made to work in the production environment. If I put the import statement from test_stuff.py into a test function, then it works fine and mongomock db will be used in the testing enviornment. I also tried change the setattr to monkeypatch.setattr('project.working_class.mongo_stuff.mongo', fake_mongo) which also does not work.
You're halfway there: you have created a mock for the db client, now you have to patch the mongo_stuff.mongo function to return the mock instead of a real connection:
#pytest.fixture(autouse=True)
def patch_mongo(monkeypatch):
db = mongomock.MongoClient()
def fake_mongo():
return db
monkeypatch.setattr('mongo_stuff.mongo', fake_mongo)
Edit:
The reason why you get the connection error is that you are importing somewhere_else on module level in test_stuff, and somewhere_else runs connection code also on module level. So patching with fixtures will come too late and will have no effect. You have to patch the mongo client before the import of somewhere_else if you want to import on module level. This will avoid the error raise, but is extremely ugly:
from project.mongo_stuff import mongo_stuff
import mongomock
import pytest
from unittest.mock import patch
with patch.object(mongo_stuff, 'mongo', return_value=mongomock.MongoClient()):
from project.working_class import somewhere_else
#patch.object(mongo_stuff, 'mongo', return_value=mongomock.MongoClient())
def test_db1(mocked_mongo):
mongo_stuff.mongo()
assert True
#patch.object(mongo_stuff, 'mongo', return_value=mongomock.MongoClient())
def test_db2(mocked_mongo):
somewhere_else.foo()
assert True
You should rather avoid running code on module level when possible, or run the imports that execute code on module level inside the tests (as you already found out in the comments).
I have my .py module which is in C:\Python_Projects\MyModules\ with the name button_generator.py.
My code goes something like this:
module_path='C:\\Python_Projects\\MyModules'
module_name='button_generator.py'
sys.path.append(module_path)
try:
limp=importlib.import_module(module_name.split('.')[0])
except:
print 'module import error'
I have tried other versions aswell:
importlib.import_module(module_name) without the split
importlib.import_module('C:\Python_Projects\MyModules\button_generator.py')
importlib.import_module('C:\Python_Projects\MyModules\button_generator')
The folder C:\Python_Projects\MyModules is in my sys.path as I checked during debug.
Why wouldn't the module import?
I suggest you to reorder your project directories and avoid calling other modules which are not in your current directory project. You'll avoid those kind of errors.
For example, let's organize our project directories and folders to look something like this:
MyProjectFolder/
├── main.py
└── modules
├── __init__.py
└── MyLib.py
NB: Don't forget to add an empty file called __init__.py
MyLib.py :
#!/usr/bin/python3
class MyLib:
def __init__(self):
self.say_hello = "Hello i'm in modules/MyLib"
def print_say_hello(self):
print(self.say_hello)
main.py:
#!/usr/bin/python3
# from folder.file import class
from modules.MyLib import MyLib
class MainClass:
def __init__(self):
my_lib = MyLib() # load MyLib class
my_lib.print_say_hello() # access to MyLib methods
### Test
if __name__ == '__main__':
app = MainClass()
In terminal when i run:
$ python3 main.py
output:
Hello i'm in modules/MyLib
So here we have successfully imported the class in modules/MyLib.py into our main.py file.
I found the error:
After treating the ImportError exception by printing it's args, I noticed that button_generator.py had an Import that was not resolving. Basically, button_generator.py could not be imported because it had a wrong import.