Develop Reference

Pytest not able to run test where script A importing another script B in the same folder level as A and giving me ModuleNotFoundError

I am trying to run the unit test using pytest in this project, here main_0.py is importing s3 file.
I am getting ModuleNotFoundError: no module named 's3'
Project Folder Structure
some_project
└───src
├───main
│ └───lambda_function
│ └───some
│ main_0.py
│ s3.py
│
└───test
└───unittest
└───lambda_function
└───some
test_main_0.py
test_s3.py
main_0.py
from s3 import PrintS3
def lambda_handler():
obj = PrintS3()
res = obj.print_txt()
return res
s3.py
class PrintS3:
def __init__(self) -> None:
self.txt = "Hello"
def print_txt(self):
print(self.txt)
return self.txt
test_main_0.py
import unittest
class TestSomeMain(unittest.TestCase):
def test_main_0(self):
from src.main.lambda_function.some.main_0 import lambda_handler
res = lambda_handler()
assert res == "Hello"
test_s3.py is empty.
I also tried adding an empty __init__.py file in both the dir but still the same error
Project Folder Structure after adding __init__.py file
some_project
└───src
├───main
│ └───lambda_function
│ └───some
│ main_0.py
│ s3.py
│ __init__.py
│
└───test
└───unittest
└───lambda_function
└───some
test_main_0.py
test_s3.py
__init__.py
the command I am using to run pytest:
python -m pytest ./src/test
and I am inside some_project folder and also using main_0.py instead of main.py because to not get confused with main folder
Edit 2:
I am to run the test case successfully by adding sys.path in the test_main_0.py file but it is breaking linting and hinting in the code editor (vscode) it didn't broke the linting and hinting, both import statement works but is there any better way.
new test_main_0.py:
import unittest
import os
import sys
sys.path.append(os.path.abspath("./src/main/lambda_function/some/"))
class TestSomeMain(unittest.TestCase):
def test_main_0(self):
from src.main.lambda_function.some.main_0 import lambda_handler # this works
from main_0 import lambda_handler # this also works but break linting and hinting in the code editor
res = lambda_handler()
assert res == "Hello"

could you please try
import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
from some.s3 import PrintS3
def lambda_handler():
obj = PrintS3()
res = obj.print_txt()
return res

I found a somewhat working solution.
added setUp() and tearDown() methods in the class for inserting and removing path in sys.path
path in sys.path is the location of the directory where the main_0.py and s3.py is located
import unittest
import os
import sys
class TestSomeMain(unittest.TestCase):
def setUp(self) -> None:
sys.path.insert(0, os.path.abspath("./src/main/lambda_function/some/"))
def tearDown(self) -> None:
sys.path.remove(os.path.abspath("./src/main/lambda_function/some/"))
def test_main_0(self):
from src.main.lambda_function.some.main_0 import lambda_handler
res = lambda_handler()
assert res == "Hello"
also update the test command in the terminal:
python -m pytest ./src/test/unittest/lambda_function/some --cov ./src/main/lambda_function/some --cov-report html

pytest testing internal functions

Let's say that I have a package qaz:
qaz/
__init__.py
qaz.py
tests/
test_qaz.py
setup.py
Now, I want to test some internal qaz.py functions:
def _abc():
return 3
def def():
return _abc() + 2
but when I run pytest with tests like this:
from qaz.qaz import *
def test_abc():
assert _abc() == 3
def test_def():
assert def() == 5
My question is how to test _abc()?
Now I'm getting:
E NameError: name '_abc' is not defined

Oh, figured it out. You need to be specific, when it comes to the internal functions:
from qaz.qaz import _abc

How to reference local import when patching - proper python testing structure

I have a Starlette app and I'm attempting to perform end to end testing on the entire application. A function defined in a.py is called by a function in b.py through a local import, and for testing purpose I would like to replace the function with a self defined function when testing.
Like others, I'm running into issues getting the path string to work. After looking at existing questions on stackoverflow, I think I'm supposed to be patching the reference in b.py, but run into the following error AttributeError: <function b at 0x7f5298087af0> does not have the attribute 'a'
Here's the relevant structure and code
Project_folder
- app
- lib
+ __init__.py
+ a.py
+ b.py
- handle
+ __init__.py
- test
+ test.py
+ main.py
#a.py
def a(id):
return stuff
#b.py
from .a import a
def b():
return a(some_stuff)
#lib.__init__py
from .a import a
from .b import b
routes refers to an async function which calls b somewhere in the code
#main.py
from starlette.applications import Starlette
from handle import routes
app = Starlette(routes=routes)
My attempt at test.py
from starlette.testclient import TestClient
from main import app
import pytest
from mock import patch
client = TestClient(app)
def mock_a(id):
return 'some value'
#patch('app.lib.b.a', new=mock_a)
def test_app(request):
response = client.post('/route', request)
assert response.status_code == 200
I'm very new to mocks and patching, and would appreciate any advice on how I should be setting this up

You import "a" and "b" as functions in init.py. So patch libs, not libs.a or libs.b. if you want to patch inside the module itself the take the imports out of init.py and the you can do something like this:
import libs
libs.b.b = libs.a.a

Mock.patch in python unittest could work for two paths

I have a dir which has such structure:
├── aaa.py
├── src
│   └── subsrc
│   ├── else.py
│   └── util.py (there is a "foo" function")
└── tests
├── __init__.py
└── unittests
├── __init__.py
└── test_aaa.py
so "aaa.py", "tests" dir and "src" dir are in project root. and in "test_aaa.py", I use mock to mock function in "util.py":
from src.subsrc.util import foo
import pytest
from unittest import mock
#mock.patch("src.subsrc.util.foo")
def test_foo(mock):
mock.return_value = 111
and then I run python3.7 -m pytest inside "unittests" dir, it worked. This makes sense to me since pytest will find the first dir without __init__.py and then add it to PATH(in this case project root dir will be added) so it could find "src.subsrc.util.foo".
But then I made a small change to "test_aaa.py", in its "mock.patch", I added "aaa" at the beginning:
from src.subsrc.util import foo
import pytest
from unittest import mock
#mock.patch("aaa.src.subsrc.util.foo")
def test_foo(mock):
mock.return_value = 111
it still worked, "aaa.py" is an executable, in "aaa.py":
#!python3.7
from src.subsrc.else import other
if __name__ = "__main__":
# ...
pass
I am very confused why #mock.patch("aaa.src.subsrc.util.foo") also worked, is Python so smart that it could ignore 'aaa' then go "src.subsrc.." to find what it needs? Thanks!
update:
I suspect if because "aaa.py"'s name is special so I changed it to different names, but it still worked. Like I change it to "bbb.py", then in mock.patch, "aaa.src..." does not work but "bbb.src..." still worked. So I am sure "mock.patch" find this executable first.
update:
I guess it could be related to how "mock.patch()" works?

Your example seems to be a bit too stripped-down, but I'll try to expand it in order to explain. When reading about mocks in Python, you will often encounter the phrase "mock it where it's used", which isn't really helpful if you are new to the topic (but here's an excellent article on this concept).
In your test_aaa.py you will probably want to test some functionality of your aaa.py module, which may call some function from src/subsrc/util.py. After importing your foo() function in the aaa.py module, that's the exact location where you should point #mock.patch to: #mock.patch("aaa.foo"). By doing this, your mock will have access to all invocations of foo() in the functions you are about to test, namely aaa.do_something(). I've expanded your example as follows:
# aaa.py
from src.subsrc.util import foo
def do_something():
return foo()
if __name__ == "__main__":
value = do_something()
print(f"value is {value}")
# src/subsrc/util.py
def foo():
return 222
# tests/unittests/test_aaa.py
from unittest import mock
from aaa import do_something
#mock.patch("aaa.foo")
def test_foo(foo_mocked):
foo_mocked.return_value = 111
value = do_something()
assert value == 111
When executing this like python aaa.py, I get the output as expected (value is 222) while the test passes with its assert value == 111.
In your example, #mock.patch("src.subsrc.util.foo") obviously worked, but probably didn't do what you intended. From your example code, I cannot see how #mock.patch("aaa.src.subsrc.util.foo") shouldn't have returned a ModuleNotFoundError.

Python: basic project structure & import paths

For my Python 3 project, I have the following directory structure:
├── myapp
│   ├── compute
│   │   └── compute.py
│   └── mymain.py
├── setup.py
└── tests
└── compute
└── compute_test.py
My goal is to be able to run the code here in three ways:
Unit tests. I've randomly chosen pytest for these but whichever framework should be fine;
python myapp/mymain.py <arguments> for when I want to do a quick "manual test";
Something like pip install and/or a Docker image for a proper deployment.
Now, the first and third of these seem to be no problem, but I'm having trouble with the middle one.
Here are the contents of the files:
compute.py:
import math
class MyComputation:
# performs an extremely difficult and relevant computation
#staticmethod
def compute(i: int) -> float:
return math.sqrt(abs(i))
compute_test.py:
import pytest
from myapp.compute.compute import MyComputation
def test_computation_normal_case():
ins = [-4, 9, -36, 121]
outs = list(map(lambda i: MyComputation.compute(i), ins))
expected = [2.0, 3.0, 6.0, 11.0]
assert outs == expected
mymain.py:
import random
from myapp.compute.compute import MyComputation
class MyApp:
#staticmethod
def main():
print("Loading data...")
i = random.randint(1, 100000)
print("Input: {}".format(i))
print("Computing...")
f = MyComputation.compute(i)
print("Output: {}".format(f))
print("Done!")
if __name__ == "__main__":
MyApp.main()
When I run, say, pytest from the command line, it works fine: finds the test, runs it, test passes.
However, when I try to run the main class:
$ python myapp/mymain.py
Traceback (most recent call last):
File "myapp/mymain.py", line 8, in <module>
from myapp.compute.compute import MyComputation
ImportError: No module named myapp.compute.compute
It makes no difference whether I add __init__.py files inside the directories or not.
But if I add the following to mymain.py, it can then be run from the command line as expected:
import os
import sys
root_path = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '../'))
sys.path.insert(0, root_path)
So, questions:
1) What is the correct, Pythonic, idiomatic way to do the main class? What I want is essentially "run this code here, in-place, as is". Where do I put my main class? Do I need to pip install my stuff locally first? Do I need to do the imports differently?
2) Surely the sys.path.insert() stuff cannot be the "official" way of accomplishing what I want to do here? There must be a less ridiculous way... right?
3) Why do the unit tests work just fine while the main class doesn't? Does the unit test framework do something similar to the sys.path.insert() stuff under the covers? Or is there a better way of handling the imports?