I have the following pytest directory structure:
system_tests/
├── conftest
├── pytest.ini
│
├── suite_1/
│ └── test_A.py
│
└── suite_2/
└── sub_suite_2a/
└── test_B.py
When each test method runs, a number of third-party libraries/processes generate artifacts in the current working directory.
When pytest is executed from the sub_suite folder (using CLI or IDE "play" button), the files are generated in the sub_suite folder, where I want them to be.
However, when pytest is run from the system_tests folder to run all tests, all artifacts are created in the system_tests folder, which is not what I want.
Is there an easy way to force pytest to always use the test class folder as the working directory so I get the same results regardless of how or where I run a test from?
EDIT: Improved Solution
Using monkeypatch as suggested by #Kound removes the boilerplate code to restore the cwd. You can also enable autouse to automatically apply this fixture to all test functions. Add the following fixture to conftest.py to change the cwd for all tests:
#pytest.fixture(autouse=True)
def change_test_dir(request, monkeypatch):
monkeypatch.chdir(request.fspath.dirname)
Any processes that are kicked off by the test will use the test case folder as their working directory and copy their logs, outputs, etc. there, regardless of where the test suite was executed.
Original Solution
The following function-level fixture will change to the test case directory, run the test (yield), then change back to the calling directory to avoid side-effects, as suggested by #hi2meuk:
#pytest.fixture
def change_test_dir(request):
os.chdir(request.fspath.dirname)
yield
os.chdir(request.config.invocation_dir)
request is a built-in pytest fixture
fspath is the LocalPath to the test module being executed
dirname is the directory of the test module
request.config.invocationdir - the folder from which pytest was executed
request.config.rootdir - pytest root, doesn't change based on where you run pytest. Not used here, but could be useful.
A different and, IMHO more robust approach: always reference your files by the complete path.
__file__ is an automatically declared Python variable that is the name of the current module. So in your test_B.py file, it would have the value: system_tests/suite_2/sub_suite_2a/test_B.py. Just get the parent and choose where to write your files.
from pathlib import Path
test_data_dir = Path(__file__).parent / "test_data"
Now you have all of them in the same place and can tell your version control system to ignore them.
If the code is inside a library, better use an absolute path:
import os
from pathlib import Path
test_data_dir = Path(__file__).parent.absolute() / "test_data"
Instead of creating a fixture for each directory like suggested by #DV82XL you can simply use monkeypatch to achieve the same:
import pytest
from pathlib import Path
#pytest.fixture
def base_path() -> Path:
"""Get the current folder of the test"""
return Path(__file__).parent
def test_something(base_path: Path, monkeypatch: pytest.MonkeyPatch):
monkeypatch.chdir(base_path / "data" )
# Do something in the data folder
Many options open to you to achieve this. Here are a few.
1.
Write a pytest fixture to check if the current working directory is equal to the desired working directory, and if not, then move all the artifact files to the desired directory. If the artifacts you are generating are all the same type of file (e.g. *.jpg, *.png, *.gif) and you just want them to be in a different directory, then this may suffice. Something like this could work
from pathlib import Path
import shutil
#pytest.fixture
def cleanup_artifacts():
yield None
cwd = Path.cwd()
desired_dir = Path.home() / 'system-tests' / 'suite-2' / 'sub_suite_2a'
if cwd != desired_dir:
for f in cwd.glob('*.jpg'):
shutil.move(f, desired_dir)
And then you can add this fixture to your tests as needed.
2.
You can configure the pytest rootdir to be the desired directory, since pytest uses the rootdir to store project/testrun specific info.
When you run pytest, run it as
pytest --rootdir=desired_path
See here for more info: https://docs.pytest.org/en/latest/customize.html#initialization-determining-rootdir-and-inifile
If both don't work for you, tell more about what your requirements are. Surely this can be done with pytest.
Related
I have a folder structure as below for my pytest files
tests/my_test1.py
tests/input/data.txt
tests/bench/bench.csv
The test my_test1.py would have to read the file tests/input/data.txt. The challenge is to find the location of the file.
The tests can be invoked in Mutiple ways as mentioned here https://docs.pytest.org/en/6.2.x/usage.html. The current working directory may be different based on the invocation. so the way to open a file the input folder data_f = open ("./input/data.txt") would be incorrect.
What would be the correct path to access the file? I tried to infer the path from os.getenv('PYTEST_CURRENT_TEST'). However this always give the value of the path as 'tests/' no matter from where the test is invoked.
You could create a fixture in the root of the tests folder (tests/conftest.py) to help you read any file relative to the tests folder:
from pathlib import Path
import pytest
#pytest.fixture()
def get_file():
def _(file_path: str):
return (Path(__file__).parent / file_path).read_text()
return _
In any test you could then use the fixture to read the file for you:
def test_a(get_file):
content = get_file('input/data.txt')
...
In this example the fixture returns a function that would read the content of any file within tests/ directory, but there are multiple other options:
The fixture can return a Path object instead of the content.
If you would always use the same file, it can return file object/content directly so within your test function you wouldn't need to call it.
I know this violates any best practices which require/assume proper packaging of Python production code: In some cases it might be helpful to beeing able to define production and test code in the same file (e.g. in case of simple scripts). How can I run all or specific tests in a file with pytest then?
EDIT - The solution for my particular use case:
file structure in <root>:
pytest.ini
scrip_with_tests.py
content of pytest.ini:
[pytest]
python_files = script_with_tests.py
content of script_with_tests.py:
import pytest # this is not required, works without as well
def test_always_pass():
pass
if __name__ == "__main__":
main()
pytest invocation in <root>:
pytest script_with_tests.py
As stated in the Customizing test collection section:
You can easily instruct pytest to discover tests from every Python file:
# content of pytest.ini
[pytest]
python_files = *.py
However, many projects will have a setup.py which they don’t want to be imported. Moreover, there may files only importable by a specific python version. For such cases you can dynamically define files to be ignored by listing them in a conftest.py file:
# content of conftest.py
import sys
collect_ignore = ["setup.py"]
if sys.version_info[0] > 2:
collect_ignore.append("pkg/module_py2.py")
I'm using Pytest to test my code and I'm running into a little, but an infuriating issue.
One of the first things my program does is checking if there are any setting files available. If there aren't any it throws an error and calls exit(). This works well during normal runtime but messes with Pytest.
The solution I came up with is to simply create a temporary settings file for the duration of the tests, by copying the template settings file. I've already written and successfully tested the code to achieve that.
The problem I'm running into is that I can't find a Pytest hook that truly fires before everything else. This results in the program throwing the error before Pytest tries to create a temporary settings file. Which results in Pytest failing before it can perform any tests.
Does anyone know of a way to fire a function before Pytest does or loads anything? Preferably within Pytest itself.
Some code context:
Throwing the error and exit
This snippet runs on the import of the settings module.
if len(cycles) == 0:
log.error("No setting files found. Please create a file " +
"in the `settings` folder using the Template.py.")
exit()
Creating the temporary settings file
This code should be the very first thing Pytest runs.
def pytest_sessionstart(session):
""" Create a test settings file """
folderPath = os.path.dirname(os.path.abspath(__file__))
folderPath = os.path.split(folderPath)[0] + "/settings"
srcfile = folderPath + '/Template.py'
dstfile = folderPath + '/test.py'
shutil.copy(srcfile, dstfile)
Removing the temporary settings file
This code should be one of the last things Pytest runs.
def pytest_sessionfinish(session, exitstatus):
""" Delete the test settings file """
folderPath = os.path.dirname(os.path.abspath(__file__))
folderPath = os.path.split(folderPath)[0] + "/settings"
os.remove(folderPath + "/test.py")
Pytest output
With the call to exit() disabled, so you can see the execution order.
Lakitna$ pytest -v -s
No setting files found. Please create a file in the `settings` folder using the Template.py.
TEMPORARY SETTINGS FILE CREATED
========================= test session starts ==========================
platform darwin -- Python 3.6.4, pytest-3.3.1, py-1.5.2, pluggy-0.6.0 -- /usr/local/opt/python3/bin/python3.6
cachedir: .cache
rootdir: /Users/lakitna/Documents/Github/Onaeri-tradfri/Onaeri, inifile:
collected 24 items
Add conftest.py to your project with below code and add your code to method pytest_configure
def pytest_configure(config):
pass # your code goes here
# other config
First, pytest tries to make all imports as relative as possible: this means that if you keep your tests in package/tests, then even your conftest will be imported as package.test.conftest. To do that import, package.__init__ will be called (unfortunately for you).
In order to run run some configuration before, the tests need to be moved out of the package tree. I ran into just such a problem recently.
As shown in a pytest issue thread, pytest_configure is actually called before pytest_sessionstart, but both occur before test collection, so either should be fine.
# contents of tests/conftest
import os
from pathlib import Path
import shutil
import pytest # if necessary
# cannot import package here (and likely not in `tests.__init__` either)
folderPath = Path.cwd()
folderPath = folderpath / 'package' / 'settings'
srcfile = folderPath / 'Template.py'
dstfile = folderPath / 'test.py'
def pytest_configure(config):
""" Create a test settings file """
# can use `config` to use command line options
shutil.copy(srcfile, dstfile)
def pytest_unconfigure(config):
""" Delete the test settings file """
os.remove(dstfile)
This should work if you have your folders structured like this:
root_folder
├── package
│ ├── settings
│ │ └── Template.py
│ ├── __init__.py
│ └── module1.py
└── tests
├── __init__.py # Does not import package
├── conftest.py # Does not import package
└── test_module1.py # Can import package
pytest can then be invoked in the normal way: pytest -flags (or pytest tests -flags) when working from root_folder.
If you want to set up your settings file in different ways depending on custom command line flags to pytest, you can do that as well in conftest.py. They can then be accessed in pytest_configure as config.getoption(...).
Note: this syntax assumes Python 3.6+ with the use of Paths as arguments to os.remove and shutil.copy.
I've managed to find a solution.
After some additional research, I found out that Pytest preloads all modules. This means that you can never run code before a module import unless you can find a hook before the collection phase. There is no such hook as far as I know. I really wanted to make this work within Pytest, but it seems to be impossible.
Instead, I created a __main__.py file in my test folder with the following content:
import pytest
import os
import shutil
"""
Create a setting file for the test procedure
"""
folderPath = os.path.dirname(os.path.abspath(__file__))
folderPath = os.path.split(folderPath)[0] + "/settings"
srcfile = folderPath + '/Template.py'
dstfile = folderPath + '/test.py'
shutil.copy(srcfile, dstfile)
"""
Actually run pytest
"""
pytest.main()
"""
Remove the test settings file
"""
os.remove(dstfile)
This code creates a temporary settings file, starts Pytest, and then removes the temporary file again.
I can now run the test procedure as follows:
$ python3 test
Pytest flags still work as normal. For example, if you want more verbose output you can do the following:
$ python3 test -v
This deviates from what you are looking for, but I would question your strategy. I suspect your code is not test friendly. The available hooks should suffice for setting up test pre-conditions like that. That is if your tests need some file, somehow setup that file first in setup, delete them with teardown, or use fixtures as per the pytest docs.
If you don't actually need the file for the tests to run, except that the code tests its existence, you can fake its existence with mocks.
But you have code running on module import that is testing (without actually being test code) a pre-condition and with user interaction.
And I think you shouldn't do this.
Remove the execution of that code running with import (make it a function or class). And invoke it in some other initialising place of your code.
Setup the file (with pytest hooks). Test code that uses the settings file.
Teardown the file (with pytest hooks).
You can in alternative completely fake the file by overriding the data read with fake data, using some data struture and functions to read/write,
simulating the file.
Test the code that tests for the settings file existence (use mocks).
Test your other code methods.
Do not mix things. Try to write methods that do one thing. Test one thing at a time. Test your business code. User interaction is another thing. If you also want to test UI, don't do it at the same time (I mean in the same code).
To run some code before and after the actual code of your test, you can use pytest fixtures:
import pytest
#pytest.fixture
def context():
"""What to do before/after the test."""
print("Entering !")
yield
print("Exiting !")
def test_stuff(context):
"""The actual code of your test"""
print("Running the test")
This displays:
Entering !
Running the test
Exiting !
To use the fixture, just pass a parameter with the same name as input of the test function. The yield instruction acts like a separator between what's done before and after the test.
I've got a large conftest.py file that I wish to split into smaller parts, for two reasons:
The file is very large (~1000 lines, including documentation)
Some of the fixtures depend on other fixtures, and I have no reason to expose those other fixtures as part of the conftest "API" when users look for relevant fixtures
I am not aware of any mechanism provided by pytest to resolve conftest files in multiple locations within the same folder, so I contrived one, below:
import sys
import os
sys.path.append(os.path.dirname(__file__))
from _conftest_private_part_1 import *
from _conftest_private_part_2 import *
from _conftest_private_part_3 import *
#pytest.fixture
def a_fixture_that_is_part_of_the_public_conftest_api():
pass
This works for my needs, but I do wonder if there is a better way.
You can put your stuff in other modules and reference them using a pytest_plugins variable in your conftest.py:
pytest_plugins = ['module1', 'module2']
This will also work if your conftest.py has hooks on them.
You shouldn't need any fancy magic for that. py.test automatically adds the path of the current test file to sys.path, as well as all parent paths up to the directory it was targeted at.
Because of that, you don't even need to put that shared code into a conftest.py. You can just put into plain modules or packages and then import it (if you want to share fixtures, those have to be in a conftest.py).
Also, there is this note about importing from conftest.py in the documentation:
If you have conftest.py files which do not reside in a python package
directory (i.e. one containing an __init__.py) then “import conftest”
can be ambiguous because there might be other conftest.py files as
well on your PYTHONPATH or sys.path. It is thus good practise for
projects to either put conftest.py under a package scope or to never
import anything from a conftest.py file.
Alternativelly, you could try:
In my_fixtures.py:
#pytest.fixture
def some_fixture():
yield
In conftest.py:
import my_fixtures
# Adding the fixture to attributes of `conftest` will register it
some_fixture = my_fixtures.some_fixture
It seems that pytest detect fixture by iterating over conftest attributes and checking for some attr._pytestfixturefunction added by #pytest.fixture.
So as long as conftest.py contains fixture attributes, it doesn't really matter which file is the fixture defined:
This works for me and seems easier/clearer:
Top level tests/conftest.py (example of re-usable print debug of requests.Response):
import pytest
import requests
from requests_toolbelt.utils import dump
#pytest.fixture(scope="session")
def print_response(response: requests.Response):
data = dump.dump_all(response)
print("========================")
print(data.decode('utf-8'))
print("========================")
print("response.url = {}".format(response.url))
print("response.request = {}".format(response.request))
print("response.status_code = {}".format(response.status_code))
print("response.headers['content-type'] = {}".format(response.headers['content-type']))
print("response.encoding = {}".format(response.encoding))
try:
print("response.json = {}".format(response.json()))
except Exception:
print("response.text = {}".format(response.text))
print("response.end")
From lower level conftest import the higher level conftest code - e.g., tests/package1/conftest.py:
from tests.conftest import *
Then, in your lower level tests within tests/package1/test_*.py, you simply import via:
from tests.package1 import conftest
And then you have the merged configtests from one conftest available. Repeat this pattern for your other lower level detailed/modular conftest.py files throughout the tests hierarchy.
I have a Python project with the following directory structure:
project/
project/src/
project/src/somecode.py
project/src/mypackage/mymodule.py
project/src/resources/
project/src/resources/datafile1.txt
In mymodule.py, I have a class (lets call it "MyClass") which needs to load datafile1.txt. This sort of works when I do:
open ("../resources/datafile1.txt")
Assuming the code that creates the MyClass instance created is run from somecode.py.
The gotcha however is that I have unit tests for mymodule.py which are defined in that file, and if I leave the relative pathname as described above, the unittest code blows up as now the code is being run from project/src/mypackage instead of project/src and the relative filepath doesn't resolve correctly.
Any suggestions for a best practice type approach to resolve this problem? If I move my testcases into project/src that clutters the main source folder with testcases.
I usually use this to get a relative path from my module. Never tried in a unittest tho.
import os
print(os.path.join(os.path.dirname(__file__),
'..',
'resources'
'datafile1.txt'))
Note: The .. tricks works pretty well, but if you change your directory structure you would need to update that part.
On top of the above answers, I'd like to add some Python 3 tricks to make your tests cleaner.
With the help of the pathlib library, you can explicit your ressources import in your tests. It even handles the separators difference between Unix (/) and Windows ().
Let's say we have a folder structure like this :
`-- tests
|-- test_1.py <-- You are here !
|-- test_2.py
`-- images
|-- fernando1.jpg <-- You want to import this image !
`-- fernando2.jpg
You are in the test_1.py file, and you want to import fernando1.jpg. With the help to the pathlib library, you can read your test resource with an object oriented logic as follows :
from pathlib import Path
current_path = Path(os.path.dirname(os.path.realpath(__file__)))
image_path = current_path / "images" / "fernando1.jpg"
with image_path.open(mode='rb') as image :
# do what you want with your image object
But there's actually convenience methods to make your code more explicit than mode='rb', as :
image_path.read_bytes() # Which reads bytes of an object
text_file_path.read_text() # Which returns you text file content as a string
And there you go !
in each directory that contains Python scripts, put a Python module that knows the path to the root of the hierarchy. It can define a single global variable with the relative path. Import this module in each script. Python searches the current directory first so it will always use the version of the module in the current directory, which will have the relative path to the root of the current directory. Then use this to find your other files. For example:
# rootpath.py
rootpath = "../../../"
# in your scripts
from rootpath import rootpath
datapath = os.path.join(rootpath, "src/resources/datafile1.txt")
If you don't want to put additional modules in each directory, you could use this approach:
Put a sentinel file in the top level of the directory structure, e.g. thisisthetop.txt. Have your Python script move up the directory hierarchy until it finds this file. Write all your pathnames relative to that directory.
Possibly some file you already have in the project directory can be used for this purpose (e.g. keep moving up until you find a src directory), or you can name the project directory in such a way to make it apparent.
You can access files in a package using importlib.resources (mind Python version compatibility of the individual functions, there are backports available as importlib_resources), as described here. Thus, if you put your resources folder into your mypackage, like
project/src/mypackage/__init__.py
project/src/mypackage/mymodule.py
project/src/mypackage/resources/
project/src/mypackage/resources/datafile1.txt
you can access your resource file in code without having to rely on inferring file locations of your scripts:
import importlib.resources
file_path = importlib.resources.files('mypackage').joinpath('resources/datafile1.txt')
with open(file_path) as f:
do_something_with(f)
Note, if you distribute your package, don't forget to include the resources/ folder when creating the package.
The filepath will be relative to the script that you initially invoked. I would suggest that you pass the relative path in as an argument to MyClass. This way, you can have different paths depending on which script is invoking MyClass.