I recently started playing with pytest and I use pytest.main() to run the tests. However it seems that pytest caches the test. Any changes made to my module or to the tests gets ignored. I am unable to run pytest from command line so pytest.main() is my only option, this is due to writing python on my ipad.
I have googled this extensively and was able to find one similar issue with advice to run pytest from command line. Any help would be greatly appreciated.
Thanks,
Pytest doesn't cache anything. A module (file) is read once and only once per instance of a Python interpreter.
There is a reload built-in, but it almost never does what you hope it will do.
So if you are running
import pytest
...
while True:
import my_nifty_app
my_nifty_app.be_nifty()
pytest.main()
my_nifty_app.py will be read once and only once even if it changes on disk. What you really need is something like
exit_code = pytest.main()
sys.exit(exit_code)
which will end that instance of the interpreter which is the only way to ensure your source files get re-read.
Related
I have a python module which is designed to be used by non-programmers, interactively. It is used within iPython, and on load, it asks for a couple of user inputs, prints some ascii art, that sort of thing. This is implemented in the __init__.py of the module.
The module also contains a utils file tld(containing setup.py)/utils.py, with some functions in it. I would like to test these. The utils file does not import anything from the rest of the module, and contains only pure functions. The tests live in tld(containing setup.py)/tests/test_utils.pyCurrently, trying to run pytest results in a failure, as the module's __init__ file is run, which hangs whilst awaiting the above mentioned user input.
Is there a way of getting pytest to run the tests in test_utils.py, without running the __init__.py of the python module? Or some other way of getting around this issue?
I have tried running pytest tests/test_utils.py, but that had the same effect.
I solved this by defining an initialize function inside the __init__.py of the module, which encapsulates all of the user interaction. If module.initialize() is not explicitly called, no user input is requested, so the pytest tests can run as expected.
The traceback provided by pytest is great and super useful for debugging.
Is there a way to run a script using the pytest api even if the script itself does not contain any test modules? Essentially, I would like a way to pinpoint and run a certain function in a script as if it were a test, but get the pytest-formatted traceback.
The pytest documentation on test discovery states that normally only functions whose name begins with test_ are run. This behaviour can be changed however with the python_functions configuration option. Try entering in the command line:
pytest [script.py] -o python_functions=[script_function]
in which you should replace [script.py] with your python script file path and replace [script_function] with the name of the function that you want to be run.
I am using py.test for unit testing my python program. I wish to debug my test code with the python debugger the normal way (by which I mean pdb.set_trace() in the code) but I can't make it work.
Putting pdb.set_trace() in the code doesn't work (raises IOError: reading from stdin while output is captured). I have also tried running py.test with the option --pdb but that doesn't seem to do the trick if I want to explore what happens before my assertion. It breaks when an assertion fails, and moving on from that line means terminating the program.
Does anyone know a way to get debugging, or is debugging and py.test just not meant to be together?
it's real simple: put an assert 0 where you want to start debugging in your code and run your tests with:
py.test --pdb
done :)
Alternatively, if you are using pytest-2.0.1 or above, there also is the pytest.set_trace() helper which you can put anywhere in your test code. Here are the docs. It will take care to internally disable capturing before sending you to the pdb debugger command-line.
I found that I can run py.test with capture disabled, then use pdb.set_trace() as usual.
> py.test --capture=no
============================= test session starts ==============================
platform linux2 -- Python 2.5.2 -- pytest-1.3.3
test path 1: project/lib/test/test_facet.py
project/lib/test/test_facet.py ...> /home/jaraco/projects/project/lib/functions.py(158)do_something()
-> code_about_to_run('')
(Pdb)
The easiest way is using the py.test mechanism to create breakpoint
http://pytest.org/latest/usage.html#setting-a-breakpoint-aka-set-trace
import pytest
def test_function():
...
pytest.set_trace() # invoke PDB debugger and tracing
Or if you want pytest's debugger as a one-liner, change your import pdb; pdb.set_trace() into import pytest; pytest.set_trace()
Similar to Peter Lyon's answer, but with the exact code you need for pytest, you can add the following to the bottom of your pytest module (my_test_module.py) :
if __name__ == "__main__":
pytest.main(["my_test_module.py", "-s"])
Then you can invoke the debugger from the command line:
pdb3 my_test_module.py
Boom. You're in the debugger and able to enter debugger commands. This method leaves your test code un-littered with set_trace() calls and will run inside pytest 'normally'.
Simply use: pytest --trace test_your_test.py.
This will invoke the Python debugger at the start of the test
I'm not familiar with py.test, but for unittest, you do the following. Maybe py.test is similar:
In your test module (mytestmodule.py):
if __name__ == "__main__":
unittest.main(module="mytestmodule")
Then run the test with
python -m pdb mytestmodule.py
You will get an interactive pdb shell.
Looking at the docs, it looks like py.test has a --pdb command line option:
https://docs.pytest.org/en/7.2.x/reference/reference.html#command-line-flags
Add and remove breakpoints without editing source files
Although you can add breakpoints by adding breakpoint() or set_trace() statements to your code, there are two issues with this approach:
Firstly, once you have started running your code, there is no way to remove your breakpoint. I often find that once I start running my code and reach an initial breakpoint, I want to place another one and remove the initial breakpoint. After breakpoint() drops me into the debugger I can add additional breakpoints, but I can't remove the initial one. Although this can be mitigated somewhat by putting the initial breakpoint statement higher up, if you have parametrised tests then even that is limited. I may find myself repeating cont very often.
Secondly, it requires changes to the source code. You need to remember to remove all breakpoint() commands before committing any code to version control, you have to remove them before switching branches, etc. I sometimes find I want to use the debugger to compare test runs between two branches, and having to edit the source code to add a breakpoint every time makes that a considerably slower and more error-prone exercise. I may even want to add a breakpoint in a library I'm calling, in which case the file I'm editing may not even me in my git repository but somewhere deep in my conda environment, increasing the risk of forgetting to remove it. Editing files to add break points is, in my humble opinion, ugly.
To add and remove breakpoints interactively without editing any source files, you can evoke pytest as follows (in the bash shell):
python -mipdb $(type -p pytest) -s test_fileset.py
The -s flag is crucial here, because it stops pytest from messing with stdin and stdout, and when running inside the debugger, pytest will fail to mess with stdin and stdout and everything will go wrong. The exact calling syntax will be different for different shells.
I am developing a Python package using a text editor and IPython. Each time I change any of the module code I have to restart the interpreter to test this. This is a pain since the classes I am developing rely on a context that needs to be re-established on each reload.
I am aware of the reload() function, but this appears to be frowned upon (also since it has been relegated from a built-in in Python 3.0) and moreover it rarely works since the modules almost always have multiple references.
My question is - what is the best/accepted way to develop a Python module/package so that I don't have to go through the pain of constantly re-establishing my interpreter context?
One idea I did think of was using the if __name__ == '__main__': trick to run a module directly so the code is not imported. However this leaves a bunch of contextual cruft (specific to my setup) at the bottom of my module files.
Ideas?
A different approach may be to formalise your test driven development, and instead of using the interpreter to test your module, save your tests and run them directly.
You probably know of the various ways to do this with python, I imagine the simplest way to start in this direction is to copy and paste what you do in the interpreter into the docstring as a doctest and add the following to the bottom of your module:
if __name__ == "__main__":
import doctest
doctest.testmod()
Your informal test will then be repeated every time the module is called directly. This has a number of other benefits. See the doctest docs for more info on writing doctests.
Ipython does allow reloads see the magic function %run iPython doc
or if modules under the one have changed the recursive dreloadd() function
If you have a complex context is it possible to create it in another module? or assign it to a global variable which will stay around as the interpreter is not restarted
How about using nose with nosey to run your tests automatically in a separate terminal every time you save your edits to disk? Set up all the state you need in your unit tests.
you could create a python script that sets up your context and run it with
python -i context-setup.py
-i When a script is passed as first argument or the -c option is
used, enter interactive mode after executing the script or the
command. It does not read the $PYTHONSTARTUP file. This can be
useful to inspect global variables or a stack trace when a
script raises an exception.
I want to run the unittests of Jinja2 whenever I change something to make sure I'm not breaking something.
There's a package full of unit tests. Basically it's a folder full of Python files with the name "test_xxxxxx.py"
How do I run all of these tests in one command?
It looks like Jinja uses the py.test testing tool. If so you can run all tests by just running py.test from within the tests subdirectory.
You could also take a look at nose too. It's supposed to be a py.test evolution.
Watch out for "test.py" in the Jinja2 package! -- Those are not unit tests! That is a set of utility functions for checking attributes, etc. My testing package is assuming that they are unit tests because of the name "test" -- and returning strange messages.
Try to 'walk' through the directories and import all from files like "test_xxxxxx.py", then call unittest.main()