I want to run the unittests of Jinja2 whenever I change something to make sure I'm not breaking something.
There's a package full of unit tests. Basically it's a folder full of Python files with the name "test_xxxxxx.py"
How do I run all of these tests in one command?
It looks like Jinja uses the py.test testing tool. If so you can run all tests by just running py.test from within the tests subdirectory.
You could also take a look at nose too. It's supposed to be a py.test evolution.
Watch out for "test.py" in the Jinja2 package! -- Those are not unit tests! That is a set of utility functions for checking attributes, etc. My testing package is assuming that they are unit tests because of the name "test" -- and returning strange messages.
Try to 'walk' through the directories and import all from files like "test_xxxxxx.py", then call unittest.main()
Related
The traceback provided by pytest is great and super useful for debugging.
Is there a way to run a script using the pytest api even if the script itself does not contain any test modules? Essentially, I would like a way to pinpoint and run a certain function in a script as if it were a test, but get the pytest-formatted traceback.
The pytest documentation on test discovery states that normally only functions whose name begins with test_ are run. This behaviour can be changed however with the python_functions configuration option. Try entering in the command line:
pytest [script.py] -o python_functions=[script_function]
in which you should replace [script.py] with your python script file path and replace [script_function] with the name of the function that you want to be run.
I have 60 functional tests in one file. I wrote them in Notepad++ and used py.test as the test framework. Today I decided to swap Notepad++ with PyCharm. I opened my file of functional tests in PyCharm and ran the tests from PyCharm, as you can see in the picture:
Now, after confirming that I could run all of the tests, I tried to run an individual test, for example test_login_with_extantUser_using_email. Logically, I right-clicked on the test, expecting a "run test" button or something similar to appear. But no such thing appeared. In fact, it appears that there is no way to run an individual test by simply right-clicking on it.
So my question is, how can I run an individual test? Must I set up a configuration for each one in the Edit Configurations menu? That would take a very long time, considering that I have 60 tests.
I have found the problem. It's a bug in Pycharm that needs to be fixed. I will explain the bug.
Right-clicking an individual test will not display the option to run the test if the test is not a member of a class that inherits from unittest.TestCase. This is true even if you are not using unittest, as in my case, in which I am using py.test.
When I made my py.test test classes inherit from unittest.TestCase, I got the option to run tests individually when I right-clicked on a test.
I have reported the bug to Pycharm. Time will tell if they fix it.
https://youtrack.jetbrains.com/issue/PY-26754
Base on pt.test doc you can specify particular test_case for running with -k (which means keywords obviously). In your case it should be like pytest -k "test_login_with_extantUser_using_email" if you running in CLI.
If you want to run particular test with Pycharm, you have to fill 'Keywords' field in Run/Debug Configurations.
Best Regards.
I'm creating a python test suite (using py.test). I'm coding the tests in Idea and I don't know how to debug a single test.
This is my setting of the debugger. It runs the whole testsuite. So I have to run all the tests before it gets to the one I'm trying to debug.
In your configuration, set:
Target to the relative path of one of your test files, i.e. testsuite/psa/test_psa_integration.py
Keywords to a keyword that identifies the test you are trying to run specifically. If tests are part of a class, Keywords should be something like: TestPsaIntegration and test_psa_integration_example
I don't use IntelliJ, but in PyCharm, you can easily debug tests without going through this tedious process of adding a Run/Debug configuration each time.
To do this with PyCharm, go to:
Preferences (or Settings) > Tools > Python Integrated Tools and set Default test runner to py.test.
Then, back in your file (i.e. test_psa_integration.py), you could just right-click anywhere within the code of a test, and select either Run 'py.test in ...' or Debug 'py.test in...' which will automatically create a new Run/Debug configuration as explained previously.
An alternative is adding --no-cov --capture=no into Additional Arguments. To make this automatic for other test file, add those into the Template part.
I have a python library that reads a config file or environment variables to set some global configuration variables.
I would like to run my test suite multiple times with different settings.
I could do this manually like
MYLIB_SETTINGS=enable_foo=True nosetests
MYLIB_SETTINGS=enable_foo=False nosetests
I'm wondering if there's a way to do this automatically using the nose api and combine the results.
Normally, you would run your test in some sort of Continuous Integration framework (like Jenkins) with --with-xunit and --xunit-file TEST_NAME_XXX.xml. Each test run will produce a separate xml file, and the CI tool will combine them together into a pretty table, showing all tests from both cases.
You can do something similar using nose API, by setting os.env variables appropriately in python and calling nose.run()
I recently started playing with pytest and I use pytest.main() to run the tests. However it seems that pytest caches the test. Any changes made to my module or to the tests gets ignored. I am unable to run pytest from command line so pytest.main() is my only option, this is due to writing python on my ipad.
I have googled this extensively and was able to find one similar issue with advice to run pytest from command line. Any help would be greatly appreciated.
Thanks,
Pytest doesn't cache anything. A module (file) is read once and only once per instance of a Python interpreter.
There is a reload built-in, but it almost never does what you hope it will do.
So if you are running
import pytest
...
while True:
import my_nifty_app
my_nifty_app.be_nifty()
pytest.main()
my_nifty_app.py will be read once and only once even if it changes on disk. What you really need is something like
exit_code = pytest.main()
sys.exit(exit_code)
which will end that instance of the interpreter which is the only way to ensure your source files get re-read.