The traceback provided by pytest is great and super useful for debugging.
Is there a way to run a script using the pytest api even if the script itself does not contain any test modules? Essentially, I would like a way to pinpoint and run a certain function in a script as if it were a test, but get the pytest-formatted traceback.
The pytest documentation on test discovery states that normally only functions whose name begins with test_ are run. This behaviour can be changed however with the python_functions configuration option. Try entering in the command line:
pytest [script.py] -o python_functions=[script_function]
in which you should replace [script.py] with your python script file path and replace [script_function] with the name of the function that you want to be run.
Related
From the question on running a single test via command line when tests are located within a sibling folder, the answer suggests using the -v option alongside the module name and test name to run a specific test.
Why does the -v option make this work? Specifying the module name and the test name makes sense since it corresponds to the unittest documnetation and obviously you need to specify which test to run. However, from what I can tell, the -v option corresponds to verbose output which shouldn't change the tests that the unittest module runs.
Apologies in advance if I've missed something obvious here.
So the reason this wasn't working was because of a pretty obvious, but stupid, error on my part 😅.
tldr; Use the full command line to run the tests (e.g. python3 -m unittest tests.module_name.TestClass.test_func) or if you're using a bash function, make sure the function accepts other arguments.
I had setup a bash function called run_tests to run unittests and I was trying to specify the module name and test name after calling that method. I.e. I had the following in .bash_profile:
run_tests ()
{
python3 -m unittest
}
and on the terminal, I did:
run_tests tests.module_name.TestClass.test_func
Since the bash function was not setup to accept arguments, the specific test I wanted to run wasn't actually being passed as an argument to unittest.
Obviously, using -v makes no difference if you use the run_tests function to try and run a specific test.
When I tested with the -v option, I used the full command python3 -m unittest -v tests.module_name.TestClass.test_func which is why I thought the -v option made it work. To test whether the -v option actually worked, I was lazy and ran run_tests tests.module_name.TestClass.test_func again since it was in my shell history instead of typing out the full command, which is what caused this confusion.
I am trying to use pytest within VS code, running in red hat linux. The environment that I am using means that I need to load modules such as pandas before running pytest. In the terminal I can run:
module load pandas
pytest
and the tests are successfully run. I can do this in both the standard terminal, in the Python Debug Console within VS code, and int he bash terminal within VS code. If, however, I press the "Run All Tests" button within VS code, then I just get an error telling me that it cannot find the pandas module.
How can I tell the test environment to run my module load pandas command before running pytest?
In this case I would create a file named conftest.py in the directory containing the tests. pytest automatically executes this file before the tests are run. In this file you could have Python execute shell commands. For the latter there are different options, but first try one of the easier ones.
More information on conftest.py:
In pytest, what is the use of conftest.py files?
By default, pytest inflates the error traceback massively and printly some information into sysout stream that are redundant: Considering that I'm using PyCharm, it is really obfuscating to see code snippet out of context, while they are already available in the IDE & debugging interface.
As a result, I intend to set pytest traceback to native permanently. However, according to the documentation, the only way to do so is to add extra command line argument when launching the test runner:
-tb=native
I would like to make my test to always use the native traceback regardless of how it was run. Is it possible to use a TestCase API to do so?
Thanks a lot for your help.
You can add this option to the pytest.ini file and it would be automatically picked by pytest. For your specific case, a pytest.ini with following contents should work:
[pytest]
addopts = --tb=native
Note the double hyphens with tb; I am using pytest 4.6.4 and that is how it works for me.
Also, Refer pytest docs for another alternative by modifying PYTEST_ADDOPTS env variable.
I'm not sure how you can do this using pytest, nor am I familiar with this package. With that being said, you can always create a bash function to accomplish this:
function pytest() {
pytest -tb=native "$#"
}
The "$#" symbol will pass all arguments following pyt to the function (kind of like *args in python), so running pyt arg1 arg2 ... argn will be the same as running
pytest -tb=native arg1 arg2 ... argn
If you are unfamiliar with creating bash shortcuts, see this question.
Update
I misunderstood and thought OP was calling pytest from the cli. Instead of creating the pyt function, if you override pytest directly, PyCharm might invoke your bash version of it instead (I'm not really sure though).
That being said, yaniv's answer seems superior to this, if it works.
I am currently developing some tests using python py.test / unittest that, via subprocess, invoke another python application (so that I can exercise the command line options, and confirm that the tool is installed correctly).
I would like to be able to run the tests in such a way that I can get a view of the code coverage metrics (using coverage.py) for the target application using pytest_cov. By default this does not work as the code coverage instrumentation does not apply to code invoked with subprocess.
Code Coverage of the code does work if I update the tests to directly invoke the entry class for the target application (rather than running via the command line).
Ideally I want to have a single set of code which can be run in two ways:
If code coverage monitoring is not enabled then use command line
Otherwise execute the main class of the target application.
Which leads to my question(s):
Is it possible for a python unit test to determine if it is being run with code coverage enabled?
Otherwise: is there any easy way to pass a command line flag from the pytest invocation that can be used to set the mode within the code.
Coverage.py has a facility to automatically measure coverage in sub-processes that are spawned: http://coverage.readthedocs.io/en/latest/subprocess.html
Coverage sets three environment flags when running tests: COV_CORE_SOURCE, COV_CORE_CONFIG and COV_CORE_DATAFILE.
So you can use a simple if-statement to verify whether the current test is being run with coverage enabled:
import os
if "COV_CORE_SOURCE" in os.environ:
# do what yo need to do when Coverage is enabled
I'm trying to setup Python to execute nose, but only on an existing application I'm developing locally. I don't want nose running around all libraries that are currently installed. I do, however, want nose to discover any tests within the current working directory and child directories.
To start with, all I'm trying to do is make sure that the arguments I'm passing are being used (solved by #need-batchelder below). However, at the moment it looks like the arguments I am passing are being ignored, and global discovery of the tests is occurring (i.e. picking up tests from the python folder too.
From the docs:
-V, --version
Output nose version and exit
Running nosetests -V from the command line produces the expected version output:
nosetests -V
nosetests-script.py version 1.2.1
However, the following test script starts running every test it can find, including those of libraries installed in the python path and not part of the current working directory , even though it is located in the root of the application:
import nose, os
def main():
print os.getcwd()
x=raw_input() #This is just so I can see the output of the cwd before it launches into testing everything it can find.
result = nose.run(argv=['-V'])
if __name__ == '__main__':
main()
Here's what I've tried:
Using nose.main() , x=nose.core.run() , x=nose.run().
Passing the arguments directly to nose.run() and using a list.
Using a nose.cfg file.
Thanks
EDIT: Trying #ned-batchelder 's suggestion allows me to run nose with given arguments, but doesn't allow discovery of tests within the application folders. So if I do that, I can pass arguments but I can't test my application.
I believe nose expects argv to be the complete argv, meaning the first element should be the name of the program:
nose.run(argv=['me.py', '-V'])
Probably, what you want is:
arguments = sys.argv[:1] + my_custom_argument_list + sys.argv[1:]
nose.run(argv=arguments)
This will allow you to use your custom arguments as well as those from the command line that invokes your script. It also address the issue that Ned points out about nose requiring that the first argument point to the script.