I use the pylint extension in Visual Studio Code within the Debug module (see image) to run my tests using Python Unittest library.
Whenever I run my tests, I execute my test.py file in Debug and it runs all of the tests in the entire file. I have my tests broken up logically by classes with several tests per class.
To reduce the time it takes evaluate test results for a single test I am actively working on, is there a way to execute just that one test rather than wait for all the tests in the test.py file to execute while in VS Code Debug mode?
For example:
test.py
import unittest
class test_TestClass(unittest.TestCase):
def test_Test1(self):
x = 1
self.assertEqual(x, 1)
def test_Test2(self):
y = 2
self.assertEqual(y, 3)
If I want to only execute test_Test2() to make sure that it fails, how do I do that without running the entire file (i.e., test_Test1 and test_Test2)?
When your tests are discovered by Python: Run All Unit Tests (or clicking on the Run Tests button in the status bar and choosing Discover Unit Tests), a code lens is provided to run individual test functions, methods, and classes either normally or under the debugger.
Clicking on Debug Test will run open the debugger and run the unit test under it. Otherwise you can modify your launch.json to add a debug configuration and specify the arguments to your test runner to just run the test(s) you're interested in.
Related
I am trying to use pytest within VS code, running in red hat linux. The environment that I am using means that I need to load modules such as pandas before running pytest. In the terminal I can run:
module load pandas
pytest
and the tests are successfully run. I can do this in both the standard terminal, in the Python Debug Console within VS code, and int he bash terminal within VS code. If, however, I press the "Run All Tests" button within VS code, then I just get an error telling me that it cannot find the pandas module.
How can I tell the test environment to run my module load pandas command before running pytest?
In this case I would create a file named conftest.py in the directory containing the tests. pytest automatically executes this file before the tests are run. In this file you could have Python execute shell commands. For the latter there are different options, but first try one of the easier ones.
More information on conftest.py:
In pytest, what is the use of conftest.py files?
I am currently developing some tests using python py.test / unittest that, via subprocess, invoke another python application (so that I can exercise the command line options, and confirm that the tool is installed correctly).
I would like to be able to run the tests in such a way that I can get a view of the code coverage metrics (using coverage.py) for the target application using pytest_cov. By default this does not work as the code coverage instrumentation does not apply to code invoked with subprocess.
Code Coverage of the code does work if I update the tests to directly invoke the entry class for the target application (rather than running via the command line).
Ideally I want to have a single set of code which can be run in two ways:
If code coverage monitoring is not enabled then use command line
Otherwise execute the main class of the target application.
Which leads to my question(s):
Is it possible for a python unit test to determine if it is being run with code coverage enabled?
Otherwise: is there any easy way to pass a command line flag from the pytest invocation that can be used to set the mode within the code.
Coverage.py has a facility to automatically measure coverage in sub-processes that are spawned: http://coverage.readthedocs.io/en/latest/subprocess.html
Coverage sets three environment flags when running tests: COV_CORE_SOURCE, COV_CORE_CONFIG and COV_CORE_DATAFILE.
So you can use a simple if-statement to verify whether the current test is being run with coverage enabled:
import os
if "COV_CORE_SOURCE" in os.environ:
# do what yo need to do when Coverage is enabled
Context
I have a python application that I'm unit testing. Half the application is working and I have a very high test accuracy.
The application requires one-time user input for installation purposes.
This means that, if you run the code, there has to be interaction with a user.
Problem
Coverage is a Python plugin for coverage reports. I use coverage with this command:
coverage run application.py
Coverage runs my application, goes through my tests, and delivers a coverage report.
The problem is that the command to run those tests, executes my application and I have to provide input. That's not that big of a deal, but I cannot do that on my CI server using Jenkins (or can I?).
Question
I want to run the coverage tool without user input. In my tests, the input function is mocked out. Running all my tests without coverage works fine. How can I prevent coverage from requiring user input?
You should probably have 2 different code paths, one for running the tests, and one for running the app:
coverage run tests.py
with tests.py importing application.py, mocking methods as necessary, then running the actual application.
Or you could allow user input via command line arguments:
coverage run application.py --user=input --other="etc."
Finally, if there truly are portions of your app that cannot be tested or reasonably mocked (it happens, say you're calling out into a third party exception tracking library/service that you can't load in your tests), you can instruct coverage to ignore those lines for the purposes of computing coverage, by adding # pragma: no cover at the end of the instruction that you won't be fully testing:
my = "code"
goes = "here"
if debug: # pragma: no cover
call_untestable(code=True)
this_portion(ignored_for_coverage=True)
covered_code = "yes, again!"
See more here:
http://coverage.readthedocs.io/en/coverage-4.2/excluding.html
I need to make a test suite incorporating slow tests from several different files, and run from command line. I am testing one file at a time, and don't want to move these slow tests, the most logical way to organize is to test them next to their neighboring functions.
I can't even find how to run test suites from command line, without which they seem to have no purpose. I'd like to run all tests in this suite, something like python -m unittest slow, and only run the tests that aren't in the slow suite when I run regular, python -m unittest.
I'm creating a python test suite (using py.test). I'm coding the tests in Idea and I don't know how to debug a single test.
This is my setting of the debugger. It runs the whole testsuite. So I have to run all the tests before it gets to the one I'm trying to debug.
In your configuration, set:
Target to the relative path of one of your test files, i.e. testsuite/psa/test_psa_integration.py
Keywords to a keyword that identifies the test you are trying to run specifically. If tests are part of a class, Keywords should be something like: TestPsaIntegration and test_psa_integration_example
I don't use IntelliJ, but in PyCharm, you can easily debug tests without going through this tedious process of adding a Run/Debug configuration each time.
To do this with PyCharm, go to:
Preferences (or Settings) > Tools > Python Integrated Tools and set Default test runner to py.test.
Then, back in your file (i.e. test_psa_integration.py), you could just right-click anywhere within the code of a test, and select either Run 'py.test in ...' or Debug 'py.test in...' which will automatically create a new Run/Debug configuration as explained previously.
An alternative is adding --no-cov --capture=no into Additional Arguments. To make this automatic for other test file, add those into the Template part.