Everything works fine with PyCharm and pytest, except if I have failing tests, then it duplicates the error output:
One of actual failures if the red one and other is white. This is really annoys, and I haven't found any way to disable such behaviour.
There is an option to disable log via py.test, however it will disable all logging.
Note: everything works as expected if I run python -m pytest test.py.
I think that is a feature not a bug. The top level is being emitted during the testing which allows you to review the failure before the testing is complete. The second copy of the results is the summary which effectively removes any of the text that was showing test progress.
You can easily view just part of the test output by clicking on the test hierarchy:
The duplicated output can be elimited by running pytest with the -q or --quiet parameter.
You can configure the parameter to be applied to all PyCharm pytest tests by setting it in Edit Configurations --> Templates --> Python Tests --> pytest --> Additional Arguments.
This will then apply those arguments to all new run configurations. If you have a bunch of existing test run configurations, deleting all of them and then re-generating them by running a test or tests using the gutter icon is the quickest way to reset the output.
Related
The results of python tests are cached for only a short time in the VSCode MS python extension.
Whether they succeeded or failed is only cached for a short while, and then they revert back to question marks again.
How to retain the results of the tests?
I think it has to do with this some logs that I see in the Output for "Python Test Log", which shows many (~30) of these lines:
python /home/.../.vscode/extensions/ms-python.python-2020.6.89148/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /home/projectdir --cache-clear -s
In particular the --cache-clear is suspicious, however I don't know what is triggering these outputs.
So my question is: how to remove --cache-clear from the call? Or maybe something else is going on entirely?
I too found myself in need of disabling this vscode behaviour.
I'm using pytest with the pytest-html plugin, and after all tests have run, as soon as i change a test file and save, this command is automatically run by vscode:
python c:\Users\username\.vscode\extensions\ms-python.python-2021.6.944021595\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\username\app\app-test-automation -s --cache-clear --html=report.html --self-contained-html tests
Which inevitably overwrite the .html test report.
I haven't found a way to remove the --cache-clear from the call.
BUT
You can disable the auto discovery of test (guilty of rerunning the command with the --cache-clear flag) by changing the following vscode setting:
Picture of the setting from the vscode settings menu
Context
I have a python application that I'm unit testing. Half the application is working and I have a very high test accuracy.
The application requires one-time user input for installation purposes.
This means that, if you run the code, there has to be interaction with a user.
Problem
Coverage is a Python plugin for coverage reports. I use coverage with this command:
coverage run application.py
Coverage runs my application, goes through my tests, and delivers a coverage report.
The problem is that the command to run those tests, executes my application and I have to provide input. That's not that big of a deal, but I cannot do that on my CI server using Jenkins (or can I?).
Question
I want to run the coverage tool without user input. In my tests, the input function is mocked out. Running all my tests without coverage works fine. How can I prevent coverage from requiring user input?
You should probably have 2 different code paths, one for running the tests, and one for running the app:
coverage run tests.py
with tests.py importing application.py, mocking methods as necessary, then running the actual application.
Or you could allow user input via command line arguments:
coverage run application.py --user=input --other="etc."
Finally, if there truly are portions of your app that cannot be tested or reasonably mocked (it happens, say you're calling out into a third party exception tracking library/service that you can't load in your tests), you can instruct coverage to ignore those lines for the purposes of computing coverage, by adding # pragma: no cover at the end of the instruction that you won't be fully testing:
my = "code"
goes = "here"
if debug: # pragma: no cover
call_untestable(code=True)
this_portion(ignored_for_coverage=True)
covered_code = "yes, again!"
See more here:
http://coverage.readthedocs.io/en/coverage-4.2/excluding.html
I have 60 functional tests in one file. I wrote them in Notepad++ and used py.test as the test framework. Today I decided to swap Notepad++ with PyCharm. I opened my file of functional tests in PyCharm and ran the tests from PyCharm, as you can see in the picture:
Now, after confirming that I could run all of the tests, I tried to run an individual test, for example test_login_with_extantUser_using_email. Logically, I right-clicked on the test, expecting a "run test" button or something similar to appear. But no such thing appeared. In fact, it appears that there is no way to run an individual test by simply right-clicking on it.
So my question is, how can I run an individual test? Must I set up a configuration for each one in the Edit Configurations menu? That would take a very long time, considering that I have 60 tests.
I have found the problem. It's a bug in Pycharm that needs to be fixed. I will explain the bug.
Right-clicking an individual test will not display the option to run the test if the test is not a member of a class that inherits from unittest.TestCase. This is true even if you are not using unittest, as in my case, in which I am using py.test.
When I made my py.test test classes inherit from unittest.TestCase, I got the option to run tests individually when I right-clicked on a test.
I have reported the bug to Pycharm. Time will tell if they fix it.
https://youtrack.jetbrains.com/issue/PY-26754
Base on pt.test doc you can specify particular test_case for running with -k (which means keywords obviously). In your case it should be like pytest -k "test_login_with_extantUser_using_email" if you running in CLI.
If you want to run particular test with Pycharm, you have to fill 'Keywords' field in Run/Debug Configurations.
Best Regards.
I have a Python project and a tests task, set up to run pytest from the project's working directory.
Doing Run 'tests' with coverage from the Run menu successfully runs the tests, and the console results shows that coverage was measured - e.g. 53% cover for mws.py.
The automatically applied coverage (as on the right) is 0% for all files, I'm not sure why. I'm using IntelliJ 2017.2.2 EAP.
NB: there is a related five year old question here, but the top rated solution there doesn't apply. There is no error message in the results console in this case.
I had a similar issue, but the accepted solution here didn't solve it.
I had pytest automatically run coverage in its configuration file.
In PyCharm, I added a Run Configuration to run all my tests with pytest.
It seemed to work, and I saw all tests running and got their results to display in PyCharm's run window.
But soon I noticed there were two problems:
When I selected "Run with coverage" I got an error like "coverage results not found", and all files showed 0% coverage.
Breakpoints in tests were not hit when running test in Debug mode.
Both problems disappeared when I added --no-cov to the "Additional Arguments" passed to to pytest (this option is in the Run Configuration).
So It seems the fix was to tell pytest to not run coverage when running it from PyCharm. Both "Run with coverage" option and the Breakpoints in tests now work.
I think the problem lies in you use pytest-cov, so Pycharm cannot parse the result which is shown in text like 53% generated by pytest-cov;
So Changes option in pytest.ini to addopts = -s -v when you want to use Pycharm built in coverage tools.
In this command -v stands for verbosity and -s for disabling all output.
Take a look at my answer for other question about same issue: https://stackoverflow.com/a/45729723/1229510
Basically if you use symlinks - coverage display won't work.
I'm creating a python test suite (using py.test). I'm coding the tests in Idea and I don't know how to debug a single test.
This is my setting of the debugger. It runs the whole testsuite. So I have to run all the tests before it gets to the one I'm trying to debug.
In your configuration, set:
Target to the relative path of one of your test files, i.e. testsuite/psa/test_psa_integration.py
Keywords to a keyword that identifies the test you are trying to run specifically. If tests are part of a class, Keywords should be something like: TestPsaIntegration and test_psa_integration_example
I don't use IntelliJ, but in PyCharm, you can easily debug tests without going through this tedious process of adding a Run/Debug configuration each time.
To do this with PyCharm, go to:
Preferences (or Settings) > Tools > Python Integrated Tools and set Default test runner to py.test.
Then, back in your file (i.e. test_psa_integration.py), you could just right-click anywhere within the code of a test, and select either Run 'py.test in ...' or Debug 'py.test in...' which will automatically create a new Run/Debug configuration as explained previously.
An alternative is adding --no-cov --capture=no into Additional Arguments. To make this automatic for other test file, add those into the Template part.