Develop Reference

How to view runtime warnings in PyCharm when running tests using pytest?

When running tests in PyCharm 2022.3.2 (Professional Edition) using pytest (6.2.4) and Python 3.9 I get the following result in the PyCharm console window:
D:\cenv\python.exe "D:/Program Files (x86)/JetBrains/PyCharm 2022.3.2/plugins/python/helpers/pycharm/_jb_pytest_runner.py" --path D:\tests\test_k.py
Testing started at 6:49 PM ...
Launching pytest with arguments D:\tests\test_k.py --no-header --no-summary -q in D:\tests
============================= test session starts =============================
collecting ... collected 5 items
test_k.py::test_init
test_k.py::test_1
test_k.py::test_2
test_k.py::test_3
test_k.py::test_4
======================= 5 passed, 278 warnings in 4.50s =======================
Process finished with exit code 0
PASSED [ 20%]PASSED [ 40%]PASSED [ 60%]PASSED [ 80%]PASSED [100%]
So the actual warnings don't show. Only the number of warnings (278) is shown.
I tried:
selecting: Pytest: do not add "--no-header --no-summary -q" in advanced settings
Setting Additional arguments to -Wall in the Run/Debug configurations window
Setting Interpreter options to -Wall in the Run/Debug configurations window
and all permutations, all to no avail. Is there a way to show all runtime warnings when running tests using pytest in PyCharm in the PyCharm Console window?
EDIT:
#Override12
When I select do not add "--no-header --no-summary -q" in advanced settings I get the following output:
D:\Projects\S\SHARK\development_SCE\cenv\python.exe "D:/Program Files (x86)/JetBrains/PyCharm 2020.3.4/plugins/python/helpers/pycharm/_jb_pytest_runner.py" --path D:\Projects\S\SHARK\development_SCE\cenv\Lib\site-packages\vistrails-3.5.0rc0-py3.9.egg\vistrails\packages\SHARK\analysis\tests\test_fairing_1_plus_k.py -- --jb-show-summary
Testing started at 10:07 AM ...
Launching pytest with arguments D:\Projects\S\SHARK\development_SCE\cenv\Lib\site-packages\vistrails-3.5.0rc0-py3.9.egg\vistrails\packages\SHARK\analysis\tests\test_fairing_1_plus_k.py in D:\Projects\S\SHARK\development_SCE\cenv\Lib\site-packages\vistrails-3.5.0rc0-py3.9.egg\vistrails\packages
============================= test session starts =============================
platform win32 -- Python 3.9.7, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- D:\Projects\S\SHARK\development_SCE\cenv\python.exe
cachedir: .pytest_cache
rootdir: D:\Projects\S\SHARK\development_SCE\cenv\Lib\site-packages\vistrails-3.5.0rc0-py3.9.egg\vistrails\packages
plugins: pytest_check-1.0.5
collecting ... collected 5 items
SHARK/analysis/tests/test_fairing_1_plus_k.py::test_init
SHARK/analysis/tests/test_fairing_1_plus_k.py::test_without_1_k_fairing
SHARK/analysis/tests/test_fairing_1_plus_k.py::test_1_k_fairing_given
SHARK/analysis/tests/test_fairing_1_plus_k.py::test_without_1_k_fairing_only_3_values_under_threshold
SHARK/analysis/tests/test_fairing_1_plus_k.py::test_1_k_fairing_given_only_3_values_under_threshold
============================== warnings summary ===============================
......\pyreadline\py3k_compat.py:8
D:\Projects\S\SHARK\development_SCE\cenv\lib\site-packages\pyreadline\py3k_compat.py:8: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
return isinstance(x, collections.Callable)
......\nose\importer.py:12
D:\Projects\S\SHARK\development_SCE\cenv\lib\site-packages\nose\importer.py:12: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
from imp import find_module, load_module, acquire_lock, release_lock
SHARK/analysis/tests/test_fairing_1_plus_k.py: 276 warnings
D:\Projects\S\SHARK\development_SCE\cenv\lib\site-packages\pymarin\objects\key.py:1101: UserWarning: siUnits is deprecated, use siUnit
warnings.warn('siUnits is deprecated, use siUnit')
-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================= 5 passed, 278 warnings in 5.79s =======================
Process finished with exit code 0
PASSED [ 20%]PASSED [ 40%]PASSED [ 60%]PASSED [ 80%]PASSED [100%]
So 4 warnings are displayed. However I would like to see all 278 warnings.
When I run pytest from the command line outside PyCharm I get the same result. So it seems to be a pytest problem and it seems that it has nothing to do with PyCharm.

I think you could also try adding verbosity or collect warnings in a programatic way, by changing how you execute the tests.

The solution is a combination of two things:
Setting 'do not add "--no-header --no-summary -q"' in advanced settings as #Override12 suggested.
When the same warning is issued multiple times, only the first time is displayed. In my case solving the first warning reduced the number of warnings from 278 to 2.

pytest - Suppress DeprecationWarning from specific 3rd party modules

When I run pytest I'm getting some deprecation warnings from a 3rd party library. I'd like to be informed about any deprecation warnings in my own code, but not in a vendored copy of a library bundled with another 3rd-party library.
This answer was helpful in getting me partway there. If I run pytest like this:
$ pytest ./tests/ I get:
$ pytest ./tests/
============================= test session starts ==============================
platform linux -- Python 3.7.4, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /home/whlt/repos/tj-image-resizer/tests, inifile: pytest.ini
collected 5 items
tests/test_file1.py . [ 20%]
tests/test_file2.py .... [100%]
=============================== warnings summary ===============================
/home/whlt/.local/lib/python3.7/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1
/home/whlt/.local/lib/python3.7/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1
/home/whlt/.local/lib/python3.7/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Mapping, MutableMapping
-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================== 5 passed, 2 warnings in 2.54s =========================
but if I run pytest like this: $ pytest ./tests/ -W ignore::DeprecationWarning I get:
============================= test session starts ==============================
platform linux -- Python 3.7.4, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /home/whlt/repos/tj-image-resizer/tests, inifile: pytest.ini
collected 5 items
tests/test_file1.py . [ 20%]
tests/test_file2.py .... [100%]
============================== 5 passed in 2.61s ===============================
This second output shows me that the filter works, but that will also hide any deprecation warnings I'd like to seeing resulting from my own code.
Part of this issue is that I'm not sure which module to try referencing in the ignore filter. I've tried $ pytest ./tests/ -W ignore::DeprecationWarning:urllib3.*: and I've tried $ pytest ./tests/ -W ignore::DeprecationWarning:botocore.*:. Both of these result in the same output as the first example with no filtering.
How can I filter out DeprecationWarnings from the version of urllib3 packaged with the vendored version of requests included with botocore (which gets called when I run commands with the boto3 library)?

You should use the warning filters options (ini or marks):
[pytest]
filterwarnings =
ignore::DeprecationWarning:botocore.*:
Source: https://docs.python.org/3/library/warnings.html#default-warning-filter
"Individual warnings filters are specified as a sequence of fields separated by colons:"
action:message:category:module:line

The answer from #Santiago Magariños does not work for me (but it put me on the correct path to find solution - so many thanks).
I use Python 3.9, pytest 6.2.1 and trying to suppress warnings from selenium 3.141.0.
I have realized that I need to prepend .* before the module name:
[pytest]
filterwarnings = ignore:::.*.selenium
or use the full "path". So to suppress the warning
../../../../../../.local/share/virtualenvs/common-bjARi2zp/lib/python3.9/site-packages/selenium/webdriver/support/wait.py:28
/home/vaclav/.local/share/virtualenvs/common-bjARi2zp/lib/python3.9/site-packages/selenium/webdriver/support/wait.py:28: DeprecationWarning: invalid escape sequence \
"""Constructor, takes a WebDriver instance and timeout in seconds.
I need to use this filter in pytest.ini file:
[pytest]
filterwarnings = ignore:::.home.vaclav..local.share.virtualenvs.common-bjARi2zp.lib.python3.9.site-packages.selenium

if you are using pyproject.toml file for pytest configuration you can use:
[tool.pytest.ini_options]
testpaths = ["./tests/unit"]
filterwarnings = ["ignore:::.*third_party_package.module:123", "ignore:::.*another_module*"]
ignore:::.*third_party_package.module:123 ignores in specific warning at specific line
ignore:::.*another_module* ignores all warnings in the module.
Notice that you can have multiple ignores. You need to list them in []. Also, you can't have *third_party_package/module:123 you must replace all / with .

How does pylint use easy_install at runtime?

Sorry this is a long question. See the sentence in bold at the bottom for the TL;DR version.
I've spent many hours trying to track down a problem where pylint sometimes doesn't report all the errors in a module. Note that it does find some errors (e.g. long lines), just not all of them (e.g. missing docstrings).
I'm running pylint 1.7.2 on Ubuntu 16.04. (The version available from apt was 1.5.2 but installing via pip gives 1.7.2.)
We typically run pylint from tox, with a tox.ini that looks something like this (this is a cut-down version):
[tox]
envlist = py35
[testenv]
setenv =
MODULE_NAME=our_module
ignore_errors = True
deps =
-r../requirements.txt
whitelist_externals = bash
commands =
pip install --editable=file:///{toxinidir}/../our_other_module
pip install -e .
bash -c \'set -o pipefail; pylint --rcfile=../linting/pylint.cfg our_module | tee pylint.log\'
Amongst other things, the ../requirements.txt file contains a line for pylint==1.7.2.
The behaviour is like this:
[wrong] When the line that imports our_other_module is present, pylint appears to complete successfully and not report any warnings, even though there are errors in the our_module code that it should pick up.
[correct] When that line is commented out, pylint generates the expected warnings.
As part of tracking this down I took two copies of the .tox folder with and without the module import, naming them .tox-no-errors-reported and .tox-with-errors-reported respectively.
So now, even without sourcing their respective tox virtualenvs, I can do the following:
$ .tox-no-errors-reported/py35/bin/pylint --rcfile=../linting/pylint.cfg our_module -- reports no linting warnings
$ .tox-with-errors-reported/py35/bin/pylint --rcfile=../linting/pylint.cfg our_module -- reports the expected linting warnings
(where I just changed the pylint script's #! line in each case to reference the python3.5 inside that specific .tox directory instead of the unrenamed .tox)
By diffing .tox-no-errors-reported and .tox-with-errors-reported, I've found that they are very similar. But I can make the "no errors" version start to report errors by removing the path to our_other_module from .tox-no-errors-reported/py35/lib/python3.5/site-packages/easy-install.pth.
So my question is why is pylint using easy_install at runtime, and what is it picking up from our other component that is causing it to fail to report some errors.
As I understand it, pylint has dependencies on astroid and logilab-common, but including these in the requirements.txt doesn't make any difference.

One possible reason for the surprising pylint behavior is the --editable option.
it creates a special .egg-link file in the deployment directory, that links to your project’s source code. And, ..., it will also update the easy-install.pth file to include your project’s source code
The pth file will then affect the sys.path which affects the module import logic of astroid and it is deeply buried in the call stack of pylint.expand_files via pylint.utils.expand_modules. Also pylint identifies the module part and function names in the AST using astroid.modutils.get_module_part.
To test the theory, you can try calling some of the affected astroid functions manually:
import sys, astroid
print(sys.path)
print(astroid.modutils.get_module_part('your_package.sub_package.module'))
astroid.modutils.file_from_modpath(['your_package', 'sub_package', 'module'])

flake8/pylint fails in Tox testing environment, raises InvocationError

I've been learning about how to do testing in tox for my python project.
I have (what should be) a fairly standard tox initialization file that looks like the following:
[tox]
envlist=py27,flake8
...
[testenv:flake8]
deps=flake8
commands=flake8 library # 'library' is temp. name of project
Everything looks normal, all the test works, and even the flake8 output comes through (output below). However, tox raises an InvocationError (it does the same for testing using pylint)
flake8 recreate: /Users/shostakovich/projects/project_templates/library/.tox/flake8
flake8 installdeps: flake8
flake8 inst: /Users/shostakovich/projects/project_templates/library/.tox/dist/library-0.1.0.zip
flake8 installed: flake8==2.4.1,library==0.1.0,mccabe==0.3,pep8==1.5.7,pyflakes==0.8.1,wheel==0.24.0
library/__main__.py:12:1: F401 'os' imported but unused
library/__main__.py:13:1: F401 're' imported but unused
...
ERROR: InvocationError: '/Users/shostakovich/projects/project_templates/library/.tox/flake8/bin/flake8 library'
I am running tox 2.0.2 on MaxOSX 10.9.5. The problem goes away if I just call flake8 or pylint directly (the version of flake8 is shown above).

tox doesn't fail, it works!
Your flake8 source code check has findings and therefore tox exits with failures, that's your test result. Fix the findings and your done!
You may configure the flake8 run to ignore specific codes with a section in your tox.ini. From the flake8 docs:
[flake8]
ignore = E226,E302,E41
There are more options you may be interested in, e.g. select = ... for whitelisting enabled code checks.

You can also tell flake8 to exit without failure, even if the test results are not perfect. This will suppress the misleading InvocationError. Just add to your command --exit-zero, so for example:
commands=flake8 library --exit-zero

Py.Test aborts when (py)ROOT is imported

I've a problem unit testing Python code, which uses the (py)ROOT package. It simply aborts collecting the tests when there is a line import ROOT in one of the scripts.
Does anybody have an idea what's going on?
foo#bar ~/project/dir [19:21:17]
(project)> $ py.test -v [±master ●]
============================= test session starts ==============================
platform darwin -- Python 2.7.6 -- py-1.4.25 -- pytest-2.6.3 -- /path/to/.virtualenvs/project/bin/python
collecting 0 items[1] 16520 abort py.test -v

I had a similar issue by adding the PyROOT bindings to export $PYTHONPATH in my .bashrc. The locally set PYTHONPATH was ignored. A solution is copying or linking the ROOT.py and libPyROOT.so files to the appropriate Python directory in /usr/lib. More details in this answer: https://stackoverflow.com/a/33130000/4753851