Develop Reference

nosetests: --nologcapture still captures INFO messages

With the code
import unittest
import logging
class LoggingTest(unittest.TestCase):
def test_that_logs(self):
logging.warning("Hello")
logging.info("World")
for a in xrange(100000000000000000):
pass
I get this unexpected output:
$ nosetests --version
nosetests version 1.3.7
$ nosetests log_from_nosetest.py --nocapture --nologcapture
WARNING:root:Hello
^C
----------------------------------------------------------------------
Ran 1 test in 105.376s
OK
As can be seen. The WARNING log level message is printed, but not the INFO level one.
I've tried using the --logging-level setting but without success.

It seems a missing feature. If --nologcapture is set, nose simply ignores all logging setup, e.g. --logging-level is not used at all. You can remedy that with a
logging.basicConfig(level=logging.INFO)
in your code - not sure if that's what you want, because this kind of setup shouldn't be in library-code at all. It would be fine within the test-setup though.

This is because the default log level is warning. Set it with
logging.getLogger('').setLevel(logging.INFO)

flake8/pylint fails in Tox testing environment, raises InvocationError

I've been learning about how to do testing in tox for my python project.
I have (what should be) a fairly standard tox initialization file that looks like the following:
[tox]
envlist=py27,flake8
...
[testenv:flake8]
deps=flake8
commands=flake8 library # 'library' is temp. name of project
Everything looks normal, all the test works, and even the flake8 output comes through (output below). However, tox raises an InvocationError (it does the same for testing using pylint)
flake8 recreate: /Users/shostakovich/projects/project_templates/library/.tox/flake8
flake8 installdeps: flake8
flake8 inst: /Users/shostakovich/projects/project_templates/library/.tox/dist/library-0.1.0.zip
flake8 installed: flake8==2.4.1,library==0.1.0,mccabe==0.3,pep8==1.5.7,pyflakes==0.8.1,wheel==0.24.0
library/__main__.py:12:1: F401 'os' imported but unused
library/__main__.py:13:1: F401 're' imported but unused
...
ERROR: InvocationError: '/Users/shostakovich/projects/project_templates/library/.tox/flake8/bin/flake8 library'
I am running tox 2.0.2 on MaxOSX 10.9.5. The problem goes away if I just call flake8 or pylint directly (the version of flake8 is shown above).

tox doesn't fail, it works!
Your flake8 source code check has findings and therefore tox exits with failures, that's your test result. Fix the findings and your done!
You may configure the flake8 run to ignore specific codes with a section in your tox.ini. From the flake8 docs:
[flake8]
ignore = E226,E302,E41
There are more options you may be interested in, e.g. select = ... for whitelisting enabled code checks.

You can also tell flake8 to exit without failure, even if the test results are not perfect. This will suppress the misleading InvocationError. Just add to your command --exit-zero, so for example:
commands=flake8 library --exit-zero

Can voluptuous code pass pylint?

I run pylint -E as part of the tests on my Python project to ensure that errors don't creep into untested code. Generally this works quite well. But recently I've been running into problems with voluptuous and pylint.
The problem is that pylint thinks that the values returned by voluptuous Schemas are lists, which is simply not the case. Here's a toy program:
import voluptuous
MyType = voluptuous.Schema({
'q': str
})
def foo(bar):
bar = MyType(bar)
q = bar.get('q')
print q
foo({'q': '1324'})
It runs just fine:
$ python toy.py
1234
pylint, however, flags the .get() call:
$ pylint -E toy.py
No config file found, using default configuration
************* Module toy
E: 11, 8: Instance of 'list' has no 'get' member (no-member)
How can I get this program to pass pylint -E?

One option is to ignore the voluptuous module entirely, e.g.
$ pylint -E --ignored-modules=voluptuous toy.py
(passes)
If would be nice if pylint understood voluptuous better, though.

Remove Python UserWarning

I just finished installing my MySQLdb package for Python 2.6, and now when I import it using import MySQLdb, a user warning appear will appear
/usr/lib/python2.6/site-packages/setuptools-0.8-py2.6.egg/pkg_resources.py:1054:
UserWarning: /home/sgpromot/.python-eggs is writable by group/others and vulnerable
to attack when used with get_resource_filename. Consider a more secure location
(set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).
warnings.warn(msg, UserWarning)
Is there a way how to get rid of this?

You can change ~/.python-eggs to not be writeable by group/everyone. I think this works:
chmod g-wx,o-wx ~/.python-eggs

You can suppress warnings using the -W ignore:
python -W ignore yourscript.py
If you want to supress warnings in your script (quote from docs):
If you are using code that you know will raise a warning, such as a deprecated function, but do not want to see the warning, then it is possible to suppress the warning using the catch_warnings context manager:
import warnings
def fxn():
warnings.warn("deprecated", DeprecationWarning)
with warnings.catch_warnings():
warnings.simplefilter("ignore")
fxn()
While within the context manager all warnings will simply be ignored. This allows you to use known-deprecated code without having to see the warning while not suppressing the warning for other code that might not be aware of its use of deprecated code. Note: this can only be guaranteed in a single-threaded application. If two or more threads use the catch_warnings context manager at the same time, the behavior is undefined.
If you just want to flat out ignore warnings, you can use filterwarnings:
import warnings
warnings.filterwarnings("ignore")

How to execute ipdb.set_trace() at will while running pytest tests

I'm using pytest for my test suite. While catching bugs in complex inter-components test, I would like to place import ipdb; ipdb.set_trace() in the middle of my code to allow me to debug it.
However, since pytest traps sys.stdin/sys.stdout ipdb fails. How can I use ipdb while testing with pytest.
I'm not interested in jumping to pdb or ipdb after a failure, but to place breaks anywhere in the code and be able to debug it there before the failure occurs.

The error is raised because pytest captures output by default.
You can run pytest with -s option (turn off capture output). For example:
py.test -s my_test.py
and then in my_test.py:
import ipdb;
ipdb.set_trace()

pytest-ipdb is unfortunately not supported anymore.
The solution is to run
pytest my_test.py --pdb --pdbcls=IPython.terminal.debugger:Pdb
From the help command:
pytest -h
--pdb start the interactive Python debugger on errors.
--pdbcls=modulename:classname
start a custom interactive Python debugger on errors.
For example:
--pdbcls=IPython.terminal.debugger:TerminalPdb
The difference is just that TerminalPdb seems to throw erros, but Pdb not (Ipython docs).

As of 2019-11 here is what should fix it:
pip install ipdb gnureadline ptpython
export PYTEST_ADDOPTS='--pdb --pdbcls=IPython.terminal.debugger:Pdb'

This is what I use
py.test tests/ --pdbcls=IPython.core.debugger:Pdb -s

You may want to give pdbpp a try. I've had more success with it, compared to ipdb when used with pytest. See my answer here: https://stackoverflow.com/a/69320311/2896799.