Develop Reference

How can I specify which positional argument to use in tox?

I have the following tox.ini:
[tox]
envlist = py37-{lint, test}
[testenv:py37-{lint, test}]
envdir = {toxworkdir}/lint_and_test_env
deps =
pylint
pytest
pytest-xdist
commands =
lint: pylint src {posargs}
test: pytest tests {posargs}
I want to run both environments in parallel and specify --jobs=4 for pylint and -n auto for pytest. Executing tox -p -- --jobs=4 -n auto fails because pylint does not recognize the -n argument and vice versa.
Is there a way to achieve my goal?

I see no way to do what you want.
That said, I would strongly recommend that you split your environments and use one for testing and one for linting.
[tox]
envlist = py37, lint
[testenv]
deps =
pytest
pytest-xdist
commands =
pytest tests {posargs}
[testenv:lint]
deps = pylint
commands = pylint src {posargs}
Then you can pass in different arguments to the commands, or run all envs via tox.
Also you could set defaults in case you do not specify {posargs}.
e.g.
[testenv:lint]
deps = pylint
commands = pylint src {--jobs=4:posargs}
If you want to run all tox envs in parallel you could run...
tox -p
P.S.: I am one of the tox maintainers, and I have contributed to hundreds of open source projects, and basically 99% use separate envs for testing and linting.
P.P.S.: If you want to run more than one linter, you should have a look at pre-commit.

tox fails on ImportError in python3.5

I am trying to run tox package on my project which worked fine so far.
I am running in Phycharm on Windows 10 Pro 64-bit, tox version 3.15.0
when I run the command:
python -m tox --recreate
tox tests pass for python3.6,3.7,3.8 but not for Python3.5.
This is the error I get when I run:
python -m tox -epy35
GLOB sdist-make: C:\Users\Or\Documents\Studies\TAU\Term2\Project2\Source_code\pylint_upload\setup.py
py35 recreate: C:\Users\Or\Documents\Studies\TAU\Term2\Project2\Source_code\pylint_upload\.tox\py35
py35 installdeps: https://github.com/PyCQA/astroid/tarball/master#egg=astroid-master-2.0, coverage<5.0, mccabe, isort>=4.2.5,<5, pytest, pytest-xdist, pytest-benchmark, pytest-p
rofiling
py35 inst: C:\Users\Or\Documents\Studies\TAU\Term2\Project2\Source_code\pylint_upload\.tox\.tmp\package\4\pylint-2.6.1.dev1.zip
py35 installed: apipkg==1.5,astroid # https://github.com/PyCQA/astroid/tarball/master,atomicwrites==1.4.0,attrs==20.2.0,colorama==0.4.3,coverage==4.5.4,execnet==1.7.1,gprof2dot=
=2019.11.30,importlib-metadata==2.0.0,iniconfig==1.0.1,isort==4.3.21,lazy-object-proxy==1.4.3,mccabe==0.6.1,packaging==20.4,pathlib2==2.3.5,pluggy==0.13.1,py==1.9.0,py-cpuinfo==
7.0.0,pylint # file:///C:/Users/Or/Documents/Studies/TAU/Term2/Project2/Source_code/pylint_upload/.tox/.tmp/package/4/pylint-2.6.1.dev1.zip,pyparsing==2.4.7,pytest==6.1.0,pytest
-benchmark==3.2.3,pytest-forked==1.3.0,pytest-profiling==1.7.0,pytest-xdist==2.1.0,six==1.15.0,toml==0.10.1,typed-ast==1.4.1,wrapt==1.12.1,zipp==1.2.0
py35 run-test-pre: PYTHONHASHSEED='454'
py35 run-test: commands[0] | python -Wignore -m coverage run -m pytest --benchmark-disable 'C:\Users\Or\Documents\Studies\TAU\Term2\Project2\Source_code\pylint_upload/tests/'
ImportError while loading conftest 'C:\Users\Or\Documents\Studies\TAU\Term2\Project2\Source_code\pylint_upload\tests\conftest.py'.
..\tests\conftest.py:9: in <module>
from pylint.testutils import MinimalTestReporter
py35\lib\site-packages\pylint\testutils.py:274: in <module>
checkers.initialize(linter)
py35\lib\site-packages\pylint\checkers\__init__.py:64: in initialize
register_plugins(linter, __path__[0])
py35\lib\site-packages\pylint\utils\utils.py:255: in register_plugins
os.path.join(directory, filename)
py35\lib\site-packages\astroid\modutils.py:284: in load_module_from_file
return load_module_from_modpath(modpath, path, use_sys)
py35\lib\site-packages\astroid\modutils.py:245: in load_module_from_modpath
module = imp.load_module(curname, mp_file, mp_filename, mp_desc)
C:\Program Files\Python 3.5\lib\imp.py:234: in load_module
return load_source(name, filename, file)
C:\Program Files\Python 3.5\lib\imp.py:172: in load_source
module = _load(spec)
py35\lib\site-packages\pylint\checkers\async.py:16: in <module>
from pylint.checkers import utils as checker_utils
py35\lib\site-packages\pylint\checkers\utils.py:640: in <module>
def is_attr_private(attrname: str) -> Optional[Match[str]]:
C:\Program Files\Python 3.5\lib\typing.py:631: in __getitem__
return Union[arg, type(None)]
C:\Program Files\Python 3.5\lib\typing.py:534: in __getitem__
dict(self.__dict__), parameters, _root=True)
C:\Program Files\Python 3.5\lib\typing.py:491: in __new__
for t2 in all_params - {t1} if not isinstance(t2, TypeVar)):
C:\Program Files\Python 3.5\lib\typing.py:490: in <genexpr>
if any(issubclass(t1, t2)
E TypeError: issubclass() arg 1 must be a class
ERROR: InvocationError for command 'C:\Users\Or\Documents\Studies\TAU\Term2\Project2\Source_code\pylint_upload\.tox\py35\Scripts\python.EXE' -Wignore -m coverage run -m pytest -
-benchmark-disable 'C:\Users\Or\Documents\Studies\TAU\Term2\Project2\Source_code\pylint_upload/tests/' (exited with code 4)
___________________________________________________________________________________ summary ____________________________________________________________________________________
ERROR: py35: commands failed
I didn't change this conftest.py file at all since it used to work, so I don't think this is the real cause of this error.
Also the same command run successfully on my Linux machine on Ubuntu, for the same project.
I tried to recreate tox again, tried to clone entire repository and run the test again, also tried this and this.
This is the tox.ini file:
[tox]
minversion = 2.4
envlist = py35, py36, py37, py38, pypy, pylint, benchmark
skip_missing_interpreters = true
[testenv:pylint]
deps =
git+https://github.com/pycqa/astroid#master
pytest
commands =
# This would be greatly simplified by a solution for https://github.com/PyCQA/pylint/issues/352
pylint -rn --rcfile={toxinidir}/pylintrc --load-plugins=pylint.extensions.docparams, pylint.extensions.mccabe \
{toxinidir}/pylint \
{toxinidir}/tests/message/ \
{toxinidir}/tests/checkers/ \
{toxinidir}/tests/extensions/ \
{toxinidir}/tests/utils/ \
{toxinidir}/tests/acceptance/ \
{toxinidir}/tests/conftest.py \
{toxinidir}/tests/test_config.py \
{toxinidir}/tests/test_func.py \
{toxinidir}/tests/test_functional.py \
{toxinidir}/tests/test_import_graph.py \
{toxinidir}/tests/test_pragma_parser.py \
{toxinidir}/tests/test_pylint_runners.py \
{toxinidir}/tests/test_regr.py \
{toxinidir}/tests/test_self.py \
{toxinidir}/tests/unittest_config.py \
{toxinidir}/tests/lint/ \
{toxinidir}/tests/unittest_pyreverse_diadefs.py \
{toxinidir}/tests/unittest_pyreverse_inspector.py \
{toxinidir}/tests/unittest_pyreverse_writer.py \
{toxinidir}/tests/unittest_reporters_json.py \
{toxinidir}/tests/unittest_reporting.py
[testenv:formatting]
basepython = python3
deps =
black==20.8b1
isort==5.5.2
commands =
black --check . --exclude="tests/functional/|tests/input|tests/extensions/data|tests/regrtest_data/|tests/data/|venv|astroid|.tox"
isort . --check-only
changedir = {toxinidir}
[testenv:mypy]
basepython = python3
deps =
typed-ast>=1.4
mypy>=0.7,<1.0
commands =
python -m mypy {toxinidir}/pylint/checkers --ignore-missing-imports {posargs:}
[testenv]
deps =
https://github.com/PyCQA/astroid/tarball/master#egg=astroid-master-2.0
coverage<5.0
mccabe
# isort 5 is not compatible with Python 3.5
py35: isort>=4.2.5,<5
pytest
pytest-xdist
pytest-benchmark
pytest-profiling
setenv =
COVERAGE_FILE = {toxinidir}/.coverage.{envname}
commands =
; Run tests, ensuring all benchmark tests do not run
python -Wignore -m coverage run -m pytest --benchmark-disable {toxinidir}/tests/ {posargs:}
; Transform absolute path to relative path
; for compatibility with coveralls.io and fix 'source not available' error.
; If you can find a cleaner way is welcome
python -c "import os;cov_strip_abspath = open(os.environ['COVERAGE_FILE'], 'r').read().replace('.tox' + os.sep + os.path.relpath('{envsitepackagesdir}', '{toxworkdir}') + os.sep, '');open(os.environ['COVERAGE_FILE'], 'w').write(cov_strip_abspath)"
changedir = {toxworkdir}
[testenv:spelling]
deps =
https://github.com/PyCQA/astroid/tarball/master#egg=astroid-master-2.0
pytest
pytest-xdist
pyenchant
commands =
python -Wi -m pytest {toxinidir}/tests/ {posargs:} -k unittest_spelling
changedir = {toxworkdir}
[testenv:coveralls]
setenv =
COVERAGE_FILE = {toxinidir}/.coverage
passenv =
*
deps =
coverage<5.0
coveralls
skip_install = true
commands =
python -m coverage combine
python -m coverage report --rcfile={toxinidir}/.coveragerc -m
- coveralls --rcfile={toxinidir}/.coveragerc
changedir = {toxinidir}
[testenv:coverage-erase]
setenv =
COVERAGE_FILE = {toxinidir}/.coverage
deps =
coverage<5
skip_install = true
commands =
python -m coverage erase
changedir = {toxinidir}
[testenv:coverage-html]
setenv =
COVERAGE_FILE = {toxinidir}/.coverage
deps =
coverage<5
skip_install = true
commands =
python -m coverage combine
python -m coverage html --ignore-errors --rcfile={toxinidir}/.coveragerc
changedir = {toxinidir}
[testenv:docs]
usedevelop = True
changedir = doc/
extras =
docs
commands =
sphinx-build -W -b html -d _build/doctrees . _build/html
[testenv:benchmark]
deps =
https://github.com/PyCQA/astroid/tarball/master#egg=astroid-master-2.0
coverage<5.0
mccabe
pytest
pytest-xdist
pygal
pytest-benchmark
commands =
; Run the only the benchmark tests, grouping output and forcing .json output so we
; can compare benchmark runs
python -Wi -m pytest --exitfirst \
--failed-first \
--benchmark-only \
--benchmark-save=batch_files \
--benchmark-save-data \
--benchmark-autosave \
{toxinidir}/tests \
--benchmark-group-by="group" \
{posargs:}
changedir = {toxworkdir}
And this is conftest.py:
# pylint: disable=redefined-outer-name
# pylint: disable=no-name-in-module
import os
import pytest
from pylint import checkers
from pylint.lint import PyLinter
from pylint.testutils import MinimalTestReporter
#pytest.fixture
def linter(checker, register, enable, disable, reporter):
_linter = PyLinter()
_linter.set_reporter(reporter())
checkers.initialize(_linter)
if register:
register(_linter)
if checker:
_linter.register_checker(checker(_linter))
if disable:
for msg in disable:
_linter.disable(msg)
if enable:
for msg in enable:
_linter.enable(msg)
os.environ.pop("PYLINTRC", None)
return _linter
#pytest.fixture(scope="module")
def checker():
return None
#pytest.fixture(scope="module")
def register():
return None
#pytest.fixture(scope="module")
def enable():
return None
#pytest.fixture(scope="module")
def disable():
return None
#pytest.fixture(scope="module")
def reporter():
return MinimalTestReporter

Your reported problem is actual a problem in Python 3.5.0!!
You can check your Python 3.5 version by entering a command like python3.5 --version or just enter python3.5 and have a look at the top of the repl output.
You can resolve the problem by installing the any higher point release of Python 3.5, e.g 3.5.1 or the even the latest one, 3.5.10.
This all said - Python 3.5 is already end of life ( https://www.python.org/downloads/ ). If there is no important reason, please consider to drop support for it.

poetry dependencies not available when running script with tox

I have a python project that uses poetry and tox. It has source code, tests and scripts (juptext notebooks). I can't import the dev dependencies in the scripts, but I can in the tests.
When I came across this problem, I created the following minimal example. At first, it didn't work, then I fiddled around with it, and now it's working. So I stripped the project that has the actual problem down so it's indistinguishable other than the project name, location, virtual env, and .git directory, but that's still not working.
UPDATE deleting all build artifacts and the virtualenv for the minimal example makes it stop working again
UPDATE adding the line scripts: poetry install to the tox commands fixed only the minimal example
The source code, tests and scripts are in the following layout
foo
+--foo
| +--__init__.py
|
+--tests
| +--__init__.py
| +--test_foo.py
|
+--scripts
| +--foo_script.py
|
+--pyproject.toml
+--tox.ini
The files are either empty or as follows:
foo_script.py
import requests
test_foo.py
import requests
import pytest
def test():
assert True
pyproject.toml
[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["foo maker"]
[tool.poetry.dependencies]
python = "^3.7"
requests = "*"
[tool.poetry.dev-dependencies]
pytest = "^4.6"
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
tox.ini
[tox]
envlist = test, scripts
isolated_build = true
skipsdist = true
[testenv]
basepython = python3.7
whitelist_externals =
pytest
bash
commands =
test: pytest
scripts: bash -c 'python3 scripts/*.py'
When I run tox, I get
test run-test-pre: PYTHONHASHSEED='4126239415'
test run-test: commands[0] | pytest
============================= test session starts ==============================
platform linux -- Python 3.6.9, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
cachedir: .tox/test/.pytest_cache
rootdir: /home/#######/foo
collected 1 item
tests/test_foo.py . [100%]
============================== 1 passed in 0.09s ===============================
scripts run-test-pre: PYTHONHASHSEED='4126239415'
scripts run-test: commands[0] | bash -c 'python3 scripts/*.py'
Traceback (most recent call last):
File "scripts/foo_script.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
ERROR: InvocationError for command /bin/bash -c 'python3 scripts/*.py' (exited with code 1)
___________________________________ summary ____________________________________
test: commands succeeded
ERROR: scripts: commands failed

I believe something like the following should work:
pyproject.toml
[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["foo maker"]
[tool.poetry.dependencies]
python = "^3.7"
requests = "*"
#
pytest = { version = "^4.6", optional = true }
[tool.poetry.extras]
test = ["pytest"]
# [tool.poetry.dev-dependencies]
# use 'test' extra instead
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
tox.ini
[tox]
envlist = test, scripts
isolated_build = true
[testenv]
basepython = python3.7
whitelist_externals =
pytest
bash
extras =
test
commands =
test: pytest
scripts: bash -c 'for f in scripts/*.py; do python "$f"; done'

Assuming you have installed poetry and tox and pytest are dependencies in your pyproject.yml (notice poetry run, see https://python-poetry.org/docs/cli/#run):
[tox]
envlist = py37
isolated_build = True
skipsdist = True
[testenv]
whitelist_externals = poetry
commands=
poetry run pytest
optionally you can make the install happen on running the tests by changing the last bit to (but then you will need to have tox installed outside of poetry, which could cause you issues down the line)
commands=
poetry install
poetry run pytest
Also depending on your root folder and where the tests are you can configure the path for tox to change directory to by adding
changedir = tests
In which case the whole file would look like this if you are in directory foo executing tox:
[tox]
envlist = py37
isolated_build = True
skipsdist = True
[testenv]
whitelist_externals = poetry
commands=
poetry run pytest
changedir = tests

This is an easy fix. First, run rm -rf .tox to delete the .tox directory, probably hasn't installed the files you wanted.
Here is an example of my tox.ini.
[tox]
isolated_build = true
skipsdist = true
envlist = py39
[testenv]
deps = -rrequirements-dev.txt
whitelist_externals =
poetry
skip_install = true
commands =
python -m pytest tests/
As you can see in the tox.ini file I have a separate requirements for dev. This can be generated from poetry via the commands below.
poetry export --format=requirements.txt --without-hashes --with dev --output=requirements.txt
poetry export --format=requirements.txt --without-hashes --output=requirements-dev.txt
Don't ask me why but the deps in [testenv] are concatenated out, as you can see with deps = -rrequirements-dev.txt, would also change your command to python -m pytest /tests
Ciao.

First I need to install the dependencies with poetry install. Then append poetry run to the beginning of the command to enable the dependencies. Also running python scripts like that will just run the first, passing the name of the others as args to the first program. Instead use for f in scripts/*.py; do python "$f"; done (see here)
All together
poetry install
poetry run bash -c 'for f in scripts/*.py; do python "$f"; done'

tox multiple tests, re-using tox environment

Is it possible to do the following using a single tox virtual environment?
[tox]
envlist = test, pylint, flake8, mypy
skipsdist = true
[testenv:lint]
deps = pylint
commands = pylint .
[testenv:flake8]
deps = flake8
commands = flake8 .
[testenv:mypy]
commands = mypy . --strict
[testenv:test]
deps = pytest
commands = pytest
As I am only testing on my python version (py3.7), I don't want tox to have to create 4 environments (.tox/test, .tox/pylint,.tox/flake8, .tox/mypy) when they could all be run on a single environment.
I also want to see what failed individually individually, thus don't want to do:
[tox]
skipsdist = true
[testenv]
commands = pylint .
flake8 .
mypy . --strict
pytest
as the output would be like this:
_____________ summary ___________
ERROR: python: commands failed
and not like this:
____________________summary _________________
ERROR: test: commands failed
ERROR: lint: commands failed
ERROR: mypy: commands failed
test: commands succeeded

Use generative names and factor-specific commands (search the tox configuration doc page for those terms for more info) to implement all of your desired environments as one so they share the same envdir:
[testenv:{lint,flake8,mypy,test}]
envdir = {toxworkdir}/.work_env
deps = pylint, flake8, pytest
commands =
lint: pylint .
flake8: flake8 .
mypy: mypy . --strict
test: pytest
In tox 4+, you may use the plugin tox-ignore-env-name-mismatch and set runner = ignore_env_name_mismatch in the testenv to allow testenvs with different names to share the same virtualenv.

How to use a testenv for every env except one in tox?

I have the following tox.ini file:
[tox]
envlist = py{27,34,35,36,37},flake8
[testenv]
changedir = {toxworkdir}/{envname}
deps =
pytest-cov
pytest
setenv =
COVERAGE_FILE = {toxinidir}/.coverage.{envname}
commands =
pytest {toxinidir}/tests --cov=SOME_DIRECTORY --cov-report=html:{toxinidir}/coverage_html_report/{envname} {posargs}
[testenv:flake8]
basepython = python3
skip_install = true
deps = flake8
commands = flake8 src tests
[flake8]
ignore: F401,E402,W503,W291,E501,W605
I would like the first testenv to be run only for py{27,34,35,36,37} and not flake8. I tried doing [testenv:py{27,34,35,36,37}] but it does change the behavior of the commands inside that testenv (the pytest command is not run).
How can I specify that the code currently in [testenv] to be run only for py{27,34,35,36,37}?