Develop Reference

How can I specify which positional argument to use in tox?

I have the following tox.ini:
[tox]
envlist = py37-{lint, test}
[testenv:py37-{lint, test}]
envdir = {toxworkdir}/lint_and_test_env
deps =
pylint
pytest
pytest-xdist
commands =
lint: pylint src {posargs}
test: pytest tests {posargs}
I want to run both environments in parallel and specify --jobs=4 for pylint and -n auto for pytest. Executing tox -p -- --jobs=4 -n auto fails because pylint does not recognize the -n argument and vice versa.
Is there a way to achieve my goal?

I see no way to do what you want.
That said, I would strongly recommend that you split your environments and use one for testing and one for linting.
[tox]
envlist = py37, lint
[testenv]
deps =
pytest
pytest-xdist
commands =
pytest tests {posargs}
[testenv:lint]
deps = pylint
commands = pylint src {posargs}
Then you can pass in different arguments to the commands, or run all envs via tox.
Also you could set defaults in case you do not specify {posargs}.
e.g.
[testenv:lint]
deps = pylint
commands = pylint src {--jobs=4:posargs}
If you want to run all tox envs in parallel you could run...
tox -p
P.S.: I am one of the tox maintainers, and I have contributed to hundreds of open source projects, and basically 99% use separate envs for testing and linting.
P.P.S.: If you want to run more than one linter, you should have a look at pre-commit.

poetry dependencies not available when running script with tox

I have a python project that uses poetry and tox. It has source code, tests and scripts (juptext notebooks). I can't import the dev dependencies in the scripts, but I can in the tests.
When I came across this problem, I created the following minimal example. At first, it didn't work, then I fiddled around with it, and now it's working. So I stripped the project that has the actual problem down so it's indistinguishable other than the project name, location, virtual env, and .git directory, but that's still not working.
UPDATE deleting all build artifacts and the virtualenv for the minimal example makes it stop working again
UPDATE adding the line scripts: poetry install to the tox commands fixed only the minimal example
The source code, tests and scripts are in the following layout
foo
+--foo
| +--__init__.py
|
+--tests
| +--__init__.py
| +--test_foo.py
|
+--scripts
| +--foo_script.py
|
+--pyproject.toml
+--tox.ini
The files are either empty or as follows:
foo_script.py
import requests
test_foo.py
import requests
import pytest
def test():
assert True
pyproject.toml
[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["foo maker"]
[tool.poetry.dependencies]
python = "^3.7"
requests = "*"
[tool.poetry.dev-dependencies]
pytest = "^4.6"
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
tox.ini
[tox]
envlist = test, scripts
isolated_build = true
skipsdist = true
[testenv]
basepython = python3.7
whitelist_externals =
pytest
bash
commands =
test: pytest
scripts: bash -c 'python3 scripts/*.py'
When I run tox, I get
test run-test-pre: PYTHONHASHSEED='4126239415'
test run-test: commands[0] | pytest
============================= test session starts ==============================
platform linux -- Python 3.6.9, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
cachedir: .tox/test/.pytest_cache
rootdir: /home/#######/foo
collected 1 item
tests/test_foo.py . [100%]
============================== 1 passed in 0.09s ===============================
scripts run-test-pre: PYTHONHASHSEED='4126239415'
scripts run-test: commands[0] | bash -c 'python3 scripts/*.py'
Traceback (most recent call last):
File "scripts/foo_script.py", line 1, in <module>
import requests
ModuleNotFoundError: No module named 'requests'
ERROR: InvocationError for command /bin/bash -c 'python3 scripts/*.py' (exited with code 1)
___________________________________ summary ____________________________________
test: commands succeeded
ERROR: scripts: commands failed

I believe something like the following should work:
pyproject.toml
[tool.poetry]
name = "foo"
version = "0.1.0"
description = ""
authors = ["foo maker"]
[tool.poetry.dependencies]
python = "^3.7"
requests = "*"
#
pytest = { version = "^4.6", optional = true }
[tool.poetry.extras]
test = ["pytest"]
# [tool.poetry.dev-dependencies]
# use 'test' extra instead
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
tox.ini
[tox]
envlist = test, scripts
isolated_build = true
[testenv]
basepython = python3.7
whitelist_externals =
pytest
bash
extras =
test
commands =
test: pytest
scripts: bash -c 'for f in scripts/*.py; do python "$f"; done'

Assuming you have installed poetry and tox and pytest are dependencies in your pyproject.yml (notice poetry run, see https://python-poetry.org/docs/cli/#run):
[tox]
envlist = py37
isolated_build = True
skipsdist = True
[testenv]
whitelist_externals = poetry
commands=
poetry run pytest
optionally you can make the install happen on running the tests by changing the last bit to (but then you will need to have tox installed outside of poetry, which could cause you issues down the line)
commands=
poetry install
poetry run pytest
Also depending on your root folder and where the tests are you can configure the path for tox to change directory to by adding
changedir = tests
In which case the whole file would look like this if you are in directory foo executing tox:
[tox]
envlist = py37
isolated_build = True
skipsdist = True
[testenv]
whitelist_externals = poetry
commands=
poetry run pytest
changedir = tests

This is an easy fix. First, run rm -rf .tox to delete the .tox directory, probably hasn't installed the files you wanted.
Here is an example of my tox.ini.
[tox]
isolated_build = true
skipsdist = true
envlist = py39
[testenv]
deps = -rrequirements-dev.txt
whitelist_externals =
poetry
skip_install = true
commands =
python -m pytest tests/
As you can see in the tox.ini file I have a separate requirements for dev. This can be generated from poetry via the commands below.
poetry export --format=requirements.txt --without-hashes --with dev --output=requirements.txt
poetry export --format=requirements.txt --without-hashes --output=requirements-dev.txt
Don't ask me why but the deps in [testenv] are concatenated out, as you can see with deps = -rrequirements-dev.txt, would also change your command to python -m pytest /tests
Ciao.

First I need to install the dependencies with poetry install. Then append poetry run to the beginning of the command to enable the dependencies. Also running python scripts like that will just run the first, passing the name of the others as args to the first program. Instead use for f in scripts/*.py; do python "$f"; done (see here)
All together
poetry install
poetry run bash -c 'for f in scripts/*.py; do python "$f"; done'

tox multiple tests, re-using tox environment

Is it possible to do the following using a single tox virtual environment?
[tox]
envlist = test, pylint, flake8, mypy
skipsdist = true
[testenv:lint]
deps = pylint
commands = pylint .
[testenv:flake8]
deps = flake8
commands = flake8 .
[testenv:mypy]
commands = mypy . --strict
[testenv:test]
deps = pytest
commands = pytest
As I am only testing on my python version (py3.7), I don't want tox to have to create 4 environments (.tox/test, .tox/pylint,.tox/flake8, .tox/mypy) when they could all be run on a single environment.
I also want to see what failed individually individually, thus don't want to do:
[tox]
skipsdist = true
[testenv]
commands = pylint .
flake8 .
mypy . --strict
pytest
as the output would be like this:
_____________ summary ___________
ERROR: python: commands failed
and not like this:
____________________summary _________________
ERROR: test: commands failed
ERROR: lint: commands failed
ERROR: mypy: commands failed
test: commands succeeded

Use generative names and factor-specific commands (search the tox configuration doc page for those terms for more info) to implement all of your desired environments as one so they share the same envdir:
[testenv:{lint,flake8,mypy,test}]
envdir = {toxworkdir}/.work_env
deps = pylint, flake8, pytest
commands =
lint: pylint .
flake8: flake8 .
mypy: mypy . --strict
test: pytest
In tox 4+, you may use the plugin tox-ignore-env-name-mismatch and set runner = ignore_env_name_mismatch in the testenv to allow testenvs with different names to share the same virtualenv.

Python - InvocationError in tox running py.test and coverage test

My project has the following structure:
my_project
|
setup.py
tox.ini
src
|
core_functions.py
client.py
server.py
sql_database.py
tests.py
The tests.py file contains the tests for all the function in files client.py, server.py, sql_database.py, core_functions.py
Next, I created the tox.ini file which I want to use for automation of py.tests and coverage tests. The structure of the tox.ini file is as follows:
[tox]
envlist = py27,cov
[testenv]
commands = py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
[testenv:py27]
commands = coverage erase
coverage run --source=my_project -m pytest
coverage report -m
deps = pytest
When I run in the command line tox the py.tests succeed, but the coverage tests fail with the following error:
============== test session starts ===================
platform darwin -- Python 2.7.13, pytest-3.1.1, py-1.4.33, pluggy-0.4.0
rootdir: /Users/xyz/Documents/ProjectGit/MyProject/my_project, inifile:
plugins: cov-2.5.1
collecting 10334 items / 167 errorsERROR: InvocationError:
'/Users/xyz/Documents/ProjectGit/MyProject/my_project/.tox/py27/bin/coverage run --source=my_project -m pytest'
cov inst-nodeps: /Users/xyz/Documents/ProjectGit/MyProject/my_project/.tox/dist/my_project-0.0.1.zip
cov installed: asn1crypto==0.22.0,attrs==17.2.0,Automat==0.6.0,bitarray==0.8.1,boto3==1.4.4,
botocore==1.5.59,cffi==1.10.0,constantly==15.1.0,coverage==4.4.1,cryptography==1.9,
cycler==0.10.0,docutils==0.13.1,enum34==1.1.6,Fabric==1.13.2,functools32==3.2.3.post2,
future==0.16.0,futures==3.1.1,idna==2.5,incremental==17.5.0,ipaddress==1.0.18,jmespath==0.9.3,
my_project==0.0.1,matplotlib==2.0.2,msgpack-python==0.4.8,numpy==1.12.1,paramiko==2.1.2,
Paver==1.2.4,petlib==0.0.41,py==1.4.33,pyasn1==0.2.3,pybloom==1.1,pycparser==2.17,
pyparsing==2.2.0,pytest==3.1.1,pytest-cov==2.5.1,python-dateutil==2.6.0,pytz==2017.2,
s3transfer==0.1.10,scapy==2.3.3,scipy==0.19.0,six==1.10.0,sphinxmix==0.0.6,
subprocess32==3.2.7,Twisted==17.1.0,zope.interface==4.4.1
cov runtests: PYTHONHASHSEED='3903710496'
cov runtests: commands[0] | py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
How can I fix my tox.ini file to make it work?

The problem is the difference between your two invocations.
In [testenv] you're essentially calling:
py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
But the equivalent with coverage you're just running:
py.test
py.test with no arguments will recurse from the current directory and attempt to discover tests. This generally isn't what you want (you don't care about testing the stdlib, and all the dependencies you've added to a virtualenv, etc.)
If you add your arguments from your other invocation to your coverage line it will act as you want:
coverage run --source=my_project -m pytest -sv --doctest-modules my_project/__init__.py my_project/tests.py

If this happens after some modification, you may remove the .lock file and let tox create it again.

How do I tell tox not to collect tests from dependencies?

By default tox will collect the test from your dependencies too and I want it to collect only the ones from my package.
How can I do this?

Tox is a tool which creates a new virtualenv for each python version you have configured, installs the module your running and then runs a user-specified command to run the tests. It doesn't actually collect the tests to run. That's up to whichever testing tool you're using: py.test, nose, etc. To do that with tox, you'll edit/create a tox.ini to use the correct command that limits the collection of tests to whatever you want.
With nose:
[tox]
envlist = py26,py27
[testenv]
deps=nose
commands=nosetests test.module
With py.test:
[tox]
envlist = py26,py27
[testenv]
deps=pytest
commands=py.test test.module