Develop Reference

How can I specify which positional argument to use in tox?

I have the following tox.ini:
[tox]
envlist = py37-{lint, test}
[testenv:py37-{lint, test}]
envdir = {toxworkdir}/lint_and_test_env
deps =
pylint
pytest
pytest-xdist
commands =
lint: pylint src {posargs}
test: pytest tests {posargs}
I want to run both environments in parallel and specify --jobs=4 for pylint and -n auto for pytest. Executing tox -p -- --jobs=4 -n auto fails because pylint does not recognize the -n argument and vice versa.
Is there a way to achieve my goal?

I see no way to do what you want.
That said, I would strongly recommend that you split your environments and use one for testing and one for linting.
[tox]
envlist = py37, lint
[testenv]
deps =
pytest
pytest-xdist
commands =
pytest tests {posargs}
[testenv:lint]
deps = pylint
commands = pylint src {posargs}
Then you can pass in different arguments to the commands, or run all envs via tox.
Also you could set defaults in case you do not specify {posargs}.
e.g.
[testenv:lint]
deps = pylint
commands = pylint src {--jobs=4:posargs}
If you want to run all tox envs in parallel you could run...
tox -p
P.S.: I am one of the tox maintainers, and I have contributed to hundreds of open source projects, and basically 99% use separate envs for testing and linting.
P.P.S.: If you want to run more than one linter, you should have a look at pre-commit.

python tox: how to use a different python envlist in environment

Here is my tox.ini:
[tox]
envlist = py27,py35
[testenv]
deps =
Flask
connexion
pytest
coverage
pytest-cov
requests
six
commands=pytest --junitxml xunit-reports/xunit-result-XXX.xml --cov {envsitepackagesdir} --cov-report=xml
[testenv:local]
#HOW DO I SPECIFY A NEW LIST OF PYENV LIKE 31,36 IN HERE????
commands=
pytest --cov {envsitepackagesdir}/XXX --cov-report html
When I run tox it runs in py27 and py35. I want tox -e local to run in a different set of multiple python environments. I can't figure out how to do this. How do I do this? Right now it does not even respect the intial envlist and only runs on Python 2.7.

So what you want is to have two different sets of environments and run them independently.
What you have to understand first is that envlist is a list of all environments that would be run if you invoke tox without the -e option.
The next thing you have to understand is that there is only one of these lists per tox.ini and that one is in the global [tox] section.
The other thing you have to understand is that the pyXX factors (factors are the parts of environment names that are separated by the - sign) have a special meaning for tox, because they instruct it to build an environment with a specific interpreter. They are also called "default environments" (see basic usage). If you don't ask for that factor when calling tox, then the basepython interpreter will be used to build the virtualenv (the interpreter you are invoking tox from).
so if you invoke tox -e local with a tox.ini like yours, it will execute what is defined in [tox:local] with the basepython, because you are not defining which python should be used to create the virtualenv, so it uses the same interpreter that you invoked tox with.
If you want to be able to invoke your local factor with other interpreters, independent from those other environments, the following could get you started (described in the v2 config docs):
[tox]
envlist = {py27,py35}-remote,{py31,py36}-local
[testenv]
deps =
Flask
connexion
pytest
coverage
pytest-cov
requests
six
[testenv:remote]
commands=pytest --junitxml xunit-reports/xunit-result-XXX.xml --cov {envsitepackagesdir} --cov-report=xml
[testenv:local]
commands= pytest --cov {envsitepackagesdir}/XXX --cov-report html
Check which envs this creates with:
$ tox -a
py27-remote
py35-remote
py31-local
py36-local
What envlist with the curly braces notation does, is create environment names by combining all factors with their permutations (this can have more dimensions also).
if you say tox without -e they will all run and all use the correct interpreter.
If you want to run the local envs only you will have to call it with:
$ tox -e py31-local,py36
Then only those two will run. The thing to take away here is that if you want to run a subset of all environments you have to ask for them with their full names. There is no "sub generation" or extra envlist magic. Just list the full names of the envorenments in a comma separated list and you are golden.
UPDATE
Today I learned that you can also use the generation syntax from the command line. So you could type:
$ tox -e 'py{31,36}'-local
Thank you #phd for pointing it out.

A possible solution to what you are attempting to do is to use tox -l to list all the environments, filter the ones you want, and then feed them back into tox -e.
For example, to run all environments that have "local" in the name (using bash):
tox -e $(tox -l | grep local | paste -sd "," -)
Step-by-step explanation:
tox -l lists all the environments, one on each line
grep local filters only the lines of the input which contain the word "local"
paste -sd "," - joins the lines of the input with commas

[tox]envlist is only a default — a list of environments to run when tox is invoked without option -e and without TOXENV environment variable. Once you use tox -e [tox]envlist is ignored.
You can run local environment with different python versions, but I don't know any way to run it multiple times. You have to list all environments explicitly:
tox -e py33-local,py34-local
You can shorten the command line using tox' conventions:
tox -e 'py3{3,4}'-local

Use generative envlist and factor-conditional settings.
[tox]
envlist = {py27,py31,py35,py36}-{default,local}
[testenv]
deps =
Flask
connexion
pytest
coverage
pytest-cov
requests
six
commands =
{default,local}: python --version
default: pytest --junitxml xunit-reports/xunit-result-XXX.xml --cov {envsitepackagesdir} --cov-report=xml
local: pytest --cov {envsitepackagesdir}/XXX --cov-report html
List all possible combinations of python version and factors using: tox -l
For your "local" case you'd invoke tox in the one of the following ways:
tox -e py31-local
tox -e py36-local
tox -e 'py3{1,6}'-local
Answer heavily influenced by #oliver-bestwalter's answer, but I couldn't get that to work properly for some reason.

OK I greatly appreciate and upvoted the other two answers here but what I ended up doing was different. It seemed onerous just to achieve a separate
python version and command.
What I ended up doing was just making a seperate tox.ini and calling it like tox -c tox-local.ini

Python - InvocationError in tox running py.test and coverage test

My project has the following structure:
my_project
|
setup.py
tox.ini
src
|
core_functions.py
client.py
server.py
sql_database.py
tests.py
The tests.py file contains the tests for all the function in files client.py, server.py, sql_database.py, core_functions.py
Next, I created the tox.ini file which I want to use for automation of py.tests and coverage tests. The structure of the tox.ini file is as follows:
[tox]
envlist = py27,cov
[testenv]
commands = py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
[testenv:py27]
commands = coverage erase
coverage run --source=my_project -m pytest
coverage report -m
deps = pytest
When I run in the command line tox the py.tests succeed, but the coverage tests fail with the following error:
============== test session starts ===================
platform darwin -- Python 2.7.13, pytest-3.1.1, py-1.4.33, pluggy-0.4.0
rootdir: /Users/xyz/Documents/ProjectGit/MyProject/my_project, inifile:
plugins: cov-2.5.1
collecting 10334 items / 167 errorsERROR: InvocationError:
'/Users/xyz/Documents/ProjectGit/MyProject/my_project/.tox/py27/bin/coverage run --source=my_project -m pytest'
cov inst-nodeps: /Users/xyz/Documents/ProjectGit/MyProject/my_project/.tox/dist/my_project-0.0.1.zip
cov installed: asn1crypto==0.22.0,attrs==17.2.0,Automat==0.6.0,bitarray==0.8.1,boto3==1.4.4,
botocore==1.5.59,cffi==1.10.0,constantly==15.1.0,coverage==4.4.1,cryptography==1.9,
cycler==0.10.0,docutils==0.13.1,enum34==1.1.6,Fabric==1.13.2,functools32==3.2.3.post2,
future==0.16.0,futures==3.1.1,idna==2.5,incremental==17.5.0,ipaddress==1.0.18,jmespath==0.9.3,
my_project==0.0.1,matplotlib==2.0.2,msgpack-python==0.4.8,numpy==1.12.1,paramiko==2.1.2,
Paver==1.2.4,petlib==0.0.41,py==1.4.33,pyasn1==0.2.3,pybloom==1.1,pycparser==2.17,
pyparsing==2.2.0,pytest==3.1.1,pytest-cov==2.5.1,python-dateutil==2.6.0,pytz==2017.2,
s3transfer==0.1.10,scapy==2.3.3,scipy==0.19.0,six==1.10.0,sphinxmix==0.0.6,
subprocess32==3.2.7,Twisted==17.1.0,zope.interface==4.4.1
cov runtests: PYTHONHASHSEED='3903710496'
cov runtests: commands[0] | py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
How can I fix my tox.ini file to make it work?

The problem is the difference between your two invocations.
In [testenv] you're essentially calling:
py.test -sv --doctest-modules my_project/__init__.py my_project/tests.py
But the equivalent with coverage you're just running:
py.test
py.test with no arguments will recurse from the current directory and attempt to discover tests. This generally isn't what you want (you don't care about testing the stdlib, and all the dependencies you've added to a virtualenv, etc.)
If you add your arguments from your other invocation to your coverage line it will act as you want:
coverage run --source=my_project -m pytest -sv --doctest-modules my_project/__init__.py my_project/tests.py

If this happens after some modification, you may remove the .lock file and let tox create it again.

Run certain Tox commands only on TravisCI

I am trying to make it so that a code coverage command is only run in Tox for TravisCI. How would I do that?
The codecov library does not work if I just stick it into the .travis.yml file, and I want it to not be run during normal tox tests as it makes the console way too messy.

check for the presence of $TRAVIS environment variable, see here for others that Travis sets
maybe you need to add this line to your tox.ini :
passenv = TRAVIS
or
you can create a custom entry in your tox.ini
[testenv:travis]
deps= ...
commands= <your travis command>
if you need need to add a step
[testenv]
commands =
....
travis: <your travis command>
if you need only to change dependencies change your deps as
[testenv]
deps =
travis: <your travis dependencies>
in your travis.ini call tox -e travis
Note do not list travis in your envlist=
one more option if you need to add a step to a matrix
[testenv]
passenv = TRAVIS
commands =
- ...
- sh -c 'if [ '{env:TRAVIS:}' = 'true' ]; then ...... ; fi'

How do I tell tox not to collect tests from dependencies?

By default tox will collect the test from your dependencies too and I want it to collect only the ones from my package.
How can I do this?

Tox is a tool which creates a new virtualenv for each python version you have configured, installs the module your running and then runs a user-specified command to run the tests. It doesn't actually collect the tests to run. That's up to whichever testing tool you're using: py.test, nose, etc. To do that with tox, you'll edit/create a tox.ini to use the correct command that limits the collection of tests to whatever you want.
With nose:
[tox]
envlist = py26,py27
[testenv]
deps=nose
commands=nosetests test.module
With py.test:
[tox]
envlist = py26,py27
[testenv]
deps=pytest
commands=py.test test.module