I am using nosetests to automatically discover and run my unittests. I would also like to have it generate coverage reports.
When I run nosetests with the following command everything works just fine
nosetests .
I looked up online that to generate the coverage, nosetests has a command line argument --with-coverage. I also double checked that this command exists using nosetests --help. However, whenever I run the following command I get the following output
nosetests --with-coverage .
Usage: nosetests [options]
nosetests: error: no such option: --with-coverage
I double checked that the coverage plugin is installed by running
nosetests --plugins
coverage shows up in the list along with a bunch of other plugins.
I also know I have coverage installed because I can manually run the coverage data collection using something along the lines of:
coverage run test.py
Am I misusing the --with-coverage option? Or is there something else I am missing?
Thanks in advance.
I never got the command line options working. I did what Janne Karila suggested and created a setup.cfg file in my projects main directory. Once I had that file I could just run nosetests with no arguments and everything would run.
One problem I had when trying to create my document was that I couldn't figure out what parameters were allowed in the config. But it turns out that any of the commands listed here https://nose.readthedocs.org/en/latest/usage.html#options can be used. Just leave off the double dashes before the command.
For reference my current config file is
[nosetests]
verbosity=1
detailed-errors=1
with-coverage=1
cover-erase=1
cover-package=application
cover-html=1
cover-html-dir=htmlcov
where=tests
This config file says to use coverage, to erase the previous run's coverage, to only report on the files in the application package, and to output an html report to the htmlcov directory.
Hopefully this will help someone else in the future.
Your syntax is correct. It maybe an issue with your environment, double check your python environment and where your have nose and coverage installed. As a sanity check, you can quickly setup a new virtualenv, install nose, and run the command with the coverage option.
As of nose 1.3.7 - the most recent version available on Pypy - that command doesn't exist:
https://github.com/nose-devs/nose/blob/release_1.3.7/nose/plugins/cover.py
It looks like the documentation is being generated from the master branch of the project that does have those options available:
https://github.com/nose-devs/nose/blob/master/nose/plugins/cover.py
What you can do is install nose from the master branch like this:
pip install git+https://github.com/nose-devs/nose#master --upgrade
It will say it just installed 1.3.7 but that's only because the version hasn't been bumped in the project setup.py file yet: https://github.com/nose-devs/nose/blob/master/setup.py#L4
Remember you have just installed an unreleased version of nose, there may be other bugs.
Related
I am trying to learn chef and very new at it.
I have a requirements.txt file which I'm trying to execute through a chef recipe to install some Python modules. I have tried different variations of the code, however I feel like I'm missing something. I have tried writing the following code in the chef recipe:
template '/etc' do
source 'requirements.txt.erb'
owner 'root'
group 'root'
mode '0644'
end
execute 'requirements.txt' do
command 'pip install -r requirements.txt'
action 'run'
end
I was expecting that the requirements file will be called when I run vagrant up and the modules/dependencies of the app will be installed. However, I get this error:
Error executing action run on resource 'execute[requirements.txt]'
Check this example from https://docs.chef.io/resource_execute.html
execute 'upgrade script' do
command 'php upgrade-application.php && touch /var/application/.upgraded'
creates '/var/application/.upgraded'
action :run
end
You need to specify the run action as shown above (:run) rather than with single quotes as shown in your posted example.
if i am not mistaken, i think that coderanger developed a cookbook named python-poise, which has pip_requirements chef resource. it looks like:
pip_requirements '/opt/myapp/requirements.txt'
I am using Pylint and Nose along with sniffer to lint and test my python application on every save. This is sniffer in case you are unaware https://pypi.python.org/pypi/sniffer
Here is the runnable responsible for running nosetests and pylint from scent.py file for sniffer
from sniffer.api import * # import the really small API
from pylint import lint
from subprocess import call
import os
import nose
#runnable
def zimmer_tests(*args):
command = "nosetests some_module/test --nologcapture --with-coverage --cover-config-file=zimmer_coverage_config"
lint.Run(['some_module/app'], exit=False)
return call(command, shell=True) == 0
Here, first lint.Run() runs pylint on my app. Then nosetest is executed on the app using call()
The Problem is that after I save the file nosetests run on updated version of the file however Pylint uses the same old version. I have to restart sniffer every time for pylint to get new version of files.
I assume this is not a problem of sniffer's configuration since nosetests is able to get the new version of file every time. Still I am not sure.
My pylintrc file is almost the same we get from generate command pylint --generate-rcfile > .pylintrc with some minor application specific tweaks.
As pointed by #KlausD. in comments, the lint.Run() was using files from cache as it was being called from a still running process. Calling pylint from command line worked as expected.
Here is the modified code
#runnable
def zimmer_tests(*args):
command_nose = "nosetests some_module/test --nologcapture --with-coverage --cover-config-file=zimmer_coverage_config"
command_lint = "pylint some_module"
call(command_lint, shell=True)
return call(command_nose, shell=True) == 0
Sorry this is a long question. See the sentence in bold at the bottom for the TL;DR version.
I've spent many hours trying to track down a problem where pylint sometimes doesn't report all the errors in a module. Note that it does find some errors (e.g. long lines), just not all of them (e.g. missing docstrings).
I'm running pylint 1.7.2 on Ubuntu 16.04. (The version available from apt was 1.5.2 but installing via pip gives 1.7.2.)
We typically run pylint from tox, with a tox.ini that looks something like this (this is a cut-down version):
[tox]
envlist = py35
[testenv]
setenv =
MODULE_NAME=our_module
ignore_errors = True
deps =
-r../requirements.txt
whitelist_externals = bash
commands =
pip install --editable=file:///{toxinidir}/../our_other_module
pip install -e .
bash -c \'set -o pipefail; pylint --rcfile=../linting/pylint.cfg our_module | tee pylint.log\'
Amongst other things, the ../requirements.txt file contains a line for pylint==1.7.2.
The behaviour is like this:
[wrong] When the line that imports our_other_module is present, pylint appears to complete successfully and not report any warnings, even though there are errors in the our_module code that it should pick up.
[correct] When that line is commented out, pylint generates the expected warnings.
As part of tracking this down I took two copies of the .tox folder with and without the module import, naming them .tox-no-errors-reported and .tox-with-errors-reported respectively.
So now, even without sourcing their respective tox virtualenvs, I can do the following:
$ .tox-no-errors-reported/py35/bin/pylint --rcfile=../linting/pylint.cfg our_module -- reports no linting warnings
$ .tox-with-errors-reported/py35/bin/pylint --rcfile=../linting/pylint.cfg our_module -- reports the expected linting warnings
(where I just changed the pylint script's #! line in each case to reference the python3.5 inside that specific .tox directory instead of the unrenamed .tox)
By diffing .tox-no-errors-reported and .tox-with-errors-reported, I've found that they are very similar. But I can make the "no errors" version start to report errors by removing the path to our_other_module from .tox-no-errors-reported/py35/lib/python3.5/site-packages/easy-install.pth.
So my question is why is pylint using easy_install at runtime, and what is it picking up from our other component that is causing it to fail to report some errors.
As I understand it, pylint has dependencies on astroid and logilab-common, but including these in the requirements.txt doesn't make any difference.
One possible reason for the surprising pylint behavior is the --editable option.
it creates a special .egg-link file in the deployment directory, that links to your project’s source code. And, ..., it will also update the easy-install.pth file to include your project’s source code
The pth file will then affect the sys.path which affects the module import logic of astroid and it is deeply buried in the call stack of pylint.expand_files via pylint.utils.expand_modules. Also pylint identifies the module part and function names in the AST using astroid.modutils.get_module_part.
To test the theory, you can try calling some of the affected astroid functions manually:
import sys, astroid
print(sys.path)
print(astroid.modutils.get_module_part('your_package.sub_package.module'))
astroid.modutils.file_from_modpath(['your_package', 'sub_package', 'module'])
I've been learning about how to do testing in tox for my python project.
I have (what should be) a fairly standard tox initialization file that looks like the following:
[tox]
envlist=py27,flake8
...
[testenv:flake8]
deps=flake8
commands=flake8 library # 'library' is temp. name of project
Everything looks normal, all the test works, and even the flake8 output comes through (output below). However, tox raises an InvocationError (it does the same for testing using pylint)
flake8 recreate: /Users/shostakovich/projects/project_templates/library/.tox/flake8
flake8 installdeps: flake8
flake8 inst: /Users/shostakovich/projects/project_templates/library/.tox/dist/library-0.1.0.zip
flake8 installed: flake8==2.4.1,library==0.1.0,mccabe==0.3,pep8==1.5.7,pyflakes==0.8.1,wheel==0.24.0
library/__main__.py:12:1: F401 'os' imported but unused
library/__main__.py:13:1: F401 're' imported but unused
...
ERROR: InvocationError: '/Users/shostakovich/projects/project_templates/library/.tox/flake8/bin/flake8 library'
I am running tox 2.0.2 on MaxOSX 10.9.5. The problem goes away if I just call flake8 or pylint directly (the version of flake8 is shown above).
tox doesn't fail, it works!
Your flake8 source code check has findings and therefore tox exits with failures, that's your test result. Fix the findings and your done!
You may configure the flake8 run to ignore specific codes with a section in your tox.ini. From the flake8 docs:
[flake8]
ignore = E226,E302,E41
There are more options you may be interested in, e.g. select = ... for whitelisting enabled code checks.
You can also tell flake8 to exit without failure, even if the test results are not perfect. This will suppress the misleading InvocationError. Just add to your command --exit-zero, so for example:
commands=flake8 library --exit-zero
I am using py test allure adaptor and trying to generate input data required for allure report. But I am not able to generate any XML's. When I execute the py file using py.test sample.py, it did create pycache dir. Then I executed "allure generate -v 1.3.9 C:\allurereports" (This is the dir where I had the sample.py). It did create an allure html report but no of test cases was 0. No details were present.
The sample.py(it is same as given in the example)
import allure
#allure.feature('Feature1')
#allure.story('Story1')
def test_minor():
assert False
#allure.feature('Feature2')
#allure.story('Story2', 'Story3')
#allure.story('Story4')
class TestBar:
# will have 'Feature2 and Story2 and Story3 and Story4'
def test_bar(self):
pass
Here's the py.test command used:
py.test sample.py --allure_features=feature1,feature2
Can anybody help me how to generate an allure report from the file? What are the commands to execute?
Lavanya.
I'll try to explain the sequence you must to perform to generate allure report of autotest.
Install pip. Download get-pip.py and perform python get-pip.py.
Install pytest and pytest-allure-adaptor via pip. Perform python -m pip install pytest allure-pytest
Generate autotest allure xml report. Perform python -m pytest sample.py --alluredir <some directory>
In <some directory> appear xml autotest report, which contain results of sample.py tests. Let's make beauty html report via an allure-cli tool.
Install allure-cli. Download last version of allure-cli. allure-cli requires java. allure-cli doesn't require installation, just unpack and use it.
Generate html report. Find allure (allure.bat for Windows) in unpacked zip. Perform allure.bat generate -o <some directory> -v 1.4.0 <some directory>
Find index.html in <some directory> and open it via a browser.
*Note <some directory> the same for all steps
There is a very simple way to generate reports via allure:
first, install allure:
allure-pytest 2.6.0
allure-python-commons 2.6.0
Then, if you are unable to generate the reports, follow below steps:
(using pytest)
pytest test_xyz.py --alluredir=path_where_you_want_to_save_reports
allure serve report_path
If it is still showing allure is not recognized command (blah -blah), then install allure using npm plugin with below command:
npm install -g allure-commandline --save-dev
then follow step (2) again, then one server will start and you will be able to see allure reports.
This is something that I found working -
python3 -m pytest [your_test_class_name] --alluredir \Report
Then perform the below line where you saved your report -
python3 -m allure serve \Report
This will open up the allure report in your default browser.
You should specify directory with your test data (the directory which contains -testsuite.xml files), not a test directory.
You can use py.test --alluredir [path_to_report_dir] to specify it.
PS. Make sure you use right version of allure (latest pytest adapter supports only allure 1.4.*).
For more information see https://github.com/allure-framework/allure-python and https://github.com/allure-framework/allure-cli
now you must use allure-command-line instead of allure-cli to generate html-report, cause the second one is deprecated.