I have a Pyramid web application which uses Jinja2 as template engine. It is tested using the Pyramid testing helpers together with py.test and the coverage plugin. But coverage works only for my Python code. Is there way to check also for coverage of Jinja templates?
As Jinja is compiled into Python code and as Armin mananged to give exact line numbers in the case of an exception, I should at least be possible in theory. Any hint would be very appreciated.
There is jinja_coverage.
However, it is incomplete and has a single developer with very little activity (not even published yet). You could also just take its code as hints for how to do it yourself.
Related
I'm evaluating test framework, lint and code coverage options for a new Python project I'll be working on.
I've chosen pytest for the testing needs. After reading a bunch of resources, I'm confused when to use Sonarcube, Sonarlint , pylint and coverage.py.
Is SonarLint and Pylint comparable? When would I use Sonarcube?
I need to be able to use this in a Jenkins build. Thanks for helping!
Sonarlint and pylint are comparable, in a way.
Sonarlint is a code linter and pylint is too. I haven't used sonarlint, but it seems that analyzes the code a bit deeper that pylint does. From my experience, pylint only follows a set of rules (that you can modify, by the way), while sonarlint goes a bit further analyzing the inner workings of your code. They are both static analyze tools, however.
Sonarcube, on the other hand, does a bit more. Sonarcube is a CI/CD tool that runs static linters, but also shows you code smells, and does a security analysis. All of what I'm saying is based purely on their website.
If you would like to run CI/CD workflows or scripts, you would use Sonarcube, but for local coding, sonarlint is enough. Pylint is the traditional way, though.
Nicholas has a great summary of Pylint vs Sonarlint.
(Personally I use the Sonarlint)
Although the question is older, I thought I'd answer the other part of your question in case anyone else has the same question; internet being eternal and all.
Coverage.py as it sounds, runs code coverage for your package. SonarQube then uses the report that coverage.py makes and does things with it and formats it in a way that the Sonar team decided was necessary. Coverage.py is needed if you want to use SonarQube for code coverage. However, if you just want the code smells from SonarQube, it is not needed.
You were also asking about when to use SonarQube, coverage.py, and Jenkins.
In Jenkins, you would create a pipeline with several stages. Something along the following lines:
Check out code (automatically done as the first step by Jenkins
Build code as it is intended to be used by user/developer
Run Unit Tests
run coverage.py
run SonarQube
Following Ned Batchelder's Coverage.py for Django templates blog post and the django_coverage_plugin plugin for measuring code coverage of Django templates.
I would really like to see template coverage reports, but the problem is - we have replaced the Django's the template engine with jinja2 through the coffin adapter. I know it is not recommended, but we had reasons. Basically, instead of Django's render_to_response() we are using:
from coffin.shortcuts import render_to_response
Currently, we are using django-coverage's test_coverage command to run tests with coverage:
python manage.py test_coverage project_name
I understand that the setup is not common. But, is it possible to gather code coverage metrics for jinja2 templates with coverage.py? If not, I would appreciate any ideas on making a plugin similar to django_coverage_plugin, but for jinja2.
The plugin support in coverage.py is still in an alpha state. I've implemented the Django template support and half of the Mako support. A Jinja plugin would be appreciated. It might need changes in Jinja to make it feasible, in particular, to map the compiled Python code execution back to template line numbers.
If you want to start work on it, read the interfaces in plugin.py in the coverage.py source (I'll eventually write real docs, of course). The source for the Mako plugin might help also: https://bitbucket.org/ned/coverage-mako-plugin
This will likely involve us working together to get past the rough parts. Drop me an email! :)
Thanks for taking it on!
Updated: I've written some docs for the plugins, though more are needed: http://coverage.readthedocs.org/en/coverage-4.0a4/plugins.html
Is it possible to use Pytest with Django without using django-pytest 3rd party app?
I had tried to set this up, but kept running into random errors, like pytest couldn't find the DJANGO_SETTINGS_MODULE. Then I fixed the path, but the normal python manage.py runserver then couldn't find the DJANGO_SETTINGS_MODULE. I'm running:
Pytest 2.5.4
Python 3.4.0
Django 1.6.2
If it is possible, would you be able to provide a setup example of where to put the tests/ directory, etc... within the project so Pytest works?
Thanks
Hmm, py.test 2.5.4 does not exist afaik.
Anyway, assuming you mean to ask whether it is possible to avoid the pytest-django plugin to test Django using py.test: the short answer is no.
The long answer is yes, but it is extremely difficult to get it all to work and you will basically write at least a minimal version of pytest-django in the conftest.py file. The pytest-django plugin was created specifically to work around all the weirdness which Django does with much global state and other hidden magic.
OTOH looking at the pytest-django source would probably help you kickstart such an effort. However you may consider thinking about what it is of pytest-django you do not like and maybe file an enhancement request to improve it.
I've been seeing and reading about a lot of people using nose to run their Django tests. I haven't been able to figure out the added benefits of using Nose to run my Django tests. If someone could fill me in on what nose is and how it adds more to a Django project, it would be helpful.
I haven't been able to find a good document/article outlining these points.
Thank you
I was curious about this too and it seems that the main advantage of django-nose using the python nose library is "Test Discovery".
In addition, from http://readthedocs.org/docs/nose/en/latest/testing.html
you can also write simple test functions, as well as test classes that are not
subclasses of unittest.TestCase. nose also supplies a number of
helpful functions for writing timed tests, testing for exceptions, and
other common use cases. See Writing tests and Testing tools for more.
From what I understand from other python developers on freenode irc, Trial test runner on Twisted Framework have these similar features like nose.
I am still not entirely convinced about using django-nose for django development but am giving a shot and report back if I find out more!
There are a lot more features overall, but I think one major reason people use nose/djano_nose is that it allows you to very easily do code coverage.
python manage.py test myapp --with-coverage --cover-package=myapp
I've started working on a project with loads of unused legacy code in it. I was wondering if it might be possible to use a tool like coverage in combination with a crawler (like the django-test-utils one) to help me locate code which isn't getting hit which we can mark with deprecation warnings. I realise that something like this won't be foolproof but thought it might help.
I've tried running coverage.py with the django debug server but it doesn't work correctly (it seems to just profile the runserver machinery rather than my views, etc).
We're improving our test coverage all the time but there's a way to go and I thought there might be a quicker way.
Any thoughts?
Thanks.
You can run the development server under coverage if you use the --noreload switch:
coverage run ./manage.py runserver --noreload
pylint is great tool for static code analysis (among others things it will detect unused imports, variables or arguments).
http://nedbatchelder.com/blog/200806/pylint.html
http://www.doughellmann.com/articles/pythonmagazine/completely-different/2008-03-linters/index.html