I want to get warnings for unused functions in PyDev, the Python IDE for Eclipse.
I found no option for that in Code Analysis. How can I do this?
PyDev by default does not have such functionality... Finding unused functions may be extremely difficult due to the Python dynamic nature -- also, it would require whole program analysis (the PyDev code analysis is meant to be fast and with as few false positives as possible without doing whole program analysis, so, this check doesn't really suit the targets on PyDev).
Now, there is a project that seems to implement it: https://github.com/jendrikseipp/vulture so, it may be an option... or maybe PyLint (PyLint itself is integrated in PyDev -- http://www.pydev.org/manual_adv_pylint.html -- but I think they may not have that functionality either).
Another option could be running with code coverage (http://www.pydev.org/manual_adv_coverage.html) -- everything not hit could potentially be unused (but that would require a suitable test suite of integration tests which actually test your whole app -- unit tests may end up calling code not used in the real app, which may skew the results -- although you really need a reliable suite of integration tests checking your whole app for this to work).
Related
I have a set of scripts and utility modules that were written for a recent version of Python 3. Now suddenly, I have a need to make sure that all this code works properly under an older version of Python 3. I can't get the user to update to a more recent Python version -- that's not an option. So I need to identify all the instances where I've used some functionality that was introduced since the old version they have installed, so I can remove it or develop workarounds.
Approach #1: eyeball all the code and compare against documentation. Not ideal when there's this much code to look at.
Approach #2: create a virtual environment locally based on the old version in question using pyenv, run everything, see where it fails, and make fixes. I'm doing this anyway, because backporting to the older Python will also mean going backwards in a number of needed third-party modules from PyPi, and I'll need to make sure that the suite still functions properly. But I don't think it's a good way to identify all my version incompatibilities, because much of the code is only exercised based on particular characteristics of input data, and it'd be hard to make sure I exercise all the code (I don't yet have good unit tests that ensure every line will get executed).
Approach #3: in my virtual environment based on the older version, I used pyenv to install the pylint module, then used this pylint module to check my code. It ran; but it didn't identify issues with standard library calls. For example, I know that several of my functions call subprocess.run() with the "check_output=" Boolean argument, which didn't become available until version 3.7. I expected the 3.6 pylint run to spot this and yell at me; but it didn't. Does pylint not check standard library calls against definitions?
Anyway, this is all I've thought of so far. Any ideas gratefully appreciated. Thanks.
If you want to use pylint to check 3.6 code the most effective way is to use a 3.6 interpreter and environment and then run pylint in it. If you want to use the latest pylint version, you can use the py-version option using 3.6 but this is probably going to catch less issue because pylint will not check what you would have in python 3.6, only some known "hard coded" issue in python 3.6 (like for example f-strings for python 3.5, not missing args in subprocess.run).
As noted in the comments, the real issue is that you do not have a proper test suite, so the question is how can you get one cheaply.
Adding unit test can be time consuming. Before doing that, you can add actual end-to-end tests (which will take some computational time and longer feedback time, but that will be easier to implement), by simply running the program with the current version of python that it is working with and storing the results and then adding a test to show you reproduce the same results.
This kind of test is usually expensive to maintain (as each time you are updating the behavior, you have to update the results). However, there are a safeguard against regression, and allow you to perform some heavy refactoring on legacy code in order to move to a more testable structure.
In your case, these end-to-end test will allow you to test against several versions of python the actual application (not only parts of it).
Once you have a better test suite, you can them decide if this heavy end-to-end tests are worth keeping based on the maintenance burden of the test suite (let's not forget that the test suite should not slow you down in your development, so if it is the bottleneck, that means you should rethink your testing)
What will take time is to generate good input data to your end-to-end tests, to help you with that, you should use some coverage tool (you might even spot unreachable code thanks to that). If there are part of your code that you don't manage to reach, I would not bother at first about it, as it means it will be unlikely to be reached by your client (and if it is the case and it fails at your client, be sure to have proper logging implemented to be able to add the test case to your test suite)
It is supposedly possible to debug a Python3/Cython project using gdb, after building gdb from source if you configure it with python2.7 as specified in the Cython debugging docu.
However, the example in the docu:
is sometimes vague (e.g. the code should be built with python3 and debugger run with python2, but I discovered python-gdb is actually linked to python2 ... so how does that work?)
is incomplete (some steps covered in code blocks, others vaguely referred to in text)
is inconsistent (e.g. references to both source.pyx and myfile.pyx)
Furthermore, it:
does not take the use of virtual environments into account
seems to assume the main function resides in the .pyx (but mine resides in a regular main.py)
does not specify what to do when your files live in different directories (e.g. like my main.py and cythonCode.pyx do)
Could someone please explain (preferably with working example) how to do debug a Python3/Cython project in a situation involving all 3 points just mentioned?
At the moment it seems I can actually get DDD to work following this Cython wiki article, but I then discovered that is the 'old' way of doing it, and it refers to the current debugging docu I also linked to. At this point however, it is unclear to me how the 'new' method functions (the old makes more sense to me) and it surely seems more complex to get it to work.
I'm evaluating test framework, lint and code coverage options for a new Python project I'll be working on.
I've chosen pytest for the testing needs. After reading a bunch of resources, I'm confused when to use Sonarcube, Sonarlint , pylint and coverage.py.
Is SonarLint and Pylint comparable? When would I use Sonarcube?
I need to be able to use this in a Jenkins build. Thanks for helping!
Sonarlint and pylint are comparable, in a way.
Sonarlint is a code linter and pylint is too. I haven't used sonarlint, but it seems that analyzes the code a bit deeper that pylint does. From my experience, pylint only follows a set of rules (that you can modify, by the way), while sonarlint goes a bit further analyzing the inner workings of your code. They are both static analyze tools, however.
Sonarcube, on the other hand, does a bit more. Sonarcube is a CI/CD tool that runs static linters, but also shows you code smells, and does a security analysis. All of what I'm saying is based purely on their website.
If you would like to run CI/CD workflows or scripts, you would use Sonarcube, but for local coding, sonarlint is enough. Pylint is the traditional way, though.
Nicholas has a great summary of Pylint vs Sonarlint.
(Personally I use the Sonarlint)
Although the question is older, I thought I'd answer the other part of your question in case anyone else has the same question; internet being eternal and all.
Coverage.py as it sounds, runs code coverage for your package. SonarQube then uses the report that coverage.py makes and does things with it and formats it in a way that the Sonar team decided was necessary. Coverage.py is needed if you want to use SonarQube for code coverage. However, if you just want the code smells from SonarQube, it is not needed.
You were also asking about when to use SonarQube, coverage.py, and Jenkins.
In Jenkins, you would create a pipeline with several stages. Something along the following lines:
Check out code (automatically done as the first step by Jenkins
Build code as it is intended to be used by user/developer
Run Unit Tests
run coverage.py
run SonarQube
In the previous project I was working on, our fabfile got out of control. While the rest of our project was well-tested, we didn't write a single test for our fabfile. Refactoring was scary, and we weren't confident a fabric command would work how we expected until we ran the command.
I'm starting a new project, and I'd like to make sure our fabfile is well-tested from the beginning. Obey the Testing Goat has a great article discussing some possible strategies, yet it has more questions than answers. Using fabtest is a possibility, although it seems to be dead.
Has anyone successfully unit tested their fabfile? If so, how?
run your Fabfile task in a Docker instance
use docker diff to verify that the right files were changed by
the Fabfile.
This is still quite a bit of work, but it allows testing without excessive Fabfile modifications.
Have you tried python-vagrant? It seems to do the same thing that fabtest does, but it includes some Fabric demos and is still used and maintained.
The slides - mentioned by Henrik Andersson - from back then are available here
Robin Kåveland Hansen replied to me:
There are some examples of the types of refactoring that we did in order to keep our fabric code well-tested there.
In general I would say the best advice is to try avoiding low-level code such as shell commands in higher level code that makes decisions about what code to run, eg. isolate effect-full code from code that makes decisions.
Branching increases the amount of test-cases that you need and it's a lot more effort to write good test-cases for code that changes state on some server.
At the time, we used mock to mock out fabric to write test-cases for branch-less code that has side-effects on the server, so the code + tests would look a lot like this
Obviously this has the weakness that it won't pick up bugs in the shell commands themselves. My experience is that this is rarely the cause of serious problems, though.
Other options using mock would be to use the following idea to run the
tests locally on your machine instead of remotely
Maybe the most robust approach is to run the tests in vagrant, but that has the disadvantage of requiring lots of setup and has a tendency to make the tests slower.
I think it's important to have fast tests, because then you can run them all the time and they give you a really nice feedback-loop.
The deploy-script I've written for my current employer has ~150 test cases and runs in less than 0.5 seconds, so the deploy-script will actually do a self-test before deploying.
This ensures that it is tested on each developer machine all the time, which has picked up a good few bugs for example for cases where linux and mac osx behave differently.
I have a python script which generates some reports based on a DB.
I am testing the script using java Db Units which call the python script.
My question is how can I verify the code coverage for the python script while I am running the DB Units?
Coverage.py has an API that you can use to start and stop coverage measurement as you need.
I'm not sure how you are invoking your Python code from your Java code, but once in the Python, you can use coverage.py to measure Python execution, and then get reports on the results.
Drop me a line if you need help.
I don't know how you can check for inter-language unit test coverage. You will have to tweak the framework yourself to achieve something like this.
That said, IMHO this is a wrong approach to take for various reasons.
Inter-language disqualifies the tests from being described as "unit". These are functional tests. and thereby shouldn't care about code coverage (See #Ned's comment below).
If you must (unit) test the Python code then I suggest that you do it using Python. This will also solve your problem of checking for test coverage.
If you do want to function test then it would be good idea to keep Python code coverage checks away from Java. This would reduce the coupling between the Java and Python code. Tests are code after all and it is usually a good idea to reduce coupling between parts.