Does anybody know how to distinguish new errors (those that were found during latest Pylint execution) and old errors (those that were alredy found during previous executions) in the Pylint report?
I'm using Pylint in one of my projects, and the project is pretty big. Pylint reports pretty many errors (even though I disabled lots of them in the rcfile). While I fix these errors with time, it is also important to not introduce new ones. But Pylint HTML and "parseable" reports don't distinguish new errors from those that were identified previously, even though I run Pylint with persistent=yes option.
As for now - I compare old and new reports manually. What would be really nice though, is if Pylint could highlight somehow those error messages which were found on a latest run, but were not found on a previous one. Is it possible to do so using Pylint or existing tools or something? Becuase if not - it seems I will end up writing my own comparison and report generation.
Two basic approaches. Fix errors as they appear so that there will be no old ones. Or, if you have no intention of fixing certain types of lint errors, tell lint to stop reporting them.
If you have a lot of files it would be a good idea to get a lint report for each file separately, commit the lint reports to revision control like svn, and then use the revision control systems diff utility to separate new lint errors from older pre-existing ones. The reason for seperate reports for each .py file is to make it easier to read the diff output.
If you are on Linux, vim -d oldfile newfile is a nice way to read diff. If you are on Windows then just use the diff capability built into Tortoise SVN.
Run pylint on dev branch, get x errors
Run pylint on master branch, get y errors
If y > x, which means you have new errors
You can do above things in CI process, before code is merged to master
Related
Is there any way to use pylint or flake8 real-time inspection on PyCharm? I don't want to push the run bottom but to real-time scan my cod to find the suggestions.
This does not exists as far as I know and for good reasons : if it did it would not be usable because it would be too demanding on the IDE. pylint is checking a lot of thing (for example code duplication which is intrinsically a hard problem to solve) so it's too slow to be run in real time for each key stroke.
I've had great success with the Pylint extension by Roberto Leinardi.
If you're using a virtual environment and have your .pylintrc file in the root of your project, the Pylint extension should pick everything up automatically, even across different platforms. Otherwise, you can go to File > Settings > Pylint to fix them on a per-machine basis.
The only caveat is the "real time" part of your question. I've not been able to get it to run entirely automatically while I'm typing or whatever. Instead I have to save the file I'm currently in (Ctrl + S or switch focus to another window) and then it'll run and make the suggestions in the IDE in the same way the built-in PyCharm linter does.
Hope this helps!
When I am writing Python in V.S. Code, I keep getting errors that highlight a good portion of my code base. IntelliCode shows an error message that reads:
"unexpected token '=' Python(parser-16)"
So, it appear to stem specifically from specific lines, and when the lines are copy & pasted they inherit the errors, however; if you type-out the line anew, the error is resolved? It is a perplexing issue.
There are 3 things that could be
The first is simple, restart your editor and see if the issue persists.
The second thing it could be is an old unmaintained extension:
Installing the wrong extensions can cause headaches and hard-to-solve issues. This is why installing extensions because it's fun, and they are free, is a bad idea. You should always have a decent reason for installing an extension, and should always avoid installing an extension under the pretense that you may, one day, need it.
When it comes to an opensource editor like VSCode, extensions are Hit & miss, which is, to say the least. Sometimes you might find a hidden-gem, or something new the surprises you unexpectedly in a good way, but you need to have a reason to be looking for a specific extension because you need to know what your look for, many extensions offer features that VSCode now supports out of the box, some extensions haven't been upgraded for 2-3 years, and are full of deprecated code. There is an example of a repository that was maintained by Microsoft up until 2020. People are still creating issues that reference parser 16 errors, and they only get a response that the repository is no longer maintained, and a suggestion to use a different tool."_
Here's the link to the repository I mentioned above. I don't know what extensions you are using, but if it uses this old language server repo, which is for some reason still active, you will want to swap out that extension for something else, or just toss it all together.
To test if it is an extension perform the simple steps below:
To confirm that it is an extension...
OPEN Your Project inside of V.S. Code
HIT F1 to open up your quick input menu
TYPE: "Reload window with extensions disabled"
SELECT the command from the menu that matches the text you typed in step-3 above ("Reload Window with Extensions Disabled") (or workbench.action.reloadWindowWithExtensionsDisabled)
After the editor reloads check your and see if the issue persists.
The problem should go away, as you shouldn't have syntax highlighting from an extension enabled. You need to accomplish having Syntax Highlighting W/O having the issue though. To do this you're going to temporarily leave all of your extensions disabled, then, if you don't have it, install the extension pack that is called:
Python (that's it, its just called "Python").
If you do have it, then just enable it. The V.S. Code Python Extension adds Microsoft support to V.S. Code for Python. Make sure that the extension ID-Name is ms-python. python, that it was developed by Microsoft, and that you have the latest version.
After you have your project open with all extensions, except for the Microsoft Python extension installed, see if the issue is gone and that your syntax highlighting works.
The third and last thing it could be is the Python Extensions parser is unable to parse your code, which may, or may not be your fault.
If the above did not solve your issue then it's likely that the issue is a problem with your code, which may, or may not, be your fault. Python syntax is funny, as it's far different than other c-style languages, yet it is still very much C-style. In the past, I have seen parser 16 errors caused by F-strings, and R-Strings, because of bugs in the parser. They either had a hard time picking up multiple quotes """ in a row, or they had a hard time, picking up a logic-operator with multiple quotes, the syntax written like the following example would cause an error in specific use cases:
"""!==
You need to go through, make sure it's not a mistake on your end, you can even post your code here, which would help the community to determine if its a bug, especially if you can confirm for us that you have only the python extension active while you are getting the error.
If you determine that your codebase is in working order, and if you are right, it is in working order, then it is likely a bug, but you need to like really, really be sure. Once you are sure, go to the following repository and create an issue.
VSCode-Python Repository (CLICK HERE)
Some characters might not be what they look like on text editor.
For example:
Non-breaking space is different from space even though they look the same.
״ is different from " even though the looks similar on the screen.
’ different from ' even though the looks similar on the screen.
So when copy a code and paste a code from somewhere and you get syntax errors you do not understand such as unexpected token, Try to replace the characters at the relevant locations.
Had similar problem with python code.
In my case restarting VSC solved problem.
Debugging the problem
Turn off extensions
To test if one of your VSCode extensions is causing the error, open the command pallatte(cmd+shift+p or ctrl+shift+p) and select Reload Window with Extensions Disabled to see if the problem still exists without any extensions
Start Extension Bisect
Also, if you go to the vscode extensions marketplace, there is an option called start extension bisect which vscode explains as
Extension Bisect will use binary search to find an extension that causes a problem. During the process the window reloads repeatedly (~7 times). Each time you must confirm if you are still seeing problems.
Advice on Extensions
The Python extension that is developed by microsoft is ms-python.python, this is usually the one you want
Don't install extensions you don't understand, it can cause hard-to-solve problems.
Any time you do install an extension, check to see if you are having any errors or problems.
I used pre-commit to check my python code style before I commit to git.
It works very well. But now I am working on an old project, most of my work just modify several lines of the file. But pre-commit always ask me fix all the pep8 issues of the file.
is there a way that let the pre-commit only check the codes I just modified not the whole file?
there is not in general. pre-commit operates on the file level and does not parse the underlying tool's output in any way
if an underlying tool supports partial changes then you can use that individual tool's features, but doing so at the framework level is difficult, error prone (there is no standardization in tool output), and most of all slow (re-parsing output and diffs is not cheap)
when adapting to an older code base, you're best to take a first pass which improves the code quality and then enable your linting tools. often you can leverage a code formatter such as autopep8 or black to make adoption easier
disclaimer: I'm the author of pre-commit
Is it possible to run the PyCharm linter / code style checks from command line and get the warnings/errors?
Extension to that: Is it possible to integrate that in my Travis tests?
To put some of the comments and some further research into an answer:
PyCharm comes with a small command-line utility bin/inspect.sh, which is documented here. This tool is quite limited though, and has some problems, e.g. it cannot run while the PyCharm IDE is running, and it reports somewhat incorrectly / different than the IDE. Related code can be seen e.g. here.
PyCharm does not do PEP8 code style checks by itself but uses the (bundled) pycodestyle tool.
Maybe these shortcomings can be fixed upstream. See e.g. this report, or this, or this.
I'm using this in Travis and GitHub Actions now, via this script.
This also does the necessary setup of a project.
This also compensates for the inspect.sh missing warnings by also using the pycodestyle tool and thus exactly matching the PyCharm IDE warnings.
Alternative, I'm thinking about writing an extended simple utility which basically does this. All the relevant PyCharm code is open source. I created a project page pychar-inspect for this. But this is just in the planing phase currently, and maybe obsolete when this will be addressed upstream.
The Story
After cleaning up my Dreamhost shared server's home folder from all the cruft accumulated over time, I decided to start afresh and compile/reinstall Python.
All tutorials and snippets I found seemed overly simplistic, assuming (or ignoring) a bunch of dependencies needed by Python to compile all modules correctly. So, starting from http://andrew.io/weblog/2010/02/installing-python-2-6-virtualenv-and-virtualenvwrapper-on-dreamhost/ (so far the best guide I found), I decided to write a set-and-forget Bash script to automate this painful process, including along the way a bunch of other things I am planning to use.
The Script
I am hosting the script on http://bitbucket.org/tmslnz/python-dreamhost-batch/src/
The TODOs
So far it runs fine, and does all it needs to do in about 900 seconds, giving me at the end of the process a fully functional Python / Mercurial / etc... setup without even needing to log out and back in.
I though this might be of use for others too, but there are a few things that I think it's missing and I am not quite sure how to go for it, what's the best way to do it, or if this just doesn't make any sense at all.
Check for errors and break
Check for minor version bumps of the packages and give warnings
Check for known dependencies
Use arguments to install only some of the packages instead of commenting out lines
Organise the code in a manner that's easy to update
Optionally make the installers and compiling silent, with error logging to file
failproof .bashrc modification to prevent breaking ssh logins and having to log back via FTP to fix it
EDIT: The implied question is: can anyone, more bashful than me, offer general advice on the worthiness of the above points or highlight any problems they see with this approach? (see my answer to Ry4an's comment below)
The Gist
I am no UNIX or Bash or compiler expert, and this has been built iteratively, by trial and error. It is somehow going towards apt-get (well, 1% of it...), but since Dreamhost and others obviously cannot give root access on shared servers, this looks to me like a potentially very useful workaround; particularly so with some community work involved.
One way to streamline this would be to make it work with one of: capistrano/fabric, puppet/chef, jhbuild, or buildout+minitage (and a lot of cmmi tasks). There are some opportunities for factoring in common code, especially with something more high-level than bash. You will run into bootstrapping issues, however, so maybe leave good enough alone.
If you want to look into userland package managers, there is autopackage (bootstraps well), nix (quickstart), and stow (simple but helps with isolation).
Honestly, I would just build packages with a name prefix for all of the pieces and have them install under /opt so that they're out of the way. That way it only takes the download time and a bit of install time to do.