Is it possible to run the PyCharm linter / code style checks from command line and get the warnings/errors?
Extension to that: Is it possible to integrate that in my Travis tests?
To put some of the comments and some further research into an answer:
PyCharm comes with a small command-line utility bin/inspect.sh, which is documented here. This tool is quite limited though, and has some problems, e.g. it cannot run while the PyCharm IDE is running, and it reports somewhat incorrectly / different than the IDE. Related code can be seen e.g. here.
PyCharm does not do PEP8 code style checks by itself but uses the (bundled) pycodestyle tool.
Maybe these shortcomings can be fixed upstream. See e.g. this report, or this, or this.
I'm using this in Travis and GitHub Actions now, via this script.
This also does the necessary setup of a project.
This also compensates for the inspect.sh missing warnings by also using the pycodestyle tool and thus exactly matching the PyCharm IDE warnings.
Alternative, I'm thinking about writing an extended simple utility which basically does this. All the relevant PyCharm code is open source. I created a project page pychar-inspect for this. But this is just in the planing phase currently, and maybe obsolete when this will be addressed upstream.
Related
Looking to find a way, if possible, to disable specific PEP8 warnings for a Python project loaded in PyCharm at the repository level (i.e. saving a repository-committed configuration file which can apply PEP8 configuration hints to any user loading a project in PyCharm).
In a situation where other developers may contribute to a project using PyCharm. I personally do not use PyCharm myself (just a text editor) and building/linting is performed through various tox environments. There are select PEP8 rules that I am particularly not fond of, such as the promotion on injecting two blank lines in specific areas of the code (e.g. E305). While linters can be configured to ignore specific PEP8 rules and developers can invoke a command (like tox) with the same linter configuration, developers using PyCharm will still see these warnings in their environment. For example:
The problem I experience is developers will make (undesired) adjustments to the implementation and submit them for changes in pull requests. While developers can dismiss warnings themselves, I do not want developers to have to assume/interpret which PEP8 rules the project follows (aside from what may be mentioned in a CONTRIBUTING document). In addition, while source files can be modified with a # noqa comment to hint to the IDE to ignore an issue on that line, I am looking for an alternative way to ignore specific PEP8 rules without peppering various # noqa hints throughout the implementation.
For example, looking for a way to disable all E305 warnings in a theoretical .pycharm file such as follows:
[pycharm]
ignore = E305
After some additional investigation, there does not appear to be support (as of 2020-09-23; v2020.2) for repository-specific PEP8 warning customization.
The IDE uses pycodestyle for PEP8 processing. Testing out options from pycodestyle's configuration has shown that user-specific configurations will disable various PEP8 hints in the IDE; however, the goal is avoid having the user to configure anything in this case. While pycodestyle indicates that it will also look at setup.cfg and tox.ini, the IDE does not invoke pycodestyle in a way to use them. Examining how the IDE invokes pycodestyle shows that it will feed a source's contents through stdin for information. pycodestyle only implicitly loads supported project-specific configurations (i.e. setup.cfg or tox.ini) based on a common prefix of provided input files (if provided) on the existing working directory pycodestyle is invoked on -- unfortunately, since PyCharm does not change the working directory when invoking pycodestyle to a project's root, project-specific configurations cannot be implicitly loaded.
An issue has been created on their issue tracker:
PY-44684 | support pycodestyle project-specific configuration settings (pep8)
https://youtrack.jetbrains.com/issue/PY-44684
There is way to do it, but at the moment it requires the corresponding PyCharm's project settings to be kept in VCS. For every reported pycodestyle.py error there is a dedicated quick fix "Ignore errors like this" that includes the respective error code in the settings of "PEP 8 coding style violation" inspection (these codes are then passed directly to pycodestyle.py via its --ignore option).
You can then find and edit these codes in the inspection settings.
If a per-project inspection profile was configured, these settings are saved in .idea/inspectionProfiles/Your_Profile_Name.xml and this way can be shared with the team or external contributors.
Obviously, it won't work if you don't want to expose IDE-specific config files in the repository, especially when there is a better, tool-independent place for such preferences. The reported PY-44684 is a legitimate issue, I agree.
I'm evaluating test framework, lint and code coverage options for a new Python project I'll be working on.
I've chosen pytest for the testing needs. After reading a bunch of resources, I'm confused when to use Sonarcube, Sonarlint , pylint and coverage.py.
Is SonarLint and Pylint comparable? When would I use Sonarcube?
I need to be able to use this in a Jenkins build. Thanks for helping!
Sonarlint and pylint are comparable, in a way.
Sonarlint is a code linter and pylint is too. I haven't used sonarlint, but it seems that analyzes the code a bit deeper that pylint does. From my experience, pylint only follows a set of rules (that you can modify, by the way), while sonarlint goes a bit further analyzing the inner workings of your code. They are both static analyze tools, however.
Sonarcube, on the other hand, does a bit more. Sonarcube is a CI/CD tool that runs static linters, but also shows you code smells, and does a security analysis. All of what I'm saying is based purely on their website.
If you would like to run CI/CD workflows or scripts, you would use Sonarcube, but for local coding, sonarlint is enough. Pylint is the traditional way, though.
Nicholas has a great summary of Pylint vs Sonarlint.
(Personally I use the Sonarlint)
Although the question is older, I thought I'd answer the other part of your question in case anyone else has the same question; internet being eternal and all.
Coverage.py as it sounds, runs code coverage for your package. SonarQube then uses the report that coverage.py makes and does things with it and formats it in a way that the Sonar team decided was necessary. Coverage.py is needed if you want to use SonarQube for code coverage. However, if you just want the code smells from SonarQube, it is not needed.
You were also asking about when to use SonarQube, coverage.py, and Jenkins.
In Jenkins, you would create a pipeline with several stages. Something along the following lines:
Check out code (automatically done as the first step by Jenkins
Build code as it is intended to be used by user/developer
Run Unit Tests
run coverage.py
run SonarQube
I'm working on an (Windows and Mac) application that uses Python as an embedded scripting language.
The application includes an internal text editor, implemented using Scintilla, and I'm using Jedi for autocompletion, which generally works great.
However, when attempting autocompletion on a computer that does not have a separate installation of Python, Jedi raises an error:
jedi.api.environment.InvalidPythonEnvironment:
Could not get version information for 'python':
FileNotFoundError(2, 'The system cannot find the file specified', None, 2, None)
Digging into the code, I can see that the underlying code that is throwing the FileNotFoundError is when Jedi attempts to run python using subprocess.Popen. Python is not installed on the computer, so this fails.
I can also reproduce the same issue on a computer that does have Python installed by editing my Path environment variable not to include the location of python.exe.
Ideally, we don't want users of our application to have to install Python just to get autocompletion working.
My questions:
Is it possible to get Jedi not to spawn subprocesses, and instead run its code inside the same instance of Python within which it itself is running? I couldn't find anything about this in the documentation or the source code that deals with Environments, and extrapolating from the discussion here I suspect the answer might be no.
Is it possible somehow to get Jedi to use the same python37.dll that our application is using for its functionality, instead of looking for a .exe file that does not exist?
Is there any way we could make some kind of minimal Python installation within our existing app installation that uses the same DLLs/Python Lib etc? How could I go about doing this?
Is there any other way to get Jedi autocompletion working in our app without requiring the user to install Python, or including a full Python installer as part of our build process?
Is it possible to get Jedi not to spawn subprocesses, and instead run its code inside the same instance of Python within which it itself is running? I couldn't find anything about this in the documentation or the source code that deals with Environments, and extrapolating from the discussion here I suspect the answer might be no.
This is definitely possible. All the tools are there. There are discussions ongoing here: https://github.com/davidhalter/jedi-vim/issues/870.
IMO a patch to Jedi is needed that uses an jedi.api.environment.InterpreterEnvironment in some cases like yours. It's definitely possible, it's just buggy at the moment.
Are there any IDEs for Python that support automatic error highlighting (like the Eclipse IDE for Java?) I think it would be a useful feature for a Python IDE, since it would make it easier to find syntax errors. Even if such an editor did not exist, it still might be possible to implement this by automatically running the Python script every few seconds, and then parsing the console output for error messages.
eclipse+pydev
pycharm
many others ....
If you use VIM or don't have a problem with it, try this extension. https://github.com/klen/python-mode
This is for Emacs as well: https://github.com/gabrielelanaro/emacs-for-python
Also pycharm and eclipse with pydev work fine.
If I don't use vim I really enjoy spyder. It is easy to use and has some really nice features, like integrated debugging and profiling, graphical variable explorer and object inspector. The latter shows, e.g., the integrated documentation for every function of class you use.
I built an extension to Eclipse and PyDev that does what you describe, it runs the Python code as you're typing, and displays all the variable values and any exceptions that occur. It's called Live Coding in Python, and the web site has a tutorial and a demo video.
PyDev can highlight some problems in your code by analysing it, and Live Coding in Python can show you problems that happen when you run it.
After using C# for long time I finally decided to switch to Python.
The question I am facing for the moment has to do about auto-complete.
I guess I am spoiled by C# and especially from resharper and I was expecting something similar to exist for Python.
My editor of choice is emacs and after doing some research I found autocomplete.pl, yasnippet and rope although it is not clear to me if and how they can be installed in a cygwin based system which is what I use since all the related documentation appears to be linux specific...
The version of emacs I currently use is 23.2.1 which bundles the python mode that although useful is far behind from whatever research has to offer.
My question to python users has to do about how common is autocomplete vs manual typing (using M-/ where possible) ?
I am thinking about just memorizing python build-in functions like len, append, extend etc. and revert close to a pre-autocomplete editing mode. How different such an approach is from what other pythonistas are doing?
I found this post
My Emacs Python environment
to be the most useful and comprehensive list of instructions and references on how to setup a decent Python development environment in Emacs regardless of OS platform. It is still a bit of work to setup but at least it covers the popular packages and components generally recommended for Python in Emacs that provide auto-completion functionality.
I loosely used this post as a guide to do the setup on my Windows machine with Emacs 23.2.1 and Python 2.6.5. Although, I also have Cygwin installed in some cases instead of running the *nix shell commands mentioned in the post, I just download the packages via a web browser, unzip them with 7zip, and copy them to my Emacs' plugin directory.
Also, to install Pymacs, Rope, and Ropemacs, I used Python's EasyInstall package manager. To use it, I downloaded and installed the setuptools package using the Windows install version. Once installed, at the command line, cd to their respective download locations and run the command
easy_install .
instead of the shell commands shown in the post.
Generally, I saved any *.el files in my ~\.emacs.d\plugins (e.g. in %USERPROFILE%\Application Data\.emacs.d\) and then updated my .emacs file to reference them as documented in the post.
Despite all this, on occasion, I've used DreamPie since it does have overall better auto-completion out of the box than my Emacs setup.
I'm spoiled by Intellisense too. The PyDev extensions for Eclipse offer a pretty good auto-complete substitute.
I find that PyDev + Eclipse can meet most of my needs. There is also PyCharm from the Intellij team. PyCharm has the added advantage of smooth integration with git.
I've been using PyScripter, an IDE for Windows, for a while now, and have found it very good. It has autocompletion among many other features. It's written in Delphi -- not that there's anything wrong with that -- it just bothers me a bit, though...
Take a look at Spyderlib, support most of the features including code completion
IMO, by far the easiest way to take advantage of the python tools available for emacs is to take advantage of the defaults that are all set up at:
https://github.com/gabrielelanaro/emacs-for-python
I actually took the time to get pymacs and ropemacs and python-mode all working independently before finding that little gem, and now I rely on it entirely for all my python based customizations. If you are new, I would definitely start there.