Looking to find a way, if possible, to disable specific PEP8 warnings for a Python project loaded in PyCharm at the repository level (i.e. saving a repository-committed configuration file which can apply PEP8 configuration hints to any user loading a project in PyCharm).
In a situation where other developers may contribute to a project using PyCharm. I personally do not use PyCharm myself (just a text editor) and building/linting is performed through various tox environments. There are select PEP8 rules that I am particularly not fond of, such as the promotion on injecting two blank lines in specific areas of the code (e.g. E305). While linters can be configured to ignore specific PEP8 rules and developers can invoke a command (like tox) with the same linter configuration, developers using PyCharm will still see these warnings in their environment. For example:
The problem I experience is developers will make (undesired) adjustments to the implementation and submit them for changes in pull requests. While developers can dismiss warnings themselves, I do not want developers to have to assume/interpret which PEP8 rules the project follows (aside from what may be mentioned in a CONTRIBUTING document). In addition, while source files can be modified with a # noqa comment to hint to the IDE to ignore an issue on that line, I am looking for an alternative way to ignore specific PEP8 rules without peppering various # noqa hints throughout the implementation.
For example, looking for a way to disable all E305 warnings in a theoretical .pycharm file such as follows:
[pycharm]
ignore = E305
After some additional investigation, there does not appear to be support (as of 2020-09-23; v2020.2) for repository-specific PEP8 warning customization.
The IDE uses pycodestyle for PEP8 processing. Testing out options from pycodestyle's configuration has shown that user-specific configurations will disable various PEP8 hints in the IDE; however, the goal is avoid having the user to configure anything in this case. While pycodestyle indicates that it will also look at setup.cfg and tox.ini, the IDE does not invoke pycodestyle in a way to use them. Examining how the IDE invokes pycodestyle shows that it will feed a source's contents through stdin for information. pycodestyle only implicitly loads supported project-specific configurations (i.e. setup.cfg or tox.ini) based on a common prefix of provided input files (if provided) on the existing working directory pycodestyle is invoked on -- unfortunately, since PyCharm does not change the working directory when invoking pycodestyle to a project's root, project-specific configurations cannot be implicitly loaded.
An issue has been created on their issue tracker:
PY-44684 | support pycodestyle project-specific configuration settings (pep8)
https://youtrack.jetbrains.com/issue/PY-44684
There is way to do it, but at the moment it requires the corresponding PyCharm's project settings to be kept in VCS. For every reported pycodestyle.py error there is a dedicated quick fix "Ignore errors like this" that includes the respective error code in the settings of "PEP 8 coding style violation" inspection (these codes are then passed directly to pycodestyle.py via its --ignore option).
You can then find and edit these codes in the inspection settings.
If a per-project inspection profile was configured, these settings are saved in .idea/inspectionProfiles/Your_Profile_Name.xml and this way can be shared with the team or external contributors.
Obviously, it won't work if you don't want to expose IDE-specific config files in the repository, especially when there is a better, tool-independent place for such preferences. The reported PY-44684 is a legitimate issue, I agree.
I used pre-commit to check my python code style before I commit to git.
It works very well. But now I am working on an old project, most of my work just modify several lines of the file. But pre-commit always ask me fix all the pep8 issues of the file.
is there a way that let the pre-commit only check the codes I just modified not the whole file?
there is not in general. pre-commit operates on the file level and does not parse the underlying tool's output in any way
if an underlying tool supports partial changes then you can use that individual tool's features, but doing so at the framework level is difficult, error prone (there is no standardization in tool output), and most of all slow (re-parsing output and diffs is not cheap)
when adapting to an older code base, you're best to take a first pass which improves the code quality and then enable your linting tools. often you can leverage a code formatter such as autopep8 or black to make adoption easier
disclaimer: I'm the author of pre-commit
Is it possible to run the PyCharm linter / code style checks from command line and get the warnings/errors?
Extension to that: Is it possible to integrate that in my Travis tests?
To put some of the comments and some further research into an answer:
PyCharm comes with a small command-line utility bin/inspect.sh, which is documented here. This tool is quite limited though, and has some problems, e.g. it cannot run while the PyCharm IDE is running, and it reports somewhat incorrectly / different than the IDE. Related code can be seen e.g. here.
PyCharm does not do PEP8 code style checks by itself but uses the (bundled) pycodestyle tool.
Maybe these shortcomings can be fixed upstream. See e.g. this report, or this, or this.
I'm using this in Travis and GitHub Actions now, via this script.
This also does the necessary setup of a project.
This also compensates for the inspect.sh missing warnings by also using the pycodestyle tool and thus exactly matching the PyCharm IDE warnings.
Alternative, I'm thinking about writing an extended simple utility which basically does this. All the relevant PyCharm code is open source. I created a project page pychar-inspect for this. But this is just in the planing phase currently, and maybe obsolete when this will be addressed upstream.
I am using flake8, flake8-docstrings and many other flake8 plugins in our project
I want to disable flake8-docstrings only for our test folder.
I want to avoid running flake8 twice because it would mean that running flake8 wouldn't be the straight forward flake8 . anymore. Not only that would mess with my ide settings, that would also be another excuse for the other developers in my project to not run flake8.
Is there a way to configure flake8 to exclude specific rules just for a specific folder?
There is not currently a native option for this.
There is a proposal to add support for this in the configuration file, though no current implementation exists.
There is flake8-per-file-ignores which is a plugin that accomplishes this feature
Update: per-file-ignores has been included in core as of flake8 3.7.x
The easiest way to use it is in the configuration file:
[flake8]
per-file-ignores =
tests/*: D101
(disclaimer: I am the current flake8 maintainer)
have more complicated case when you need to ignore rule for ALL same named folders of a project
When you have a monorepo with several services and need to ignore a rule for for all files in all tests folders accross the "globe"
this way not works:
per_file_ignores =
*/tests/*: S101
I would very much like to integrate pylint into the build process for
my python projects, but I have run into one show-stopper: One of the
error types that I find extremely useful--:E1101: *%s %r has no %r
member*--constantly reports errors when using common django fields,
for example:
E1101:125:get_user_tags: Class 'Tag' has no 'objects' member
which is caused by this code:
def get_user_tags(username):
"""
Gets all the tags that username has used.
Returns a query set.
"""
return Tag.objects.filter( ## This line triggers the error.
tagownership__users__username__exact=username).distinct()
# Here is the Tag class, models.Model is provided by Django:
class Tag(models.Model):
"""
Model for user-defined strings that help categorize Events on
on a per-user basis.
"""
name = models.CharField(max_length=500, null=False, unique=True)
def __unicode__(self):
return self.name
How can I tune Pylint to properly take fields such as objects into account? (I've also looked into the Django source, and I have been unable to find the implementation of objects, so I suspect it is not "just" a class field. On the other hand, I'm fairly new to python, so I may very well have overlooked something.)
Edit: The only way I've found to tell pylint to not warn about these warnings is by blocking all errors of the type (E1101) which is not an acceptable solution, since that is (in my opinion) an extremely useful error. If there is another way, without augmenting the pylint source, please point me to specifics :)
See here for a summary of the problems I've had with pychecker and pyflakes -- they've proven to be far to unstable for general use. (In pychecker's case, the crashes originated in the pychecker code -- not source it was loading/invoking.)
Do not disable or weaken Pylint functionality by adding ignores or generated-members.
Use an actively developed Pylint plugin that understands Django.
This Pylint plugin for Django works quite well:
pip install pylint-django
and when running pylint add the following flag to the command:
--load-plugins pylint_django
Detailed blog post here.
I use the following: pylint --generated-members=objects
If you use Visual Studio Code do this:
pip install pylint-django
And add to VSC config:
"python.linting.pylintArgs": [
"--load-plugins=pylint_django"
],
My ~/.pylintrc contains
[TYPECHECK]
generated-members=REQUEST,acl_users,aq_parent,objects,_meta,id
the last two are specifically for Django.
Note that there is a bug in PyLint 0.21.1 which needs patching to make this work.
Edit: After messing around with this a little more, I decided to hack PyLint just a tiny bit to allow me to expand the above into:
[TYPECHECK]
generated-members=REQUEST,acl_users,aq_parent,objects,_meta,id,[a-zA-Z]+_set
I simply added:
import re
for pattern in self.config.generated_members:
if re.match(pattern, node.attrname):
return
after the fix mentioned in the bug report (i.e., at line 129).
Happy days!
django-lint is a nice tool which wraps pylint with django specific settings : http://chris-lamb.co.uk/projects/django-lint/
github project: https://github.com/lamby/django-lint
Because of how pylint works (it examines the source itself, without letting Python actually execute it) it's very hard for pylint to figure out how metaclasses and complex baseclasses actually affect a class and its instances. The 'pychecker' tool is a bit better in this regard, because it does actually let Python execute the code; it imports the modules and examines the resulting objects. However, that approach has other problems, because it does actually let Python execute the code :-)
You could extend pylint to teach it about the magic Django uses, or to make it understand metaclasses or complex baseclasses better, or to just ignore such cases after detecting one or more features it doesn't quite understand. I don't think it would be particularly easy. You can also just tell pylint to not warn about these things, through special comments in the source, command-line options or a .pylintrc file.
I resigned from using pylint/pychecker in favor of using pyflakes with Django code - it just tries to import module and reports any problem it finds, like unused imports or uninitialized local names.
This is not a solution, but you can add objects = models.Manager() to your Django models without changing any behavior.
I myself only use pyflakes, primarily due to some dumb defaults in pylint and laziness on my part (not wanting to look up how to change the defaults).
Try running pylint with
pylint --ignored-classes=Tags
If that works, add all the other Django classes - possibly using a script, in say, python :P
The documentation for --ignore-classes is:
--ignored-classes=<members names>
List of classes names for which member
attributes should not be checked
(useful for classes with attributes
dynamicaly set). [current: %default]
I should add this is not a particular elegant solution in my view, but it should work.
For neovim & vim8 use w0rp's ale plugin. If you have installed everything correctly including w0rp's ale, pylint & pylint-django. In your vimrc add the following line & have fun developing web apps using django.
Thanks.
let g:ale_python_pylint_options = '--load-plugins pylint_django'
The solution proposed in this other question it to simply add get_attr to your Tag class. Ugly, but works.
So far I have found no real solution to that but work around:
In our company we require a pylint
score > 8. This allows coding
practices pylint doesn't understand
while ensuring that the code isn't
too "unusual". So far we havn't seen
any instance where E1101 kept us
from reaching a score of 8 or
higher.
Our 'make check' targets
filter out "for has no 'objects'
member" messages to remove most of
the distraction caused by pylint not
understanding Django.
For heroku users, you can also use Tal Weiss's answer to this question using the following syntax to run pylint with the pylint-django plugin (replace timekeeping with your app/package):
# run on the entire timekeeping app/package
heroku local:run pylint --load-plugins pylint_django timekeeping
# run on the module timekeeping/report.py
heroku local:run pylint --load-plugins pylint_django timekeeping/report.py
# With temporary command line disables
heroku local:run pylint --disable=invalid-name,missing-function-docstring --load-plugins pylint_django timekeeping/report.py
Note: I was unable to run without specifying project/package directories.
If you have issues with E5110: Django was not configured., you can also invoke as follows to try to work around that (again, change timekeeping to your app/package):
heroku local:run python manage.py shell -c 'from pylint import lint; lint.Run(args=["--load-plugins", "pylint_django", "timekeeping"])'
# With temporary command line disables, specific module
heroku local:run python manage.py shell -c 'from pylint import lint; lint.Run(args=["--load-plugins", "pylint_django", "--disable=invalid-name,missing-function-docstring", "timekeeping/report.py"])'