Ignore the rest of the python file - python

My python scripts often contain "executable code" (functions, classes, &c) in the first part of the file and "test code" (interactive experiments) at the end.
I want python, py_compile, pylint &c to completely ignore the experimental stuff at the end.
I am looking for something like #if 0 for cpp.
How can this be done?
Here are some ideas and the reasons they are bad:
sys.exit(0): works for python but not py_compile and pylint
put all experimental code under def test():: I can no longer copy/paste the code into a python REPL because it has non-trivial indent
put all experimental code between lines with """: emacs no longer indents and fontifies the code properly
comment and uncomment the code all the time: I am too lazy (yes, this is a single key press, but I have to remember to do that!)
put the test code into a separate file: I want to keep the related stuff together
PS. My IDE is Emacs and my python interpreter is pyspark.

Use ipython rather than python for your REPL It has better code completion and introspection and when you paste indented code it can automatically "de-indent" the pasted code.
Thus you can put your experimental code in a test function and then paste in parts without worrying and having to de-indent your code.
If you are pasting large blocks that can be considered individual blocks then you will need to use the %paste or %cpaste magics.
eg.
for i in range(3):
i *= 2
# with the following the blank line this is a complete block
print(i)
With a normal paste:
In [1]: for i in range(3):
...: i *= 2
...:
In [2]: print(i)
4
Using %paste
In [3]: %paste
for i in range(10):
i *= 2
print(i)
## -- End pasted text --
0
2
4
In [4]:
PySpark and IPython
It is also possible to launch PySpark in IPython, the enhanced Python interpreter. PySpark works with IPython 1.0.0 and later. To use IPython, set the IPYTHON variable to 1 when running bin/pyspark:1
$ IPYTHON=1 ./bin/pyspark

Unfortunately, there is no widely (or any) standard describing what you are talking about, so getting a bunch of python specific things to work like this will be difficult.
However, you could wrap these commands in such a way that they only read until a signifier. For example (assuming you are on a unix system):
cat $file | sed '/exit(0)/q' |sed '/exit(0)/d'
The command will read until 'exit(0)' is found. You could pipe this into your checkers, or create a temp file that your checkers read. You could create wrapper executable files on your path that may work with your editors.
Windows may be able to use a similar technique.
I might advise a different approach. Separate files might be best. You might explore iPython notebooks as a possible solution, but I'm not sure exactly what your use case is.

Follow something like option 2.
I usually put experimental code in a main method.
def main ():
*experimental code goes here *
Then if you want to execute the experimental code just call the main.
main()

With python-mode.el mark arbitrary chunks as section - for example via py-sectionize-region.
Than call py-execute-section.
Updated after comment:
python-mode.el is delivered by melpa.
M-x list-packages RET
Look for python-mode - the built-in python.el provides 'python, while python-mode.el provides 'python-mode.
Developement just moved hereto: https://gitlab.com/python-mode-devs/python-mode

I think the standard ('Pythonic') way to deal with this is to do it like so:
class MyClass(object):
...
def my_function():
...
if __name__ == '__main__':
# testing code here
Edit after your comment
I don't think what you want is possible using a plain Python interpreter. You could have a look at the IEP Python editor (website, bitbucket): it supports something like Matlab's cell mode, where a cell can be defined with a double comment character (##):
## main code
class MyClass(object):
...
def my_function():
...
## testing code
do_some_testing_please()
All code from a ##-beginning line until either the next such line or end-of-file constitutes a single cell.
Whenever the cursor is within a particular cell and you strike some hotkey (default Ctrl+Enter), the code within that cell is executed in the currently running interpreter. An additional feature of IEP is that selected code can be executed with F9; a pretty standard feature but the nice thing here is that IEP will smartly deal with whitespace, so just selecting and pasting stuff from inside a method will automatically work.

I suggest you use a proper version control system to keep the "real" and the "experimental" parts separated.
For example, using Git, you could only include the real code without the experimental parts in your commits (using add -p), and then temporarily stash the experimental parts for running your various tools.
You could also keep the experimental parts in their own branch which you then rebase on top of the non-experimental parts when you need them.

Another possibility is to put tests as doctests into the docstrings of your code, which admittedly is only practical for simpler cases.
This way, they are only treated as executable code by the doctest module, but as comments otherwise.

Related

Unexpected keyword arg in decorator : Python [duplicate]

I'm trying to disable warning C0321 ("more than one statement on a single line" -- I often put if statements with short single-line results on the same line), in Pylint 0.21.1 (if it matters: astng 0.20.1, common 0.50.3, and Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)).
I've tried adding disable=C0321 in the Pylint configuration file, but Pylint insists on reporting it anyway. Variations on that line (like disable=0321 or disable=C321) are flagged as errors, so Pylint does recognize the option properly. It's just ignoring it.
Is this a Pylint bug, or am I doing something wrong? Is there a way around this?
I'd really like to get rid of some of this noise.
pylint --generate-rcfile shows it like this:
[MESSAGES CONTROL]
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time.
#enable=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once).
#disable=
So it looks like your ~/.pylintrc should have the disable= line/s in it inside a section [MESSAGES CONTROL].
Starting from Pylint v. 0.25.3, you can use the symbolic names for disabling warnings instead of having to remember all those code numbers. E.g.:
# pylint: disable=locally-disabled, multiple-statements, fixme, line-too-long
This style is more instructive than cryptic error codes, and also more practical since newer versions of Pylint only output the symbolic name, not the error code.
The correspondence between symbolic names and codes can be found here.
A disable comment can be inserted on its own line, applying the disable to everything that comes after in the same block. Alternatively, it can be inserted at the end of the line for which it is meant to apply.
If Pylint outputs "Locally disabling" messages, you can get rid of them by including the disable locally-disabled first as in the example above.
I had this problem using Eclipse and solved it as follows:
In the pylint folder (e.g. C:\Python26\Lib\site-packages\pylint), hold Shift, right-click and choose to open the windows command in that folder. Type:
lint.py --generate-rcfile > standard.rc
This creates the standard.rc configuration file. Open it in Notepad and under [MESSAGES CONTROL], uncomment
disable= and add the message ID's you want to disable, e.g.:
disable=W0511, C0321
Save the file, and in Eclipse → Window → Preferences → PyDev → *pylint, in the arguments box, type:
--rcfile=C:\Python26\Lib\site-packages\pylint\standard.rc
Now it should work...
You can also add a comment at the top of your code that will be interpreted by Pylint:
# pylint: disable=C0321
Pylint message codes.
Adding e.g. --disable-ids=C0321 in the arguments box does not work.
All available Pylint messages are stored in the dictionary _messages, an attribute of an instance of the pylint.utils.MessagesHandlerMixIn class. When running Pylint with the argument --disable-ids=... (at least without a configuration file), this dictionary is initially empty, raising a KeyError exception within Pylint (pylint.utils.MessagesHandlerMixIn.check_message_id().
In Eclipse, you can see this error-message in the Pylint Console (windows* → show view → Console, select Pylint console from the console options besides the console icon.)
To disable a warning locally in a block, add
# pylint: disable=C0321
to that block.
There are several ways to disable warnings & errors from Pylint. Which one to use has to do with how globally or locally you want to apply the disablement -- an important design decision.
Multiple Approaches
In one or more pylintrc files.
This involves more than the ~/.pylintrc file (in your $HOME directory) as described by Chris Morgan. Pylint will search for rc files, with a precedence that values "closer" files more highly:
A pylintrc file in the current working directory; or
If the current working directory is in a Python module (i.e. it contains an __init__.py file), searching up the hierarchy of Python modules until a pylintrc file is found; or
The file named by the environment variable PYLINTRC; or
If you have a home directory that isn’t /root:
~/.pylintrc; or
~/.config/pylintrc; or
/etc/pylintrc
Note that most of these files are named pylintrc -- only the file in ~ has a leading dot.
To your pylintrc file, add lines to disable specific pylint messages. For example:
[MESSAGES CONTROL]
disable=locally-disabled
Further disables from the pylint command line, as described by Aboo and Cairnarvon. This looks like pylint --disable=bad-builtin. Repeat --disable to suppress additional items.
Further disables from individual Python code lines, as described by Imolit. These look like some statement # pylint: disable=broad-except (extra comment on the end of the original source line) and apply only to the current line. My approach is to always put these on the end of other lines of code so they won't be confused with the block style, see below.
Further disables defined for larger blocks of Python code, up to complete source files.
These look like # pragma pylint: disable=bad-whitespace (note the pragma key word).
These apply to every line after the pragma. Putting a block of these at the top of a file makes the suppressions apply to the whole file. Putting the same block lower in the file makes them apply only to lines following the block. My approach is to always put these on a line of their own so they won't be confused with the single-line style, see above.
When a suppression should only apply within a span of code, use # pragma pylint: enable=bad-whitespace (now using enable not disable) to stop suppressing.
Note that disabling for a single line uses the # pylint syntax while disabling for this line onward uses the # pragma pylint syntax. These are easy to confuse especially when copying & pasting.
Putting It All Together
I usually use a mix of these approaches.
I use ~/.pylintrc for absolutely global standards -- very few of these.
I use project-level pylintrc at different levels within Python modules when there are project-specific standards. Especially when you're taking in code from another person or team, you may find they use conventions that you don't prefer, but you don't want to rework the code. Keeping the settings at this level helps not spread those practices to other projects.
I use the block style pragmas at the top of single source files. I like to turn the pragmas off (stop suppressing messages) in the heat of development even for Pylint standards I don't agree with (like "too few public methods" -- I always get that warning on custom Exception classes) -- but it's helpful to see more / maybe all Pylint messages while you're developing. That way you can find the cases you want to address with single-line pragmas (see below), or just add comments for the next developer to explain why that warning is OK in this case.
I leave some of the block-style pragmas enabled even when the code is ready to check in. I try to use few of those, but when it makes sense for the module, it's OK to do as documentation. However I try to leave as few on as possible, preferably none.
I use the single-line-comment style to address especially potent errors. For example, if there's a place where it actually makes sense to do except Exception as exc, I put the # pylint: disable=broad-except on that line instead of a more global approach because this is a strange exception and needs to be called out, basically as a form of documentation.
Like everything else in Python, you can act at different levels of indirection. My advice is to think about what belongs at what level so you don't end up with a too-lenient approach to Pylint.
This is a FAQ:
4.1 Is it possible to locally disable a particular message?
Yes, this feature has been added in Pylint 0.11. This may be done by
adding
# pylint: disable=some-message,another-one at the desired
block level or at the end of the desired line of code.
4.2 Is there a way to disable a message for a particular module only?
Yes, you can disable or enable (globally disabled) messages at the
module level by adding the corresponding option in a comment at the
top of the file:
# pylint: disable=wildcard-import, method-hidden
# pylint: enable=too-many-lines
You can disable messages by:
numerical ID: E1101, E1102, etc.
symbolic message: no-member, undefined-variable, etc.
the name of a group of checks. You can grab those with pylint --list-groups.
category of checks: C, R, W, etc.
all the checks with all.
See the documentation (or run pylint --list-msgs in the terminal) for the full list of Pylint's messages. The documentation also provide a nice example of how to use this feature.
You can also use the following command:
pylint --disable=C0321 test.py
My Pylint version is 0.25.1.
You just have to add one line to disable what you want to disable.
E.g.,
#pylint: disable = line-too-long, too-many-lines, no-name-in-module, import-error, multiple-imports, pointless-string-statement, wrong-import-order
Add this at the very beginning of your module.
In case this helps someone, if you're using Visual Studio Code, it expects the file to be in UTF-8 encoding. To generate the file, I ran pylint --generate-rcfile | out-file -encoding utf8 .pylintrc in PowerShell.
As per Pylint documentation, the easiest is to use this chart:
C convention-related checks
R refactoring-related checks
W various warnings
E errors, for probable bugs in the code
F fatal, if an error occurred which prevented Pylint from doing further processing.
So one can use:
pylint -j 0 --disable=I,E,R,W,C,F YOUR_FILES_LOC
Sorry for diverging a bit from the initial question, about poster's general preference, which would be better addressed by a global configuration file.
But, as in many popular answers, I tend to prefer seeing in my code what could trigger warnings, and eventually inform contributors as well.
My comment to answer from #imolit needs to stay short, here are some details.
For multiple-statements message, it's probably better to disable it at block or module level, like this
# pylint: disable=multiple-statements
My use-case being now attribute-defined-outside-init in a unittest setup(), I opted for a line-scoped message disabling, using the message code to avoid the line-too-long issue.
class ParserTest(unittest.TestCase):
def setUp(self):
self.parser = create_parser() # pylint: disable=W0201
The correspondance can be found locally with a command like
$ pylint --list-msgs | grep 'outside-init'
:attribute-defined-outside-init (W0201): *Attribute %r defined outside __init__*
Of course, you would similarly retrieve the symbolic name from the code.
Python syntax does permit more than one statement on a line, separated by semicolon (;). However, limiting each line to one statement makes it easier for a human to follow a program's logic when reading through it.
So, another way of solving this issue, is to understand why the lint message is there and not put more than one statement on a line.
Yes, you may find it easier to write multiple statements per line, however, Pylint is for every other reader of your code not just you.
My pylint kept ignoring the disable list in my .pylintrc. Finally, I realized that I was executing:
pylint --disable=all --enable=F,E,W
which was overriding the disable list in my .pylintrc.
The correct command to show only Fatal, Errors, Warnings, is:
pylint --disable=C,R
Edit "C:\Users\Your User\AppData\Roaming\Code\User\settings.json"
and add 'python.linting.pylintArgs' with its lines at the end as shown below:
{
"team.showWelcomeMessage": false,
"python.dataScience.sendSelectionToInteractiveWindow": true,
"git.enableSmartCommit": true,
"powershell.codeFormatting.useCorrectCasing": true,
"files.autoSave": "onWindowChange",
"python.linting.pylintArgs": [
"--load-plugins=pylint_django",
"--errors-only"
],
}

Python script entry point: How to call "__main__2"?

I have inherited a python script which appears to have multiple distinct entry points. For example:
if __name__ == '__main__1':
... Do stuff for option 1
if __name__ == '__main__2':
... Do stuff for option 2
etc
Google has turned up a few other examples of this syntax (e.g. here) but I'm still no wiser on how to use it.
So the question is: How can I call a specific entry point in a python script that has multiple numbered __main__ sections?
Update:
I found another example of it here, where the syntax appears to be related to a specfic tool.
https://github.com/brython-dev/brython/issues/163
The standard doc mentions only main as a reserved module namespace. After looking at your sample I notice that every main method seems separate, does its imports, performs some enclosed functionality. My suspicion is that the developer wanted to quickly swap functionalities and didn't bother to use command line arguments for that, opting instead to swap 'main2' to 'main' as needed.
This is by no means proven, though - any chance of contacting the one who wrote this in the first place?

What is the correct way (if any) to use Python 2 and 3 libraries in the same program?

I wish to write a python script for that needs to do task 'A' and task 'B'. Luckily there are existing Python modules for both tasks, but unfortunately the library that can do task 'A' is Python 2 only, and the library that can do task 'B' is Python 3 only.
In my case the libraries are small and permissively-licensed enough that I could probably convert them both to Python 3 without much difficulty. But I'm wondering what is the "right" thing to do in this situation - is there some special way in which a module written in Python 2 can be imported directly into a Python 3 program, for example?
The "right" way is to translate the Py2-only module to Py3 and offer the translation upstream with a pull request (or equivalent approach for non-git upstream repos). Seriously. Horrible hacks to make py2 and py3 packages work together are not worth the effort.
I presume you know of tools such as 2to3, that aim to make the job of porting code to py3k easier, just repeating it here for others' reference.
In situations where I have to use libraries from python3 and python2, I've been able to work around it using the subprocess module. Alternatively, I've gotten around this issue with shell scripts that pipes output from the python2 script to the python3 script and vice-versa. This of course covers only a tiny fraction of use cases, but if you're transferring text (or maybe even picklable objects) between 2 & 3, it (or a more thought out variant) should work.
To the best of my knowledge, there isn't a best practice when it comes to mixing versions of python.
I present to you an ugly hack
Consider the following simple toy example, involving three files:
# py2.py
# file uses python2, here illustrated by the print statement
def hello_world():
print 'hello world'
if __name__ == '__main__':
hello_world()
# py3.py
# there's nothing py3 about this, but lets assume that there is,
# and that this is a library that will work only on python3
def count_words(phrase):
return len(phrase.split())
# controller.py
# main script that coordinates the work, written in python3
# calls the python2 library through subprocess module
# the limitation here is that every function needed has to have a script
# associated with it that accepts command line arguments.
import subprocess
import py3
if __name__ == '__main__':
phrase = subprocess.check_output('python py2.py', shell=True)
num_words = py3.count_words(phrase)
print(num_words)
# If I run the following in bash, it outputs `2`
hals-halbook: toy hal$ python3 controller.py
2

Copy code from IPython without leading triple dots

I'm using IPython Qt Console and when I copy code FROM Ipython it comes out like that:
class notathing(object):
...:
...: def __init__(self):
...: pass
...:
Is there any way to copy them without those leading triple dots and doublecolon?
P.S. I tried both Copy and Copy Raw Text in context menu and it's still the same. OS: Debian Linux 7.2 (KDE).
How about using %hist n to print line n (or a range of lines) without prompts (including line continuations), and doing your copy from that? (Simply scrolling back to that line is nearly as good).
In [1]: def foo():
...: return 1+2
...:
In [6]: %history 1
def foo():
return 1+2
One of the cool features of ipython is session logging. If you enable it, the code you input in your session is logged to a file. It's very useful, I use it all the time.
To make things even niftier for me, I have a shell alias ipy_log_cat, which prints the entire file. You can do something like: ipy_log_cat | tail to get the most recent input lines. (this is also useful for greping session history, etc.). You can also save a few keyboard/mouse strokes by piping it into xclip!
This QTconsole copy regression has been fixed, see https://github.com/ipython/ipython/issues/3206 - I can confirm that the desired behavior is again present in the QtConsole in the Canopy 1.2 GUI and, I suspect, in the ipython egg installable by free users from the Enthought egg repo.
This may be too roundabout for you, but you could use the %save magic function to save the lines in question and then copy them from the save file.
I tend to keep an open gvim window for this kind of things. Paste your class definition as is and then do something like:
:%s/^.*\.://

Printing Variable names and contents as debugging tool; looking for emacs/Python shortcut

I find myself adding debugging "print" statements quite often -- stuff like this:
print("a_variable_name: %s" % a_variable_name)
How do you all do that? Am I being neurotic in trying to find a way to optimize this? I may be working on a function and put in a half-dozen or so of those lines, figure out why it's not working, and then cut them out again.
Have you developed an efficient way of doing that?
I'm coding Python in Emacs.
Sometimes a debugger is great, but sometimes using print statements is quicker, and easier to setup and use repeatedly.
This may only be suitable for debugging with CPython (since not all Pythons implement inspect.currentframe and inspect.getouterframes), but I find this useful for cutting down on typing:
In utils_debug.py:
import inspect
def pv(name):
record=inspect.getouterframes(inspect.currentframe())[1]
frame=record[0]
val=eval(name,frame.f_globals,frame.f_locals)
print('{0}: {1}'.format(name, val))
Then in your script.py:
from utils_debug import pv
With this setup, you can replace
print("a_variable_name: %s' % a_variable_name)
with
pv('a_variable_name')
Note that the argument to pv should be the string (variable name, or expression), not the value itself.
To remove these lines using Emacs, you could
C-x ( # start keyboard macro
C-s pv('
C-a
C-k # change this to M-; if you just want to comment out the pv call
C-x ) # end keyboard macro
Then you can call the macro once with C-x e
or a thousand times with C-u 1000 C-x e
Of course, you have to be careful that you do indeed want to remove all lines containing pv(' .
Don't do that. Use a decent debugger instead. The easiest way to do that is to use IPython and either to wait for an exception (the debugger will set off automatically), or to provoke one by running an illegal statement (e.g. 1/0) at the part of the code that you wish to inspect.
I came up with this:
Python string interpolation implementation
I'm just testing it and its proving handy for me while debugging.

Categories

Resources