I would like to have
from __future__ import (absolute_import, division, print_function, unicode_literals)
loaded in each interactive session, using a PYTHONSTARTUP file. This works with basic python (2.7.5, installed with Anaconda), but does not work with ipython (1.0.0, Anaconda). Other import statements work with ipython, but __future__ ones are just ignored (although they work if I enter them on the command line). Is this an ipython bug, or is there something I'm missing?
There are two issues here.
The first one is more general than __future__ statements: At least in my tests, ipython 1.0.0 and earlier just don't process the PYTHONSTARTUP environment variable at all. You can see this pretty easily:
$ echo -e 'print "PYTHONSTARTUP!"\n' > pythonstartup.py
$ PYTHONSTARTUP=./pythonstartup.py ipython
Nothing extra gets printed out.
#2706 suggested that it should do so, #3569 patched it, and 1.1.0 seems to be the first version with the change.
So, the fix is to upgrade to 1.1.0. Or, if you're stuck with an older version, do what was suggested in #2706, and add this to your first $IPYTHONDIR/profile_default/startup/*py file:
import os
if os.environ['PYTHONSTARTUP']:
execfile(os.environ['PYTHONSTARTUP'])
However, that still won't fix the problem.
The way $PYTHONSTARTUP gets run (either explicitly by you or implicitly by iPython) is equivalent to an exec. It does explicitly give the appropriate globals to the exec, which ensures that you end up with the print_function tuple available… but that won't affect the parser. (Compare typing exec('from __future__ import print_function)` at the interactive shell.)
And the same is true for the startup files described above, the backward-compat ipython.rc file (if you have that enabled), and any other files that are supposed to be executed in your interactive environment—they're actually just exec'd in your globals, which isn't quite the same thing.
Even files executed as part of the exec_files mechanism in your ipython_config.py or other app-config script are handled this way.
However, lines executed as part of the exec_lines mechanism are not. So, that's the solution.
Edit or create ~/.ipython/profile_default/ipython_config.py. (If you're using a different profile, ipythondir, app name, etc., you presumably know that, and know how to adjust.)
If it's not already present, add this line:
c = get_config()
Then add this:
c.InteractiveShellApp.exec_lines = ['from __future__ import print_function']
See Configuring the ipython command line application for more details.
If you really wanted to, you could probably do something like this:
import os
try:
# Make sure to pop it so it won't get exec'd later in the startup
pythonstartup = os.environ.pop('PYTHONSTARTUP')
with open(pythonstartup) as f:
c.InteractiveShellApp.exec_lines.append(list(f))
except KeyError:
pass
But that seems pretty hacky.
Related
For writing Python I currently use the excellent PyCharm IDE. I love the way it has code completion so that you often only need to type the first 2 letters and then hit enter.
For easy testing I am of course also often on the command line. The only thing is that I miss the convenient features of the IDE on the command line. Why is there no code completion on the command line? And when I fire up a new Python interactive interpreter, why doesn't it remember commands I inserted earlier (like for example sqlite3 does)?
So I searched around, but I can't find anything like it, or I'm simply not searching for the right words.
So my question; does anybody know of an improved and more convenient version of the Python interactive command line interpreter? All tips are welcome!
bpython is one of the many choices for alternative interactive Python interpreters that sports both of the features you mentioned (tab completion and persistent readline history).
Another very commonly used one would by IPython, though I personally don't like it very much (just a personal preference, many people are very fond of it).
Last but not least you can also enable those features for the standard Python interpreter:
Tab completion: See the docs on the rlcompleter module.
Create a file ~/.pythonrc in your home directory containing this script:
try:
import readline
except ImportError:
print "Module readline not available."
else:
import rlcompleter
readline.parse_and_bind("tab: complete")
This will try to import the readline module, and bind its default completion function to the tab key. In order to execute this script every time you start a Python interpreter, set the environment variable PYTHONSTARTUP to contain the path to this script. How you do this depends on your operating system - on Linux you could do it in your ~/.bashrc for example:
export PYTHONSTARTUP="/home/lukas/.pythonrc"
(The file doesn't need to be called .pythonrc or even be in your home directory - all that matters is that it's the same path you set in PYTHONSTARTUP)
Persistent history: See the .pythonrc file in Marius Gedminas's dotfiles. The concept is the same as above: You add the code that saves and loads the history to your ~/.pythonrc, and configure the PYTHONSTARTUP environment variable to contain the path to that script, so it gets executed every time you start a Python interpreter.
His script already contains the tab completion part. So since you want both, you could save his script called python to ~/.python and add the contents of his bashrc.python to your ~/.bashrc.
I'm working on a wrapper script written in Python. The wrapper is supposed to choose another Python script based on the system state and execute it (using the absolute path). There is no need to return to the parent script.
It should be noted that I have no control over the scripts being run. They can use __name__ checks, access sys.argv and it should all behave like if the script was run directly.
Right now, I'm using os.execl():
import os, sys
# ...
os.execl(sys.executable, sys.executable, new_script, *sys.argv[1:])
But I can count at least three issues with that:
Any options passed to the Python interpreter are not preserved (e.g. python -v wrapper stops being verbose on re-exec);
The Python interpreter is re-executed unnecessarily (with PyPy it adds 0,7s on my system);
It relies on sys.executable being useful, and the docs say that:
If Python is unable to retrieve the real path to its executable, sys.executable will be an empty string or None.
I'm wondering what replacement I should use for the os.execl call to solve all the issues. So far, I can tell that:
execfile() would probably work but it was removed in Python3 and re-implementing it by hand AFAICS is ugly (because of the encoding issues). I'm not sure what other implications can execfile() have;
imp.load_module() would probably work but it's a bit hacky and was deprecated in Python3.3. It probably can suffer Python3 encoding issues as well.
Which solution do you suggest I use?
Edit: I'd forget. The solution has to work with Python 2.5+, PyPy and Jython 2.5+.
I would just use execfile() instead of imp.load_module(). Although control will be returned to the executive script, one big advantage is that, quoting the docs:
It is different from the import statement in that it does not use the
module administration — it reads the file unconditionally and does not
create a new module.
This means the script file can be anywhere, can have any (or no) file extension, and resources aren't wasted doing module-import related tasks.
Doing so automatically accomplishes or avoids the things you desire:
interpreter options will be preserved
interpreter will not be re-executed unnecessarily
it does not rely on the value of sys.executable
Have you tried something like this?
### wrapped script ###
import sys
print("__name__: {0}\nsys.argv: {1}".format(__name__, sys.argv))
### wrapper script ###
import builtins, sys
my_script = "wrapped_script.py"
print("Executing the wrapped script...")
sys.argv[0] = my_script
with open(my_script, 'r') as fd:
for line in fd:
exec(line)
Result:
$ python3 wrapped_script.py --foo --bar=quux
__name__: __main__
sys.argv: ['wrapped_script.py', '--foo', '--bar=quux']
$ python3 wrapper.py --foo --bar=quux
Executing the wrapped script...
__name__: __main__
sys.argv: ['wrapped_script.py', '--foo', '--bar=quux']
I find myself typing import numpy as np almost every single time I fire up the python interpreter. How do I set up the python or ipython interpreter so that numpy is automatically imported?
For ipython, there are two ways to achieve this. Both involve ipython's configuration directory which is located in ~/.ipython.
Create a custom ipython profile.
Or you can add a startup file to ~/.ipython/profile_default/startup/
For simplicity, I'd use option 2. All you have to do is place a .py or .ipy file in the ~/.ipython/profile_default/startup directory and it will automatically be executed. So you could simple place import numpy as np in a simple file and you'll have np in the namespace of your ipython prompt.
Option 2 will actually work with a custom profile, but using a custom profile will allow you to change the startup requirements and other configuration based on a particular case. However, if you'd always like np to be available to you then by all means put it in the startup directory.
For more information on ipython configuration. The docs have a much more complete explanation.
Use the environment variable PYTHONSTARTUP. From the official documentation:
If this is the name of a readable file, the Python commands in that
file are executed before the first prompt is displayed in interactive
mode. The file is executed in the same namespace where interactive
commands are executed so that objects defined or imported in it can be
used without qualification in the interactive session.
So, just create a python script with the import statement and point the environment variable to it. Having said that, remember that 'Explicit is always better than implicit', so don't rely on this behavior for production scripts.
For Ipython, see this tutorial on how to make a ipython_config file
I use a ~/.startup.py file like this:
# Ned's .startup.py file
print("(.startup.py)")
import datetime, os, pprint, re, sys, time
print("(imported datetime, os, pprint, re, sys, time)")
pp = pprint.pprint
Then define PYTHONSTARTUP=~/.startup.py, and Python will use it when starting a shell.
The print statements are there so when I start the shell, I get a reminder that it's in effect, and what has been imported already. The pp shortcut is really handy too...
As a simpler alternative to the accepted answer, on linux:
just define an alias, e.g. put alias pynp='python -i -c"import numpy as np"' in your ~/.bash_aliases file. You can then invoke python+numpy with pynp, and you can still use just python with python. Python scripts' behaviour is left untouched.
While creating a custom startup script like ravenac95 suggests is the best general answer for most cases, it won't work in circumstances where you want to use a from __future__ import X. If you sometimes work in Python 2.x but want to use modern division, there is only one way to do this. Once you create a profile, edit the profile_default (For Ubuntu this is located in ~/.ipython/profile_default) and add something like the following to the bottom:
c.InteractiveShellApp.exec_lines = [
'from __future__ import division, print_function',
'import numpy as np',
'import matplotlib.pyplot as plt',
]
You can create a normal python script as import_numpy.py or anything you like
#!/bin/env python3
import numpy as np
then launch it with -i flag.
python -i import_numpy.py
Way like this will give you flexibility to choose only modules you want for different projects.
As ravenac95 mentioned in his answer, you can either create a custom profile or modify the default profile. This answer is quick view of Linux commands needed to import numpy as np automatically.
If you want to use a custom profile called numpy, run:
ipython profile create numpy
echo 'import numpy as np' >> $(ipython locate profile numpy)/startup/00_imports.py
ipython --profile=numpy
Or if you want to modify the default profile to always import numpy:
echo 'import numpy as np' >> $(ipython locate profile default)/startup/00_imports.py
ipython
Check out the IPython config tutorial to read more in depth about configuring profiles. See .ipython/profile_default/startup/README to understand how the startup directory works.
My default ipython invocation is
ipython --pylab --nosep --InteractiveShellApp.pylab_import_all=False
--pylab has been a ipython option for some time. It imports numpy and (parts of) matplotlib. I've added the --Inter... option so it does not use the * import, since I prefer to use the explicit np.....
This can be a shortcut, alias or script.
I created a little script to get ipython initialized with the code you want.
Create a start.ipy file at your project root folder.
Edit the created file with all the things you need to get into ipython.
ipython profile create <your_profile_name>. Tip, do not add the word "profile" to the name because ipython already includes it.
cp start.ipy ~/.ipython/profile_<your_profile_name>/startup/start.ipy
Run ipython --profile=<your_profile_name> everytime you need everything loaded in ipython.
With this solution, you don't need to set any env variable up. You will need to copy the start.ipy file to the ipython folder every time you modify it, though.
Let's say I have this python script script.py and I load it in the interpreter by typing
import script
and then I execute my function by typing:
script.testFunction(testArgument)
OK so far so good, but when I change script.py, if I try to import again the script doesn't update. I have to exit from the interpreter, restart the interpreter, and then import the new version of the script for it to work.
What should I do instead?
You can issue a reload script, but that will not update your existing objects and will not go deep inside other modules.
Fortunately this is solved by IPython - a better python shell which supports auto-reloading.
To use autoreloading in IPython, you'll have to type import ipy_autoreload first, or put it permanently in your ~/.ipython/ipy_user_conf.py.
Then run:
%autoreload 1
%aimport script
%autoreload 1 means that every module loaded with %aimport will be reloaded before executing code from the prompt. This will not update any existing objects, however.
See http://ipython.org/ipython-doc/dev/config/extensions/autoreload.html for more fun things you can do.
http://docs.python.org/library/functions.html#reload
reload(module)
Reload a previously imported module. The argument must
be a module object, so it must have been successfully imported before.
This is useful if you have edited the module source file using an
external editor and want to try out the new version without leaving
the Python interpreter. The return value is the module object (the
same as the module argument).
An alternative solution that has helped me greatly is to maintain a copy of sys.modules keys and pop the new modules after the import to force re-imports of deep imports:
>>> oldmods = set(sys.modules.keys())
>>> import script
>>> # Do stuff
>>> for mod in set(sys.modules.keys()).difference(oldmods): sys.modules.pop(mod)
>>> import script
When I start the interactive django shell through manage.py, by executing
python -v manage.py shell
from the project directory, I see a lot of modules of format django.package.module getting imported in the verbose output but still I have to import them to use it in the shell.
The same happens when I just run the Python shell (with the -v argument). For example I see this in the verbose output,
import os # precompiled from /usr/local/gdp/lib/python2.4/os.pyc
but still i have to do import os to import and use the os module. What is being imported that am seeing in the verbose output and why I have to import them explicitly again to use them in the shell? Does Python load some essential modules while starting the shell or is it some kind of behind-the-scene magic?
-v traces the first import of a module -- the one that actually loads the module (executes its code, and so may take a bit of time) and sticks it into sys.modules.
That has nothing to do whether your interactive session (module __main__) gets the module injected into its namespace, of course. To ensure module 'goo' does get into the namespace of module 'X' (for any X, so of course including __main__... among many, many others), module 'X' just needs to import goo itself (a very fast operation indeed, if sys.modules['goo'] is already defined!-).
Python loads the site module implicitly when starting up, which may in turn import other modules for its own use. You can pass -S to disable this behavior.
They are getting imported (look at sys.modules) and references to the module are created in whichever modules have imported it.
When you do an import in your shell, if the module has already been imported, you will just get a copy of the reference to it in sys.modules