When I start the interactive django shell through manage.py, by executing
python -v manage.py shell
from the project directory, I see a lot of modules of format django.package.module getting imported in the verbose output but still I have to import them to use it in the shell.
The same happens when I just run the Python shell (with the -v argument). For example I see this in the verbose output,
import os # precompiled from /usr/local/gdp/lib/python2.4/os.pyc
but still i have to do import os to import and use the os module. What is being imported that am seeing in the verbose output and why I have to import them explicitly again to use them in the shell? Does Python load some essential modules while starting the shell or is it some kind of behind-the-scene magic?
-v traces the first import of a module -- the one that actually loads the module (executes its code, and so may take a bit of time) and sticks it into sys.modules.
That has nothing to do whether your interactive session (module __main__) gets the module injected into its namespace, of course. To ensure module 'goo' does get into the namespace of module 'X' (for any X, so of course including __main__... among many, many others), module 'X' just needs to import goo itself (a very fast operation indeed, if sys.modules['goo'] is already defined!-).
Python loads the site module implicitly when starting up, which may in turn import other modules for its own use. You can pass -S to disable this behavior.
They are getting imported (look at sys.modules) and references to the module are created in whichever modules have imported it.
When you do an import in your shell, if the module has already been imported, you will just get a copy of the reference to it in sys.modules
Related
Say I have a module foo.py, with some code, and a script main.py that includes foo and is executed with python -m main.
At which point changing the code of foo affects the outcome of python -m main?
Specifically, does calling import "freeze" the file in the sense that future execution is not affected by changing it?
Example of main.py:
input()
import foo
input()
import foo
print(foo.f())
On which circumstances the modification of a module file can affect the outcome of the execution?
My question is related with the following:
If I have a code under version control and run it, and checkout a different branch, the code from the different branch will be run if some import is called lazily (e.g. on a function to avoid circular depedencies). Is this true?
From the documentation:
A module can contain executable statements as well as function definitions. These statements are intended to initialize the module. They are executed only the first time the module name is encountered in an import statement.
So changing the module on disk will not have any effect once the module has been imported once. You can see this yourself: have a file foo.py that prints "foo" when imported:
print("foo")
and a file main.py that imports foo multiple times:
import foo
import foo
import foo
and you can see that when you run main.py, the output is only one foo, so foo.py only runs once.
(Note that there is a function importlib.reload that attempts to reload the module, but it is not guaranteed to replace all references to the old module.)
With regard to your edit, yes, that's correct.
In python documentation
When a module is first imported, Python searches for the module and if
found, it creates a module object 1, initializing it.
Once this object is created it will be used even if in another module is reimporting it, python keeps track of already imported modules.
If you want to reload them you have to do it manually, check the built-in reload for python2 or imp.reload for python3
I would like to have
from __future__ import (absolute_import, division, print_function, unicode_literals)
loaded in each interactive session, using a PYTHONSTARTUP file. This works with basic python (2.7.5, installed with Anaconda), but does not work with ipython (1.0.0, Anaconda). Other import statements work with ipython, but __future__ ones are just ignored (although they work if I enter them on the command line). Is this an ipython bug, or is there something I'm missing?
There are two issues here.
The first one is more general than __future__ statements: At least in my tests, ipython 1.0.0 and earlier just don't process the PYTHONSTARTUP environment variable at all. You can see this pretty easily:
$ echo -e 'print "PYTHONSTARTUP!"\n' > pythonstartup.py
$ PYTHONSTARTUP=./pythonstartup.py ipython
Nothing extra gets printed out.
#2706 suggested that it should do so, #3569 patched it, and 1.1.0 seems to be the first version with the change.
So, the fix is to upgrade to 1.1.0. Or, if you're stuck with an older version, do what was suggested in #2706, and add this to your first $IPYTHONDIR/profile_default/startup/*py file:
import os
if os.environ['PYTHONSTARTUP']:
execfile(os.environ['PYTHONSTARTUP'])
However, that still won't fix the problem.
The way $PYTHONSTARTUP gets run (either explicitly by you or implicitly by iPython) is equivalent to an exec. It does explicitly give the appropriate globals to the exec, which ensures that you end up with the print_function tuple available… but that won't affect the parser. (Compare typing exec('from __future__ import print_function)` at the interactive shell.)
And the same is true for the startup files described above, the backward-compat ipython.rc file (if you have that enabled), and any other files that are supposed to be executed in your interactive environment—they're actually just exec'd in your globals, which isn't quite the same thing.
Even files executed as part of the exec_files mechanism in your ipython_config.py or other app-config script are handled this way.
However, lines executed as part of the exec_lines mechanism are not. So, that's the solution.
Edit or create ~/.ipython/profile_default/ipython_config.py. (If you're using a different profile, ipythondir, app name, etc., you presumably know that, and know how to adjust.)
If it's not already present, add this line:
c = get_config()
Then add this:
c.InteractiveShellApp.exec_lines = ['from __future__ import print_function']
See Configuring the ipython command line application for more details.
If you really wanted to, you could probably do something like this:
import os
try:
# Make sure to pop it so it won't get exec'd later in the startup
pythonstartup = os.environ.pop('PYTHONSTARTUP')
with open(pythonstartup) as f:
c.InteractiveShellApp.exec_lines.append(list(f))
except KeyError:
pass
But that seems pretty hacky.
This question already has answers here:
Auto-load a module on python startup
(5 answers)
Closed 9 years ago.
I'm using python on Windows. And I'm trying to find a way to import some default module when starting python. It means, when starting python, some modules should be already imported, just like builtins. Is there any way?
Thanks.
If you mean while going into the interactive mode, just make a small startup script. Instead of just launching python with:
python
try instead:
python -i startupImports.py
startupImports.py:
import time
from collections import defaultdict
from customclass import myclass
#...etc
Define a PYTHONSTARTUP environment variable and point it to a python file and it will be loaded when you start python interactively without a command line script.
I have my PYTHONSTARTUP point to a .pythonrc where I try to import a __boot__.py (bootstrap) in the current directory for any interactive python session.
try:
from __boot__ import *
except Exception:
pass
There's a bunch of other things in my pythonrc like __future__ imports, readline setup, etc.
One option is adding your import to sitecustomize.py. This file is automatically imported at startup. This will be loaded whether Python is started in interactive mode or not. To quote the docs:
[In site.py] an attempt is made to import a module named sitecustomize, which can perform arbitrary site-specific customizations. It is typically created by a system administrator in the site-packages directory. If this import fails with an ImportError exception, it is silently ignored.
Another option is to set the PYTHONSTARTUP environment variable to a startup script with your import. This will only be executed if Python is started in interactive mode, though.
Let's say I have this python script script.py and I load it in the interpreter by typing
import script
and then I execute my function by typing:
script.testFunction(testArgument)
OK so far so good, but when I change script.py, if I try to import again the script doesn't update. I have to exit from the interpreter, restart the interpreter, and then import the new version of the script for it to work.
What should I do instead?
You can issue a reload script, but that will not update your existing objects and will not go deep inside other modules.
Fortunately this is solved by IPython - a better python shell which supports auto-reloading.
To use autoreloading in IPython, you'll have to type import ipy_autoreload first, or put it permanently in your ~/.ipython/ipy_user_conf.py.
Then run:
%autoreload 1
%aimport script
%autoreload 1 means that every module loaded with %aimport will be reloaded before executing code from the prompt. This will not update any existing objects, however.
See http://ipython.org/ipython-doc/dev/config/extensions/autoreload.html for more fun things you can do.
http://docs.python.org/library/functions.html#reload
reload(module)
Reload a previously imported module. The argument must
be a module object, so it must have been successfully imported before.
This is useful if you have edited the module source file using an
external editor and want to try out the new version without leaving
the Python interpreter. The return value is the module object (the
same as the module argument).
An alternative solution that has helped me greatly is to maintain a copy of sys.modules keys and pop the new modules after the import to force re-imports of deep imports:
>>> oldmods = set(sys.modules.keys())
>>> import script
>>> # Do stuff
>>> for mod in set(sys.modules.keys()).difference(oldmods): sys.modules.pop(mod)
>>> import script
I'm just getting started with twisted.web, and I'm having trouble importing a Python module into a .rpy script.
in C:\py\twisted\mysite.py, I have this:
from twisted.web.resource import Resource
from twisted.web import server
class MySite(Resource):
def render_GET(self, request):
request.write("<!DOCTYPE html>")
request.write("<html><head>")
request.write("<title>Twisted Driven Site</title>")
request.write("</head><body>")
request.write("<h1>Twisted Driven Website</h1>")
request.write("<p>Prepath: <pre>{0}</pre></p>".format(request.prepath))
request.write("</body></html>")
request.finish()
return server.NOT_DONE_YET
and in C:\py\twisted\index.rpy, I have this:
import mysite
reload(mysite)
resource = mysite.MySite()
I ran twistd -n web --port 8888 --path C:\py\twisted in command prompt and the server started successfully. But when I requested localhost:8888 I got a (huge) stack trace originating from an ImportError:
<type 'exceptions.ImportError'>: No module named mysite
I can import the module from the interpreter, and if i just execute index.rpy as a python script, I don't get the import error. The documentation on this subject is a bit vague, it just says "However, it is often a better idea to define Resource subclasses in Python modules. In order for changes in modules to be visible, you must either restart the Python process, or reload the module:" (from here).
Does anyone know the proper way to do this?
Short answer: you need to set PYTHONPATH to include C:\py\twisted.
Long answer...
An rpy script is basically just some Python code, like any other Python code. So an import in a rpy script works just like an import in any other Python code. For the most common case, this means that the directories in sys.path are visited one by one, in order, and if a .py file matching the imported name is found, that file is used to define the module.
sys.path is mostly populated from a static definition including things like C:\Python26\Lib\ and from the PYTHONPATH environment variable. However, there's one extra thing worth knowing about. When you run "python", the current working directory is added to the front of sys.path. When you run "python C:\foo\bar\baz.py", C:\foo\bar\' is added to the front ofsys.path. But when you run "twistd ...", nothing useful is added tosys.path`.
This last behavior probably explains why your tests work if you run the rpy script directly, or if you run python and try to import the module interactively, but fail when you use twistd. Adding C:\py\twisted to the PYTHONPATH environment variable should make the module importable when the rpy script is run from the server you start with twistd.