I have a Django app and I'm getting an error whenever I try to run my code:
Error: No module named django_openid
Let me step back a bit and tell you how I came about this:
I formatted my computer and completely re-installed everything -- including virtualenv, and all dependent packages (in addition to Django) required for my project based on settings in my requirements.txt folder
I tried doing python manage.py syncdb and got the error
I googled the issue, and many people say it could be a path problem.
I'm confused as to how I go about changing the path variables though, and what exactly they mean. I found some documentation, but being somewhat of a hack-ish noob, it kind of goes over my head.
So my questions are:
What exactly is their purpose -- and are they on a system based level based on the version of Python or are they project dependent?
How can I see what mine are set to currently?
How can I change them (ie. where is this .profile file they talk of and can I just use a text editor)
Any input you would have would be great as this one is stumping me and I just want to get back to writing code :-)
The path is just the locations in your filesystem in which python will search for the modules you are trying to import. For example, when you run import somemodule, Python will perform a search for somemodule in all the locations contained in the path (sys.path variable).
You should check the path attribute in sys module:
import sys
print sys.path
It is just a regular list, sou you could append/remove elements from it:
sys.path.append('/path/to/some/module/folder/')
If you want to change your path for every python session you start, you should create a file to be loaded at startup, doing so:
Create a PYTHONSTARTUP environment variable and setting it to your startup file. E.g.: PYTHONSTARTUP=/home/user/.pythonrc (in a unix shell);
Edit the startup file so it contains the commands you want to be auto-executed when python is loaded;
An example of a .pythonrc could be:
import sys
sys.path.append('/path/to/some/folder/')
Do you really need to alter the path? It's always best to actually think about your reasons first. If you're only going to be running a single application on the server or you just don't care about polluting the system packages directory with potentially unnecessary packages, then put everything in the main system site-packages or dist-packages directory. Otherwise, use virtualenv.
The system-level package directory is always on the path. Virtualenv will add its site-packages directory to the path when activated, and Django will add the project directory to the path when activated. There shouldn't be a need to add anything else to the path, and really it's something you should never really have to worry about in practice.
Related
On windows 7, I currently don't have a python path. Can I safely make one? If so, how do I do it?
Upon making this variable, I can no longer load Spyder (IDE) without it crashing. Does anyone know why?
I would like to edit my existing python path if possible, but just don't know why it isn't already there in environmental variables.
I would ultimately like to be able to run "python myscript.py" and have myscript be in a different directory from the call directory.
PYTHONPATH adds new paths to the ones Python uses by default. The path in total determines where Python will look for modules when you import them.
Look at sys.path to see the combination of the defaults with your PYTHONPATH environment variable. It's likely that Spyder is loading a module that exists in two different places and the wrong one comes first.
When you import modules in python, python searches for the module in the directories in PYTHONPATH, in addition to some other directories.
In order to be able to run your script as > myscript.py, you want to put your script somewhere on PATH (here are some instructions for viewing or updating PATH), this is where the OS looks for scripts and programs when you give it a command. I believe that in windows the .py extension must be associated with python for windows to know that myscript.py should be run using python. This should happen automatically when python in installed, but maybe someone with more windows knowledge can comment on this.
it has role similar to path. this variable tells the python interpreter where to
locate the module files imported into a program. it should include the python source library directory and the directories contain in python source code
The first entry of sys.path is the directory of the current script, according to the docs. In the following setup, I would like to change this default. Imagine the following directory structure:
src/
core/
stuff/
tools/
tool1.py
tool2.py
gui/
morestuff/
gui.py
The scripts tool*.py and gui.py are intended to be run as scripts, like the following:
python src/core/tools/tool2.py
python src/gui/gui.py
Now all tools import from src.core.stuff, and the GUI needs gui.morestuff. This means that sys.path[0] should point to src/, but it points to src/core/tools/ or src/gui/ by default.
I can adjust sys.path[0] in every script (with a construct like the following, e.g., at the beginning of gui.py):
if __name__ == '__main__':
if sys.path[0]: sys.path[0] = os.path.dirname(os.path.abspath(sys.path[0]))
However, this is sort of redundant, and it becomes tedious for a mature code base with thousands of scripts. I also know the -m switch:
python -m gui.gui
But this requires the current directory to be src/.
Is there a better way to achieve the desired result, e.g. by modifying the __init__.py files?
EDIT: This is for Python 2.7:
~$ python -V
Python 2.7.3
The only officially approved way to run a script that is in a package is by using the -m flag. While you could run a script directly and try to do sys.path manipulations yourself in each script, it's likely to be a big pain. If you move a script between folders, the logic for rewriting sys.path may also need to be changed to reflect the new location. Even if you get sys.path right, explicit relative imports will not work correctly.
Now, making python -m mypackage.mymodule work requires that either you be in the project's top level folder (src in your case), or for that top level folder to be on the Python search path. Requiring you to be in a specific folder is awkward, and you've said that you don't want that. Getting src into the search path is our goal then.
I think the best approach is to use the PYTHONPATH environment variable to point the interpreter to your project's src folder so that it can find your packages from anywhere.
This solution is simple to set up (the environment variable can be be set automatically in your .profile, .bashrc or some other equivalent place), and will work for any number of scripts. If you move your project, just update your environment settings and you'll be all set, without needing to do any more work for each script.
You've got three basic options here. I've been through all three in both a production environment and personal projects. In many ways they build on each other. However, my advice is to just skip to the last one.
The fundamental problem is that you need your ./src directory to be in the python search path. This is really what python packaging is all about.
PYTHONPATH
The most straightforward, user defined way to adjust your python path is through the environment variable PYTHONPATH. You can set it at run time, doing something like:
PYTHONPATH=/src python src/gui/gui.py
You can of course also set this up in your global environment so hopefully all processes that need it will find the correct PYTHONPATH. But, just remember, you'll always forget one. Usually at 3 AM when your cron task finally runs.
Site Packages
To avoid needing an environment variable, your options are pretty much to include your software in an existing entry in the source path, or find some additional way to add a new search path. So this can mean dropping the contents of your src directory into /usr/lib/python2.7/site-packages or wherever your system site-packages is located.
Since you may not want to actually include the code in site-packages, you can create a symlink for your two sub-packages.
This is of course less than ideal for a number of reasons. If you're not careful with naming then suddenly every python program on the machine is exposed to potential name conflicts. You're exposing your software to every user on the machine. You might run into issues if python get's updated. If you add a new sub-package, now you have to create a new symlink.
A slightly better approach is to include a .pth file somewhere in your site-packages. When python encounters these files, it adds the contents (which is supposed to be the name of a directory) to the search path. This avoids the problem of having to remember to add a new symlink for each new sub-package.
virtualenv and packaging
The best solution is to just bite the bullet and do real python packaging. This, combined with great tools like virtualenv and pip let you have an isolated (or semi-isolated) python environment.
Under virtualenv, you would have a custom site-packages for just your project where you can easily install your software into it, avoiding all the problems of the earlier solutions. virtualenv also makes it easy to maintain executable scripts so that the python environment it runs under is exactly as you expect.
The one downside is that you have to write and maintain a setup.py which will instruct pip (the python installer) to include your software in the virtualenv. The contents would be something like:
!/usr/bin/env python
# -*- coding: utf-8 -*-
from distutils.core import setup
setup(
name='myproject',
package_dir={'myproject': 'src'},
scripts=['src/gui/gui.py', 'src/core/tools/tool1.py', 'src/core/tools/tool2.py']
)
So, to setup this environment, it's going to look something like this:
virtualenv env
env/bin/pip install -e setup.py
To run your script, then you'd just do something like:
env/bin/tool1.py
I wanted to do this to avoid having to set PYTHONPATH in the first place
There are other places you can hook into Python's sys.path initialization, using the site module, which is (by default) automatically imported when Python initializes.
Based on the this code in site.py...
# Prefixes for site-packages; add additional prefixes like /usr/local here
PREFIXES = [sys.prefix, sys.exec_prefix]
...it looks as if the intention was that this file was designed to be modified after installation, which is one option, although it also provides other ways you can influence sys.path, e.g. by placing a .pth file somewhere inside your site-packages directory.
Assuming the desired result is to make the code work 'out of the box', this would work, but only for all users on a single system.
If you need it to work on multiple systems, then you'd have to apply the same changes to all systems.
For deployment, this is no big deal. Indeed, many Python packages already do something like this. e.g. on Ubuntu...
~$ dpkg -L python-imaging | grep pth
/usr/share/pyshared/PIL.pth
/usr/lib/python2.7/dist-packages/PIL.pth
...but if your intention is to make it easy for multiple concurrent developers, each using their own system, you may be better off sticking with the current option of adding some 'boilerplate' code to every Python module which is intended to be run as a script.
There may be another option, but it depends on exactly what you're trying to achieve.
I am tinkering with some pet projects with Python in Linux (Mint 13) and I plan to do the following:
Create a Dropbox subfolder named "pybin" where I put all my home-made python modules;
Put a symlink to this folder somewhere in the system (first candidate: /usr/lib/python2.7/dist-packages, which is in sys.path, or some similar path);
Then I just do import mymodule from any python session, and the module is imported.
I tried it and it didn't work. I suspect this has to do with differences between modules and packages, and __init__.py files, but I confess that everytime I read something about this stuff I get pretty confused. Besides learning a bit more about this, all I really want to do is find a way to import my modules the described way. It is crucial that the actual folder is inside Dropbox (or any other file-syncing folder), not in a system folder.
Thanks for any help!
Why not simply set the PYTHONPATH envvar in your .bash_profile. That way every time you execute a bash shell (normally happens upon login), this environment variable will be set the wherever you place your user defined modules. The python interpreter uses this variable to determine where to search for module imports:
PYTHONPATH="${PYTHONPATH}:/path/to/some/cool/python/package/:/path/to/another/cool/python/package/"
export PYTHONPATH
I am having a bit of a time installing django on my local machine due to the fact that my local machine is windows. One thing I am having issues with is the includes
#!/usr/bin/env python
Every file has this inlclude. I set python in my global varibles so I can call on it like
python
or I can include it like
#!python
but not like
#!/usr/bin/env python
my assumption is that in order for this to work. I probably might have to move my interpreter to a folder called
c:/user/bin/env
Is this the case or is there a better way. I damn for sure dont want to change all the paths in all the folders.
You just need to set PATH environment variable (add path to python) and leave django files as them are.
I'm trying to run Python scripts using Xcode's User Scripts menu.
The issue I'm having is that my usual os.sys.path (taken from ~/.profile) does not seem to be imported when running scripts from XCode the way it is when running them at the Terminal (or with IPython). All I get is the default path, which means I can't do things like
#!/usr/bin/python
import myScript
myScript.foo()
Where myScript is a module in a folder I've added to my path.
I can append a specific path to os.sys.path manually easily enough, but I have to do it in every single script for every single path I want to use modules from
Is there a way to set this up so it uses the same path I use everywhere else?
EDIT: After looking into things a bit more, it seems like scripts executed from Xcode use a completely different PATH than normal. The path I get by running a script in Xcode is:
PATH=/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin
and I'm sure my regular path doesn't have /Developer/usr/bin in it. Does anybody have any idea where this path is coming from?
On the mac, environment variables in your .profile aren't visible to applications outside of the terminal.
If you want an environment variable (like PATH, PYTHONPATH, etc) to be available to xcode apps, you should add it to a new plist file that you create at ~/.MacOSX/environment.plist.
See the EnvironmentVars doc on the apple developer website for more details.
A quick but hackish way is to have a wrapper script for python.
cat > $HOME/bin/mypython << EOF
#!/usr/bin/python
import os
os.path = ['/list/of/paths/you/want']
EOF
and then start all your XCode scripts with
#!/Users/you/bin/mypython
Just add the paths to sys,path.
>>> import sys
>>> sys.path
['', ... lots of stuff deleted....]
>>> for i in sys.path:
... print i
...
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages
/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload
/Library/Python/2.5/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC
>>> sys.path.append("/Users/crm/lib")
>>> for i in sys.path:
... print i
...
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-darwin
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/plat-mac/lib-scriptpackages
/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-tk
/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-dynload
/Library/Python/2.5/site-packages
/System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/PyObjC
/Users/crm/lib
>>>
I tend to use pth files. From the docs.
The most convenient way is to add a
path configuration file to a directory
that’s already on Python’s path,
usually to the .../site-packages/
directory. Path configuration files
have an extension of .pth, and each
line must contain a single path that
will be appended to sys.path. (Because
the new paths are appended to
sys.path, modules in the added
directories will not override standard
modules. This means you can’t use this
mechanism for installing fixed
versions of standard modules.)
So the simplest thing to do is to do the following:
echo "/some/path/I/want/to/add" > /Library/Python/2.5/site-packages/custom.pth
HTH
Forgive me if my answer seems ignorant, I'm not totally familiar with Mac and I also may have misunderstood your question.
On Windows and Linux, when I want to refer to a script I've written, I set the PYTHONPATH environment variable. It is what os.sys.path gets its values from, if I remember correctly.
Let's say myScript.py is in /Somewhere. Set PYTHONPATH to:
PYTHONPATH = /Somewhere
Now you should be able to "import myScript".
If you start doing sub-folders as python packages, look into usage of init.py files in each folder.
If you plan on re-using this and other scripts all the time, you should leave PYTHONPATH set as an environment variable.
Not sure if Xcode counts as launching the script through Finder or not, but if it does, apparently Finder doesn't read .profile or .cshrc files when it starts for a user the way Terminal does.
If your question is unanswered still, check out Apple's knowledge base: QA1067 and set up your environment using the plist.