i am a django newbie, and trying to get my projects and apps in a directory on my desktop.. I need to add these directories to the python path so it knows where they live and can interact with eachother.. how can I do this best? I found several ways which might have downsides???
through shell, import sys, sys.path.append('/my/dir') isn't permanent?
through sitecustomize.py, this is permanent but in my python IDE it seems modules get loaded multiple times? because my python in running in a framework (macosx)
through commandline export PYTHONPATH:$ etc etc
So whats the way to go ?
I tend to use the simple
import sys
path = '/var/www/Django/'
if path not in sys.path:
sys.path.append(path)
In my WSGI scripts which "power" my application. I also have a custom management command that does exactly this before calling shell. Previously to that, I simply dumped the above in
settings.py.
/var/www/Django is where all my projects belong - thus if I'm referring to a project I always use project.app. Once I've created a portable app for re-use, it graduates and lives in /var/www/Django itself and gets included via other project's settings.py. Does that make sense?
Related
I have a code project with various python scripts and modules. The folder structure of the github project is something like this:
/data_collection
/analysis
/modules
/helpers
Most of the scripts in data_collection and analysis will import stuff from modules or helpers. The code for doing this, in an example script /data_collection/pull_data.py, would be something like this:
import sys
sys.path.insert(0, '..')
from modules import my_module
from helpers import my_helper
now, if i simply run this code from the shell (from the dir that the script is in) - easy, it works just fine.
BUT: I want to run this from the crontab. It doesn't work, because crontab's PWD is always the cron user's home dir.
Now, I realise that I could add PWD=/path/to/project at the top of cron. But, what if I also have other project's scripts firing from cron?
I also realise that I could reorganise the whole folder structure of the project, perhaps putting all these folders into a folder called app and adding __init__.py to each folder -- but I'm not really in a position to do that at this moment.
So - I wonder, is there a possibility to achieve the following:
retain the relative paths in sys.path.insert within scripts (or perhaps get some solution that avoids the sys.path business altogether (so that it can run without modification on other systems)
be able to run these scripts from the crontab while also running scripts that live in other project directories from the crontab
Many thanks in advance!
I've created an experimental import library: ultraimport
It allows to do file system based imports with relative (or absolute) paths that will always work, no matter how you run the code or which user is running the code (given the user has read access to the files you're trying to import).
In your example script /data_collection/pull_data.py you would then write:
import ultraimport
my_module = ultraimport('__dir__/../modules/my_module.py')
my_helper = ultraimport('__dir__/../helpers/my_helper.py')
No need for any __init__.py and no need to change your directory structure and no need to change sys.path. Also your current working directory (which you can get running pwd) does not play any role for finding the import files.
I have a Django app and I'm getting an error whenever I try to run my code:
Error: No module named django_openid
Let me step back a bit and tell you how I came about this:
I formatted my computer and completely re-installed everything -- including virtualenv, and all dependent packages (in addition to Django) required for my project based on settings in my requirements.txt folder
I tried doing python manage.py syncdb and got the error
I googled the issue, and many people say it could be a path problem.
I'm confused as to how I go about changing the path variables though, and what exactly they mean. I found some documentation, but being somewhat of a hack-ish noob, it kind of goes over my head.
So my questions are:
What exactly is their purpose -- and are they on a system based level based on the version of Python or are they project dependent?
How can I see what mine are set to currently?
How can I change them (ie. where is this .profile file they talk of and can I just use a text editor)
Any input you would have would be great as this one is stumping me and I just want to get back to writing code :-)
The path is just the locations in your filesystem in which python will search for the modules you are trying to import. For example, when you run import somemodule, Python will perform a search for somemodule in all the locations contained in the path (sys.path variable).
You should check the path attribute in sys module:
import sys
print sys.path
It is just a regular list, sou you could append/remove elements from it:
sys.path.append('/path/to/some/module/folder/')
If you want to change your path for every python session you start, you should create a file to be loaded at startup, doing so:
Create a PYTHONSTARTUP environment variable and setting it to your startup file. E.g.: PYTHONSTARTUP=/home/user/.pythonrc (in a unix shell);
Edit the startup file so it contains the commands you want to be auto-executed when python is loaded;
An example of a .pythonrc could be:
import sys
sys.path.append('/path/to/some/folder/')
Do you really need to alter the path? It's always best to actually think about your reasons first. If you're only going to be running a single application on the server or you just don't care about polluting the system packages directory with potentially unnecessary packages, then put everything in the main system site-packages or dist-packages directory. Otherwise, use virtualenv.
The system-level package directory is always on the path. Virtualenv will add its site-packages directory to the path when activated, and Django will add the project directory to the path when activated. There shouldn't be a need to add anything else to the path, and really it's something you should never really have to worry about in practice.
I run into this problem pretty consistently... keep in mind I am quite new to Django and a total Python amateur.
It seems that, for example, whenever I check out my Django project on a new computer after a clean install of Python and Django, it can never find the project/apps I create or copy in.
So right now I have an app that is working, and I downloaded a 3rd party Django module and installed it into my app directory, include it in my settings, and the web server quits because it cannot find the module.
This is the first time I've imported an third party module. In the past when it couldn't find modules I created, I would just rename the folder and run "manage.py startapp appname", delete the folder it created, and name my original folder back, boom, problem solved...
But that's obviously a hack, I am wondering if anyone can explain the the heck is going on here and how best to approach it.
I can't be the only one who has run into this, but I couldn't find any other questions on this site that seemed to match my issue.
Happens on both OS X and Windows 7.
They way Django works is pretty much how Python works. At default the folder you create when you run django-admin.py startproject name is added to your python path. That means that anything you put into there you can get to. But you have to mind that when you write the app into the installed app list. If you have an app at project/apps/appname, you would have to write 'app.appname' in the installed apps list.
Now there are some ways to go about adding 3rd party apps located somewhere else to your project. You can either add them to your python path, put in your python path, or make a link to your python path. However, you can also add a sys.path.insert(...) in your manage.py file where you add the folder of your liking to your python path. Doing this will allow you to add folders to your python path for that project only, and will keep your python path more clean.
Your third party django module should be searchable by PYTHONPATH, because django module is no other than a python module. Now there are two ways to do this:
Create a folder (anywhere you want), put your third party django module under there. Now set that directory to environment variable $PYTHONPATH
e.g (on Linux box):
export PYTHONPATH = /home/me/pythonmodules/
Create a folder (anywhere you want), put the third party django module under there. Now if you are on Unix box, create a symlink to that directory to python site-packages. Use this command to find out where your python site packages is:
python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"
I have a python project with this directory structure and these files:
/home/project_root
|---__init__.py
|---setup
|---__init__.py
|---configs.py
|---test_code
|---__init__.py
|---tester.py
The tester script imports from setup/configs.py with the reference "setup.configs". It runs fine on my development machine.
This works on the development (Linux) computer. When I move this to another (Linux) computer, I set the PYTHONPATH with
PYTHONPATH = "/home/project_root"
But when I run tester.py, it can't find the configs module. And when I run the interactive Python interpreter, sys.path doesn't include the /home/project_root directory. But /home/project_root does appear when I echo $PYTHPATH.
What am I doing wrong here?
(I don't want to rely on the .bashrc file to set the PYTHONPATH for the target machine -- the code is for a Django application, and will eventually be run by www-data. And, I know that the apache configuration for Django includes a specification of the PYTHONPATH, but I don't want to use that here as I'm first trying to make sure the code passes its unit tests in the target machine environment.)
CURIOUSER AND CURIOUSER
This seems to be a userid and permissions problem.
- When launched by a command from an ordinary user, the interpreter can import modules as expected.
- When launched by sudo (I'm running Ubuntu here), the interpreter cannot import modules as expected.
- I've been calling the test script with sudo, as the files are owned by www-data (b/c they'll be called by the user running apache as part of the Django application).
- After changing the files' ownership to that of an ordinary user, the test script does run without import errors (albeit, into all sorts of userid related walls).
Sorry to waste your time. This question should be closed.
Stick this in the tester script right before the import setup.configs
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), os.path.pardir))
sys.path is a list of all the directories the python interpreter looks for when importing a python module.
This will add the parent directory which contains setup module to the beginning of that list which means that the local directory will be checked first. That is important if you have your module installed system wide. More info on that here: sys doc.
EDIT: You could also put a .pth file in /usr/local/lib/python2.X/site-packages/ A .pth file is simply a text file with a directory path on each line that the python interpreter will search in. So just add a file with this line in it:
/home/project_root
Try explicitly setting your python path in your scripts. If you don't want to have to change it, you could always add something like "../" to the path in tester. That is to say:
sys.path.append("../")
(I don't want to rely on the .bashrc file to set the PYTHONPATH for the target machine -- the code is for a Django application, and will eventually be run by www-data. And, I know that the apache configuration for Django includes a specification of the PYTHONPATH, but I don't want to use that here as I'm first trying to make sure the code passes its unit tests in the target machine environment.)
If the code is for a Django application, is there a reason you're not testing it in the context of a Django project? Testing it in the context of a Django project gives a couple benefits:
Django's manage.py will set up your Python environment for you. It'll add the appropriate project paths to sys.path, and it'll set the environment variable DJANGO_SETTINGS_MODULE correctly.
Django's libraries include ample unit testing facilities, and you can easily extend that functionality to include your own testing facilities. Executing tests in a Django project is as easy as executing a single command via manage.py.
I am an experienced PHP programmer using Django for the first time, and I think it is incredible!
I have a project that has a lot of apps, so I wanted to group them in an apps folder.
So the structure of the project is:
/project/
/project/apps/
/project/apps/app1/
/project/apps/app2
Then in Django settings I have put this:
INSTALLED_APPS = (
'project.apps.app1',
'project.apps.app2',
)
This does not seem to work?
Any ideas on how you can put all your apps into a seprate folder and not in the project root?
Many thanks.
Make sure that the '__init__.py' file is in your apps directory, if it's not there it won't be recognized as part of the package.
So each of the folders here should have '__init__.py' file in it. (empty is fine).
/project/
/project/apps/
/project/apps/app1/
/project/apps/app2
Then as long as your root 'module' folder is in your PYTHONPATH you'll be able to import from your apps.
Here's the documentation regarding the python search path for your reading pleasure:
http://docs.python.org/install/index.html#modifying-python-s-search-path
And a nice simple explanation of what __init__.py file is for:
http://effbot.org/pyfaq/what-is-init-py-used-for.htm
As long as your apps are in your PYTHONPATH, everything should work. Try setting that environment variable to the folder containing your apps.
PYTHONPATH="/path/to/your/apps/dir/:$PYTHONPATH"
Your top-level urls.py (also named in your settings.py) must be able to use a simple "import" statement to get your applications.
Does import project.apps.app1.urls work? If not, then your PYTHONPATH isn't set up properly, or you didn't install your project in Python's site-packages directory.
I suggest using the PYTHONPATH environment variable, instead of installing into site-packages. Django applications (to me, anyway) seem easier to manage when outside site-packages.
We do the following:
Django projects go in /opt/project/.
PYTHONPATH includes /opt/project.
Our settings.py uses apps.this and apps.that (note that the project part of the name is part of the PYTHONPATH, not part of the import.