I have a python project with this directory structure and these files:
/home/project_root
|---__init__.py
|---setup
|---__init__.py
|---configs.py
|---test_code
|---__init__.py
|---tester.py
The tester script imports from setup/configs.py with the reference "setup.configs". It runs fine on my development machine.
This works on the development (Linux) computer. When I move this to another (Linux) computer, I set the PYTHONPATH with
PYTHONPATH = "/home/project_root"
But when I run tester.py, it can't find the configs module. And when I run the interactive Python interpreter, sys.path doesn't include the /home/project_root directory. But /home/project_root does appear when I echo $PYTHPATH.
What am I doing wrong here?
(I don't want to rely on the .bashrc file to set the PYTHONPATH for the target machine -- the code is for a Django application, and will eventually be run by www-data. And, I know that the apache configuration for Django includes a specification of the PYTHONPATH, but I don't want to use that here as I'm first trying to make sure the code passes its unit tests in the target machine environment.)
CURIOUSER AND CURIOUSER
This seems to be a userid and permissions problem.
- When launched by a command from an ordinary user, the interpreter can import modules as expected.
- When launched by sudo (I'm running Ubuntu here), the interpreter cannot import modules as expected.
- I've been calling the test script with sudo, as the files are owned by www-data (b/c they'll be called by the user running apache as part of the Django application).
- After changing the files' ownership to that of an ordinary user, the test script does run without import errors (albeit, into all sorts of userid related walls).
Sorry to waste your time. This question should be closed.
Stick this in the tester script right before the import setup.configs
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), os.path.pardir))
sys.path is a list of all the directories the python interpreter looks for when importing a python module.
This will add the parent directory which contains setup module to the beginning of that list which means that the local directory will be checked first. That is important if you have your module installed system wide. More info on that here: sys doc.
EDIT: You could also put a .pth file in /usr/local/lib/python2.X/site-packages/ A .pth file is simply a text file with a directory path on each line that the python interpreter will search in. So just add a file with this line in it:
/home/project_root
Try explicitly setting your python path in your scripts. If you don't want to have to change it, you could always add something like "../" to the path in tester. That is to say:
sys.path.append("../")
(I don't want to rely on the .bashrc file to set the PYTHONPATH for the target machine -- the code is for a Django application, and will eventually be run by www-data. And, I know that the apache configuration for Django includes a specification of the PYTHONPATH, but I don't want to use that here as I'm first trying to make sure the code passes its unit tests in the target machine environment.)
If the code is for a Django application, is there a reason you're not testing it in the context of a Django project? Testing it in the context of a Django project gives a couple benefits:
Django's manage.py will set up your Python environment for you. It'll add the appropriate project paths to sys.path, and it'll set the environment variable DJANGO_SETTINGS_MODULE correctly.
Django's libraries include ample unit testing facilities, and you can easily extend that functionality to include your own testing facilities. Executing tests in a Django project is as easy as executing a single command via manage.py.
Related
I have a python3 script that I am calling in terminal; I do not use Python prefix to run it, since I did add #!/usr/local/bin/python3 in my script (I have python3 from brew, on OSX).
The interesting thing is that if I run the script in terminal, I get an import error because one of my custom module hasn't been found. If I run the same script in pycharm, it works fine.
I assume Python launch and read all the various path that I use for modules in the same way, in both pycharm and terminal, but it seems not the case. How do I set up my scripts so the modules are found, independently from their path?
I may run the same script from other machines too, so I want to be prepared and do the right thing from the start.
EDIT
I am running pycharm on OSX; Python3 is installed via Brew, but the symlink is in /usr/local/bin.
My script is running from a folder inside my home directory, so
/Users/tester/git/python_test_app/main/base/app_main.py
The custom modules are in the same folder of the main py script, but one level above: /Users/tester/git/python_test_app/main/pyutils.py
The import statement from app_main.py is
import main.pyutils as utilities
This is the stack trace that I get when running the script:
Traceback (most recent call last):
File "main/base/app_main.py", line 13, in <module>
import main.pyutils as utilities
ModuleNotFoundError: No module named 'main'
EDIT 2 and solution
Thanks to The answers, I was able to figure out that the issue is related to how Pycharm handle projects. Basically it somehow save the path where the project is; so calling an import will result in the project folder being parsed, and that's why it works fine from Pycharm.
In Python, unless PYTHONPATH has the path to my project or other modules that I wrote, it won't be able to find them, hence, raise the error.
FIX:
in my main module that I use to run the application, I did retrieve the path of the file; which I know being one level below the modules I need; so I can explicitly add the folder to the current sys.path. This will end up making possible for me to call the import successfully.
import sys
current_dir = os.path.dirname(__file__)
sys.path.insert(0, , current_dir)
The only downside is that every file and resource that I use in my project, has to be directly referred by full path; so I have to pass the current_dir around the various files in the project.
PyCharm has project interpreter settings. Verify these are the same as your system Python. Go to:
File menu
Settings
Project: <project name>
Project Interpreter
View the path to the Python executable/binary being used by the project in PyCharm and verify it matches what your system is calling (e.g., which python3)
Alternatively, it may be that you declared your sources root within PyCharm and the system cannot properly run the module as it exists in the path you're running it from (especially if inside a package). You can get around this using the -m parameter and calling it from Python.
You can also try running it from the Terminal inside PyCharm and see what it adds to the path before initializing the shell session (you can sometimes see this in your Run configurations also). If you are referring to modules not installed via pip / into the Python path but rather loaded into your project path, then this may be the culprit.
On PyCharm, next to the green "RUN" arrow press the box and then press edit configurations (see image)
There you'll have Working Directory - that path is where PyCharm is running that script from (without errors).
Try running it from the terminal within that path - that should solve your import errors.
I have a Django app and I'm getting an error whenever I try to run my code:
Error: No module named django_openid
Let me step back a bit and tell you how I came about this:
I formatted my computer and completely re-installed everything -- including virtualenv, and all dependent packages (in addition to Django) required for my project based on settings in my requirements.txt folder
I tried doing python manage.py syncdb and got the error
I googled the issue, and many people say it could be a path problem.
I'm confused as to how I go about changing the path variables though, and what exactly they mean. I found some documentation, but being somewhat of a hack-ish noob, it kind of goes over my head.
So my questions are:
What exactly is their purpose -- and are they on a system based level based on the version of Python or are they project dependent?
How can I see what mine are set to currently?
How can I change them (ie. where is this .profile file they talk of and can I just use a text editor)
Any input you would have would be great as this one is stumping me and I just want to get back to writing code :-)
The path is just the locations in your filesystem in which python will search for the modules you are trying to import. For example, when you run import somemodule, Python will perform a search for somemodule in all the locations contained in the path (sys.path variable).
You should check the path attribute in sys module:
import sys
print sys.path
It is just a regular list, sou you could append/remove elements from it:
sys.path.append('/path/to/some/module/folder/')
If you want to change your path for every python session you start, you should create a file to be loaded at startup, doing so:
Create a PYTHONSTARTUP environment variable and setting it to your startup file. E.g.: PYTHONSTARTUP=/home/user/.pythonrc (in a unix shell);
Edit the startup file so it contains the commands you want to be auto-executed when python is loaded;
An example of a .pythonrc could be:
import sys
sys.path.append('/path/to/some/folder/')
Do you really need to alter the path? It's always best to actually think about your reasons first. If you're only going to be running a single application on the server or you just don't care about polluting the system packages directory with potentially unnecessary packages, then put everything in the main system site-packages or dist-packages directory. Otherwise, use virtualenv.
The system-level package directory is always on the path. Virtualenv will add its site-packages directory to the path when activated, and Django will add the project directory to the path when activated. There shouldn't be a need to add anything else to the path, and really it's something you should never really have to worry about in practice.
Disclaimer: I am new to python and django but have Drupal programming experience. I'm using Windows 7 (same issues on Windows XP)
On python 2.7 and Django-1.3.1, I successfully created a default project
django-admin.py startproject djsite
Now, I need to "bootstrap" djsite.manage as explained here (http://www.pyinstaller.org/wiki/Recipe/DjangoApplication) in a file called bootstrap.py located in djsite's parent directory as follows:
import djsite.manage
djsite.manage.execute_manager(djsite.manage.settings,['manage.py', 'runserver'])
Yet, as soon as the compiler sees:
import djsite.manage
I get this:
"Error: Can't find the file 'settings.py' in the directory containing 'C:\Python27\Lib\site-packages\djsite\manage.pyc'. It appears you've customized things... You'll have to run django-admin.py, passing it your settings module." And, I don't know how to follow the error's advice in this situation.
However, if I instead issue the following in bootstrap.py:
import os, sys
sys.path.append(os.path.abspath('djsite'))
import djsite.manage
djsite.manage.execute_manager(djsite.manage.settings,['manage.py', 'runserver'])
the script works correctly, but it breaks Pyinstaller (I've already asked this question on that software's mailing list (http://groups.google.com/group/pyinstaller/browse_thread/thread/174a72e26c26a44c). Even if I add the path to the djsite in my PATH variable, I get the same error.
So my question here is this: Why does importing the manage.py module fail with this approach and how can I proceed? Thanks!
Try adding this to your bootstrap.py to inform it where your settings file lives:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'djsite.settings'
This will assume that djsite is in your pythonpath. That is, if its located here: /path/to/my/djsite, then this should be in your pythonpath: /path/to/my
Actually the best way to be doing this from the start is to being using virtualenv which will ensure that your environment is correct. I feel like that had to have been part of your tutorial if I remember bootstrap at all. If you are using virtualenv, make sure you remembered to source bin/activate
If that doesn't work, you can try altering the runserver command:
args = ['manage.py', 'runserver', '--settings=/path/to/my/djsite/settings.py']
djsite.manage.execute_manager(djsite.manage.settings, args)
i am a django newbie, and trying to get my projects and apps in a directory on my desktop.. I need to add these directories to the python path so it knows where they live and can interact with eachother.. how can I do this best? I found several ways which might have downsides???
through shell, import sys, sys.path.append('/my/dir') isn't permanent?
through sitecustomize.py, this is permanent but in my python IDE it seems modules get loaded multiple times? because my python in running in a framework (macosx)
through commandline export PYTHONPATH:$ etc etc
So whats the way to go ?
I tend to use the simple
import sys
path = '/var/www/Django/'
if path not in sys.path:
sys.path.append(path)
In my WSGI scripts which "power" my application. I also have a custom management command that does exactly this before calling shell. Previously to that, I simply dumped the above in
settings.py.
/var/www/Django is where all my projects belong - thus if I'm referring to a project I always use project.app. Once I've created a portable app for re-use, it graduates and lives in /var/www/Django itself and gets included via other project's settings.py. Does that make sense?
I have written a small DB access module that is extensively reused in many programs.
My code is stored in a single directory tree /projects for backup and versioning reasons, and so the module should be placed within this directory tree, say at /projects/my_py_lib/dbconn.py.
I want to easily configure Python to automatically search for modules at the /projects/my_py_lib directory structure (of course, __init__.py should be placed within any subdirectory).
What's the best way to do this under Ubuntu?
Thanks,
Adam
You can add a PYTHONPATH environment variable to your .bashrc file. eg.
export PYTHONPATH=/projects/my_py_lib
on linux, this directory will be added to your sys.path automatically for pythonN.M
~/.local/lib/pythonN.M/site-packages/
So you can put your packages in there for each version of python you are using.
You need a copy for each version of python, otherwise the .pyc file will be recompiled every time you import the module with a different python version
This also allows fine grained control if the module only works for some of the versions of python you have installed
If you create this file
~/.local/lib/pythonN.M/site-packages/usercustomize.py
it will be imported each time you start the python interpreter
Another option is to create a soft link in /usr/lib*/python*/site-packages/:
ln -s /projects/my_py_lib /usr/lib*/python*/site-packages/
That will make the project visible to all Python programs plus any changes will be visible immediately, too.
The main drawback is that you will eventually have *.pyc files owned by root or another user unless you make sure you compile the files yourself before you start python as another user.