i have a following solution structure in python:
main_app
main_app/template_processor/
main_app/template_processor/models
main_app/template_processor/views
everything works just fine on my local machine. as soon as code gets to server (it stopped working after i removed all .pyc files from svn), it doesn't see the assembly (if it could be called like that in terms of python) with models. during syncdb command, it creates no tables except for admin ones. but in runtime it finds models themselves, but does not find tables (since they were not created by syncdb)
i've added application to settings.py as installed app, everything seems to be fine.
the only difference that i can see for now is that on my local machine i have main_app/template_processor/models/models.pyc file, but it doesn't precompile it on server for some reason (might be a hint??)
__init__.py files are existing in every folder/subfolder.
have anyone faced an issue like that?
Hm, looking at your directory structure, I think that you need to import all your models in template_processor/models/__init__.py, because Django looks only in <app_name>.models module when loading models automatically (i.e. for syncdb).
My guess would be that you renamed a file, and didn't delete the oldname.pyc file.
So if you try to
import oldname
then you rename oldname to rename, but don't update your import statement, the code will work on systems where oldname.pyc exists, however python won't be able to recreate oldname.pyc if oldname.py doesn't exist.
try
find . | grep py | xargs grep import | grep -v django | sort -u
that should give you a list of all imports in your project, see if one of those imports is pointing at a module for which you have a pyc file but not a .py file.
In general, python quickly compiles .py files into .pyc files, and it does this once. I wouldn't worry about the time it takes to generate new .pyc files.
Sounds like Django isn't seeing that module (folder, in this case) for some reason. Make sure all the folders have a file called __init__.py (notice the two underscores before and after). Once that's done, make sure it's listed in your installed apps.
Maybe you made some change to it that's causing it to stop loading. You can also try moving the .pyc files out of the directory on your local machine and see whether they're regenerated or not when you runserver.
Maybe the most helpful thing: ./manage.py shell to bring up an interactive shell and then 'import app_name' (without quotes) to see whether django is having trouble finding the module.
Ok, if anyone's interested in what really was happening, i've got a story to tell ya:
http://code.djangoproject.com/ticket/4470
that's basically what i was going to implement.
in order to really get this thing work i still should have a file models.py, which will have a proper list of classess, with Pass inside of it. Then i should have taken all the files (my models) and changed their meta for syncdb to understand they are from the certain "assembly".
source code is available (pls see url above).
thx for helping out!
Related
Core Problem:
I've run into situations multiple times where pyc files break my local application (especially unit tests).
For example I have a folder utils/ which contains a poorly named sqlalchemy.py (which just contains utilities for sql alchemy). I add another file utilities/whatever.py which contains
import sqlalchemy.orm.session
and that breaks because well...the python2 importer at least thinks "I'm going to look at the relative path first". So I rename or delete utilities/sqlalchemy.py and everything works hunky dory.
Until my teammates pull down changes and everything breaks. Because while their copy no longer has utilities/sqlalchemy.py, it still has the git ignored utilities/sqlalchemy.pyc file
What I Want:
Is there a way to get the python importer to ignore pyc files? I would of course only want this to be active locally when running unit test, so I'm hoping for like an environment variable or a configuration for the python importer, but hoping other people have run into this problem enough that there's some sort of "quasi-official" solution.
Note that I specifically don't want to do something like find . -name '*.pyc' -delete in a post-checkout git hook since our repo is large enough that this takes several minutes and would really slow us down every time someone switches branches.
I wrote a small python program and again I am struggling with producing a good structure.
A few days ago I read a blog and many other websites, which advise against from a import b imports and recommend to use always import a.b "project absolute" path (the whole path from the project root to what I want to import) imports. That as a background.
I included a __main__.py so that I can run my program with python -m <directory>, which was another recommendation of someone on stackoverflow in one of the hundreds of python import questions. It is supposed to help keeping code runnable and testable with the same import structure, which was a problem in another project of mine.
Now what I want it, that from anywhere in my system, I can run python -m <dir of my code> and not only from one directory up the RSTCitations directory.
How can I achieve that, without:
python path manipulations (which are a dirty hack)
changing my imports somehow and getting a not recommended import structure
doing other dark magic to my code
I want to stick to best practices in organizing my code but still want it to be runnable from wherever I am in the terminal.
Example of fail
When I run the program as described from another directory completely unrelated to my program, I get for example the following error:
/home/user/development/anaconda3/bin/python: No module named /home/user/development/rst-citations-to-raw-latex/RSTCitations
However the path is correct. That is exactly where the code is.
You can :
install your program with pip ( see Installing Python packages from local file system folder with pip )
put your module in the python import path (and/or edit PYTHONPATH in the environment for the users that need to use it)
If you don't need other users to 'import' your library but just use it as a standalone program, you can also just put a symlink/script to your program, makeing it runnable from a directory which is in your PATH.
This is a seemingly simple problem but it proves to be harder than expected.
I've created a project in pycharm in the following layout
bin
main
helpers
userhelper
models
user
session
tests
userTest
in my main I run the code that calls everything and this works like a charm in pycharm. Now I want to run this on a server and start it with cron. How do I start this from cron while keeping all the module references in place?
I guess I need to add the root of my project to the python path. To do this I added the following bash script to invoke my project with:
PYTHONPATH="${PYTHONPATH}:/home/steven/projectX"
export PYTHONPATH
python bin/main.py
But this does not seem to do anything, what would be the best way to periodically run the bin/main.py within this project and have all my modules and things like 'ConfigParser.RawConfigParser().read(os.path.abspath("../configuration.cfg"))' in place relative to my project?
EDIT: I am not trying to fix my imports or debugging my code, I have a large project in pycharm that runs a simulation that I want to invoke on the server an maintain within my development setup. The question is how do I run this in the same way pycharm does?
It sounds like you're interested in making a distributable Python package. You should read through the tutorial here. Ultimately, you're going to want to write a setup.py (sure you could call it something else, but it's like renaming self -- why do it?) that will configure your project. Now a word of advice since I've seen many people go down a wrong path here. You NEVER want to modify your PYTHONPATH directly. It might be the quickest solution to get something up and working, but it WILL cause lasting problems.
So I've found the way of dealing with mu issue here. The best option would obviously be to create a distributable python package and use this. However since I am developing a simulation that has loads of features and desired outcomes there is a lot of tuning going on and having to create a package every time I want to run the project is a bit out of the scope for my project.
What I do want to do is being able to run the files in a similar way to how I am doing this on my development machine with PyCharm. The way I resolved this is to have my main.py in the root of my project and to have all references relative to the executed file. This means my userhelper looks for the datastore as follows:
path = os.path.join(os.path.dirname(os.path.dirname(__file__)),
"resources/" + settings['datastore_filename']
This resolves my issue of not being able to run my project on the server the same way it runs on my PC.
I have a Django app and I'm getting an error whenever I try to run my code:
Error: No module named django_openid
Let me step back a bit and tell you how I came about this:
I formatted my computer and completely re-installed everything -- including virtualenv, and all dependent packages (in addition to Django) required for my project based on settings in my requirements.txt folder
I tried doing python manage.py syncdb and got the error
I googled the issue, and many people say it could be a path problem.
I'm confused as to how I go about changing the path variables though, and what exactly they mean. I found some documentation, but being somewhat of a hack-ish noob, it kind of goes over my head.
So my questions are:
What exactly is their purpose -- and are they on a system based level based on the version of Python or are they project dependent?
How can I see what mine are set to currently?
How can I change them (ie. where is this .profile file they talk of and can I just use a text editor)
Any input you would have would be great as this one is stumping me and I just want to get back to writing code :-)
The path is just the locations in your filesystem in which python will search for the modules you are trying to import. For example, when you run import somemodule, Python will perform a search for somemodule in all the locations contained in the path (sys.path variable).
You should check the path attribute in sys module:
import sys
print sys.path
It is just a regular list, sou you could append/remove elements from it:
sys.path.append('/path/to/some/module/folder/')
If you want to change your path for every python session you start, you should create a file to be loaded at startup, doing so:
Create a PYTHONSTARTUP environment variable and setting it to your startup file. E.g.: PYTHONSTARTUP=/home/user/.pythonrc (in a unix shell);
Edit the startup file so it contains the commands you want to be auto-executed when python is loaded;
An example of a .pythonrc could be:
import sys
sys.path.append('/path/to/some/folder/')
Do you really need to alter the path? It's always best to actually think about your reasons first. If you're only going to be running a single application on the server or you just don't care about polluting the system packages directory with potentially unnecessary packages, then put everything in the main system site-packages or dist-packages directory. Otherwise, use virtualenv.
The system-level package directory is always on the path. Virtualenv will add its site-packages directory to the path when activated, and Django will add the project directory to the path when activated. There shouldn't be a need to add anything else to the path, and really it's something you should never really have to worry about in practice.
I added a folder to my PYTHONPATH where I can put all of my Django Apps. I print sys.path, and everything looks good, the folder I want is there. However, when I go to import a module, it tells me that there's no module by that name. All the site-packages modules work fine. In all of my Django apps, there's an "_____init_____.py" like there's supposed to be. I heard that if those are created on windows there can be problems, but I couldn't dig up much more than that.
Fixed it. Sorry Santa for not including the console print out. Windows was dumb and added an extra .py to the file when I created it. So everything was actually "pythonfile.py.py". Awesome.