I'm not sure what's going on, but on my own laptop, everything works okay. When I upload to my host with Python 2.3.5, my views.py can't find anything in my models.py. I have:
from dtms.models import User
from dtms.item_list import *
where my models, item_list, and views files are in /mysite/dtms/
It ends up telling me it can't find User. Any ideas?
Also, when I use the django shell, I can do "from dtms.models import *" and it works just fine.
Okay, after doing the suggestion below, I get a log file of:
syspath = ['/home/victor/django/django_projects', '/home/victor/django/django_projects/mysite']
DEBUG:root:something <module 'dtms' from '/home/victor/django/django_projects/mysite/dtms/__init__.pyc'>
DEBUG:root:/home/victor/django/django_projects/mysite/dtms/__init__.pyc
DEBUG:root:['/home/victor/django/django_projects/mysite/dtms']
I'm not entirely sure what this means - my file is in mysite/dtms/item_list.py. Does this mean it's being loaded? I see the dtms module is being loaded, but it still can't find dtms.models
The fact that from X import * works does not guarantee that from X import Wowie will work too, you know (if you could wean yourself away from that import * addiction you'd be WAY happier on the long run, but, that's another issue;-).
My general advice in import problems is to bracket the problematic import with try/except:
try:
from blah import bluh
except ImportError, e:
import sys
print 'Import error:', e
print 'sys.path:', sys.path
blah = __import__('blah')
print 'blah is %r' % blah
try:
print 'blah is at %s (%s)' % (blah.__file__, blah.__path__)
except Exception, e:
print 'Cannot give details on blah (%s)' % e
and the like. That generally shows you pretty quickly that your sys.path isn't what you thought it would be, and/or blah is at some weird place or with weird path, and the like.
To check your sys.path you can do what Alex said, but instead of using print you can use the logging module:
import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME,level=logging.DEBUG,)
logging.debug('This message should go to the log file')
Make sure your project (or the folder above your "dtms" app) is in python's module search path.
This is something you may need to set in your web server's configuration. The reason it works in the django shell is probably because you are in your project's folder when you run the shell.
This is explained here if you're using apache with mod_python.
I could be way off with this, but did you set the DJANGO_SETTINGS_MODULE environment variable yet? It affects what you can import. Set it to ".settings". It's also something that gets set when you fire up manage.py, so things work there that won't work in other situations with setting the variable beforehand.
Here's what I do on my system:
export DJANGO_SETTINGS_MODULE=<project name>.settings
or
import os
os.environ['DJANGO_SETTINGS_MODULE']='<project name>.settings'
Sorry if this misses the point, but when I hear of problems importing models.py, I immediately thing of environment variables. Also, the project directory has to be on PYTHONPATH, but you probably already know that.
Related
I'm currently dealing with a strange bug when running a Python script from SSIS/a SQL Server Agent Job after promoting to a production server. This issue didn't arise in dev/staging environments, only when promoting to production. When importing several other files from the same folder the main script is located in, no error is raised, but on the final import I'm getting an error of the form 'FileNotFoundError(2, 'No such file or directory')' ; the relevant portions of the script looks something like this:
(The issues arise before loading the config file for the application which parameterizes the logging location, hence the ugly fallback logging here)
#!Python3.8
import sys
import os
sys.path.append(site-packages path)
from xyz import obj1, obj2
from abc import obj3
etc.
# Additional try/except block to figure out specific error for this import
try:
import script_in_question
except Exception as e:
with open(fallback log path, 'w') as f:
f.write(f"{sys.argv}")
f.write(f"Could not load script_in_question: {repr(e)} \n {os.getcwd()}")
f.write(f"\n {repr(e)}")
Weirdly, when logging sys.path, I can see the folder containing the successfully imported scripts and the script in question; os.getcwd() reports the correct working directory, and this error cropped up whether I was directly importing the script or using noation like 'from script_in_question import main_function'
I've tried switching between from script_in_question import main_function and import script_in_question, explicitly changing to the correct working directory.
I know this has been asked dozens of times but I can't see what in the world I'm doing wrong. I'm trying to import a module in python 2.7 from a different directory. I would greatly appreciate some input to help me understand why this method doesn't work. I have the following directory structure on my raspbian system:
/home/pi/
...projects/__init__.py
...projects/humid_temp.py
...python_utilities/__init.py__
...python_utilities/tools.py
I'm calling humid_temp.py and I need to import a function within tools.py This is what their contents look like:
humid_temp.py:
import os
import sys
sys.path.append('home/pi/python_utilities')
print sys.path
from python_utilities.tools import *
tools.py:
def tail(file):
#function contents
return stuff
The print sys.path output contains /home/pi/python_utilities
I'm not messing up my __init__.py's am I?
I've also ruled out possible permission issues with that path as I gave it full 777 access and I still hit the
ImportError: No module named python_utilities.tools.
What did I miss?
When you want to import something like -
from python_utilities.tools import *
You need to add the parent directory of python_utilities to sys.path , not python_utilities itself. So, you should add something like -
sys.path.append('/home/pi') #Assuming the missing of `/` at start was not a copy/paste mistake
Also, just a note, from <module> import * is bad , you should consider only importing the required items, you can check the question - Why is "import *" bad? - for more details.
In humid_temp.py, just write:
from python_utilities import tools
There is no need for appending subfolder to sys.path.
Then when you want to use functions from tools, just
tools.function()
I am working on project euler and wanted to time all of my code. What I have is directory of files in the form 'problemxxx.py' where xxx is the problem number. Each of these files has a main() function that returns the answer. So I have created a file called run.py, located in the same directory as the problem files. I am able to get the name of the file through command prompt. But when I try to import the problem file, I continue to get ImportError: No module named problem. Below is the code for run.py so far, along with the command prompt used.
# run.py
import sys
problem = sys.argv[1]
import problem # I have also tired 'from problem import main' w/ same result
# will add timeit functions later, but trying to get this to run first
problem.main()
The command prompts that I have tried are the following: (both of which give the ImportError stated above)
python run.py problem001
python run.py problem001.py
How can I import the function main() from the file problem001.py? Does importing not work with the file name stored as a variable? Is there a better solution than trying to get the file name through command prompt? Let me know if I need to add more information, and thank you for any help!
You can do this by using the __import__() function.
# run.py
import sys
problem = __import__(sys.argv[1], fromlist=["main"]) # I have also tired 'from problem import main' w/ same result
problem.main()
Then if you have problem001.py like this:
def main():
print "In sub_main"
Calling python run.py problem001 prints:
In sub_main
A cleaner way to do this (instead of the __import__ way) is to use the importlib module. Your run.py needs to changes:
import importlib
problem = importlib.import_module(sys.argv[1])
Alternatives are mentioned in this question.
For sure! You can use __ import_ built-in function like __import__(problem). However this is not recommended to use, because it is not nice in terms of coding-style. I think if you are using this for testing purposes then you should use unittest module, either way try to avoid these constructions.
Regards
You can use exec() trick:
import sys
problem = sys.argv[1]
exec('import %s' % problem)
exec('%s.main()' % problem)
I have to code something in Jython, for CCPS (programm using jython as scripting interface). However Jython does not update the submodules if I change them in an editor, unless I restart the programm (startup time is prohibitive). SO testing and adjusting is relatively slow.
I have googled and found a post indicating, that one should import or reload the submodules. The basic outline thus is:
def loader(module, part=None):
if not module in sys.modules :
if part == None:
exec("import "+module)
else:
exec("from %s import %s" % (module, part))
else :
exec("reload "+module)
however I have an issue with this, the module is loaded locally, meaning i can access the module within the loader() function, but not in my main code.
Two questions:
What is the right way to test something with submodules in Jython without restarting Jython after each submodule change?
Is there a way to generate globals dynamically so I can import into the global namespace?
(e.g. exec("global %(mod)s = %(mod)s" % ({'mod':module}))
How about just unloading all modules so they are reloaded on the next import:
import sys
sys.modules.clear()
Summary: when a certain python module is imported, I want to be able to intercept this action, and instead of loading the required class, I want to load another class of my choice.
Reason: I am working on some legacy code. I need to write some unit test code before I start some enhancement/refactoring. The code imports a certain module which will fail in a unit test setting, however. (Because of database server dependency)
Pseduo Code:
from LegacyDataLoader import load_me_data
...
def do_something():
data = load_me_data()
So, ideally, when python excutes the import line above in a unit test, an alternative class, says MockDataLoader, is loaded instead.
I am still using 2.4.3. I suppose there is an import hook I can manipulate
Edit
Thanks a lot for the answers so far. They are all very helpful.
One particular type of suggestion is about manipulation of PYTHONPATH. It does not work in my case. So I will elaborate my particular situation here.
The original codebase is organised in this way
./dir1/myapp/database/LegacyDataLoader.py
./dir1/myapp/database/Other.py
./dir1/myapp/database/__init__.py
./dir1/myapp/__init__.py
My goal is to enhance the Other class in the Other module. But since it is legacy code, I do not feel comfortable working on it without strapping a test suite around it first.
Now I introduce this unit test code
./unit_test/test.py
The content is simply:
from myapp.database.Other import Other
def test1():
o = Other()
o.do_something()
if __name__ == "__main__":
test1()
When the CI server runs the above test, the test fails. It is because class Other uses LegacyDataLoader, and LegacydataLoader cannot establish database connection to the db server from the CI box.
Now let's add a fake class as suggested:
./unit_test_fake/myapp/database/LegacyDataLoader.py
./unit_test_fake/myapp/database/__init__.py
./unit_test_fake/myapp/__init__.py
Modify the PYTHONPATH to
export PYTHONPATH=unit_test_fake:dir1:unit_test
Now the test fails for another reason
File "unit_test/test.py", line 1, in <module>
from myapp.database.Other import Other
ImportError: No module named Other
It has something to do with the way python resolves classes/attributes in a module
You can intercept import and from ... import statements by defining your own __import__ function and assigning it to __builtin__.__import__ (make sure to save the previous value, since your override will no doubt want to delegate to it; and you'll need to import __builtin__ to get the builtin-objects module).
For example (Py2.4 specific, since that's what you're asking about), save in aim.py the following:
import __builtin__
realimp = __builtin__.__import__
def my_import(name, globals={}, locals={}, fromlist=[]):
print 'importing', name, fromlist
return realimp(name, globals, locals, fromlist)
__builtin__.__import__ = my_import
from os import path
and now:
$ python2.4 aim.py
importing os ('path',)
So this lets you intercept any specific import request you want, and alter the imported module[s] as you wish before you return them -- see the specs here. This is the kind of "hook" you're looking for, right?
There are cleaner ways to do this, but I'll assume that you can't modify the file containing from LegacyDataLoader import load_me_data.
The simplest thing to do is probably to create a new directory called testing_shims, and create LegacyDataLoader.py file in it. In that file, define whatever fake load_me_data you like. When running the unit tests, put testing_shims into your PYTHONPATH environment variable as the first directory. Alternately, you can modify your test runner to insert testing_shims as the first value in sys.path.
This way, your file will be found when importing LegacyDataLoader, and your code will be loaded instead of the real code.
The import statement just grabs stuff from sys.modules if a matching name is found there, so the simplest thing is to make sure you insert your own module into sys.modules under the target name before anything else tries to import the real thing.
# in test code
import sys
import MockDataLoader
sys.modules['LegacyDataLoader'] = MockDataLoader
import module_under_test
There are a handful of variations on the theme, but that basic approach should work fine to do what you describe in the question. A slightly simpler approach would be this, using just a mock function to replace the one in question:
# in test code
import module_under_test
def mock_load_me_data():
# do mock stuff here
module_under_test.load_me_data = mock_load_me_data
That simply replaces the appropriate name right in the module itself, so when you invoke the code under test, presumably do_something() in your question, it calls your mock routine.
Well, if the import fails by raising an exception, you could put it in a try...except loop:
try:
from LegacyDataLoader import load_me_data
except: # put error that occurs here, so as not to mask actual problems
from MockDataLoader import load_me_data
Is that what you're looking for? If it fails, but doesn't raise an exception, you could have it run the unit test with a special command line tag, like --unittest, like this:
import sys
if "--unittest" in sys.argv:
from MockDataLoader import load_me_data
else:
from LegacyDataLoader import load_me_data