I am having some trouble importing a class from a particular module. The class is in the module my_module1.my_module2.my_module3.my_module4.my_module5.my_module6.my_module7
This code works
import my_module1.my_module2.my_module3.my_module4.my_module5.my_module6.my_module7
which means to access my class I have to do
my_module1.my_module2.my_module3.my_module4.my_module5.my_module6.my_module7.MyClass
But this does not
from my_module1.my_module2.my_module3.my_module4.my_module5.my_module6.my_module7 import MyClass
Neither does this
import my_module1.my_module2.my_module3.my_module4.my_module5.my_module6.my_module7 as my_name
Both Give this error saying
AttributeError: 'module' object has no attribute my_module7'
This has me completely stumped and I have been working on it for a couple of weeks now. Any suggestions?
EDIT - I cant change the structure unfortunately as it is imposed by the system I am using
Looks like a circular import.
Gordon McMillan says:
Circular imports are fine where both modules use the “import ” form of import. They fail when the 2nd module wants to grab a name out of the first (“from module import name”) and the import is at the top level. That’s because names in the 1st are not yet available, because the first module is busy importing the 2nd.
I think you may want to consider an alternate design in the first place (redesigning your module breakdown so it's a flatter setup), but as that isn't your question try:
import my_module1.my_module2.my_modu...<etc>
my_name = my_module1.my_module2.my_modu...<etc>
my_name.MyClass
A module is a first class object in python, so you can alias them with variables.
Related
I'm working with a project that contains about 30 unique modules. It wasn't designed too well, so it's common that I create circular imports when adding some new functionality to the project.
Of course, when I add the circular import, I'm unaware of it. Sometimes it's pretty obvious I've made a circular import when I get an error like AttributeError: 'module' object has no attribute 'attribute' where I clearly defined 'attribute'. But other times, the code doesn't throw exceptions because of the way it's used.
So, to my question:
Is it possible to programmatically detect when and where a circular import is occuring?
The only solution I can think of so far is to have a module importTracking that contains a dict importingModules, a function importInProgress(file), which increments importingModules[file], and throws an error if it's greater than 1, and a function importComplete(file) which decrements importingModules[file]. All other modules would look like:
import importTracking
importTracking.importInProgress(__file__)
#module code goes here.
importTracking.importComplete(__file__)
But that looks really nasty, there's got to be a better way to do it, right?
To avoid having to alter every module, you could stick your import-tracking functionality in a import hook, or in a customized __import__ you could stick in the built-ins -- the latter, for once, might work better, because __import__ gets called even if the module getting imported is already in sys.modules, which is the case during circular imports.
For the implementation I'd simply use a set of the modules "in the process of being imported", something like (benjaoming edit: Inserting a working snippet derived from original):
beingimported = set()
originalimport = __import__
def newimport(modulename, *args, **kwargs):
if modulename in beingimported:
print "Importing in circles", modulename, args, kwargs
print " Import stack trace -> ", beingimported
# sys.exit(1) # Normally exiting is a bad idea.
beingimported.add(modulename)
result = originalimport(modulename, *args, **kwargs)
if modulename in beingimported:
beingimported.remove(modulename)
return result
import __builtin__
__builtin__.__import__ = newimport
Not all circular imports are a problem, as you've found when an exception is not thrown.
When they are a problem, you'll get an exception the next time you try to run any of your tests. You can change the code when this happens.
I don't see any change required from this situation.
Example of when it's not a problem:
a.py
import b
a = 42
def f():
return b.b
b.py
import a
b = 42
def f():
return a.a
Circular imports in Python are not like PHP includes.
Python imported modules are loaded the first time into an import "handler", and kept there for the duration of the process. This handler assigns names in the local namespace for whatever is imported from that module, for every subsequent import. A module is unique, and a reference to that module name will always point to the same loaded module, regardless of where it was imported.
So if you have a circular module import, the loading of each file will happen once, and then each module will have names relating to the other module created into its namespace.
There could of course be problems when referring to specific names within both modules (when the circular imports occur BEFORE the class/function definitions that are referenced in the imports of the opposite modules), but you'll get an error if that happens.
import uses __builtin__.__import__(), so if you monkeypatch that then every import everywhere will pick up the changes. Note that a circular import is not necessarily a problem though.
Let's say I have imported two modules like this:
from module0 import hello_func
from directory.module1 import hello_var
Where in module0:
def hello_func(): return "hello from module0"
And module1:
hello_var = "hello from module1"
How can I know from which file is each object being imported?
I tried to check locals() function but nothing in there giving reference to the file...
Actually, you kind of answered your question yourself:
Let's say I have imported two modules
(insert "from xxx import *" here)
How can I know from which file is each object being imported?
One of the reasons of NOT using wildcard imports is precisely to make it clear where names are imported from (the other one being to avoind one imported name to shadow a previously imported one - something that tends to break you code in the most unexpected - and sometimes quite hard to spot - ways).
Note that in your edited question:
from module0 import hello_func
from directory.module1 import hello_var
you already have a much better idea where a name comes from. Not the exact files path yet, but at least the name of the package/module.
And that's one of the main reasons why one should NOT use wildcard imports.
Now if you want to know the exact files path, you have two distinct cases.
Some objects keep trace of where they were created (mostly modules, classes, functions, etc - cf the list of types supported by inspect.getfile()), and then, well, you already know the answer (use inspect.getfile() xD).
But most types wont (because there's no reason for it). In this case, you have to know which module they were imported from and call inspect.getfile() on the module itself. In this case, if you used wildcard imports, you will have to manually inspect all the modules you imported from to find out which one defined this name. Enjoy. Specially if one of those modules did also use wildcard imports...
one question please: where they does keep traces? and how these traces look like?
Modules keep it in their __file__ attribute. Functions and classes keep a reference to their module's name in their __module__ attribute, and from this you can use it to retrieve the module from the sys.modules dict (a cache of all modules already imported in the current process), which will gives you the file.
I never had a need to search this info for tracebacks, frames, code objects etc so you'll have to check it yourself I'm afraid ;-)
You can define in each module a constant with its path, something like this should work:
import os
FILE_PATH = os.path.abspath(__file__)
When you import that module you can access its location like this:
import module
print(module.FILE_PATH)
Another solution using the inspect and os modules.
import module0
import os
import inspect
print(os.path.abspath(inspect.getfile(module0.hello_func)))
If you are looking for the absolute path of where the script is being run, this should work for sure:
import os
abs_path = os.path.dirname(os.path.abspath(__file__))
print(abs_path)
I have these Python files
project/packages/foo/job.py
project/packages/foo/models.py
project/packages/foo/stuff/Thing.py
models.py contains class Thing and Thing.py contains functions related to Thing
job.py tries to do this:
from . import models
from . import stuff
def job ():
x = models.Thing (123)
stuff.Thing.related_function (x)
This yields an error:
AttributeError: module 'foo.stuff' has no attribute 'Thing'
I've tried variations on the import spellings but can't get it to work. I want to not bring Thing into the namespace, but always have to refer to it as models.Thing or stuff.Thing
How do I do this?
Add the __init__ files:
project/packages/foo/__init__.py
project/packages/foo/job.py
project/packages/foo/models.py
project/packages/foo/stuff/__init__.py
project/packages/foo/stuff/Thing.py
In the stuff package, initialise the submodule by adding this line:
# in project/packages/foo/stuff/__init__.py file
from foo.stuff import Thing
The other __init__.py can be empty.
Now in your job.py code, the models.Thing attribute should resolve (it's a class) and stuff.Thing attribute should resolve (it's a submodule).
Note: It is not a good naming convention to have a module name matching the class, within the same project - that's unnecessarily confusing. I recommend to rename the Thing submodule for "functions related to Thing" to something else, maybe thing_helpers.
I've written a basic program with a couple of classes, and I've had two issues I would like help with. All of the files are in the same directory, and my classes are in files with the same name as the class.
First, my class files can only import with the format
from module import class
I can't use the format
import module
Second, I've only been able to use my classes if I do the import inside of main. When I import at the beginning of the file, I get an unboundlocalerror when creating an object. I've had these issues (especially the 1st one) on more than one program. Any ideas?
Thanks!
You cannot, as you found out, use
import class
You either have to use
from module import class
And you'd call the class simply as
class # note you don't have the module namespace
Or if you'd like to keep the namespace (which I'd recommend)
import module
Then you can say
module.class
module.otherclass
...etc
As you found, you cannot just type:
import class
as that would lead python to believe that you wanted to import a module named class, when what you want is the class inside the module. That is why
from module import class
does work, as it shows python where 'class' is.
Google App Engine just gave me an error I don't understand. Given a module "X" that contains the file "Car.py" which contains a class "Car",
and given this block of code:
import X
class Passenger(db.Model):
car = db.ReferenceProperty(X.Car.Car)
I get the error:
AttributeError: 'module' object has no attribute 'Car'
But if I change it to:
from X import Car
class Passenger(db.Model):
car = db.ReferenceProperty(Car.Car)
It works. They look the same to me, but they're clearly not. What's the difference?
As Lattyware points out, X is a package, and that's just the way packages work. Importing the outer level doesn't automatically give you access to the modules within it. You could do import X.Car if you wanted to refer to the whole thing as X.Car.Car.
(Also please note Python is not Java: there's no reason to have each class in a separate file, and even if you do then modules and packages usually have lower case names.)
The problem here is that when the package X is loaded, it contains modules but they are not in it's namespace.
To put the module into the package's namespace, add import module (where module is the name of the module, naturally) into the __init__.py file for the package. It will then be in the package's namespace, and you can use the first way of accessing Car.