I am trying to do a python script that it is divided in multiple files, so I can maintain it more easily instead of making a very-long single file script.
Here is the directory structure:
wmlxgettext.py
<pywmlx>
|- __init__.py
|- (some other .py files)
|- <state>
|- __init__.py
|- state.py
|- machine.py
|- lua_idle.py
if I reach the main directory of my project (where the wmlxgettext.py script is stored) and if I try to "import pywmlx" I have an import error (Attribute Error: 'module' object has no attribute 'state')
Here is the complete error message:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/__init__.py", line 9, in <module>
import pywmlx.state as statemachine
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/__init__.py", line 1, in <module>
from pywmlx.state.machine import setup
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/machine.py", line 2, in <module>
from pywmlx.state.lua_idle import setup_luastates
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/lua_idle.py", line 3, in <module>
import pywmlx.state.machine as statemachine
AttributeError: 'module' object has no attribute 'state'
Since I am in the "project main directory" pywmlx should be on PYTHONPATH (infact I have no troubles when I tried to import pywmlx/something.py)
I'm not able to figure where is my error and how to solve this problem.
Here is the pywmlx/__init__.py source:
# all following imports works well:
from pywmlx.wmlerr import ansi_setEnabled
from pywmlx.wmlerr import wmlerr
from pywmlx.wmlerr import wmlwarn
from pywmlx.postring import PoCommentedString
from pywmlx.postring import WmlNodeSentence
from pywmlx.postring import WmlNode
# this is the import that does not work:
import pywmlx.state as statemachine
Here is the pywmlx/state/__init__.py source:
from pywmlx.state.machine import setup
from pywmlx.state.machine import run
But I think that the real problem is somewhat hidden in the "imports" used by one (or all) python modules stored in pywmlx/state directory.
Here is the pywmlx/state/machine.py source:
# State is a "virtual" class
from pywmlx.state.state import State
from pywmlx.state.lua_idle import setup_luastates
import pywmlx.nodemanip as nodemanip
def addstate(self, name, value):
# code is not important for this question
pass
def setup():
setup_luastates()
def run(self, *, filebuf, fileref, fileno, startstate, waitwml=True):
# to do
pass
Finally here is the pywmlx/state/lua_idle.py source:
import re
import pywmlx.state.machine as statemachine
# State is a "virtual" class
from pywmlx.state.state import State
# every state is a subclass of State
# all proprieties were defined originally on the base State class:
# self.regex and self.iffail were "None"
# the body of "run" function was only "pass"
class LuaIdleState (State):
def __init__(self):
self.regex = re.compile(r'--.*?\s*#textdomain\s+(\S+)', re.I)
self.iffail = 'lua_checkpo'
def run(xline, match):
statemachine._currentdomain = match.group(1)
xline = None
return (xline, 'lua_idle')
def setup_luastates():
statemachine.addstate('lua_idle', LuaIdleState)
Sorry if I posted so much code and so many files... but I fear that the files, in directory, hides more than a single import problem, so I published them all, hoping that I could explain the problem avoiding confusion.
I think that I miss something about how import works in python, so I hope this question can be useful also to other programmers, becouse I think I am not the only one who found the official documentation very difficult to understand when explaining import.
Searches Done:
Not Useful: I am already explicitly using import x.y.z all times I need to import something
Not Useful: Even if the question asks about import errors, it seems not useful for the same reason as (1)
Not Useful: As far as I know, pywmlx should be located into PYTHONPATH since "current working directory" on my tests is the directory that contains the main python script and pywmlx directory. Correct me if I am wrong
Python does several things when importing packages:
Create an object in sys.modules for the package, with the name as key: 'pywmlx', 'pywmlx.state', 'pywmlx.state.machine', etc.
Run the bytecode loaded for that module; this may create more modules.
Once a module is fully loaded and it is located inside another package, set the module as an attribute of the parent module object. Thus the sys.modules['pywmlx.state'] module is set as the state attribute on the sys.modules['pywmlx'] module object.
That last step hasn't taken place yet in your example, but the following line only works when it has been set:
import pywmlx.state.machine as statemachine
because this looks up both state and machine as attributes first. Use this syntax instead:
from pywmlx.state import machine as statemachine
Alternatively, just use
import pywmlx.state.machine
and replace statemachine. everywhere else with pywmlx.state.machine.. This works because all that is added to your namespace is a reference to the sys.modules['pywmlx'] module object and the attribute references won't need to be resolved until you use that reference in the functions and methods.
You are having a circular import in your framework. Circular imports do not work well with aliases. When importing a module with an alias and then, during the circular import, importing it again without an alias, python complains. The solution is to not use aliases (the "import module as" syntax) but always use the full "import module" statement.
Related
I have split a project that i write from 1 file to several files
The thing is, that i have around 50 classes that I created and are called from the main file, I don't want to rewrite all those class references and add the module name before each class.
I tried to make all those classes accessible through 1 package (Tokens)
So I have
Main.py
Tokens /
__init__.py
Gen.py
BuilltIns.py
The Idea was to populate the package namespace with all the classes, and then import the package inside Main.py
__init__.py:
from Gen import *
from BuilltIns import *
Main.py:
from Tokens import *
When I ran __init__ it works perfectly, and dir() reveals that all class names are imported to package namepace.
However, when i run Main.py, I get the error message:
Traceback (most recent call last):
File "../Main.py", line 1, in <module>
from Tokens import *
File "..\Tokens\__init__.py", line 1, in <module>
from Gen import *
ModuleNotFoundError: No module named 'Gen'
How should I extract the 60+ classes from Main.py to other modules without rewriting all the calls for those classes?
You should use from Tokens.Gen import ... since Tokens is the package where the Gen module resides. The import path should be relative to the main script, unless you've modified the sys.path to specify additional directories to be searched during imports.
Alternatively, in Tokens/__init__.py you can do from .Gen import * (note the . before Gen). This denotes a relative import. What happens when you run the main script is that the current working directory is added to the paths to be searched during imports, so when from Gen import ... is encountered only the default locations are searched (which doesn't include the Tokens directory). By using a relative import you tell Python where it can find that module relative to the current one.
Note that you can define your classes in __all__ in order to constrain what will be imported during from ... import *. This way you don't leak other names in your main script's namespace.
Let me first establish what working scenario.
main.py
module/file1.py
module/file2.py
main.py
import module.file1
print(module.file1)
module/file1.py
import module.file2
module/file2.py
import module.file1
Running python3 main.py gives me the following, which is fine.
<module 'module.file1' from '/project/module/file1.py'>
Now, if I change module/file2.py to have the following:
import module.file1 as testtt
I get this new output (error):
Traceback (most recent call last):
File "main.py", line 1, in <module>
import module.file1
File "/project/module/file1.py", line 1, in <module>
import module.file2
File "/project/module/file2.py", line 2, in <module>
import module.file1 as testtt
AttributeError: module 'module' has no attribute 'file2'
I'm guessing that python doesn't fully evaluate the imported module when simply importing, causing the circular reference to blow up only when you immediately use it within either of the two files.
I'd imagine I also would not get the error if I used the module in a function, since that would be evaluate when the function is actually called, like this:
import module.file1
def test():
print(module.file1)
What is the recommendation here? Should I just work to remove the circular reference? It seems like code smell anyway (existing code base).
Its an implementation detail. The import statement uses the __import__ function to do the work of finding and importing the module and then binds its returned module to the as testtt variable.
When doing a nested import like import module.file1 as testtt, __import__ returns the base module ("module"). Since the importer still needs to bind "file1" to the local namespace, it has to look up the submodule name "file1" on that object. Since the import of file1 is still in progress, it hasn't been bound to the "module" module yet.
It works in the import module.file1 case because file1 isn't bound to the local namespace and doesn't need a lookup.
There are many pitfalls with circular imports that will bedevil you throughout your code's life cycle. Good luck!
"import" is an executable statement, so you can just do the import inside the function
def test():
import module.file1
print(module.file1)
So I am having trouble importing classes in the same directory and getting them to work properly.
I currently have the following hiearchy
BBDriver.py
bbsource:
BouncyBallEnv.py
Console.py
resources:
misc:
objects:
Ball.py
Platform.py
My problem is between the 2 files in the bbsource directory. I have figured out how to get access from the bbsource directory down to the classes in the objects directory and vice versa but when I try to from BouncyBallEnv import BouncyBallEnv in the Console class I get the following error:
File "E:\PycharmProjects\BouncyBallPythonV0\bbsource\Console.py", line 5, in
from BouncyBallENV import BouncyBallEnv
ImportError: cannot import name 'BouncyBallEnv'
I have tried several things like:
from bbsource import BouncyBallEnv
from bbsource.BouncyBallEnv import BouncyBallEnv
But I can't get it to work.
The only time I could get it to work is when I did the following:
import bbsource.BouncyBallEnv
#Extra
print(bbsource.BouncyBallEnv.BouncyBallEnv.WIDTH)
But there must be a better way to do it than that so that I wouldn't have to type that lengthy statement that is in the print statement every time that I want to use a static variable in BouncyBallEnv.
I am still quite confused on how the Python importing works so I'm not sure how to go about doing this. Thank you.
NOTE: Running Python 3.5.1
the thing you need is aliases :
import bbsource.BouncyBallEnv as bbe
#Extra
print(bbe.WIDTH)
and you can't import a module with the from ... import ... syntax. Only attributes. It work like this :
import <module> [as <alias>]
or
from <module> import <attribute> [, <attribute2>...] # import some attributes
from <module> import * # import everything
with the second one, you could have done :
from bbsource.BouncyBallEnv import WIDTH
# the variable WIDTH is directly loaded : watch out for collision !
print(WIDTH)
It is abosolue_import rule.
try
from .BouncyBallENV import BouncyBallEnv
to access module in relative position.
besides, there should be an __init__.py file under bbsource directory
EDIT:
I couldn't use my own modules. It was a stupid waste of time. If you ever have the same problem, try reading this first:
http://docs.python-guide.org/en/latest/writing/structure/
I just started OOP with Python and I am confused by modules and classes.
I work with a Mac and I can write my own modules and load them from the site-packages folder.
Now I would like to create modules with useful classes.
import custom_module works.
But if custom_module has a class Custom_class, things don't work.
I tried doing:
(EDIT: I am sorry, I am removing old code which was made up, this is what I just used and doesn't work)
in custommodule.py:
class Customclass:
def __init__(self, name):
self.name = name
This module loads without errors.
Then I get:
new = custommodule.Customclass('foo')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'Customclass'
BTW, I started trying to do this using code from this tutorial
I was not able to get over this. Please advise, I must be doing something wrong.
With this file layout
site-packages/custommodule/__init__.py
site-packages/custommodule/custommodule.py
you are creating a package named custommodule that contains a module also named custommodule. Your code needs to look like
import custommodule
# or more specifically,
# import custommodule.custommodule
new = custommodule.custommodule.Customclass('foo')
or
from custommmodule import custommodule
new = custommodule.Customclass('foo')
You can also put custommodule.py directly in site-packages to avoid creating the package, in which case your original code should work.
For me, at least, this works from mod_name import ClassName
When I run this code I don't get any errors. Hope this helped
EDIT: also make sure that the module you want to import is in the project directory. If you view the left panel in the images both modules are in Stack. I hope this is obvious but, the class needs to be in the module you import as well. Make sure that the class you're importing does not import the class your importing in because then you get circular dependencies.
Try this way
File custommodule.py in the directory custommodule
class Customclass:
def __init__(self, name):
self.name = name
File __init__.py int he custommodule directory
from .custommodule import CustomClass
Note the dot before custommodule. This force the init to load the module from the same directory.
Without the dot, it work under python2 but not under python3
I know that Python discourages any situation which can get you into a circular import. But I wanted understand the Python internals of why from-imports are seemingly arbitrarily less forgiving than normal imports in circular import situations.
For example, this code compiles:
# main.py
import CommonUtil
# commonutil.py
import util
class CommonUtil:
# some code that uses util.Util
pass
# util.py
import commonutil
class Util:
# some code that uses commonutil.CommonUtil
pass
But this code does not:
# main.py
import CommonUtil
# commonutil.py
import util
class CommonUtil:
# some code that uses util.Util
pass
# util.py
from commonutil import CommonUtil
class Util:
# some code that uses CommonUtil
pass
Traceback (most recent call last):
File "main.py", line 1, in <module>
import CommonUtil
File "commonutil.py", line 1, in <module>
import util
File "util.py", line 1, in <module>
from commonutil import CommonUtil
ImportError: cannot import name CommonUtil
You don't hit compiler errors as long as you don't try to use the relevant classes before all the imports have completed. But when you try to do some aliasing, then it fails. Can someone explain what's going on internally in the Python that causes this error to rear its head only when from-import is used? And secondarily, is there any easy way around this? (Besides the obvious "pull shared code out to a third module," which I'll likely do anyways.)
Modules are executed from top to bottom. When an import is seen for the first time, execution of the current module is suspended so that the other module can be imported. When the other module attempts to import the first module, it gets a reference to the currently partially-executed module. Since code located after the import of the other module hasn't yet been executed, any names contained within it cannot yet exist.
main.py
import a
a.py
var1 = 'foo'
import b
var2 = 'bar'
b.py
import a
print a.var1 # works
print a.var2 # fails
The way around it is to not access the names in the imported module until its execution has completed.
see http://effbot.org/zone/import-confusion.htm#circular-imports for an explanation of what is happening.
I assume you launch the main.py file. Python will first try to load commonutil. It will create a module object, and start filling it with class and function and global variable when encountering their definition. The first statement is an import, so now python creates the util module and start filling it. The common module exist, but is empty. In the first version, you do not access any commonutil object at load time, so everything is fine. In the second one, you try to fetch a specific variable in commonutil which does not exist at this moment. If you had used something like f(commonutil.CommonUtil) in the first version, it would have also crashed.