I need little help with importing modules in python. So.. Without unnecessary chatter:
I have specific structure of directories and modules:
multisoft/
Core/
__init__.py
Soft1/
__init__.py
Controls.py
Rigs.py
MySoft/
__init__.py
Controls.py
Rigs.py
Now from the user point of view, you only import:
import multisoft.Core as MSC
And e.g. you should get access to submodules by:
MSC.Controls.someFunction()
This module can be used in different softwares, and from user from point of view it doesn't matter in which software you are. Using of this module is the same. Of course the implementation of submodules, are differ from one software to another.
And I have some constraints which I have to keep. Basically every module can have more or less files. And the __all__
variable must be taken into consideration in every software implementation:
MySoft/__init__.py
Soft1/__init__.py
(in those init's files you put __all__ variable: __all__ = ['Controls', 'Rigs'])
I produced a part of code which basically do what I want to do:
Core/__init__.py
import os
import sys
def _softRecognize():
if youAreInSoft1():
directory = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0,directory)
module = __import__('Soft1', fromlist=['*'])
globals().update(vars(module))
_softRecognize()
But I can't reload this module, and this is one of constraint which I must keep.
import multisoft.Core as MSC
reload(MSC)
# Error: reload(): module Soft1 not in sys.modules
# Traceback (most recent call last):
# File "<maya console>", line 1, in <module>
# ImportError: reload(): module Soft1 not in sys.modules #
Thanks for any help
All the Best
Kamil
So, you are trying to abstract the interface. There are two ways to do this, either conditional imports, or variable assignments:
# Conditional
if youAreInSoft1():
import Soft1.Controls as Controls
import Soft1.Rigs as Rigs
else:
import Soft2.Controls as Controls
import Soft2.Rigs as Rigs
reload(Controls)
reload(Rigs)
#Variable
import Soft1 as _soft1
import Soft2 as _soft2
Controls = None
Rigs = None
global Controls, Rigs
if youAreInSoft1():
Controls = _soft1.Controls
Rigs = _soft1.Controls
else:
Controls = _soft2.Controls
Rigs = _soft2.Controls
reload(Controls)
reload(Rigs)
In either case, you also need to have the submodules reload whatever they need as well, i.e. reloading Controls will have to reload its submodules on down as well, if you want that behaviour.
Related
I have split a project that i write from 1 file to several files
The thing is, that i have around 50 classes that I created and are called from the main file, I don't want to rewrite all those class references and add the module name before each class.
I tried to make all those classes accessible through 1 package (Tokens)
So I have
Main.py
Tokens /
__init__.py
Gen.py
BuilltIns.py
The Idea was to populate the package namespace with all the classes, and then import the package inside Main.py
__init__.py:
from Gen import *
from BuilltIns import *
Main.py:
from Tokens import *
When I ran __init__ it works perfectly, and dir() reveals that all class names are imported to package namepace.
However, when i run Main.py, I get the error message:
Traceback (most recent call last):
File "../Main.py", line 1, in <module>
from Tokens import *
File "..\Tokens\__init__.py", line 1, in <module>
from Gen import *
ModuleNotFoundError: No module named 'Gen'
How should I extract the 60+ classes from Main.py to other modules without rewriting all the calls for those classes?
You should use from Tokens.Gen import ... since Tokens is the package where the Gen module resides. The import path should be relative to the main script, unless you've modified the sys.path to specify additional directories to be searched during imports.
Alternatively, in Tokens/__init__.py you can do from .Gen import * (note the . before Gen). This denotes a relative import. What happens when you run the main script is that the current working directory is added to the paths to be searched during imports, so when from Gen import ... is encountered only the default locations are searched (which doesn't include the Tokens directory). By using a relative import you tell Python where it can find that module relative to the current one.
Note that you can define your classes in __all__ in order to constrain what will be imported during from ... import *. This way you don't leak other names in your main script's namespace.
I am developping a program which runs on two different plattforms. Depending on which plattform I want to run it, the import directories and the names of the libraries change. For this reason i set a variable called RUN_ON_PC to True or False.
I want to implement a helper which sets the paths correctly and imports the libraries with the correct name depending of the platform and gives an interface with the same name of the libraries to the main program. The module myimporthelper is either in the "/mylib" or in the "/sd/mylib" directory. The other module names in these directories differ.
I try to do the following which is not working, since the imported modules from myimporthelper.py are not visible to main.py:
main.py:
RUN_ON_PC = True
import sys
if RUN_ON_PC:
sys.path.append("/mylib1")
else:
sys.path.append("/sd/mylib1")
import myimporthelper
myimporthelper.importall(RUN_ON_PC)
a = moduleA.ClassA() -> produces NameError: name not defined
myimporthelper.py:
import sys
def importall(run_on_pc):
if (run_on_pc == True):
sys.path.append("C:\\Users\\.....\\mylib")
import module1 as moduleA
else:
sys.path.append("/sd/mylib")
import module_a as moduleA
I want to keep the main.py short and want to outsource the platform dependent importing stuff to other module. I was not able to find a solution for this and would aprecciate any help.
Thanks a lot in advance.
You just have to qualify the name with the helper module name
a = myimporthelper.moduleA.ClassA()
But the moduleA name has to be accessible. If you import it inside a function in the helper it won't be, because of scope, unless you assign it to a name you previously declared as global in the helper module function.
I am trying to do a python script that it is divided in multiple files, so I can maintain it more easily instead of making a very-long single file script.
Here is the directory structure:
wmlxgettext.py
<pywmlx>
|- __init__.py
|- (some other .py files)
|- <state>
|- __init__.py
|- state.py
|- machine.py
|- lua_idle.py
if I reach the main directory of my project (where the wmlxgettext.py script is stored) and if I try to "import pywmlx" I have an import error (Attribute Error: 'module' object has no attribute 'state')
Here is the complete error message:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/__init__.py", line 9, in <module>
import pywmlx.state as statemachine
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/__init__.py", line 1, in <module>
from pywmlx.state.machine import setup
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/machine.py", line 2, in <module>
from pywmlx.state.lua_idle import setup_luastates
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/lua_idle.py", line 3, in <module>
import pywmlx.state.machine as statemachine
AttributeError: 'module' object has no attribute 'state'
Since I am in the "project main directory" pywmlx should be on PYTHONPATH (infact I have no troubles when I tried to import pywmlx/something.py)
I'm not able to figure where is my error and how to solve this problem.
Here is the pywmlx/__init__.py source:
# all following imports works well:
from pywmlx.wmlerr import ansi_setEnabled
from pywmlx.wmlerr import wmlerr
from pywmlx.wmlerr import wmlwarn
from pywmlx.postring import PoCommentedString
from pywmlx.postring import WmlNodeSentence
from pywmlx.postring import WmlNode
# this is the import that does not work:
import pywmlx.state as statemachine
Here is the pywmlx/state/__init__.py source:
from pywmlx.state.machine import setup
from pywmlx.state.machine import run
But I think that the real problem is somewhat hidden in the "imports" used by one (or all) python modules stored in pywmlx/state directory.
Here is the pywmlx/state/machine.py source:
# State is a "virtual" class
from pywmlx.state.state import State
from pywmlx.state.lua_idle import setup_luastates
import pywmlx.nodemanip as nodemanip
def addstate(self, name, value):
# code is not important for this question
pass
def setup():
setup_luastates()
def run(self, *, filebuf, fileref, fileno, startstate, waitwml=True):
# to do
pass
Finally here is the pywmlx/state/lua_idle.py source:
import re
import pywmlx.state.machine as statemachine
# State is a "virtual" class
from pywmlx.state.state import State
# every state is a subclass of State
# all proprieties were defined originally on the base State class:
# self.regex and self.iffail were "None"
# the body of "run" function was only "pass"
class LuaIdleState (State):
def __init__(self):
self.regex = re.compile(r'--.*?\s*#textdomain\s+(\S+)', re.I)
self.iffail = 'lua_checkpo'
def run(xline, match):
statemachine._currentdomain = match.group(1)
xline = None
return (xline, 'lua_idle')
def setup_luastates():
statemachine.addstate('lua_idle', LuaIdleState)
Sorry if I posted so much code and so many files... but I fear that the files, in directory, hides more than a single import problem, so I published them all, hoping that I could explain the problem avoiding confusion.
I think that I miss something about how import works in python, so I hope this question can be useful also to other programmers, becouse I think I am not the only one who found the official documentation very difficult to understand when explaining import.
Searches Done:
Not Useful: I am already explicitly using import x.y.z all times I need to import something
Not Useful: Even if the question asks about import errors, it seems not useful for the same reason as (1)
Not Useful: As far as I know, pywmlx should be located into PYTHONPATH since "current working directory" on my tests is the directory that contains the main python script and pywmlx directory. Correct me if I am wrong
Python does several things when importing packages:
Create an object in sys.modules for the package, with the name as key: 'pywmlx', 'pywmlx.state', 'pywmlx.state.machine', etc.
Run the bytecode loaded for that module; this may create more modules.
Once a module is fully loaded and it is located inside another package, set the module as an attribute of the parent module object. Thus the sys.modules['pywmlx.state'] module is set as the state attribute on the sys.modules['pywmlx'] module object.
That last step hasn't taken place yet in your example, but the following line only works when it has been set:
import pywmlx.state.machine as statemachine
because this looks up both state and machine as attributes first. Use this syntax instead:
from pywmlx.state import machine as statemachine
Alternatively, just use
import pywmlx.state.machine
and replace statemachine. everywhere else with pywmlx.state.machine.. This works because all that is added to your namespace is a reference to the sys.modules['pywmlx'] module object and the attribute references won't need to be resolved until you use that reference in the functions and methods.
You are having a circular import in your framework. Circular imports do not work well with aliases. When importing a module with an alias and then, during the circular import, importing it again without an alias, python complains. The solution is to not use aliases (the "import module as" syntax) but always use the full "import module" statement.
There are several posts around this error I have already read, but I still don't get what I am doing wrong.
I put it into a minimal example:
Imagine I have a Doc.py, and the package Tools which includes Tool1.py and Tool2.py.
Doc.py:
from Tools import *
import sys
def __main__():
TOOL_REPORT("Tool1","Test")
def TOOL_REPORT(tool, path):
if(tool == 'Tool1'):
Tool1.REPORT(path)
elif(tool == 'Tool2'):
Tool2.REPORT(path)
else:
sys.stderr.write("This tool is not yet included in Doc. Please check TOOLS for more information.")
if __name__=="__main__": __main__()
Tool1.py:
def REPORT(path):
print("Tool1 "+path)
Tool2.py:
def REPORT(path):
print("Tool2 "+path)
If I run this, I always end up with this error:
File "Doc.py", line 15, in TOOL_REPORT
Tool1.REPORT(path)
NameError: global name 'Tool1' is not defined
I'd appreciate any hint to what is going wrong!
Your Tool1 and Tool2 submodules are not visible until explicitly imported somewhere.
You can import them in the Tools/__init__.py package file:
import Tool1, Tool2
at which point they become available for import from Tools.
Another option is to import the modules from your own code:
import Tools.Tool1, Tools.Tool2
from Tools import *
Only when explicitly imported are submodules also set as attributes of the package.
Python will treat any folder as a module when there is __init__.py file present in it. Otherwise it will just be another folder for python and not a module from which it can import things. So just add init.py file in your Tool folder (so it will become module in pythonic terms) and then you can import that module in other python scripts.
One more things for better practice instead of using
from Tools import *
Always provide the file name of library specifically which you want to import like in your case you should use it like this
from Tools import Tool1, Tool2
This will enhance the code readbility for others and for you too.
I know that Python discourages any situation which can get you into a circular import. But I wanted understand the Python internals of why from-imports are seemingly arbitrarily less forgiving than normal imports in circular import situations.
For example, this code compiles:
# main.py
import CommonUtil
# commonutil.py
import util
class CommonUtil:
# some code that uses util.Util
pass
# util.py
import commonutil
class Util:
# some code that uses commonutil.CommonUtil
pass
But this code does not:
# main.py
import CommonUtil
# commonutil.py
import util
class CommonUtil:
# some code that uses util.Util
pass
# util.py
from commonutil import CommonUtil
class Util:
# some code that uses CommonUtil
pass
Traceback (most recent call last):
File "main.py", line 1, in <module>
import CommonUtil
File "commonutil.py", line 1, in <module>
import util
File "util.py", line 1, in <module>
from commonutil import CommonUtil
ImportError: cannot import name CommonUtil
You don't hit compiler errors as long as you don't try to use the relevant classes before all the imports have completed. But when you try to do some aliasing, then it fails. Can someone explain what's going on internally in the Python that causes this error to rear its head only when from-import is used? And secondarily, is there any easy way around this? (Besides the obvious "pull shared code out to a third module," which I'll likely do anyways.)
Modules are executed from top to bottom. When an import is seen for the first time, execution of the current module is suspended so that the other module can be imported. When the other module attempts to import the first module, it gets a reference to the currently partially-executed module. Since code located after the import of the other module hasn't yet been executed, any names contained within it cannot yet exist.
main.py
import a
a.py
var1 = 'foo'
import b
var2 = 'bar'
b.py
import a
print a.var1 # works
print a.var2 # fails
The way around it is to not access the names in the imported module until its execution has completed.
see http://effbot.org/zone/import-confusion.htm#circular-imports for an explanation of what is happening.
I assume you launch the main.py file. Python will first try to load commonutil. It will create a module object, and start filling it with class and function and global variable when encountering their definition. The first statement is an import, so now python creates the util module and start filling it. The common module exist, but is empty. In the first version, you do not access any commonutil object at load time, so everything is fine. In the second one, you try to fetch a specific variable in commonutil which does not exist at this moment. If you had used something like f(commonutil.CommonUtil) in the first version, it would have also crashed.