I have split a project that i write from 1 file to several files
The thing is, that i have around 50 classes that I created and are called from the main file, I don't want to rewrite all those class references and add the module name before each class.
I tried to make all those classes accessible through 1 package (Tokens)
So I have
Main.py
Tokens /
__init__.py
Gen.py
BuilltIns.py
The Idea was to populate the package namespace with all the classes, and then import the package inside Main.py
__init__.py:
from Gen import *
from BuilltIns import *
Main.py:
from Tokens import *
When I ran __init__ it works perfectly, and dir() reveals that all class names are imported to package namepace.
However, when i run Main.py, I get the error message:
Traceback (most recent call last):
File "../Main.py", line 1, in <module>
from Tokens import *
File "..\Tokens\__init__.py", line 1, in <module>
from Gen import *
ModuleNotFoundError: No module named 'Gen'
How should I extract the 60+ classes from Main.py to other modules without rewriting all the calls for those classes?
You should use from Tokens.Gen import ... since Tokens is the package where the Gen module resides. The import path should be relative to the main script, unless you've modified the sys.path to specify additional directories to be searched during imports.
Alternatively, in Tokens/__init__.py you can do from .Gen import * (note the . before Gen). This denotes a relative import. What happens when you run the main script is that the current working directory is added to the paths to be searched during imports, so when from Gen import ... is encountered only the default locations are searched (which doesn't include the Tokens directory). By using a relative import you tell Python where it can find that module relative to the current one.
Note that you can define your classes in __all__ in order to constrain what will be imported during from ... import *. This way you don't leak other names in your main script's namespace.
Related
My project path is like:
main.py
modules/
__init__.py
databaseManager.py
sync.py
excel.py
in main.py:
from modules.databaseManger import addExcelToDb, searchInDbAll
from modules.excel import search, showExcelDirContents
from modules.sync import syncExcelAndDB
and for example in database.py :
from modules.excel import showExcelDirContents
from modules.sync import insertExcelNameToSyncDb
but when I run main.py I get this error:
Traceback (most recent call last):
File "main.py", line 6, in <module>
from modules.databaseManger import searchIn
ImportError: cannot import name 'searchInDbAll'
and also having error when trying to import a function from each file in modules directory to others.
I need some examples of importing.
This is circular import issue.
Explanation:
You start by triggering import of databaseManager module.
During this databaseManager code starts to import excel.
During excel importing, excel code tries to retrieve function searchInDbAll() from databaseManager. But at that moment this function does not exist - because databaseManager is in process of importing excel and he hasn't started defining any functions.
How to fix:
In modules where circular import conflicts exist, import modules instead of functions. For example, change this:
from modules.excel import showExcelDirContents
to that:
from modules import excel
And of course, you must then change corresponding function calls, from showExcelDirContents() to excel.showExcelDirContents().
You must do this in your databaseManger, excel and sync modules. With this fix, I actually could run your code.
And yeah, remove appends to sys.path, that is wrong
You can append to your path where you put your modules like this:
import sys
sys.path.append('modules/')
or
import sys
sys.path.append('c:/mycode/stuff/modules/')
note those are forward slashes, or you can use double backslashes like \\
Then just have your databaseManger.py file in /modules
You'll also need to have a file in the /modules folder named:
__init__.py
Which is just an empty file
Then you should be able to do:
from databaseManger import addExcelToDb, searchInDbAll
I think I'm misunderstanding something about the way the import system works when you throw __init__.py into the mix. Here's my (very streamlined for this example) layout (Python 3.5):
adminproj/
__init.__py
main.py
functions/
password.py
In adminproj.__init__.py, I've defined two components with the intention of using them globally (in main.py and functions.password.py):
with open((os.path.dirname(__file__)) + "/config/dev.yml", 'r') as ymlfile:
config = yaml.load(ymlfile)
logger = logging.getLogger("adminproj")
# followed by a bunch of log customization
Now in my scripts, both main.py and password.py, I can do this:
from adminproj import config
from adminproj import logger
logger.info("config values: " + str(config))
Horray!
But I also want to make my function calls not require fully qualified module names, by adding this into adminproj.__init__.py:
from adminproj.functions.password import *
Now when I do this, everything blows to hell, specifically I get problems with importing "logger" and "config":
>>> import adminproj
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/admin/sandbox/adminproj/adminproj/__init__.py", line 12, in <module>
from adminproj.functions.password import *
File "/admin/sandbox/adminproj/adminproj/functions/password.py", line 3, in <module>
from adminproj import logger
ImportError: cannot import name 'logger'
I'm pretty sure it has something to do with a name collision or something, e.g. when __init__.py imports something which itself imports something that __init__.py declares. Even if that's the right frame of mind, I can't seem to figure out a solution to do what I want to do. Perhaps using __init__.py as a global namespace is in bad form.
The simple solution is to just forget from functions.password import * and use fully qualified imports. But I want to make my function calls short and sweet, yet still utilize the global variables in __init__.py.
adminproj.contains_profanity("badword")
versus.
adminproj.functions.password.contains_profanity("badword")
I am trying to do a python script that it is divided in multiple files, so I can maintain it more easily instead of making a very-long single file script.
Here is the directory structure:
wmlxgettext.py
<pywmlx>
|- __init__.py
|- (some other .py files)
|- <state>
|- __init__.py
|- state.py
|- machine.py
|- lua_idle.py
if I reach the main directory of my project (where the wmlxgettext.py script is stored) and if I try to "import pywmlx" I have an import error (Attribute Error: 'module' object has no attribute 'state')
Here is the complete error message:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/__init__.py", line 9, in <module>
import pywmlx.state as statemachine
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/__init__.py", line 1, in <module>
from pywmlx.state.machine import setup
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/machine.py", line 2, in <module>
from pywmlx.state.lua_idle import setup_luastates
File "/home/user/programmi/my/python/wmlxgettext/true/pywmlx/state/lua_idle.py", line 3, in <module>
import pywmlx.state.machine as statemachine
AttributeError: 'module' object has no attribute 'state'
Since I am in the "project main directory" pywmlx should be on PYTHONPATH (infact I have no troubles when I tried to import pywmlx/something.py)
I'm not able to figure where is my error and how to solve this problem.
Here is the pywmlx/__init__.py source:
# all following imports works well:
from pywmlx.wmlerr import ansi_setEnabled
from pywmlx.wmlerr import wmlerr
from pywmlx.wmlerr import wmlwarn
from pywmlx.postring import PoCommentedString
from pywmlx.postring import WmlNodeSentence
from pywmlx.postring import WmlNode
# this is the import that does not work:
import pywmlx.state as statemachine
Here is the pywmlx/state/__init__.py source:
from pywmlx.state.machine import setup
from pywmlx.state.machine import run
But I think that the real problem is somewhat hidden in the "imports" used by one (or all) python modules stored in pywmlx/state directory.
Here is the pywmlx/state/machine.py source:
# State is a "virtual" class
from pywmlx.state.state import State
from pywmlx.state.lua_idle import setup_luastates
import pywmlx.nodemanip as nodemanip
def addstate(self, name, value):
# code is not important for this question
pass
def setup():
setup_luastates()
def run(self, *, filebuf, fileref, fileno, startstate, waitwml=True):
# to do
pass
Finally here is the pywmlx/state/lua_idle.py source:
import re
import pywmlx.state.machine as statemachine
# State is a "virtual" class
from pywmlx.state.state import State
# every state is a subclass of State
# all proprieties were defined originally on the base State class:
# self.regex and self.iffail were "None"
# the body of "run" function was only "pass"
class LuaIdleState (State):
def __init__(self):
self.regex = re.compile(r'--.*?\s*#textdomain\s+(\S+)', re.I)
self.iffail = 'lua_checkpo'
def run(xline, match):
statemachine._currentdomain = match.group(1)
xline = None
return (xline, 'lua_idle')
def setup_luastates():
statemachine.addstate('lua_idle', LuaIdleState)
Sorry if I posted so much code and so many files... but I fear that the files, in directory, hides more than a single import problem, so I published them all, hoping that I could explain the problem avoiding confusion.
I think that I miss something about how import works in python, so I hope this question can be useful also to other programmers, becouse I think I am not the only one who found the official documentation very difficult to understand when explaining import.
Searches Done:
Not Useful: I am already explicitly using import x.y.z all times I need to import something
Not Useful: Even if the question asks about import errors, it seems not useful for the same reason as (1)
Not Useful: As far as I know, pywmlx should be located into PYTHONPATH since "current working directory" on my tests is the directory that contains the main python script and pywmlx directory. Correct me if I am wrong
Python does several things when importing packages:
Create an object in sys.modules for the package, with the name as key: 'pywmlx', 'pywmlx.state', 'pywmlx.state.machine', etc.
Run the bytecode loaded for that module; this may create more modules.
Once a module is fully loaded and it is located inside another package, set the module as an attribute of the parent module object. Thus the sys.modules['pywmlx.state'] module is set as the state attribute on the sys.modules['pywmlx'] module object.
That last step hasn't taken place yet in your example, but the following line only works when it has been set:
import pywmlx.state.machine as statemachine
because this looks up both state and machine as attributes first. Use this syntax instead:
from pywmlx.state import machine as statemachine
Alternatively, just use
import pywmlx.state.machine
and replace statemachine. everywhere else with pywmlx.state.machine.. This works because all that is added to your namespace is a reference to the sys.modules['pywmlx'] module object and the attribute references won't need to be resolved until you use that reference in the functions and methods.
You are having a circular import in your framework. Circular imports do not work well with aliases. When importing a module with an alias and then, during the circular import, importing it again without an alias, python complains. The solution is to not use aliases (the "import module as" syntax) but always use the full "import module" statement.
In python 2 I can create a module like this:
parent
->module
->__init__.py (init calls 'from file import ClassName')
file.py
->class ClassName(obj)
And this works. In python 3 I can do the same thing from the command interpreter and it works (edit: This worked because I was in the same directory running the interpreter). However if I create __ init __.py and do the same thing like this:
"""__init__.py"""
from file import ClassName
"""file.py"""
class ClassName(object): ...etc etc
I get ImportError: cannot import name 'ClassName', it doesn't see 'file' at all. It will do this as soon as I import the module even though I can import everything by referencing it directly (which I don't want to do as it's completely inconsistent with the rest of our codebase). What gives?
In python 3 all imports are absolute unless a relative path is given to perform the import from. You will either need to use an absolute or relative import.
Absolute import:
from parent.file import ClassName
Relative import:
from . file import ClassName
# look for the module file in same directory as the current module
Try import it this way:
from .file import ClassName
See here more info on "Guido's decision" on imports in python 3 and complete example on how to import in python 3.
I need little help with importing modules in python. So.. Without unnecessary chatter:
I have specific structure of directories and modules:
multisoft/
Core/
__init__.py
Soft1/
__init__.py
Controls.py
Rigs.py
MySoft/
__init__.py
Controls.py
Rigs.py
Now from the user point of view, you only import:
import multisoft.Core as MSC
And e.g. you should get access to submodules by:
MSC.Controls.someFunction()
This module can be used in different softwares, and from user from point of view it doesn't matter in which software you are. Using of this module is the same. Of course the implementation of submodules, are differ from one software to another.
And I have some constraints which I have to keep. Basically every module can have more or less files. And the __all__
variable must be taken into consideration in every software implementation:
MySoft/__init__.py
Soft1/__init__.py
(in those init's files you put __all__ variable: __all__ = ['Controls', 'Rigs'])
I produced a part of code which basically do what I want to do:
Core/__init__.py
import os
import sys
def _softRecognize():
if youAreInSoft1():
directory = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0,directory)
module = __import__('Soft1', fromlist=['*'])
globals().update(vars(module))
_softRecognize()
But I can't reload this module, and this is one of constraint which I must keep.
import multisoft.Core as MSC
reload(MSC)
# Error: reload(): module Soft1 not in sys.modules
# Traceback (most recent call last):
# File "<maya console>", line 1, in <module>
# ImportError: reload(): module Soft1 not in sys.modules #
Thanks for any help
All the Best
Kamil
So, you are trying to abstract the interface. There are two ways to do this, either conditional imports, or variable assignments:
# Conditional
if youAreInSoft1():
import Soft1.Controls as Controls
import Soft1.Rigs as Rigs
else:
import Soft2.Controls as Controls
import Soft2.Rigs as Rigs
reload(Controls)
reload(Rigs)
#Variable
import Soft1 as _soft1
import Soft2 as _soft2
Controls = None
Rigs = None
global Controls, Rigs
if youAreInSoft1():
Controls = _soft1.Controls
Rigs = _soft1.Controls
else:
Controls = _soft2.Controls
Rigs = _soft2.Controls
reload(Controls)
reload(Rigs)
In either case, you also need to have the submodules reload whatever they need as well, i.e. reloading Controls will have to reload its submodules on down as well, if you want that behaviour.