Python imports. Again...
I have this file structure:
[test]
start.py (from scripts import main)
[scripts]
__init__.py (empty)
main.py (from . import install)
install.py (from scripts import main # or # from . import main)
I get import error:
vic#wic:~/projects/test$ python3 start.py
Traceback (most recent call last):
File "start.py", line 2, in <module>
from scripts import main
File "/home/vic/projects/test/scripts/main.py", line 1, in <module>
from . import install
File "/home/vic/projects/test/scripts/install.py", line 1, in <module>
from scripts import main
ImportError: cannot import name main
vic#wic:~/projects/test$
I don't understand: first time from scripts import main worked (by "worked" I mean it didn't fail with ImportError), and second time the same code gives ImportError: cannot import name main
What is going on?
UPDATE:
My question is not about circular imports. I am confused by the fact that exactly the same code from scripts import main first time works ok and then second time fails.
I guess there is some internal import mechanism which i don't understand.
First time a statement imports a module, second time exactly the same code tries to import a name from a module. How this works?
You created a circular import. You can't do that. Avoid importing main from install.
What is happening is that a module, while being imported, is incomplete until the whole top level has been executed. If during that execution it imports another module that (directly or indirectly) tries to import the original module again, this will fail. The original module was not yet done importing yet.
In other words, you are creating a circular graph:
start -> scripts.main -> scripts.install
^ |
| |
----------------
You need to re-arrange your code to not need to import from main from within your scripts package.
See What are the “best practices” for using import in a module? for some tips on how to handle this.
main imports install and install imports main:
from scripts import main in start.py leads to
from . import install in main.py leads to
from scripts import main in install.py -- circular import
To solve the circular import either combine main and install into a single module or create the 3rd module that both modules can import.
This should work:
start.py:
from scripts import main
scripts/main.py:
import install
scripts/install.py:
import main
This is a patch I use to override the default behavior:
import sys
import builtins
import importlib
def _import(name, globals=None, locals=None, fromlist=None, level=0, __import__=__import__):
"""A hack to to make names of imported modules to be available in the parent package before
they are fully imported. If a module is present in sys.modules event if it's not fully
imported, it should not be a problem.
I.e. ``from package import module1, module2`` will create variables ``package.module1`` and
``package.module2`` at start of the import, not when it's finished.
This helps in some situations with circular import.
"""
module = __import__(name, globals, locals, fromlist, level)
for attr in fromlist or ():
sub_module = sys.modules.get(module.__name__ + '.' + attr)
if sub_module: # is this a module?
# if subpackage is already being imported, even if not finished,
# inject its name into the parent package
setattr(module, attr, sub_module)
return module
builtins.__import__ = _import
Here is a project to test this: https://github.com/warvariuc/python_import_improver
Related
How do I import a module and use it as if I installed it without actually installing it? More specifically, when I do imports on an installed module, all the imports are relative to the module. However when I just import it, all the imports are relative to the module it is called from.
As a specific example, in the below I would like run_import.py to print module not parent. But because it imports cfg from the parent not the module, it prints the wrong one. I would like to be able to run module1 on a standalone basis and have it isolated from any code that is calling it, much like you would in an installed package.
Directory structure is
module1/
__init__.py
cfg.py
tasks.py
run_module.py
utils.py
cfg.py
run_import.py
utils.py
Code
# cfg.py
level = 'parent'
# module1/cfg.py
level='module'
# module1/tasks.py
import cfg
def run_it():
print(cfg.level)
# module1/run_module.py
import tasks
tasks.run_it() # prints 'module'
# module1/utils.py
def util_fun():
print('util module')
# utils.py
def util_fun():
print('util parent')
# run_import.py
import module1.tasks as tasks
tasks.run_it() # prints 'parent', would like it to print 'module'
import utils
utils.util_fun() # prints util parent, should not print util module
This give me what I was looking for but has the unintended consequence that everything gets imported relative to that module
import sys
sys.path.insert(0, "module1")
tasks.run_it() # prints 'parent'
import utils
utils.util_fun() # prints util module, not util parent
Here are a couple of answers I have looked it, they didn't seem to solve it.
What's the purpose of the "__package__" attribute in Python? this sounds like what is needed but not sure how that can be set
Python3 importlib.util.spec_from_file_location with relative path? look potentially promising but i'm not sure how to use it, particularly not sure what the file path should be
How to use a packed python package without installing it
Import a module from a relative path
Can a python module be imported without installing
Import module using relative paths
When you execute run_import.py, the imports are searching in the root folder (module1/).
So you can try changing module1/tasks.py
from module1 import cfg
def run_it():
print(cfg.level)
Try this, adding the module to the path works if you would only like to change run_import.py
import sys
sys.path.insert(0, "module1")
from module1 import tasks
tasks.run_it()
I know this question has been asked before but I couldn't get to an answer.
My package folder looks like this, no sub folders, just a flat package folder with .py files in it.
+Package
∣
∣--__init__.py
∣--moduleA.py
∣--moduleB.py
If I run my test.py script from inside the package folder, the imports, classes and methods work fine:
import moduleA.py
import moduleB.py
# ...stuff
Now, if I try to run my package from outside importing import Package, outside being \site-packages I get
File "defaultPathTo\Python\Python38\lib\site-packages\Package\__init__.py", line 1, in <module>
import moduleA
ModuleNotFoundError: No module named 'moduleA'
This is my init file
import moduleA
import moduleB
I tried changing the content to from moduleA import *, from . import moduleA
from .moduleA import (whatever class) seems to work, but I don't want to change all my classes from moduleA.ClassA because it clashes with class names from the other modules.
I think I summed up all the information neede. Thanks for the help
Using from . import moduleA for all imports from within my package did the trick.
I may have done something wrong the first time I tried because I got no known parent folder
I don't know how to be clear and precise about explaining this question...
But I'm trying to run a main script (main.py), that imports several libraries (like the 'os' module) and this script should import another child script (child.py).
The problem is the 'child.py' must use this 'os' module but it can't get from the 'main.py'.
I tried to run 'main.py' and get:
NameError: name 'os' is not definied.
My directories structure:
main/
|__ main.py
|__ sub/
|__ child.py
|__ __init__.py
[main.py] content:
import os
from sub.child import function
function()
[child.py] content:
def function:
os.system('clear')
<more code that require 'os' module>
What am I missing here?
I'm trying to import all libraries from the 'main.py' to avoid waiting to much when running another scripts (don't want them importing a lot of libraries after, I want to import everything from the main file).
I'm trying to import all libraries from the 'main.py' to avoid waiting to much when running another scripts (don't want them importing a lot of libraries after, I want to import everything from the main file).
This is not how it works. You should import a module in the module that actually uses it and only there.
So you need to import the os module in the child module to be able to use it:
# child.py
import os
def function():
os.system('clear')
# more code that uses `os`
And if don't actually use os in main.py, you shouldn't import it there:
# main.py
from sub.child import function
function()
Also as #user10987432 pointed; if you are worried about import statements slowing your code down, then you've got your priorities mixed up. Not only is it a form of premature optimization, but on top of that, it's hard to even think of a time where an import statement was the bottleneck - that is to say - it's probably not possible and it's a non-problem.
I think that your code didn't originally have the import os line in it but you corrected this in the source and re-imported the file.
The problem is that Python caches modules. If you import more than once, each time you get back the same module - it isn't re-read. The mistake you had when you did the first import will persist.
To re-import the imtools.py file after editing, you must use reload(imtools).
I get an ImportError because of wrong path when doing multiple imports in Python. For example with these files:
folder1
first.py
folder2
second.py
folder3
third.py
first.py imports the class in second.py
second.py imports the class in third.py
There's no problem with python ./folder2/second.py but python first.py gives me an ImportError
Traceback (most recent call last):
File "first.py", line 1, in <module>
from folder2.second import SecondClass
File "home/test/folder2/second.py", line 1, in <module>
from folder3.third import ThirdClass
ImportError: No module named 'folder3'
It seems first.py executes the import folder3.third.ThirdClass of second.py when it tries to import SecondClass from second.py, and because folder3 isn't in its path it raises an error.
If I change the import path in second.py from from folder3.third import ThirdClass to from folder2.folder3.third import ThirdClass, first.py works, but obviously second.py doesn't work anymore.
Is there a way to solve this?
Edit: Adding
import sys
sys.path.append("./folder1")
in first.py solves the problem.
It would have worked as is with Python 2, in Python 3 you need to use relative imports withing the package, on other word your import line the second.py should read:
from .folder3.third import ThirdClass
Now a slight complication. I am not sure where your package boundary is, if your first.py is still part of it or is the main module (meant to be passed directly to the python interpreter). Relative imports are based on module name (__main__ for the main one), main module must use absolute imports.
This has two implications for you. What do import statements in first.py look like. And also another form your import in second.py could look like, because based on where you package start, apart from the above give example, you could also say (first.py is your main module):
from folder2.folder3.third import ThirdClass
or (folder1 and first.py are already part of the package and imported from a main module):
from folder1.folder2.folder3.third import ThirdClass
All the fun details are as always in the corresponding PEP-328.
I'm working from the basis of this previously posted problem:
module importing itself
Essentially, that problem is solved, but in the "modulename.py" file, there is a class defined, with an init function, and a ui function. Inside the class, any line of the form:
import submodule
Will function just fine. However..
import submodule.subsubmodule
or
import subsubmodule
Will produce an ImportError.
All submodules and subsubmodules have an
__init__.py
file.
This often happens if you have multiple modules with the same name inside a package.
For example, consider:
mypkg/
__init__.py
toplevel.py
mypkg.py
If the toplevel.py file calls import mypkg.mypkg, it will actually be importing the mypkg.py file and not the package.
You can solve this by including from __future__ import absolute_import as the first line in toplevel.py, which will force it to import the top-level package.
Alternatively, you can use from . import mypkg in toplevel.py, which will explicitly import the mypkg.py file.