Assume the following structure...
root/test.py
root/myapp/__init__.py
root/myapp/myapp.py # contains My_App_Class()
root/myapp/SomeObject.py # contains My_Obj_Class()
If my __init__.py contains the following structure:
from myapp import *
import SomeObject
__all__ = ['SomeObject']
I want to be able to call myapp and have the routines pre-extracted from myapp/myapp.py so that I can do the following in test.py:
import myapp
if 'My_App_Class' in myapp.__dict__.keys():
print 'success'
else:
print 'fail'
if 'My_Obj_Class' in myapp.SomeObject.__dict__.keys():
print 'success'
else:
print 'fail'
so that I can effectively collapse from myapp.myapp import * into from myapp import *
In [1]: %run test.py
fail
success
The __all__ attribute in your __init__.py file is what is preventing other names (imported from myapp/myapp.py) from being visible when using the myapp package.
Just don't use the all - and maybe, to avoid ambiguity, change the import written as:
from myapp import *
to
from .myapp import *
In other words, your __init__.py file should be simply:
from .myapp import *
import SomeObject
and no other line.
Your test.py on the other hand if further incorrect - if it starts with import myapp , all of myapp/myapp.py's names will be in the myapp namespace. You should change that first line to be from myapp import * as you put further down in the question.
Mandatory note: You should avoid doing import * and variants in Python projects that import names from packages. Python programs and projects should always import individual names from packages/modules - with the name written explicitly, anyone reading your code will know exactly from were each function or class you use came from. Moreover, IDE's and other programming tools will be able to check for non-existing or inconsistent names and warn you.
However, note it is ok to use from module import * inside a package's __init__.py to get the names exposed (in the __all__ export) in each of the package's modules into the package namespace itself - which is what you want.
Related
The project has the same structure as in the picture: I'm trying to import from "mod.py " in "index.py "
from .. import mod
However, it gives the error: "ImportError: attempted relative import with no known parent package" If you use this option:
from pack1 import mod
Then error: "ModuleNotFoundError error: there is no module named 'pack1'"
enter image description here
PROJECT/
pack1/
__init__.py
mod.py
pack2/
__init__.py
index.py
What is the problem?
This is a recurring question on StackOverflow. And much of the confusion (in my opinion) comes from how Python interprets the files and folders it sees is based on where Python is run from. First, some terminology:
module: a file containing Python code.
package: a folder containing files with Python code and other folders.
When you start Python in a directory (folder), it doesn't "know" what the namespace of that directory should be. I.e., if you are working in Z:\path\to_my\project\ when you start Python:
it does NOT consider project to be a package.
any .py files you want to import from will be in their own namespace as modules.
any folders you want to import from will also be in their own namespace as packages.
What about __init__.py? Since version 3.3, Python has implicit namespace packages, which allows importing without needing to create an empty __init__.py file.
Consider #2: if you have two files: first.py and second.py:
path/
to_my/
project/
>>Python is running here<<
first.py
second.py
with these contents:
# first.py
first_var = 'hello'
# second.py
from .first import first_var
second_var = first_var + ' world'
if you try to import like this:
>>> import second
Python basically does the following:
"ok, I see second.py"
"Reading that in as a module, chief!"
"Ok, it wants to import .first
"The . means get the package (folder) that contains first.py"
"Wait, I don't have a parent package for first.py!"
"Better raise an error."
The same rules apply for #3 as well. If we add a few packages to the project like this:
path/
to_my/
project/
>>Python is running here<<
first.py
second.py
pack1/
mod.py
other_mod.py
pack2/
index.py
with the following contents:
# pack1/mod.py
mod_var = 1234
# pack1/other_mod.py
from .mod import mod_var
other_var = mod_var * 10
# pack2/index.py
from ..pack1 import mod
and when you try to import like this:
>>> from pack2 import index.py
The import in pack2/index.py is going to fail for the same reason second.py, Python will work its way up the import chain of dots like this:
"Reading in in index.py as a module."
"Looks like it wants to import mod from ..pack1.
"Ok, . is the pack2 parent package namespace of index.py, found that."
"So, .. is the parent package of pack2."
"But, I don't have a parent package for pack2!"
"Better raise an error."
How do we make it work? Two thing.
First, move where Python is running up one level so that all of the .py files and subfolders are considered to be part of the same package namespace, which allows the file to reference each other using relative references.
path/
to_my/
>>Python is running here now<<
project/
first.py
second.py
pack1/
mod.py
other_mod.py
pack2/
index.py
So now Python sees project as a package namespace, and all of the files within can use relative references up to that level.
This changes how you import when you are in the Python interpreter:
>>> from project.pack2 import index.py
Second, you make explicit references instead of relative references. That can make the import statements really long, but if you have several top-level modules that need to pull from one another, this is how you can do it. This is useful when you are defining your functions in one file and writing your script in another.
# first.py
first_var = 'hello'
# second.py
from first import first_var # we dropped the dot
second_var = first_var + ' world'
I hope this helps clear up some of the confusion about relative imports.
My code is structured as follows:
main.py
utils.py
blah.py
The main module uses argparse to read in the location of a configurations yaml file which is then loaded as a dictionary. Is there a way for utils and blah to import this built-up dictionary?
Edit: I tried using from main import config (config being the dictionary I built) but I get ImportError: cannot import name 'config' from 'main'
Edit2: Main imports the other 2 modules - apologies for leaving out this very important detail
I would recommend making another file, say, globals.py. Import this in main, utils, and blah, and set properties in it to be recalled by the other modules. For example:
globals.py
configs = {}
main.py
import .globals
...
user_configs = yaml.load('user/entered/path.yml')
globals.configs.update(user_configs) # modifies the global `configs` variable
utils.py
import .globals
...
# need to use one of the configs for something:
try:
relevant_config = globals.configs['relevant_config']
except KeyError:
print("User did not input the config field 'relevant_config'")
All modules will be able to see the same globals instance, thus allowing you to use what are effectively global variables across your program.
You could simply save configs as a gobal variable in main.py and have utils.py and blah.py import .main, but having a designated module for this is cleaner and clearer than to have other modules importing the main module.
Just do
import main
and use it as
main.dictionary
That should do it!
I have the following package structure
package
__init__.py
sub1
__init__.py
foo.py # Contains class Foo
sub2
__init__.py
bar.py # Contains class Bar
I want to be able to just import package and have package.Foo and package.Bar, i.e. I want to have the subpackages be transparent to users.
The catch is that importing sub2 takes a long time, and many users don't care at all about the stuff in sub2 and only want the stuff in sub1. Thus I want users to be able to say import package.sub1 or from package import sub1 to just import sub1 and skip the import of sub2.
I know I can achieve the first part by having package/__init__.py contain
from .sub1 import *
from .sub2 import *
and having package/sub1/__init__.py be from .foo import Foo and similarly for sub2. However, this will always import sub1 and sub2 even if the user tries to import only package.sub1.
Correspondingly, I can achieve the second part by having package/__init__.py be empty and using the same sub1/__init__.py as above. However, then just saying import package doesn't load sub1 or sub2, so users would have to explicitly load them and then refer to package.sub1.Foo.
Ideally a solution would work both in 2.7.10 and 3.5.0, but I'll accept one or the other if both isn't possible.
The LazyLoader class is provided for exactly this kind of situation: postponing loading of the module when it is actually used, instead of at the point of importing it.
To build a lazy loader you can follow the example in the documentation:
suffixes = importlib.machinery.SOURCE_SUFFIXES
loader = importlib.machinery.SourceFileLoader
lazy_loader = importlib.util.LazyLoader.factory(loader)
finder = importlib.machinery.FileFinder(path, [(lazy_loader, suffixes)])
then you can use finder.find_spec to obtain the spec of a module and pass the result to Loader.create_module to load it.
This is a bit cumbersome to do manually for just one module.
Note that searching for "lazy import python" you'll find quite a few solutions that have different pro and cons, some of which run in python2.x. However the LazyLoader class above is the official way of doing it in python3.5+
you could add your shortcuts to the __init__.py of your module:
package/__init__.py
__all__ = [
... add everything you want to be listed for this module
'Foo',
'Bar',
...
]
from package.sub1.foo import Foo
from package.sub2.bar import Bar
now you should be able to call:
from package import Bar
I'm new to Python and I still can't get my head around why we need a __init__.py file to import modules. I have gone through other questions and answers, such as this.
What confuses me is that I can import my modules without __init__py, so why do I need it at all?
My example,
index.py
modules/
hello/
hello.py
HelloWorld.py
index.py,
import os
import sys
root = os.path.dirname(__file__)
sys.path.append(root + "/modules/hello")
# IMPORTS MODULES
from hello import hello
from HelloWorld import HelloWorld
def application(environ, start_response):
results = []
results.append(hello())
helloWorld = HelloWorld()
results.append(helloWorld.sayHello())
output = "<br/>".join(results)
response_body = output
status = '200 OK'
response_headers = [('Content-Type', 'text/html'),
('Content-Length', str(len(response_body)))]
start_response(status, response_headers)
return [response_body]
modules/hello/hello.py,
def hello():
return 'Hello World from hello.py!'
modules/hello/HelloWorld.py,
# define a class
class HelloWorld:
def __init__(self):
self.message = 'Hello World from HelloWorld.py!'
def sayHello(self):
return self.message
Result,
Hello World from hello.py!
Hello World from HelloWorld.py!
What it takes is just these two lines,
root = os.path.dirname(__file__)
sys.path.append(root + "/modules/hello")
Without any of __init__py. Can someone explain why it works in this way?
If __init__py is the proper way, what should I do/change in my code?
Based on this link: Since Python 3.3
Allowing implicit namespace packages means that the requirement to provide an __init__.py file can be dropped completely
__init__.py is for packages. A package contains a collection of related modules. If you just have a single module you want to use, you don't need to use __init__.py; just put the single .py file somewhere on the system path and you can import it.
The purpose of packages is not just to allow you to import the modules inside them. It's to group the modules together. The main benefit of this is that, if a module is inside a package, then that module can import other modules from the package using relative imports. If you have foo.py and bar.py in the same package, then foo can just do from . import bar. This makes intra-package imports more compact and easier to reorganize if you restructure the package or change its name.
Also, an obvious benefit is. . . if you make it a package, you don't have to do that sys.path stuff every time you want to import something from it.
I think that this might be due to Python version you are using. I did some experimentation and found out that having following structure:
jedrzej#jedrzej-UX303LB ~/temp $ tree .
.
├── main.py
└── packages
├── file.py
└── file.pyc
1 directory, 5 files
content of main.py:
import packages.file as p
p.fun()
and content of file.py:
import sys
def fun():
print(sys.path)
When I am executing main.py with Python 2.7.12 I get ImportError while execution of main.py with Python 3.5.2 simply works.
After adding __init__.py in packages directory, code works with both versions of Python.
Files named __init__.py are used to mark directories on disk as Python package directories. If you have the files
modules/spam/__init__.py
modules/spam/module.py
and modules is in your path, you can import the code in module.py as
import spam.module
or
from spam import module
If you remove the __init__.py file, Python will no longer look for submodules inside that directory, so attempts to import the module will fail.
The __init__.py file is usually empty, but can be used to export selected portions of the package under a more convenient name, hold convenience functions, etc. Given the example above, the contents of the init module can be accessed with
import spam
And finally here is what the official documentation has to say about this file:
The __init__.py files are required to make Python treat the
directories as containing packages; this is done to prevent
directories with a common name, such as string, from
unintentionally hiding valid modules that occur later on the
module search path. In the simplest case, __init__.py can just
be an empty file, but it can also execute initialization code
for the package or set the __all__ variable, described later.
I think this is a good 'answer' for what I didn't understand.
myMath/
__init__.py
adv/
__init__.py
sqrt.py
fib.py
add.py
subtract.py
multiply.py
divide.py
myMath/__init__.py
from add import add
from divide import division
from multiply import multiply
from subtract import subtract
from adv.fib import fibonacci
from adv.sqrt import squareroot
index.py
import sys
sys.path.append('C:\Users\mdriscoll\Documents')
import mymath
print mymath.add(4,5)
print mymath.division(4, 2)
print mymath.multiply(10, 5)
print mymath.fibonacci(8)
print mymath.squareroot(48)
I have a directory structure that looks like this:
project/
__init__.py
foo/
__init.py__
first.py
second.py
third.py
plum.py
In project/foo/__init__.py I import classes from first.py, second.py and third.py and put them in __all__.
There's a class in first.py named WonderfulThing which I'd like to use in second.py, and want to import by importing * from foo. (It's outside of the scope of this question why I'd like to do so, assume I have a good reason.)
In second.py I've tried from .foo import *, from foo import * and from . import * and in none of these cases is WonderfulThing imported. I also tried from ..foo import *, which raises an error "Attempted relative import beyond toplevel package".
I've read the docs and the PEP, and I can't work out how to make this work. Any assistance would be appreciated.
Clarification/Edit: It seems like I may have been misunderstanding the way __all__ works in packages. I was using it the same as in modules,
from .first import WonderfulThing
__all__ = [ "WonderfulThing" ]
but looking at the docs again it seems to suggest that __all__ may only be used in packages to specify the names of modules to be imported by default; there doesn't seem to be any way to include anything that's not a module.
Is this correct?
A non-wildcard import failed (cannot import name WonderfulThing). Trying from . import foo failed, but import foo works. Unfortunately, dir(foo) shows nothing.
Edit: I did misunderstand the question: No __all__ is not restricted to just modules.
One question is why you want to do a relative import. There is nothing wrong with doing from project.foo import *, here. Secondly, the __all__ restriction on foo won't prevent you from doing from project.foo.first import WonderfulThing, or just from .first import WonderfulThing, which still will be the best way.
And if you really want to import a a lot of things, it's probably best to do from project import foo, and then use the things with foo.WonderfulThing instead for doing an import * and then using WonderfulThing directly.
However to answer your direct question, to import from the __init__ file in second.py you do this:
from . import WonderfulThing
or
from . import *