Dir structure:
main.py
my_modules/
module1.py
module2.py
module1.py:
class fooBar():
....
class pew_pew_FooBarr()
....
...
How can I add all classes from module* to main without prefixes (i.e. to use them like foo = fooBar(), not foo = my_modules.module1.fooBar()).
An obvious decision is to write in main.py something like this:
from my_modules.module1 import *
from my_modules.module2 import *
from my_modules.module3 import *
...
But I don't want to change main.py when I create new moduleN. Is there solution for that?
I do know it's not a good idea to import classes like this, but I'm still curious about that.
UPD: This question differs from this one Loading all modules in a folder in Python, because my problem is to load modules without namespaces.
In the my_modules folder, add a __init__.py file to make it a proper package. In that file, you can inject the globals of each of those modules in the global scope of the __init__.py file, which makes them available as your module is imported (after you've also added the name of the global to the __all__ variable):
__all__ = []
import pkgutil
import inspect
for loader, name, is_pkg in pkgutil.walk_packages(__path__):
module = loader.find_module(name).load_module(name)
for name, value in inspect.getmembers(module):
if name.startswith('__'):
continue
globals()[name] = value
__all__.append(name)
Now, instead of doing:
from my_modules.class1 import Stuff
You can just do:
from my_modules import Stuff
Or to import everything into the global scope, which seems to be what you want to do:
from my_modules import *
The problem with this approach is classes overwrite one another, so if two modules provide Foo, you'll only be able to use the one that was imported last.
Related
I have a python package of the form:
package
├── __init__.py
├── module_1.py
└── module_2.py
Inside __init__.py I have:
__all__ = ["module_1", "module_2"]
Module_1 is:
__all__ = ["foo"]
def foo(): pass
Module_2 is:
__all__ = ["Bar"]
class Bar: pass
From the documentation I thought that the following code would import foo and Bar:
import package as pk
However, when I run pk.foo() it throws an error: AttributeError: module 'package' has no attribute 'foo' (same for Bar).
Thanks to this answer, I know that to get the desired behavior, I can change __init__.py into:
from .module_1 import *
from .module_2 import *
The above works.
However, I do not understand the documentation. The lines:
if a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered
Sound like my original __init__.py should have worked (the one with __all__ = ["module_1", "module_2"]). That is, the line import package as pk should have imported the modules module_1 and module_2, which in turn make foo and Bar available.
What am I missing?
EDIT:
I have also tried using exactly what the documentation mentions. That is, using from package import *, and then tried with package.foo() and foo(), but neither worked.
The first, package.foo(), throws the error NameError: 'package' is not defined.. The second, foo(), throws the error NameError: 'foo' is not defined..
How would a working example look like?.. One where __init__.py is of the form __all__ = ["module_1", "module_2"].
I will try to summarize the knowledge gained from the comments to my question, the documentation, my own tests, and this post.
1) __all__ behaves differently on __init__ and modules
1.1) Within a module
When __all__ is within a module, it determines what objects are made available when running from module import *.
Given this package structure:
package
├── __init__.py
├── module_1.py
└── module_2.py
And given the following code for module_1:
__all__ = ["foo"]
def foo(): pass
def baz(): pass
Running from package.module_1 import * will make foo available but not baz.
Moreover, foo can then be called using foo(), i.e., there is no need to reference the module.
1.2) Within __init__ (my original question)
When __all__ is within __init__,
it is taken to be the list of module names that should be imported when from package import * is encountered.
Running from package import * will then have two effects:
The scripts of the modules in __all__ will be ran (they are imported).
These modules are made available in the namespace.
This means that if __init__ is of the form:
__all__ = ["module_1", "module_2"]
Then, running from package import * will run the scripts of module_1 and module_2 and make both modules available. So, now, the function foo inside module_1 can be called as module_1.foo() instead of package.module_1.foo().
However, if this is the intent, then using from package.module_1 import foo might be better. As it makes foo accessible as foo().
2) from package import * is not the same as import package
Running from package import * has the effect mentioned in 1.2). However, this is not true for running import package: i.e. module_1.foo() would not work in this scenario.
3) The alternative approach to __init__
(The following is based on this post) As I mentioned in my question, there is an alternative approach to __init__, in which the objects that you want to make available when the user calls from package import * are directly imported into __init__.
As an example, __init__ could contain the following code:
from .module_1 import *
from .module_2 import *
Then, when the user calls from package import * the objects from modules 1 and 2 would be available on the namespace.
If module_1 was that of 1.1), then the function foo could be called without referencing the module. i.e. foo(). However, the same would not work for baz.
As mentioned in 2), from package import * is different to import package. Calling import package in this scenario (with this __init__), makes foo available through package.foo(), instead of just foo(). Similarly import package as pk makes foo available as pk.foo().
This approach could be preferable to that of 1.2), in which foo is made available through module_1.foo().
The key problem in your code is that you have defined __all__ in sub modules. Which allows to be exported only that module, but such module is not available in your directory. They are definitions you want to use.
So, just remove them from sub modules and you'll get it working.
After reading a decent amount in this area, I've come up with the following for myself.
__init__.py has imports of all submodules which are intended for outside use; an __all__ variable which contains a list of all.
Classes are typically one to a module, though if it makes sense to bundle a few classes into single module, then that's allowed.
Modules not meant for public use, are not imported in __init__.py, and are not listed in its __all__ member. This deters but does not prevent user code from accessing these non-public modules.
Any 'runnable' submodules in the package, must to explicit, non-relative imports of any required submodules.
Running these runnable submodules should be done with -m. Eg python3 -m mypackage.mySubModule
Any submodule MAY import the entire module via from mypackage import *. It seems fair to allow submodule authors, of having an easy way to get at all the (public) submodules, and fair to trust them to intelligently handle collision issues, to recognize the names of submodules, etc. It also puts the burden on __init__.py's __all__ This is the only use of from foo import *I tolerate :)
The biggest drawbacks I can think of are ...
Non-public submodules are still accessible.
Submodules explicitly use the package name, so if the package name changes, modules break
I'm allowing from foo import * just this one time, which is about as 'famous last words' as it gets.
I'm happy to hear other issues as well.
Below is a simplified but thorough example. Thanks very much to all.
File & Directory Layout
mypackage/
__init__.py
A.py # class A
B.py # class B
_Helper.py # Helper class used by A and B
api.py # public API for mypackage
runMe.py # A runnable class inside mypackage (possibly for testing)
userCode.py # Code that uses mypackage; could be located anywhere
__init__.py
from mypackage.A import A
from mypackage.B import B
from mypackage import api
__all__ = ['A', 'B', 'api']
A.py (and B.py as well)
from mypackage import _Helper
class A:
def __init__ (self): pass
def test (self): _Helper.printMessage()
_Helper.py
def printMessage ():
print("*** _Helper.printMessage() is reachable ***")
api.py
from mypackage import *
def createA (): return A()
def createB (): return B()
runMe.py
from mypackage import api
a = api.createA()
a.test()
b = api.createB()
b.test()
userCode.py
import mypackage
a = mypackage.api.createA()
a.test()
b = mypackage.api.createB()
b.test()
i'm searching how to do
from myLib import *
inside my python code in order to do a import loop
__import__() method does not seems to provide the * feature, as i have to specify every content i want to import.
Is there a way to do the * ?
Thank's a lot for your help
EDIT:
to clarify, the goal is to import a bunch of classes, that stand inside a bunch of modules in a package to access them directly through there classes name, not like myPacakge.myModule.myClass(), nor myModule.myClass() but just myClass()
imagine you have:
myScript.py
myPackage
\__init__.py
\myModule_0.py
\myModule_1.py
\myModule_2.py
\myModule_3.py
\myModule_4.py
each myModule contains a bunch of classes and you are editing myScript.py, you want to have access to all classes in each myModule_X.py like:
myClass()
myOtherClass()
myOtherOtherClass()
etc... not like myModule_X.myClass() nor myPackage.myModyle_X.myClass()
__import__ returns the imported module's namespace. If you want to do import * from it, then you can iterate over that namespace and stuff all the module's names into your module's globals, which is what from modulename import * does. You probably shouldn't, just like you shouldn't use import * (except more so because you don't even know what module you're importing) but you can.
module = __import__("modulename")
if hasattr(module, "__all__"): # module tells us which names are all names
globals().update((name, getattr(module, name)) for name in module.__all__)
else: # import all non-private names
globals().update((name, getattr(module, name)) for name in dir(module)
if not name.startswith("_"))
You could also write it like so, which is a little safer since it avoids clobbering any global names already defined (at the risk of potentially not having a name you need):
module = __import__("modulename")
if hasattr(module, "__all__"): # module tells us which names are all names
globals().update((name, getattr(module, name)) for name in module.__all__
if name not in globals())
else: # import all non-private names
globals().update((name, getattr(module, name)) for name in dir(module)
if not (name.startswith("_") or name in globals()))
import myLib
will import everything but I advise against an import all
to use it you'd preface all imports with
myLib.my_module
If you only want to import certain things on the fly you'd want to do a conditional import, eg:
if condition_met:
import myLib.my_module
import * wreaks havoc with static code checking and debugging, so I don't recommend using it in a script. Assuming you're not trying to do something ill-advised with this, you might consider using the __all__ attribute to get a list of strings of the members of the package.
import my_package
for sub_package in my_package.__all__:
print "found " + sub_package
I have a package in my project containing many *.py source files (each consisting of one class in most cases, and named by the class). I would like it so that when this package is imported, all of the files in the package are also imported, so that I do not have to write
import Package.SomeClass.SomeClass
import Package.SomeOtherClass.SomeOtherClass
import ...
just to import every class in the package. Instead I can just write
import Package
and every class is available in Package, so that later code in the file can be:
my_object = Package.SomeClass()
What's the easiest way of doing this in __init__.py? If different, what is the most general way of doing this?
The usual method is inside package/__init__.py
from .SomeClass import SomeClass
As Winston put it, the thing to do is to have an __init__.py file where all your classes are available in the module (global) namespace.
One way to do is, is to have a
from .myclasfile import MyClass
line for each class in your package, and that is not bad.
But, of course, this being Python, you can "automagic" this by doing something like this in __init__.py
import glob
for module_name in glob.glob("*.py"):
class_name = module_name.split(".")[0]
mod = __import__(module_name)
globals()[class_name] = getattr(mod, class_name)
del glob
If I have a directory structure like this:
package/
__init__.py
functions.py #contains do()
classes.py #contains class A()
And I want to be able to call
import package as p
How do I make the contents of functions, classes, accessible as:
p.do()
p.A()
in stead of:
p.functions.do()
p.classes.A()
The subdivision in files is only there for convenience (allowing easier collaboration), but I'd prefer to have all the contents in the same namespace.
You could do this in __init__.py (because that's what you import when you import package):
from package.functions import *
from package.classes import *
However, import * is nearly always a bad idea and this isn't one of the exceptions. Instead, many packages explicitly import a limited set of commonly-used names - say,
from package.functions import do
from package.classes import A
This too allows accessing do or A directly, but it's not nearly as prone to name collisions and the other problems that come from import *.