i'm searching how to do
from myLib import *
inside my python code in order to do a import loop
__import__() method does not seems to provide the * feature, as i have to specify every content i want to import.
Is there a way to do the * ?
Thank's a lot for your help
EDIT:
to clarify, the goal is to import a bunch of classes, that stand inside a bunch of modules in a package to access them directly through there classes name, not like myPacakge.myModule.myClass(), nor myModule.myClass() but just myClass()
imagine you have:
myScript.py
myPackage
\__init__.py
\myModule_0.py
\myModule_1.py
\myModule_2.py
\myModule_3.py
\myModule_4.py
each myModule contains a bunch of classes and you are editing myScript.py, you want to have access to all classes in each myModule_X.py like:
myClass()
myOtherClass()
myOtherOtherClass()
etc... not like myModule_X.myClass() nor myPackage.myModyle_X.myClass()
__import__ returns the imported module's namespace. If you want to do import * from it, then you can iterate over that namespace and stuff all the module's names into your module's globals, which is what from modulename import * does. You probably shouldn't, just like you shouldn't use import * (except more so because you don't even know what module you're importing) but you can.
module = __import__("modulename")
if hasattr(module, "__all__"): # module tells us which names are all names
globals().update((name, getattr(module, name)) for name in module.__all__)
else: # import all non-private names
globals().update((name, getattr(module, name)) for name in dir(module)
if not name.startswith("_"))
You could also write it like so, which is a little safer since it avoids clobbering any global names already defined (at the risk of potentially not having a name you need):
module = __import__("modulename")
if hasattr(module, "__all__"): # module tells us which names are all names
globals().update((name, getattr(module, name)) for name in module.__all__
if name not in globals())
else: # import all non-private names
globals().update((name, getattr(module, name)) for name in dir(module)
if not (name.startswith("_") or name in globals()))
import myLib
will import everything but I advise against an import all
to use it you'd preface all imports with
myLib.my_module
If you only want to import certain things on the fly you'd want to do a conditional import, eg:
if condition_met:
import myLib.my_module
import * wreaks havoc with static code checking and debugging, so I don't recommend using it in a script. Assuming you're not trying to do something ill-advised with this, you might consider using the __all__ attribute to get a list of strings of the members of the package.
import my_package
for sub_package in my_package.__all__:
print "found " + sub_package
Related
Let's say i have 3 modules within the same directory. (module1,module2,module3)
Suppose the 2nd module imports the 3rd module then if i import module2 in module 1. Does that automatically import module 3 to module 1 ?
Thanks
No. The imports only work inside a module. You can verify that by creating a test.
Saying,
# module1
import module2
# module2
import module3
# in module1
module3.foo() # oops
This is reasonable because you can think in reverse: if imports cause a chain of importing, it'll be hard to decide which function is from which module, thus causing complex naming conflicts.
No, it will not be imported unless you explicitly specify python to, like so:
from module2 import *
What importing does conceptually is outlined below.
import some_module
The statement above is equivalent to:
module_variable = import_module("some_module")
All we have done so far is bind some object to a variable name.
When it comes to the implementation of import_module it is also not that hard to grasp.
def import_module(module_name):
if module_name in sys.modules:
module = sys.modules[module_name]
else:
filename = find_file_for_module(module_name)
python_code = open(filename).read()
module = create_module_from_code(python_code)
sys.modules[module_name] = module
return module
First, we check if the module has been imported before. If it was, then it will be available in the global list of all modules (sys.modules), and so will simply be reused. In the case that the module is not available, we create it from the code. Once the function returns, the module will be assigned to the variable name that you have chosen. As you can see the process is not inefficient or wasteful. All you are doing is creating an alias for your module. In most cases, transparency is prefered, hence having a quick look at the top of the file can tell you what resources are available to you. Otherwise, you might end up in a situation where you are wondering where is a given resource coming from. So, that is why you do not get modules inherently "imported".
Resource:
Python doc on importing
I have a system that collects all classes that derive from certain base classes and stores them in a dictionary. I want to avoid having to specify which classes are available (I would like to discover them programatically), so have used a from ModuleName import * statement. The user is then directed to place all tests to be collected in the ModuleName module. However, I cannot find a way to programatically determine what symbols were imported with that import statement. I have tried using dir() and __dict__ as indicated in the following example, but to no avail. How does one programatically find symbols imported in this manner (with import *)? I am unable to find them with the above methods.
testTypeFigureOuterrer.py:
from testType1 import *
from testType2 import *
class TestFigureOuterrer(object):
def __init__(self):
self.existingTests = {'type1':{},'type2':{}}
def findAndSortTests(self):
for symbol in dir(): # Also tried: dir(self) and __dict__
try:
thing = self.__getattribute__(symbol)
except AttributeError:
continue
if issubclass(thing,TestType1):
self.existingTests['type1'].update( dict(symbol,thing) )
elif issubclass(thing,TestType3):
self.existingTests['type2'].update( dict(symbol,thing) )
else:
continue
if __name__ == "__main__":
testFigureOuterrer = TestFigureOuterrer()
testFigureOuterrer.findAndSortTests()
testType1.py:
class TestType1(object):
pass
class TestA(TestType1):
pass
class TestB(TestType1):
pass
testType2.py:
class TestType2:
pass
class TestC(TestType2):
pass
class TestD(TestType2):
pass
Since you know the imports yourself, you should just import the module manually again, and then check the contents of the module. If an __all__ property is defined, its contents are imported as names when you do from module import *. Otherwise, just use all its members:
def getImportedNames (module):
names = module.__all__ if hasattr(module, '__all__') else dir(module)
return [name for name in names if not name.startswith('_')]
This has the benefit that you do not need to go through the globals, and filter everything out. And since you know the modules you import from at design time, you can also check them directly.
from testType1 import *
from testType2 import *
import testType1, testType2
print(getImportedNames(testType1))
print(getImportedNames(testType2))
Alternatively, you can also look up the module by its module name from sys.modules, so you don’t actually need the extra import:
import sys
def getImportedNames (moduleName):
module = sys.modules[moduleName]
names = module.__all__ if hasattr(module, '__all__') else dir(module)
return [name for name in names if not name.startswith('_')]
print(getImportedNames('testType1'))
print(getImportedNames('testType2'))
Take a look at this SO answer, which describes how to determine the name of loaded classes, you can get the name of all the classes defined within the context of the module.
import sys, inspect
clsmembers = inspect.getmembers(sys.modules['testType1'], inspect.isclass)
which is now defined as
[('TestA', testType1.TestA),
('TestB', testType1.TestB),
('TestType1', testType1.TestType1)]
You can also replace testType1 with __name__ when you're within the function of interest.
Don't use the * form of import. This dumps the imported names into your script's global namespace. Not only could they clobber some important bit of data by using the same name, you don't have any easy way to fish out the names you just imported. (Easiest way is probably to take a snapshot of globals().keys() before and after.)
Instead, import just the module:
import testType1
import testType2
Now you can easily get a list of what's in each module:
tests = dir(testType1)
And access each using getattr() on the module object:
for testname in tests:
test = getattr(testType1, testname)
if callable(test):
# do something with it
I want my_module to export __all__ as empty list, i.e.
from my_module import *
assert '__all__' in dir() and __all__ == []
I can export __all__ like this (in 'my_module.py'):
__all__ = ['__all__']
However it predictably binds __all__ to itself , so that
from my_module import *
assert '__all__' in dir() and __all__ == ['__all__']
How can I export __all__ as an empty list? Failing that, how can I hook into import process to put __all__ into importing module's __dict__ on every top level import my_module statement, circumventing module caching.
I'll start with saying this is, in my mind, a terrible idea. You really should not implicitly alter what is exported from a module, this goes counter to the Zen of Python: Explicit is better than implicit..
I also agree with the highest-voted answer on the question you cite; Python already has a mechanism to mark functions 'private', by convention we use a leading underscore to indicate a function should not be considered part of the module API. This approach works with existing tools, vs. the decorator dynamically setting __all__ which certainly breaks static code analysers.
That out of the way, here is a shotgun pointing at your foot. Use it with care.
What you want here is a way to detect when names are imported. You cannot normally do this; there are no hooks for import statements. Once a module has been imported from source, a module object is added to sys.modules and re-used for subsequent imports, but that object is not notified of imports.
What you can do is hook into attribute access. Not with the default module object, but you can stuff any object into sys.modules and it'll be treated as a module. You could just subclass the module type even, then add a __getattribute__ method to that. It'll be called when importing any name with from module import name, for all names listed in __all__ when using from module import *, and in Python 3, __spec__ is accessed for all import forms, even when doing just import module.
You can then use this to hack your way into the calling frame globals, via sys._getframe():
import sys
import types
class AttributeAccessHookModule(types.ModuleType):
def __getattribute__(self, name):
if name == '__all__':
# assume we are being imported with from module import *
g = sys._getframe(1).f_globals
if '__all__' not in g:
g['__all__'] = []
return super(AttributeAccessHook, self).__getattribute__(name)
# replace *this* module with our hacked-up version
# this part goes at the *end* of your module.
replacement = sys.module[__name__] = AttributeAccessHook(__name__, __doc__)
for name, obj in globals().items():
setattr(replacement, name, obj)
The guy there sets __all__ on first decorator application, so not explicitly exporting anything causes it to implicitly export everything. I am trying to improve on this design: if the decorator is imported, then export nothing my default, regardless of it's usage.
Just set __all__ to an empty list at the start of your module, e.g.:
# this is my_module.py
from utilitymodule import public
__all__ = []
# and now you could use your #public decorator to optionally add module to it
A Python namespace package can be spread over many directories, and zip files or custom importers. What's the correct way to iterate over all the importable submodules of a namespace package?
Here is a way that works well for me. Create a new submodule all.py, say, in one of the packages in the namespace.
If you write
import mynamespace.all
you are given the object for the mynamespace module. This object contains all of the loaded modules in the namespace, irrespective of where they were loaded, since there is only one instance of mynamespace around.
So, just load all the packages in the namespace in all.py!
# all.py
from pkgutil import iter_modules
# import this module's namespace (= parent) package
pkg = __import__(__package__)
# iterate all modules in pkg's paths,
# prefixing the returned module names with namespace-dot,
# and import the modules by name
for m in iter_modules(pkg.__path__, __package__ + '.'):
__import__(m.name)
Or in a one-liner that keeps the all module empty, if you care about that sort of thing:
# all.py
(lambda: [__import__(_.name) for _ in __import__('pkgutil').iter_modules(__import__(__package__).__path__, __package__ + '.')])() # noqa
After importing the all module from your namespace, you then actually receive a fully populated namespace module:
import mynamespace.all
mynamespace.mymodule1 # works
mynamespace.mymodule2 # works
...
Of course, you can use the same mechanism to enumerate or otherwise process the modules in the namespace, if you do not want to import them immediately.
Please read import confusion.
It very clearly distinguishes all the different ways you can import packages and its sub modules and in the process answers your question. When you need a certain submodule from a package, it’s often much more convenient to write from io.drivers import zip than import io.drivers.zip, since the former lets you refer to the module simply as zip instead of its full name.
from modname import *, this provides an easy way to import all the items from a module into the current namespace; however, this statement should be used sparingly.
I have a package in my project containing many *.py source files (each consisting of one class in most cases, and named by the class). I would like it so that when this package is imported, all of the files in the package are also imported, so that I do not have to write
import Package.SomeClass.SomeClass
import Package.SomeOtherClass.SomeOtherClass
import ...
just to import every class in the package. Instead I can just write
import Package
and every class is available in Package, so that later code in the file can be:
my_object = Package.SomeClass()
What's the easiest way of doing this in __init__.py? If different, what is the most general way of doing this?
The usual method is inside package/__init__.py
from .SomeClass import SomeClass
As Winston put it, the thing to do is to have an __init__.py file where all your classes are available in the module (global) namespace.
One way to do is, is to have a
from .myclasfile import MyClass
line for each class in your package, and that is not bad.
But, of course, this being Python, you can "automagic" this by doing something like this in __init__.py
import glob
for module_name in glob.glob("*.py"):
class_name = module_name.split(".")[0]
mod = __import__(module_name)
globals()[class_name] = getattr(mod, class_name)
del glob