I have bunch of modules to import and run. I have dynamically imported the modules using Dynamic module import in Python. This is in the main code. Once imported, I'm trying to run the functions in the modules.
All of the modules look like this structure
#function foo
def run(a,b)
c=a+b
return c
foo has been imported, I need to say something like bar=foo.run(a,b) dynamically
from this example:
How to call Python functions dynamically. I have already tried the following:
i='foo'
bar = getattr(sys.modules[__name__], i+'.run()')(a,b)
traceback AttributeError: 'module' object has no attribute 'foo.run()'
I'm confused, about the attribute error. The calling functions dynamically example is clearly calling functions.
If you have imported foo already, but don't have a reference to it, use:
sys.modules['foo'].run(a,b)
the_module.run(a, b)
Regardless of what magic made the module come into existence, it's an ordinary module object with ordinary attributes, and you know that the function is called run.
If you always know you'll use module foo, you're done.
You may also need to find the module object dynamically, because the module to choose varies.
If you imported the module properly, under the name you use to refer to it (e.g. foo) rather than some other name, you can also use sys.modules[mod_name].
Otherwise, you should probably have a dictionary of modules so that you can say, the_module = modules[mod_name].
Related
From Dynamic linking in C/C++ (dll) vs JAVA (JAR)
when i want to use this jar file in another project we use "package" or "import" keyword
You don't have to. This is just a short hand. You can use full
package.ClassName and there is no need for an import. Note: this
doesn't import any code or data, just allow you to use a shorter name
for the class.
e.g. there is no difference between
java.util.Date date = new java.util.Date();
and
import java.util.Date();
Date date = new Date(); // don't need to specify the full package name.
Is it the same case for import in Python3?
Can we use a identifier defined in a module, without importing its module? Did I miss something in the following to make that happen?
What differences are between Java and Python's import?
>>> random.randint(1,25)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'random' is not defined
>>> import random
>>> random.randint(1,25)
18
Python is not Java. In Python you can only access names that are either builtins or defined in the current scope or it's parents scopes - the "top-level" scope being the module namespace (AKA "global" namespace).
The import statement (which is an executable statement FWIW) does two things: first load the module (this actually happens only once per process, then the module is cached in sys.modules), then bind the imported name(s) in the current scope. IOW this:
import foo
is syntaxic sugar for
foo = __import__("foo")
and
from foo import bar
is syntaxic sugar for
foo = __import__("foo")
bar = getattr(foo, "bar")
del foo
Also you have to understand what "loading a module" really means: executing all the code at the module's top-level.
As I mentionned, import is an executable statement, but so are class and def - the def statement creates a code object from the function's body and signature, then creates a function object with this code object, and finally bind this function object to the function's name in the current scope, the class statement does the same thing for a class (executing all the code at the "class" statement's top-level in a temporary namespace and using this namespace to create the "class" object, then binding the class object to it's name).
IOW, all happens at runtime, everything is an object (including functions, classes and modules), and everything you do with an import, class or def statement can be done "manually" too (more or less easily though - manually creating a function is quite an involved process).
So as you can see, this really has nothing to do with how either Java or C++ work.
Short answer: No, you can't implicitly import a module in Python by using a fully qualified name.
Slightly longer answer:
In Python, importing a module can have side effects: a module can have module-level code, not all of its code has to be wrapped in functions or classes. Therefore, importing modules at arbitrary and unexpected locations could be confusing, since it would trigger those side effects when you don't expect them.
The recommended style (see https://www.python.org/dev/peps/pep-0008/ for details) is to put all your imports at the top of your module, and not to hide imports at unexpected places.
This question concerns when you should have imports for Python modules and how it all interacts when you are trying to take an OOP approach to what you're making.
Let's say we have the following Modules:
ClassA.py:
class Class_A:
def doSomething(self):
#doSomething
ClassB.py
class Class_B:
def doSomethingElse(self):
#doSomethingElse
ClassC.py
class Class_C:
def __init__(self, ClassAobj, ClassBobj):
self.a = ClassAobj
self.b = ClassBobj
def doTheThing(self):
self.a.doSomething()
self.b.doSomethingElse()
Main.py:
from ClassA import Class_A
from ClassB import Class_B
from ClassC import Class_C
a = Class_A()
b = Class_B()
c = Class_C(a,b)
In here Class_C uses objects of Class_A and Class_B however it does not have import statements for those classes. Do you see this creating errors down the line, or is this fine? Is it bad practice to do this?
Would having imports for Class_A and Class_B inside of Class_C cause the program as a whole to use more memory since it would be importing them for both Main.py and ClassC.py? Or will the Python compiler see that those modules have already been imported and just skip over them?
I'm just trying to figure out how Python as a language ticks with concerns to importing and using modules. Basically, if at the topmost level of your program (your Main function) if you import everything there, would import statements in other modules be redundant?
You don't use Class_A or Class_B directly in Class_C, so you don't need to import them there.
Extra imports don't really use extra memory, there is only a single instance of each module in memory. Import just creates a name for the module in the current module namespace.
In Python, it's not idiomatic to have a single class per file. It's normal to have closely related classes all in the same file. A module name "ClassA" looks silly, that is the name of a class, not of a module.
You can only use a module inside another one if it's imported there. For instance the sys module is probably already in memory after Python starts, as so many things use it, including import statements.
An import foo statement does two things:
If the foo module is not in memory yet, it is loaded, parsed, executed and then placed in sys.modules['foo'].
A local name foo is created that also refers to the module in sys.modules.
So if you have say a print() in your module (not inside a function), then that is only executed the first time the module is imported.
Then later statements after the import can do things with foo, like foo.somefunc() or print(foo.__name__).
C does not need the import statements; all it uses is a pair of object handles (i.e. pointers). As long as it does not try to access any method or attribute of those objects, the pure assignment is fine. If you do need such additions, then you need the import statements.
This will not cause additional memory usage in Main: Python checks (as do most languages) packages already imported, and will not import one multiple times. Note that this sometimes means that you have to be careful of package dependencies and importation order.
Importing a module does two things: it executes the code stored in the module, and it adds name bindings to the module doing the importing. ClassC.py doesn't need to import ClassA or ClassB because it doesn't know or care what types the arguments to ClassC.__init__ have, as long as they behave properly when used. Any references to code needed by either object is stored in the object itself.
Suppose I have two nearly identical versions of a python package mymod, ie mymod0 and mymod1. Each of these packages has files init.py and foo.py, and foo.py has a single function printme(). Calling mymod0.foo.printme() will print "I am mymod0" and calling mymod1.foo.printme() will print "I am mymod1". So far so good.
But now I need to dynamically import either mymod0 or mymod1. The user will input either 0 or 1 to the script (as variable "index"), and then I can create packageName="mymod"+str(index)
I tried this:
module=importlib.import_module(module_name)
module.foo.printme()
But I get this error:
AttributeError: 'module' object has no attribute 'foo'
How can I specify the the package should now be referred to as module so that module.foo.printme() will work?
UPDATE: So it looks like the easiest solution is to use the exec() function. This way I can dynamically create an import statement like this:
cmdname="from mymod%s import foo" % index
exec(cmdname)
Then:
foo.printme()
This seems to work.
How can I specify the the package should now be referred to as module so that module.foo.printme() will work?
You have to make sure <module_name>.__init__ imports foo into the module's namespace:
#./<module_name>/__init__.py
import foo
then you can
module=importlib.import_module(module_name)
module.foo.printme()
But now I need to dynamically import either mymod0 or mymod1.
Note this only works the first time because python caches loaded modules. If the module has changed since that start of the program, use the reload function function. Just a word of caution: There are several caveats associated with this, and it may end up not doing what you intended.
How can I recreate this dynamically?
for i in range(0,2):
module_name = 'mymod%s' % i
module=importlib.import_module(module_name)
module.foo.printme()
In Python, when we import something:
import Module
when we later want to use functions created in the module we have to say
Module.foo()
Is there any way to "attach" the module so that if I simply call
foo()
It knows that I mean to use the foo defined in Module, as long as the name does not conflict with any name in the current file?
from Module import *
This imports all symbols in Module unless overriden by __all__.
You can also explicitly import (which is better) only the symbols you actually need.
from Module import foo
It's typically preferred to use the later. Even better is to use the module as namespacing. There's nothing wrong with Module.foo() vs. foo(). Once your program gets fairly large, this will help you quite a bit with refactoring.
You can just do from module import foo, and then refer to foo() directly.
I'm noticing some weird situations where tests like the following fail:
x = <a function from some module, passed around some big application for a while>
mod = __import__(x.__module__)
x_ref = getattr(mod, x.__name__)
assert x_ref is x # Fails
(Code like this appears in the pickle module)
I don't think I have any import hooks, reload calls, or sys.modules manipulation that would mess with python's normal import caching behavior.
Is there any other reason why a module would be loaded twice? I've seen claims about this (e.g, https://stackoverflow.com/a/10989692/1332492), but I haven't been able to reproduce it in a simple, isolated script.
I believe you misunderstood how __import__ works:
>>> from my_package import my_module
>>> my_module.function.__module__
'my_package.my_module'
>>> __import__(my_module.function.__module__)
<module 'my_package' from './my_package/__init__.py'>
From the documentation:
When the name variable is of the form package.module, normally, the
top-level package (the name up till the first dot) is returned, not
the module named by name. However, when a non-empty fromlist
argument is given, the module named by name is returned.
As you can see __import__ does not return the sub-module, but only the top package. If you have function also defined at package level you will indeed have different references to it.
If you want to just load a module you should use importlib.import_module instead of __import__.
As to answer you actual question: AFAIK there is no way to import the same module, with the same name, twice without messing around with the importing mechanism. However, you could have a submodule of a package that is also available in the sys.path, in this case you can import it twice using different names:
from some.package import submodule
import submodule as submodule2
print(submodule is submodule2) # False. They have *no* relationships.
This sometimes can cause problems with, e.g., pickle. If you pickle something referenced by submodule you cannot unpickle it using submodule2 as reference.
However this doesn't address the specific example you gave us, because using the __module__ attribute the import should return the correct module.