I'm trying to import a function from another module; however, I can't use import because the module's name needs looking up in a list.
If I try to call the imported function ExampleFunc normally I get:
NameError: global name 'ExampleFunc' is not defined
However; if I explicitly tell python to look in locals, it finds it.
File module.py
def ExampleFunc(x):
print x
File code.py
def imprt_frm(num,nam,scope):
for key, value in __import__(num,scope).__dict__.items():
if key==nam:
scope[key]=value
def imprt_nam(nam,scope):
imprt_frm("module",nam,scope)
def MainFunc(ary):
imprt_nam("ExampleFunc",locals())
#return ExampleFunc(ary) #fails
return locals()["ExampleFunc"](ary) #works
MainFunc("some input")
The locals() dictionary is but a reflection of the actual locals array. You cannot add new names to the locals through it, nor can you alter existing locals.
It is a dictionary created on demand from the actual frame locals, and is one-way only. From the locals() function documentation:
Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.
Function locals are highly optimised and determined at compile time, Python builds on not being able to alter the known locals dynamically at runtime.
Rather than try and stuff into locals directly, you can return the one object from the dynamic import. Use the importlib module rather than __import__ here:
import importlib
def import_frm(module_name, name):
module = importlib.import_module(module_name)
return getattr(module, name)
then just assign to a local name:
def MainFunc(ary):
ExampleFunc = import_from('module' , 'ExampleFunc')
ExampleFunc(ary)
Related
In file "A.py" I wrote a function runPyFile which simply is
exec(open(file).read())
But now when I write in file "B.py":
from A import *
runPyFile(myFile)
then the values defined in this myFile file are not available in "B" file. What can I do to be able to use them in "B" file?
exec takes dictionaries to hold the global and local variables in the executed code. Pass in globals() to use the globals of the module it's in.
exec(open(file).read(), globals())
SSince you need to be able to call this from other modules, you can write runPyFile so it accepts a globals dictionary passed by the caller. Then the caller passes its globals().
def runPyFile(file, globals):
exec(open(file).read(), globals)
runPyFile(myFile, globals())
With a little stack inspection you can get the caller's globals without passing them in explicitly. This is "magic" and relies on details specific to CPython, so use with caution. (The caller can still pass in its own globals if it wants to.)
from inspect import currentframe
def runPyFile(file, globals=None):
if globals is None:
globals = currentframe().f_back.f_globals
exec(open(file).read(), globals)
Finally, there's the technique of just using your own dictionary rather than the module's global namespace. This isolates the execed code's variables from any module's and allows you to avoid overwriting values and even classes and functions in your module. You can make a dict subclass that lets you access elements as attributes to make it easier to get to those variables.
from inspect import currentframe
class Variables(dict):
__getattr__ = dict.__getitem__
def runPyFile(file, globals=None):
if globals is None:
globals = Variables()
exec(open(file).read(), globals)
return globals
vars = runPyFile(myFile)
print(vars.a) # `a` defined in myFile
vars.func(...) # calls `func` defined in myFile
I am trying to create a dynamic method executor, where I have a list that will always contain two elements. The first element is the name of the file, the second element is the name of the method to execute.
How can I achieve this?
My below code unfortunately doesn't work, but it will give you an good indication of what I am trying to achieve.
from logic.intents import CenterCapacity
def method_executor(event):
call_reference = ['CenterCapacity', 'get_capacity']
# process method call
return call_reference[0].call_reference[1]
Thanks!
You can use __import__ to look up the module by name and then then use getattr to find the method. For example if the following code is in a file called exec.py then
def dummy(): print("dummy")
def lookup(mod, func):
module = __import__(mod)
return getattr(module, func)
if __name__ == "__main__":
lookup("exec","dummy")()
will output
dummy
Addendum
Alternatively importlib.import_module can be used, which although a bit more verbose, may be easier to use.
The most important difference between these two functions is that import_module() returns the specified package or module (e.g. pkg.mod), while __import__() returns the top-level package or module (e.g. pkg).
def lookup(mod, func):
import importlib
module = importlib.import_module(mod)
return getattr(module, func)
starting from:
from logic.intents import CenterCapacity
def method_executor(event):
call_reference = ['CenterCapacity', 'get_capacity']
# process method call
return call_reference[0].call_reference[1]
Option 1
We have several options, the first one is using a class reference and the getattr. For this we have to remove the ' around the class and instantiate the class before calling a reference (you do not have to instantiate the class when the method is a staticmethod.)
def method_executor(event):
call_reference = [CenterCapacity, 'get_capacity'] # We now store a class reference
# process method call
return getattr(call_reference[0](), call_reference[1])
option 2
A second option is based on this answer. It revolves around using the getattr method twice. We firstly get module using sys.modules[__name__] and then get the class from there using getattr.
import sys
def method_executor(event):
call_reference = ['CenterCapacity', 'get_capacity']
class_ref = getattr(sys.modules[__name__], call_reference[0])
return getattr(class_ref, call_reference[1])
Option 3
A third option could be based on a full import path and use __import__('module.class'), take a look at this SO post.
(Note: This answer assumes that the necessary imports have already happened, and you just need a mechanism to invoke the functions of the imported modules. If you also want the import do be done by some program code, I will have to add that part, using importlib library)
You can do this:
globals()[call_reference[0]].__dict__[call_reference[1]]()
Explanation:
globals() returns a mapping between global variable names and their referenced objects. The imported module's name counts as one of these global variables of the current module.
Indexing this mapping object with call_reference[0] returns the module object containing the function to be called.
The module object's __dict__ maps each attribute-name of the module to the object referenced by that attribute. Functions defined in the module also count as attributes of the module.
Thus, indexing __dict__ with the function name call_reference[1] returns the function object.
A little easy problem:
exec("a=3")
print(a)
# This will print 3
If I use this:
def func():
exec("a=3")
print(a)
func()
# NameError: name 'a' is not defined.
What happened?How could I use exec() to assign it a value in a function?
Edit:I found a question with the same trouble but still didn't solved.
why do you want to do that?
I know using exec() is bad and unsafe.But recently I try to solve a OP's problem.I met it.
Python knows several kinds of scope: module global, function local, nonlocal closures, class body. Notably, scope resolution is defined statically at byte code compile time – most importantly, whether names refer to local/nonlocal or global scope cannot be changed.
Of these scopes, only global scope is guaranteed to behave similar to a dict, and as such writeable. The local/nonlocal scope is generally not writeable, and new variables cannot be added to it.
exec will write to the global scope if locals is not passed in; globals must then explicitly be set to its default of globals().
def func():
exec("a='exec'", globals()) # access only global scope
print(a)
a = 'global'
func() # prints exec
However, once a name is local to a function, exec cannot modify it.
def func():
a = 'local' # assignment makes name local
exec("a='exec global'", globals())
exec("a='exec locals'", globals(), locals())
print(a)
a = 'global'
func() # prints local
While a dict-like representation of local/nonlocal scope exists, the interpreter is not required to honour changes to it.
locals()
Update and return a dictionary representing the current local symbol table. Free variables are returned by locals() when it is called in function blocks, but not in class blocks. Note that at the module level, locals() and globals() are the same dictionary.
Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.
Even though exec does take locals as a dict, these are not treated like function locals/nonlocals. Attempts to modify the default locals (the result of locals()) are not defined.
exec()
... If globals and locals are given, they are used for the global and local variables, respectively. If provided, locals can be any mapping object. Remember that at module level, globals and locals are the same dictionary. If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition.
Note: The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. ...
execute help(exec) in your python3 REPL, you will get the following docuement:
Help on built-in function exec in module builtins:
exec(source, globals=None, locals=None, /)
Execute the given source in the context of globals and locals.
The source may be a string representing one or more Python statements
or a code object as returned by compile().
The globals must be a dictionary and locals can be any mapping,
defaulting to the current globals and locals.
If only globals is given, locals defaults to it.
so there are at least 2 options to provide the value of argument 'a':
assign a value to the variable 'a' in the module's global scope:
a = 1
def func():
exec("global a;a=3")
print(a)
pass a customized global or local context to exec:
def func():
my_context = {'a': 1}
exec("a=3", None, my_context)
print(my_context['a'])
NOTE: DONT USE eval or exec IN YOUR SERIOUS CODE UNLESS YOU KNOW WHAT YOU ARE DOING.
EDIT NOTE
the following solution(2th solution mentioned in the comments) wont work:
def func():
a = 1
exec("a=3")
print(a) # still get 1 here
Obviously this doesn't work:
#module.py
def modifyglobals():
global a
a = 12
#main.py
from module import modifyglobals
modifyglobals()
print a # NameError: name 'a' is not defined
since I think it modifies module.py's globals, but not main.py's.
On the other hand, I was expecting that this would work:
#module.py
def modifyglobals(g):
g()['a'] = 12
#main.py
from module import modifyglobals
modifyglobals(g=globals) # pass a reference to **main's globals** to modifyglobals
print a
But still it gives NameError: name 'a' is not defined.
Question: Why doesn't main.py pass a reference to its own globals, such that the called function can modify main.py's globals?
Why doesn't main.py pass a reference to its own globals, such that the called function can modify main.py's globals?
Because g=globals passes the globals function, not any particular global variable dict. When globals is called, it returns the global variable dict of whatever code called globals. You still end up calling globals in module.py rather than in main.py, so you get module.py's globals.
Pass the actual global variable dict:
#module.py
def modifyglobals(g):
g['a'] = 12
#main.py
from module import modifyglobals
modifyglobals(g=globals())
print a
Of course, most of the time, having a function modify its caller's global variables directly is a bad idea. Don't actually do this.
In case you want to access module's globals, you could do something like this:
# module.py
def modifyglobals():
global a
a = 12
# main.py
import module
module.modifyglobals()
print(module.a)
In case you want to modify your main.py's global, you could do something like this instead:
# module.py
def modifyglobals(mod):
mod['a'] = 12
# main.py
import module
a = None
module.modifyglobals(globals())
print(a)
That said, I really discourage you to use neither these 2 ways at all. Personally I rarely consider the usage of module globals at all. If I decide to define module globals is either because I'll use them to be private at the module level, or as a singletons, modules's constants or maybe module's cache. There are much better ways to do data transfer between modules and the above ones shouldn't be used.
There's a part of __import__ in Python documentation, which I don't understand:
__import__(name[, globals[, locals[, fromlist[, level]]]])
The function imports the module name, potentially using the given globals and locals to determine how to interpret the name in a package context. The standard implementation does not use its locals argument at all, and uses its globals only to determine the package context of the import statement.
What is there to "interpret" about the module name? What is package context?
An example call using those parameters looks like this:
spam = __import__('spam', globals(), locals(), [], -1)
Why does the example provide globals() and locals() to the function? What happens when I only provide globals()? Or neither?
I am probably missing some part of the namespace logic with relation to importing modules. Could you point me to an article that explains this/has examples with __import__ function?
The standard implementation does not use its locals argument at all, and uses its globals only to determine the package context of the import statement.
(from docs.python.org)
I still have no idea how globals are used; what global variable can ever affect the way import statement works?
EDIT: After looking at import.c in Python 2.5 source I found that __import__ expects to find either __name__ or __path__ in globals in order to augment import search path relative to path(s) found in one of these variables, in that order.
globals is used to determine the current context on where the import is being called. For example:
"""
/myproject/a/b.py
/myproject/a/foo.py
/myproject/c/d.py
/myproject/c/foo.py
"""
# Which foo gets imported?
import foo #1
foo = __import__('foo') #2
They are not the same, since there is no (easy) way on #2 to know from which module the import is being called from. The __import__ function needs to know which is the current module to actually import the correct foo.
Internally on __import__(), globals is used to get the reference on the current module invoking the import. From the __import__ source code:
Return the package that an import is being performed in. If globals comes
from the module foo.bar.bat (not itself a package), this returns the
sys.modules entry for foo.bar. If globals is from a package's init.py,
the package's entry in sys.modules is returned, as a borrowed reference.
What is there to "interpret" about the module name? What is package context?
When you enter
>>> a
Python must "interpret" that name. Is it a global? Is it a local?
>>> def f(x):
... return x * a
Now, x is clearly local. a has to be "interpreted". Global? Local?
Why does the example provide globals() and locals() to the function? What happens when I only provide globals()? Or neither?
Try it and see. Seriously. It's easier to play with it than it is to ask.
What's important is that things you do at the >>> prompt are global.
You'll need to define functions that will create a local context so you can see the differences between global and local.