Say I have a string: 'src.utils.random_num'
How can I import random_num and actually execute it. What I have thus far:
from importlib import import_module
def parse_module(module_path):
package = '.'.join(module_path.split('.')[:-1])
module = module_path.split('.')[-1]
return package, module
req_import = parse_module('src.utils.random_num')
# import the package, works fine
module = import_module(req_import[0])
# how can I access and execute `random_num` via `req_import[1]`?
module.random_num()
How can I evaluate that random_num is a function and execute it dynamically?
Related
I am trying to modify a better_profanity library to include an additional argument to get_replacement_for_swear_word function. To do so I first import the necessary parts of the library and test its functionality before:
from better_profanity import profanity, Profanity
text = "Nice c0ck"
censored = profanity.censor(text)
print(censored)
Now I get the source code of the class method, modify it and execute it to __main___:
from inspect import getsource
new_hide_swear_words = getsource(profanity._hide_swear_words).replace(
'get_replacement_for_swear_word(censor_char)', 'get_replacement_for_swear_word(censor_char, cur_word)').replace(
'ALLOWED_CHARACTERS', 'self.ALLOWED_CHARACTERS'
)
# fixing the indent
new_hide_swear_words = '\n'.join(i[4:] for i in new_hide_swear_words.split('\n'))
exec(new_hide_swear_words)
Now I replace this function inside the class:
profanity._hide_swear_words = _hide_swear_words.__get__(profanity, Profanity)
Note that I swap ALLOWED_CHARACTERS for self.ALLOWED_CHARACTERS. This is because the author of the library has imported ALLOWED_CHARACTERS in the same file where the class is defined, so when I swap the function and try to run the first piece of code again, it sais that this variable is not defined. It just so happens that it is stored in self as well, but there is no such luck with several other imported modules. Any ideas how to tackle this?
Here is the class definition on github.
When you run exec(new_hide_swear_words), you define the function _hide_swear_words in your current module (that's why you can access it later with just _hide_swear_words).
That however means, that the function lives fully in your module, so when you call it indirectly with profanity.censor(some_text) it will run the function inside this module and look for all dependent global symbols in your module.
That's why it can't access the variable ALLOWED_CHARACTERS or the function any_next_words_form_swear_words. They are defined in the profanity module, but not in your module where you run the exec.
One way to solve this, would be to just import all symbols into your module.
from inspect import getsource
from better_profanity import Profanity, profanity
from better_profanity.constants import ALLOWED_CHARACTERS
from better_profanity.utils import *
new_hide_swear_words = getsource(profanity._hide_swear_words)
new_hide_swear_words = "\n".join(i[4:] for i in new_hide_swear_words.split("\n"))
exec(new_hide_swear_words)
profanity._hide_swear_words = _hide_swear_words.__get__(profanity, Profanity)
text = "Nice c0ck"
censored = profanity.censor(text)
print(censored)
Another way would be to execute the function in the profanity module itself (then all the symbols are already defined).
However that also has a little overhead. E.g. you have to import the module and pass it to the exec function, and afterwards you need to extract the function from the module (as it will be defined in that module).
from importlib import import_module
from inspect import getsource
from better_profanity import Profanity, profanity
new_hide_swear_words = getsource(profanity._hide_swear_words)
# fixing the indent
new_hide_swear_words = "\n".join(i[4:] for i in new_hide_swear_words.split("\n"))
profanity_module = import_module(Profanity.__module__)
exec(new_hide_swear_words, vars(profanity_module))
profanity._hide_swear_words = profanity_module._hide_swear_words.__get__(
profanity, Profanity
)
text = "Nice c0ck"
censored = profanity.censor(text)
print(censored)
profanity_module = import_module(Profanity.__module__) is the same thing as import better_profanity.better_profanity as profanity_module.
I have just started a python project. The directory structure is as follows:
/algorithms
----/__init__.py
----/linkedlist
--------/__init__.py
--------/file1.py
--------/file2.py
/tests
----/test_linkedlist
You can also check the Github repository.
In each of the sub folders under algorithms, in the __init__ file I am including the following for all the files one by one:
from .file1 import *
from .file2 import *
And so on.
The task that I am trying to achieve is running all tests together using the query:
python3 -m unittest discover tests
Each file in the tests directory starts as follows:
from algorithms.linkedlist import *
import unittest
Right now if I want to add a new file to the linkedlist directory, I create the file and then add another from .filename import * in the __init__ file.
How do I write a script in the __init__ file so that each time I create a new file, I do not have to manually insert the import command?
So the __init__ is in the same folder? As the docs say The import statement is syntactic sugar for the __import__ function.
So we can use:
import importlib
import glob
for file in glob.iglob('*.py'):
importlib.__import__(file)
Some reasons why this does not work:
You want to load the functions in the module - the import * from syntax. With this code you can only run file1.test.
You run the script loading from another directory, which confuses glob. We have to specify the actual working directory.
__import__ prefers to know the module name.
To find the solution I combine the import * from function from this answer with pkgutil.walk_packages from this blog.
import importlib
import pkgutil
def custom_import_all(module_name):
""" Use to dynamically execute from module_name import * """
# get a handle on the module
mdl = importlib.import_module(module_name)
# is there an __all__? if so respect it
if "__all__" in mdl.__dict__:
names = mdl.__dict__["__all__"]
else:
# otherwise we import all names that don't begin with _
names = [x for x in mdl.__dict__ if not x.startswith("_")]
# now drag them in
globals().update({k: getattr(mdl, k) for k in names})
__path__ = pkgutil.extend_path(__path__, __name__)
for importer, modname, ispkg in pkgutil.walk_packages(path=__path__, prefix=__name__+'.'):
custom_import_all(modname)
Importing a module can be done in a function, then passed back to the caller, take this simple utility.
def import_reload_or_none(name, reload=True):
try:
mod = __import__(name)
if reload:
import importlib
mod = importlib.reload(mod)
return mod
except Exception:
import traceback
traceback.print_exc()
return None
This works for root-level modules, however I'm not sure how this could be done when the function is in another module or when relative imports are used.
How could this function be made to work to replace for eg:
from . import my_package_module
Try passing the path to the module you want to import to your function for example:
def import_reload_or_none(name, reload=True, path=None):
mod = __import__(path + '/' + name) if path is not None else __import__(name)
if reload:
import importlib
mod = importlib.reload(mod)
return mod
Then, if you want to import with a relative path you can use
import_reload_or_none(my_package_module, path = '.')
I want to let the user to choose which file to open using a string variable. Basically I want to learn how can I tell Python to use a variable in a code section.
I have the following code:
def call_file(fn1):
import fn1
filename = input("Name of the file to import")
call_file(filename)
And inside the same folder i have the helloWorld.py with:
print("hello world")
As you have found, the import statement does not accomplish what you need. Try this instead:
from importlib import import_module
def call_file(fn1):
return import_module(fn1)
filename = input("Name of the file to import: ")
usermodule = call_file(filename)
The import_module function allows you to import a module given as an argument. The python docs have more information on this function.
Example
Running under ipython, we can use the code above to import the os module and access it under the name usermodule:
In [3]: run t.py
Name of the file to import: os
In [4]: usermodule.stat('t.py')
Out[4]: os.stat_result(st_mode=33200, st_ino=97969455, st_dev=2066, st_nlink=1, st_uid=5501, st_gid=5501, st_size=196, st_atime=1462081283, st_mtime=1462081283, st_ctime=1462081283)
Improvement
If the file the user asks for cannot be imported, the code should handle the error, possibly like this:
try:
usermodule = call_file(filename)
except ImportError:
print('Sorry, that file could not be imported.')
Alternative
It is also possible to import modules from a variable name using __import__:
>>> mod = 'math'
>>> new = __import__(mod)
>>> new.cos(0)
1.0
Note, however, that the python documentation frowns on this:
Direct use of __import__() is also discouraged in favor of
importlib.import_module().
You could also use the sys module to achieve the same effect as importing a module as some other name.
import sys
def my_import(name):
__import__(name)
return sys.modules[name]
module = my_import('random') #just for testing
print module.randint(0,1) #just for testing
Below code can be used to grab module at certain depths!
def my_import(name):
m = __import__(name)
for n in name.split(".")[1:]:
m = getattr(m, n)
return m
m = __import__("xml.etree.ElementTree") # returns xml
m = my_import("xml.etree.ElementTree") # returns ElementTree
I have a string, say: abc.def.ghi.jkl.myfile.mymethod. How do I dynamically import mymethod?
Here is how I went about it:
def get_method_from_file(full_path):
if len(full_path) == 1:
return map(__import__,[full_path[0]])[0]
return getattr(get_method_from_file(full_path[:-1]),full_path[-1])
if __name__=='__main__':
print get_method_from_file('abc.def.ghi.jkl.myfile.mymethod'.split('.'))
I am wondering if the importing individual modules is required at all.
Edit: I am using Python version 2.6.5.
From Python 2.7 you can use the importlib.import_module() function. You can import a module and access an object defined within it with the following code:
from importlib import import_module
p, m = name.rsplit('.', 1)
mod = import_module(p)
met = getattr(mod, m)
met()
You don't need to import the individual modules. It is enough to import the module you want to import a name from and provide the fromlist argument:
def import_from(module, name):
module = __import__(module, fromlist=[name])
return getattr(module, name)
For your example abc.def.ghi.jkl.myfile.mymethod, call this function as
import_from("abc.def.ghi.jkl.myfile", "mymethod")
(Note that module-level functions are called functions in Python, not methods.)
For such a simple task, there is no advantage in using the importlib module.
For Python < 2.7 the builtin method __ import__ can be used:
__import__('abc.def.ghi.jkl.myfile.mymethod', fromlist=[''])
For Python >= 2.7 or 3.1 the convenient method importlib.import_module has been added. Just import your module like this:
importlib.import_module('abc.def.ghi.jkl.myfile.mymethod')
Update: Updated version according to comments (I must admit I didn't read the string to be imported till the end and I missed the fact that a method of a module should be imported and not a module itself):
Python < 2.7 :
mymethod = getattr(__import__("abc.def.ghi.jkl.myfile", fromlist=["mymethod"]))
Python >= 2.7:
mymethod = getattr(importlib.import_module("abc.def.ghi.jkl.myfile"), "mymethod")
from importlib import import_module
name = "file.py".strip('.py')
# if Path like : "path/python/file.py"
# use name.replaces("/",".")
imp = import_module(name)
# get Class From File.py
model = getattr(imp, "classNameImportFromFile")
NClass = model() # Class From file
It's unclear what you are trying to do to your local namespace. I assume you want just my_method as a local, typing output = my_method()?
# This is equivalent to "from a.b.myfile import my_method"
the_module = importlib.import_module("a.b.myfile")
same_module = __import__("a.b.myfile")
# import_module() and __input__() only return modules
my_method = getattr(the_module, "my_method")
# or, more concisely,
my_method = getattr(__import__("a.b.myfile"), "my_method")
output = my_method()
While you only add my_method to the local namespace, you do load the chain of modules. You can look at changes by watching the keys of sys.modules before and after the import. I hope this is clearer and more accurate than your other answers.
For completeness, this is how you add the whole chain.
# This is equivalent to "import a.b.myfile"
a = __import__("a.b.myfile")
also_a = importlib.import_module("a.b.myfile")
output = a.b.myfile.my_method()
# This is equivalent to "from a.b import myfile"
myfile = __import__("a.b.myfile", fromlist="a.b")
also_myfile = importlib.import_module("a.b.myfile", "a.b")
output = myfile.my_method()
And, finally, if you are using __import__() and have modified you search path after the program started, you may need to use __import__(normal args, globals=globals(), locals=locals()). The why is a complex discussion.
This website has a nice solution: load_class. I use it like this:
foo = load_class(package.subpackage.FooClass)()
type(foo) # returns FooClass
As requested, here is the code from the web link:
import importlib
def load_class(full_class_string):
"""
dynamically load a class from a string
"""
class_data = full_class_string.split(".")
module_path = ".".join(class_data[:-1])
class_str = class_data[-1]
module = importlib.import_module(module_path)
# Finally, we retrieve the Class
return getattr(module, class_str)
Use importlib (2.7+ only).
The way I tend to to this (as well as a number of other libraries, such as pylons and paste, if my memory serves me correctly) is to separate the module name from the function/attribute name by using a ':' between them. See the following example:
'abc.def.ghi.jkl.myfile:mymethod'
This makes the import_from(path) function below a little easier to use.
def import_from(path):
"""
Import an attribute, function or class from a module.
:attr path: A path descriptor in the form of 'pkg.module.submodule:attribute'
:type path: str
"""
path_parts = path.split(':')
if len(path_parts) < 2:
raise ImportError("path must be in the form of pkg.module.submodule:attribute")
module = __import__(path_parts[0], fromlist=path_parts[1])
return getattr(module, path_parts[1])
if __name__=='__main__':
func = import_from('a.b.c.d.myfile:mymethod')
func()
How about this :
def import_module(name):
mod = __import__(name)
for s in name.split('.')[1:]:
mod = getattr(mod, s)
return mod