First of all, a little bit of context I was using importlib to modify an import and was trying to find a solution for modifying an imported file.
I saw an answer suggesting that for arbitrarily modifying an imported module you can use this code
def modify_and_import(module_name, package):
spec = importlib.util.find_spec(module_name, package)
source = spec.loader.get_source(module_name)
new_source = tokens.tokenize(source)
module = importlib.util.module_from_spec(spec)
codeobj = compile(new_source, module.__spec__.origin, 'exec')
exec(codeobj, module.__dict__)
sys.modules[module_name] = module
return module
Now I ran it with the following call
o = modify_and_import('o', "./o.xran")
but, running it through an exec gives me the following error
AttributeError: 'NoneType' object has no attribute 'loader'
Then I tried to print 'spec' but got None so, I think spec is returning while a file named o.xran exists.
Ok, after a day I finally made it work and since I was only working with local files, I didn't need to use imp at all. Here's my final code
with open(module_name) as f:
source = f.read()
source = tokens.tokenize(source)
module = types.ModuleType(module_name)
exec(source, module.__dict__)
sys.modules[module_name] = module
return module
Thanks to anyone who answered or tried to contribute.
bro,open the modified module in your text editor,in the same window in which your project is opened,then in your project import it using,'from import *'..hope it will works..thanx
I am trying to do exception handling mainly from Python's documentation. I am facing a weird problem: the directory contains main.py and exceptions.py.
In exceptions.py (with no import statements) I define my exception class as follows:
class ConvergenceError(AssertionError):
def __init__(self,dc_places):
self.decimal_places = dc_places
def __str__(self):
return 'convergence of R_inf is not correct to ', self.decimal_places
In main.py I have this import statement:
import exceptions as ex
and I call the exception as follows:
try:
np.testing.assert_array_almost_equal(R_inf,R_itr,4)
raise ex.ConvergenceError(4)
except ex.ConvergenceError as ce :
print str(ce)
I receive the following error:
except ex.ConvergenceError as ce :
AttributeError: 'module' object has no attribute 'ConvergenceError'
I can not figure out why the interpreter can't see Convergence in the exceptions.py module.
As you have discussed with jonrsharpe, your problem is that you are actually importing a built-in module exceptions. Try the following:
>>> import exceptions
>>> exceptions
<module 'exceptions' (built-in)>
So what you can do is simply rename your module (e.g. to ex.py) and you'll be good:
>>> import ex
...
I would just like to elaborate that exceptions is actually a C-builtin, and is loaded upon the start of the interpreter (this is only true on Python2 and not Python3, for Python3 it is found in builtins):
>>> import sys
>>> [i for i in sys.modules if 'exceptions' in i]
['exceptions']
>>> import exceptions # dictionary lookup to builtin
<module 'exceptions' (built-in)>
Exceptions has no path and is automatically in the current path for fairly obvious reasons: the interpreter itself, from the core libraries, requires exception handling. This means if you try to import exceptions via any standard approach, you will import the builtin and not your custom module you want.
For example, here I have exceptions.py in the path and try to load it via a standard import.
>>> [i for i in os.listdir(os.getcwd()) if 'except' in i]
['exceptions.py']
>>> import exceptions
<module 'exceptions' (built-in)>
exceptions = __import__("exceptions")
>>> exceptions
<module 'exceptions' (built-in)>
You cannot even reload it to force it into the interpreter.
>>> reload(exceptions)
<module 'exceptions' (built-in)>
You can absolutely force it if you use IMP and a custom full path.
>>> imp.load_source('exceptions', os.path.join(os.getcwd(), 'exceptions.py'))
<module 'exceptions' from '/home/alex/exceptions.py'>
EDIT: I should also mention that this does not override the builtin exceptions class except in the current namespace. Doing a from module import package still refers to the builtin even after the exceptions name has been overridden. Python wisely does not let you shoot yourself in the foot.
>>> exceptions
<module 'exceptions' from '/home/alex/exceptions.pyc'>
>>> from exceptions import BaseException
>>> BaseException
<type 'exceptions.BaseException'>
PyCharm shows me that imp is deprecated so I wonder if there any analogue of imp.new_module for importlib.
Quoting from documentation (Emphasis mine) -
imp.new_module(name)
Return a new empty module object called name. This object is not inserted in sys.modules.
Deprecated since version 3.4: Use types.ModuleType instead.
Example -
>>> import types
>>> types.ModuleType('name')
<module 'name'>
To show how they are synonymous -
>>> import imp
>>> imp.new_module('name')
<module 'name'>
Python documentation suggests to us:
Note Use importlib.util.module_from_spec() to create a new module if
you wish to set the various import-controlled attributes.
importlib.util.module_from_spec(spec) is preferred over using types.ModuleType to create a new module as spec is used to set as many import-controlled attributes on the module as possible.
To import a Python source file directly, we can use the following snippet:
import importlib.util
import sys
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
The example below, we can use to implement lazy imports:
import importlib.util
import sys
def lazy_import(name):
spec = importlib.util.find_spec(name)
loader = importlib.util.LazyLoader(spec.loader)
spec.loader = loader
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
loader.exec_module(module)
return module
>>> lazy_typing = lazy_import("typing")
>>> #lazy_typing is a real module object,
>>> #but it is not loaded in memory yet.
>>> lazy_typing.TYPE_CHECKING
False
I'm writing a Python application that takes a command as an argument, for example:
$ python myapp.py command1
I want the application to be extensible, that is, to be able to add new modules that implement new commands without having to change the main application source. The tree looks something like:
myapp/
__init__.py
commands/
__init__.py
command1.py
command2.py
foo.py
bar.py
So I want the application to find the available command modules at runtime and execute the appropriate one.
Python defines an __import__() function, which takes a string for a module name:
__import__(name, globals=None, locals=None, fromlist=(), level=0)
The function imports the module name, potentially using the given globals and locals to determine how to interpret the name in a package context. The fromlist gives the names of objects or submodules that should be imported from the module given by name.
Source: https://docs.python.org/3/library/functions.html#__import__
So currently I have something like:
command = sys.argv[1]
try:
command_module = __import__("myapp.commands.%s" % command, fromlist=["myapp.commands"])
except ImportError:
# Display error message
command_module.run()
This works just fine, I'm just wondering if there is possibly a more idiomatic way to accomplish what we are doing with this code.
Note that I specifically don't want to get in to using eggs or extension points. This is not an open-source project and I don't expect there to be "plugins". The point is to simplify the main application code and remove the need to modify it each time a new command module is added.
See also: How do I import a module given the full path?
With Python older than 2.7/3.1, that's pretty much how you do it.
For newer versions, see importlib.import_module for Python 2 and Python 3.
Or using __import__ you can import a list of modules by doing this:
>>> moduleNames = ['sys', 'os', 're', 'unittest']
>>> moduleNames
['sys', 'os', 're', 'unittest']
>>> modules = map(__import__, moduleNames)
Ripped straight from Dive Into Python.
The recommended way for Python 2.7 and 3.1 and later is to use importlib module:
importlib.import_module(name, package=None)
Import a module. The name argument specifies what module to import in absolute or relative terms (e.g. either pkg.mod or ..mod). If the name is specified in relative terms, then the package argument must be set to the name of the package which is to act as the anchor for resolving the package name (e.g. import_module('..mod', 'pkg.subpkg') will import pkg.mod).
e.g.
my_module = importlib.import_module('os.path')
Note: imp is deprecated since Python 3.4 in favor of importlib
As mentioned the imp module provides you loading functions:
imp.load_source(name, path)
imp.load_compiled(name, path)
I've used these before to perform something similar.
In my case I defined a specific class with defined methods that were required.
Once I loaded the module I would check if the class was in the module, and then create an instance of that class, something like this:
import imp
import os
def load_from_file(filepath):
class_inst = None
expected_class = 'MyClass'
mod_name,file_ext = os.path.splitext(os.path.split(filepath)[-1])
if file_ext.lower() == '.py':
py_mod = imp.load_source(mod_name, filepath)
elif file_ext.lower() == '.pyc':
py_mod = imp.load_compiled(mod_name, filepath)
if hasattr(py_mod, expected_class):
class_inst = getattr(py_mod, expected_class)()
return class_inst
Using importlib
Importing a source file
Here is a slightly adapted example from the documentation:
import sys
import importlib.util
file_path = 'pluginX.py'
module_name = 'pluginX'
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
# Verify contents of the module:
print(dir(module))
From here, module will be a module object representing the pluginX module (the same thing that would be assigned to pluginX by doing import pluginX). Thus, to call e.g. a hello function (with no parameters) defined in pluginX, use module.hello().
To get the effect "importing" functionality from the module instead, store it in the in-memory cache of loaded modules, and then do the corresponding from import:
sys.modules[module_name] = module
from pluginX import hello
hello()
Importing a package
To import a package instead, calling import_module is sufficient. Suppose there is a package folder pluginX in the current working directory; then just do
import importlib
pkg = importlib.import_module('pluginX')
# check if it's all there..
print(dir(pkg))
Use the imp module, or the more direct __import__() function.
You can use exec:
exec("import myapp.commands.%s" % command)
If you want it in your locals:
>>> mod = 'sys'
>>> locals()['my_module'] = __import__(mod)
>>> my_module.version
'2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit (Intel)]'
same would work with globals()
Similar as #monkut 's solution but reusable and error tolerant described here http://stamat.wordpress.com/dynamic-module-import-in-python/:
import os
import imp
def importFromURI(uri, absl):
mod = None
if not absl:
uri = os.path.normpath(os.path.join(os.path.dirname(__file__), uri))
path, fname = os.path.split(uri)
mname, ext = os.path.splitext(fname)
if os.path.exists(os.path.join(path,mname)+'.pyc'):
try:
return imp.load_compiled(mname, uri)
except:
pass
if os.path.exists(os.path.join(path,mname)+'.py'):
try:
return imp.load_source(mname, uri)
except:
pass
return mod
The below piece worked for me:
>>>import imp;
>>>fp, pathname, description = imp.find_module("/home/test_module");
>>>test_module = imp.load_module("test_module", fp, pathname, description);
>>>print test_module.print_hello();
if you want to import in shell-script:
python -c '<above entire code in one line>'
The following worked for me:
import sys, glob
sys.path.append('/home/marc/python/importtest/modus')
fl = glob.glob('modus/*.py')
modulist = []
adapters=[]
for i in range(len(fl)):
fl[i] = fl[i].split('/')[1]
fl[i] = fl[i][0:(len(fl[i])-3)]
modulist.append(getattr(__import__(fl[i]),fl[i]))
adapters.append(modulist[i]())
It loads modules from the folder 'modus'. The modules have a single class with the same name as the module name. E.g. the file modus/modu1.py contains:
class modu1():
def __init__(self):
self.x=1
print self.x
The result is a list of dynamically loaded classes "adapters".
import v_framework as framework
framework.loadModules(["Maintenance"])
framework.Maintenance.showPage()
In framework I have:
def loadModules(aModules):
d_utility = {"Maintenance":"COOl_M_PAGE"}
for module in a_aModules:
exec("import " + d_utility[module] + " as " + module)
When loadModules is executed, it imports the modules in the v_framework namespace. Since I am importing v_framework as framework, I think I should be able to use the imported module using framework.Maintenance. But it does not work that way.
Is there a way to do way to do what I'm trying to do? Alternatively, is there any way to import modules in a namespace other than the one where exec is executed?
There are libraries for importing modules dynamically. You could use importlib (and another one that might be useful is pkgutil). Now, for your case, I guess this would do the job:
import importlib
mods = {}
def loadModules(aModule):
global mods
mods[module] = importlib.import_module(d_utility[module])
# or maybe globals()[module] = ... would work also (exactly as you expect it to
UPDATE: exec modifies the function's local namespace, not the global one (I think).
Hope it helps. :)
When you import inside a function, the module is imported/executed as normal, but the name you import under is local to the function, just like any other variable assigned inside a function.
>>> def test_import():
... import os
... print os
...
>>> test_import()
<module 'os' from '/usr/lib/python2.7/os.pyc'>
>>> os
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'os' is not defined
os has been imported though, and you can still access it through sys.modules:
>>> import sys
>>> sys.modules['os']
<module 'os' from '/usr/lib/python2.7/os.pyc'>
>>> os = sys.modules['os']
>>> os
<module 'os' from '/usr/lib/python2.7/os.pyc'>
A quick and dirty way to do what you want would be something like this; exec takes an optional mapping to be used as the local and global variables. So you could do
def loadModules(aModules):
d_utility = {"Maintenance":"COOl_M_PAGE"}
for module in aModules:
exec ('import %s as %s' % (d_utility[module], module)) in globals()
Though this is ugly and probably has security implications or something. As jadkik94 mentions, there are libraries that provide cleaner ways to deal with this.