I want to write a function which takes the name of a variable, a file name, and a third string, and tries to import the given variable from the file and if it can not do that, it sets the variable to the third string. Let me show you. This is in my config.py:
variable = 'value'
This is my function (it doesn't work):
#!/usr/bin/python
def importvar (var, fname, notfound) :
try:
from fname import var
except:
var = notfound
return var;
value = importvar ('variable', 'config', 'value not found')
print value #prints 'value not found'
This is what I am trying to achieve:
from config import variable
print variable #prints 'value'
This question is similar to "How to use a variable name as a variable in python?", but the answers I found to those didn't seem to work for me. I don't necessarily need to store them in a variable, but I couldn't come up with anything better. I know this is a perfect example of "What you shouldn't do in python", but I still need this. Thanks for the help!
What you want to do is dynamically importing a module starting from a string describing the path of the module. You can do this by using import_module from the importlib package.
import importlib
def importvar (var, fname, notfound) :
try:
return getattr(importlib.import_module(fname), var)
except:
return notfound
This should give you the clue:
>>> from importlib import import_module
>>> config = import_module('config')
>>> print( getattr(config, 'variable') )
value
See the docs for getattr.
Basically, getattr(x, 'variable') is equivalent to x.variable
a function for import & return imported variable:
def importvar (var, fname, notfound):
try:
exec('from {f} import {v}'.format(f=fname, v=var))
return locals().get(var, notfound)
except:
return notfound
If you just want a simple import from a string, the __import__ builtin may be good enough. It takes the module name as a string and returns it. If you also need to get an attribute from it programmatically use the builtin getattr, which takes the attribute name as a string.
If you're trying to import a package submodule, though, importlib.import_module is easier--you can import a name with a dot in it and get the module directly. This just calls __import__ for you. Compare __import__("logging.config").config vs import_module("logging.config").
If you're trying to import an arbitrary file not on the Python path, it gets a little more involved. The Python docs have a recipe for this.
import importlib.util
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
Unlike __import__, this doesn't add the module to the cache, because it doesn't have a canonical import name. But you can add it yourself (using whatever name you want) if you want to import it normally later, e.g.
import sys
sys.modules["foo_module"] = module
After running this, it allows you to get the same module instance again with a simple
import foo_module
Related
Let's say I have a number in Python project and I want to find out in what package/module a particular class is defined (I know the class name). Is there an efficient way to do it?
You can use the inspect module to get the location where a module/package is defined.
inspect.getmodule(my_class)
Sample Output:
<module 'module_name' from '/path/to/my/module.py'>
As per the docs,
inspect.getmodule(object)
Try to guess which module an object was defined in.
Let's explain by example
import numpy
print numpy.__file__
gives
/usr/lib64/python2.7/site-packages/numpy/__init__.pyc
on my machine.
If you only have a class, you can use that class with the python2-ic imp module:
#let unknownclass be looked for
import imp
modulename = unknownclass.__module__
tup = imp.find_module(modulename)
#(None, '/usr/lib64/python2.7/site-packages/numpy', ('', '', 5))
print "path to module", modulename, ":", tup[1]
#path to module numpy : /usr/lib64/python2.7/site-packages/numpy
As you can see, the __module__ property is probably what you're looking for.
One way to get the location of a class is using a combination of the __module__ attribute of a class and the sys.modules dictionary.
Example:
import sys
from stl import stl
module = sys.modules[stl.BaseStl.__module__]
print module.__file__
I should note that this doesn't always give the correct module, it just gives the module where you got it from. A more thorough (but slower) method is using the inspect module:
from stl import stl
import inspect
print inspect.getmodule(stl.BaseStl).__file__
My_Module=input()
My_Method=input()
from My_Module import My_Method
I'm writing a program and I need to import a method first.
But when I type what I did in above it sends an error and says can't "import name My_Method from My_Module".
I tried 2 other different codes but they didn't work either.I know that My_Method is a name and can't be imported but what can I do to solve this problem and have it worked?
You need to use __import__ to get the module in a dynamic way (using a variable content as name)
Then, you need to use getattr to get the function within the module.
import importlib
modulename = input('module ')
funcname = input('func ')
mod = importlib.import_module(modulename)
func = getattr(mod, funcname)
print(func)
# call the function
func()
Thx to Patrick Haugh and his comment
I have a system that collects all classes that derive from certain base classes and stores them in a dictionary. I want to avoid having to specify which classes are available (I would like to discover them programatically), so have used a from ModuleName import * statement. The user is then directed to place all tests to be collected in the ModuleName module. However, I cannot find a way to programatically determine what symbols were imported with that import statement. I have tried using dir() and __dict__ as indicated in the following example, but to no avail. How does one programatically find symbols imported in this manner (with import *)? I am unable to find them with the above methods.
testTypeFigureOuterrer.py:
from testType1 import *
from testType2 import *
class TestFigureOuterrer(object):
def __init__(self):
self.existingTests = {'type1':{},'type2':{}}
def findAndSortTests(self):
for symbol in dir(): # Also tried: dir(self) and __dict__
try:
thing = self.__getattribute__(symbol)
except AttributeError:
continue
if issubclass(thing,TestType1):
self.existingTests['type1'].update( dict(symbol,thing) )
elif issubclass(thing,TestType3):
self.existingTests['type2'].update( dict(symbol,thing) )
else:
continue
if __name__ == "__main__":
testFigureOuterrer = TestFigureOuterrer()
testFigureOuterrer.findAndSortTests()
testType1.py:
class TestType1(object):
pass
class TestA(TestType1):
pass
class TestB(TestType1):
pass
testType2.py:
class TestType2:
pass
class TestC(TestType2):
pass
class TestD(TestType2):
pass
Since you know the imports yourself, you should just import the module manually again, and then check the contents of the module. If an __all__ property is defined, its contents are imported as names when you do from module import *. Otherwise, just use all its members:
def getImportedNames (module):
names = module.__all__ if hasattr(module, '__all__') else dir(module)
return [name for name in names if not name.startswith('_')]
This has the benefit that you do not need to go through the globals, and filter everything out. And since you know the modules you import from at design time, you can also check them directly.
from testType1 import *
from testType2 import *
import testType1, testType2
print(getImportedNames(testType1))
print(getImportedNames(testType2))
Alternatively, you can also look up the module by its module name from sys.modules, so you don’t actually need the extra import:
import sys
def getImportedNames (moduleName):
module = sys.modules[moduleName]
names = module.__all__ if hasattr(module, '__all__') else dir(module)
return [name for name in names if not name.startswith('_')]
print(getImportedNames('testType1'))
print(getImportedNames('testType2'))
Take a look at this SO answer, which describes how to determine the name of loaded classes, you can get the name of all the classes defined within the context of the module.
import sys, inspect
clsmembers = inspect.getmembers(sys.modules['testType1'], inspect.isclass)
which is now defined as
[('TestA', testType1.TestA),
('TestB', testType1.TestB),
('TestType1', testType1.TestType1)]
You can also replace testType1 with __name__ when you're within the function of interest.
Don't use the * form of import. This dumps the imported names into your script's global namespace. Not only could they clobber some important bit of data by using the same name, you don't have any easy way to fish out the names you just imported. (Easiest way is probably to take a snapshot of globals().keys() before and after.)
Instead, import just the module:
import testType1
import testType2
Now you can easily get a list of what's in each module:
tests = dir(testType1)
And access each using getattr() on the module object:
for testname in tests:
test = getattr(testType1, testname)
if callable(test):
# do something with it
How do I do from some.module import * where name of the module is defined in a string variable?
This code imports all symbols from os:
import importlib
# Load the module as `module'
module = importlib.import_module("os")
# Now extract the attributes into the locals() namespace, as `from ..
# import *' would do
if hasattr(module, "__all__"):
# A module can define __all__ to explicitly define which names
# are imported by the `.. import *' statement
attrs = { key: getattr(module, key) for key in module.__all__ }
else:
# Otherwise, the statement imports all names that do not start with
# an underscore
attrs = { key: value for key, value in module.__dict__.items() if
key[0] != "_" }
# Copy the attibutes into the locals() namespace
locals().update(attrs)
See e.g. this question for more information on the logic behind the from ... import * operation.
Now while this works, you should not use this code. It is already considered bad practice to import all symbols from a named module, but it certainly is even worse to do this with a user-given name. Search for PHP's register_globals if you need a hint for what could go wrong.
In Python the built-in import function accomplishes the same goal as using the import statement, but it's an actual function, and it takes a string as an argument.
sys = __import__('sys')
The variable sys is now the sys module, just as if you had said import sys.
Reference
I am trying to dynamically import a configuration file in Python.
My code works fine when I use:
import conf.config as config
but doesn't work correctly when I use:
config = __import__("conf.config")
Below are to sample programs and the results I get when running them:
#regularimport.py
import conf.config as config
def read_values(cfg):
for varname in cfg.__dict__.keys():
if varname.startswith('__'):
continue
value = getattr(cfg, varname)
yield (varname, value)
for name,value in read_values(config):
print "Current config: %s = %s" % (name, value)
Results:
$python regularimport.py
Current config: SETTING_TWO = another setting
Current config: SETTING_ONE = some setting
Dynamic import:
#dynamicimport.py
conf_str = "conf.config"
config = __import__(conf_str)
def read_values(cfg):
for varname in cfg.__dict__.keys():
if varname.startswith('__'):
continue
value = getattr(cfg, varname)
yield (varname, value)
for name,value in read_values(config):
print "Current config: %s = %s" % (name, value)
Results:
$ python dynamicimport.py
Current config: config = <module 'conf.config' from '/home/ubuntu/importex/conf/config.pyc'>
My question is why the difference? And more importantly, how can I make the dynamic import example work as it does with the regular import?
As the documentation explains:
When the name variable is of the form package.module, normally, the top-level package (the name up till the first dot) is returned, not the module named by name.
So, when you do this:
config = __import__("conf.config")
That's not the same as:
import conf.config as config
But rather something more like:
import conf.config
config = conf
Why?
Because conf, not conf.config, is the thing that gets bound by an import statement. (Sure, in import foo as bar, obviously bar gets bound… but __import__ isn't meant to be an equivalent of import foo as bar, but of import foo.) The docs explain further. But the upshot is that you probably shouldn't be using __import__ in the first place.
At the very top of the function documentation it says:
Note: This is an advanced function that is not needed in everyday Python programming, unlike importlib.import_module().
And at the bottom, after explaining how __import__ works with packages and why it works that way, it says:
If you simply want to import a module (potentially within a package) by name, use importlib.import_module().
So, as you might guess, the simple solution is to use importlib.import_module.
If you have to use Python 2.6, where importlib doesn't exist… well, there just is no easy solution. You can build something like import_module yourself out of imp. Or use __import__ and then dig through sys.modules. Or __import__ each piece and then getattr your way through the results. Or in various other hacky ways. And yes, that sucks—which is why 3.0 and 2.7 fixed it.
The 2.6 docs give an example of the second hack. Adapting it to your case:
__import__("conf.config")
config = sys.modules["conf.config"]
config = __import__("conf.config") is not equivalent to import conf.config as config.
For example:
>>> import os.path as path
>>> path
<module 'posixpath' from '/usr/lib/python2.7/posixpath.pyc'>
>>> __import__('os.path')
<module 'os' from '/usr/lib/python2.7/os.pyc'>
Instead of __import__ use importlib.import_module to get the subpackage / submodule.
>>> import importlib
>>> importlib.import_module('os.path')
<module 'posixpath' from '/usr/lib/python2.7/posixpath.pyc'>