Suppose I have two nearly identical versions of a python package mymod, ie mymod0 and mymod1. Each of these packages has files init.py and foo.py, and foo.py has a single function printme(). Calling mymod0.foo.printme() will print "I am mymod0" and calling mymod1.foo.printme() will print "I am mymod1". So far so good.
But now I need to dynamically import either mymod0 or mymod1. The user will input either 0 or 1 to the script (as variable "index"), and then I can create packageName="mymod"+str(index)
I tried this:
module=importlib.import_module(module_name)
module.foo.printme()
But I get this error:
AttributeError: 'module' object has no attribute 'foo'
How can I specify the the package should now be referred to as module so that module.foo.printme() will work?
UPDATE: So it looks like the easiest solution is to use the exec() function. This way I can dynamically create an import statement like this:
cmdname="from mymod%s import foo" % index
exec(cmdname)
Then:
foo.printme()
This seems to work.
How can I specify the the package should now be referred to as module so that module.foo.printme() will work?
You have to make sure <module_name>.__init__ imports foo into the module's namespace:
#./<module_name>/__init__.py
import foo
then you can
module=importlib.import_module(module_name)
module.foo.printme()
But now I need to dynamically import either mymod0 or mymod1.
Note this only works the first time because python caches loaded modules. If the module has changed since that start of the program, use the reload function function. Just a word of caution: There are several caveats associated with this, and it may end up not doing what you intended.
How can I recreate this dynamically?
for i in range(0,2):
module_name = 'mymod%s' % i
module=importlib.import_module(module_name)
module.foo.printme()
Related
I am using https://github.com/sciunto/python-bibtexparser as a library module for my code(installed with pip3). So, I don't have writing access to the module.
But, the set it is using is rather limited, as defined in standard_types in https://github.com/sciunto/python-bibtexparser/blob/master/bibtexparser/bibdatabase.py.
Is it possible, that, I redefine/append the set from the calling code? e.g.
#foo.py
import bibtexparser
#redefine standard_types
bibtexparser.STANDARD_TYPES = set([the complete set I need])
#or append the standard_types
bibtexparser.STANDARD_TYPES.update = set([append to the set])
?
Update: Actually I cant access the variable STANDARD_TYPES. I am trying to do:
from bibtexparser import bibdatabase as bd
class Window(Gtk.ApplicationWindow):
def __init__(self, application, giofile=None):
Gtk.ApplicationWindow.__init__(self,
application=application,
default_width=1000,
default_height=200,
border_width=5)
print(bd)
print(bd.STANDARD_TYPES)
Which is yielding:
<module 'bibtexparser.bibdatabase' from '/usr/lib/python3.5/site-packages/bibtexparser/bibdatabase.py'>
......
print(bd.STANDARD_TYPES)
AttributeError: module 'bibtexparser.bibdatabase' has no attribute 'STANDARD_TYPES'
It looks like other parts of the package perform from bibtextparser import STANDARD_TYPES, so you can't just replace it; the modules that have already imported it will keep the old version. Using update would work fine though you need to fix your syntax:
bibtexparser.STANDARD_TYPES.update({append to the set})
That said, it looks like BibTexParser can be initialized passing it ignore_nonstandard_types=True and it won't even check STANDARD_TYPES, which is probably what you really want; it doesn't involve hacky monkey-patching at all.
I have bunch of modules to import and run. I have dynamically imported the modules using Dynamic module import in Python. This is in the main code. Once imported, I'm trying to run the functions in the modules.
All of the modules look like this structure
#function foo
def run(a,b)
c=a+b
return c
foo has been imported, I need to say something like bar=foo.run(a,b) dynamically
from this example:
How to call Python functions dynamically. I have already tried the following:
i='foo'
bar = getattr(sys.modules[__name__], i+'.run()')(a,b)
traceback AttributeError: 'module' object has no attribute 'foo.run()'
I'm confused, about the attribute error. The calling functions dynamically example is clearly calling functions.
If you have imported foo already, but don't have a reference to it, use:
sys.modules['foo'].run(a,b)
the_module.run(a, b)
Regardless of what magic made the module come into existence, it's an ordinary module object with ordinary attributes, and you know that the function is called run.
If you always know you'll use module foo, you're done.
You may also need to find the module object dynamically, because the module to choose varies.
If you imported the module properly, under the name you use to refer to it (e.g. foo) rather than some other name, you can also use sys.modules[mod_name].
Otherwise, you should probably have a dictionary of modules so that you can say, the_module = modules[mod_name].
I'm noticing some weird situations where tests like the following fail:
x = <a function from some module, passed around some big application for a while>
mod = __import__(x.__module__)
x_ref = getattr(mod, x.__name__)
assert x_ref is x # Fails
(Code like this appears in the pickle module)
I don't think I have any import hooks, reload calls, or sys.modules manipulation that would mess with python's normal import caching behavior.
Is there any other reason why a module would be loaded twice? I've seen claims about this (e.g, https://stackoverflow.com/a/10989692/1332492), but I haven't been able to reproduce it in a simple, isolated script.
I believe you misunderstood how __import__ works:
>>> from my_package import my_module
>>> my_module.function.__module__
'my_package.my_module'
>>> __import__(my_module.function.__module__)
<module 'my_package' from './my_package/__init__.py'>
From the documentation:
When the name variable is of the form package.module, normally, the
top-level package (the name up till the first dot) is returned, not
the module named by name. However, when a non-empty fromlist
argument is given, the module named by name is returned.
As you can see __import__ does not return the sub-module, but only the top package. If you have function also defined at package level you will indeed have different references to it.
If you want to just load a module you should use importlib.import_module instead of __import__.
As to answer you actual question: AFAIK there is no way to import the same module, with the same name, twice without messing around with the importing mechanism. However, you could have a submodule of a package that is also available in the sys.path, in this case you can import it twice using different names:
from some.package import submodule
import submodule as submodule2
print(submodule is submodule2) # False. They have *no* relationships.
This sometimes can cause problems with, e.g., pickle. If you pickle something referenced by submodule you cannot unpickle it using submodule2 as reference.
However this doesn't address the specific example you gave us, because using the __module__ attribute the import should return the correct module.
I am currently doing a python tutorial, but they use IDLE, and I opted to use the interpreter on terminal. So I had to find out how to import a module I created. At first I tried
import my_file
then I tried calling the function inside the module by itself, and it failed. I looked around and doing
my_file.function
works. I am very confused why this needs to be done if it was imported. Also, is there a way around it so that I can just call the function? Can anyone point me in the right direction. Thanks in advance.
If you wanted to use my_file.function by just calling function, try using the from keyword.
Instead of import my_file try from my_file import *.
You can also do this to only import parts of a module like so :
from my_file import function1, function2, class1
To avoid clashes in names, you can import things with a different name:
from my_file import function as awesomePythonFunction
EDIT:
Be careful with this, if you import two modules (myfile, myfile2) that both have the same function inside, function will will point to the function in whatever module you imported last. This could make interesting things happen if you are unaware of it.
This is a central concept to python. It uses namespaces (see the last line of import this). The idea is that with thousands of people writing many different modules, the likelihood of a name collision is reasonably high. For example, I write module foo which provides function baz and Joe Smith writes module bar which provides a function baz. My baz is not the same as Joe Smiths, so in order to differentiate the two, we put them in a namespace (foo and bar) so mine can be called by foo.baz() and Joe's can be called by bar.baz().
Of course, typing foo.baz() all the time gets annoying if you just want baz() and are sure that none of your other modules imported will provide any problems... That is why python provides the from foo import * syntax, or even from foo import baz to only import the function/object/constant baz (as others have already noted).
Note that things can get even more complex:
Assume you have a module foo which provides function bar and baz, below are a few ways to import and then call the functions contained inside foo...
import foo # >>> foo.bar();foo.baz()
import foo as bar # >>> bar.bar();bar.baz()
from foo import bar,baz # >>> bar(); baz()
from foo import * # >>> bar(); baz()
from foo import bar as cow # >>> cow() # This calls bar(), baz() is not available
...
A basic import statement is an assignment of the module object (everything's an object in Python) to the specified name. I mean this literally: you can use an import anywhere in your program you can assign a value to a variable, because they're the same thing. Behind the scenes, Python is calling a built-in function called __import__() to do the import, then returning the result and assigning it to the variable name you provided.
import foo
means "import module foo and assign it the name foo in my namespace. This is the same as:
foo = __import__("foo")
Similarly, you can do:
import foo as f
which means "import module foo and assign it the name f in my namespace." This is the same as:
f = __import__("foo")
Since in this case, you have only a reference to the module object, referring to things contained by the module requires attribute access: foo.bar etc.
You can also do from foo import bar. This creates a variable named bar in your namespace that points to the bar function in the foo module. It's syntactic sugar for:
bar = __import__("foo").bar
I don't really understand your confusion. You've imported the name my_file, not anything underneath it, so that's how you reference it.
If you want to import functions or classes inside a module directly, you can use:
from my_file import function
I'm going to incorporate many of the comments already posted.
To have access to function without having to refer to the module my_file, you can do one of the following:
from my_file import function
or
from my_file import *
For a more in-depth description of how modules work, I would refer to the documentation on python modules.
The first is the preferred solution, and the second is not recommended for many reasons:
It pollutes your namespace
It is not a good practice for maintainability (it becomes more difficult to find where specific names reside.
You typically don't know exactly what is imported
You can't use tools such as pyflakes to statically detect errors in your code
Python imports work differently than the #includes/imports in a static language like C or Java, in that python executes the statements in a module. Thus if two modules need to import a specific name (or *) out of each other, you can run into circular referencing problems, such as an ImportError when importing a specific name, or simply not getting the expected names defined (in the case you from ... import *). When you don't request specific names, you don't run into the, risk of having circular references, as long as the name is defined by the time you actually want to use it.
The from ... import * also doesn't guarantee you get everything. As stated in the documentation on python modules, a module can defined the __all__ name, and cause from ... import * statements to miss importing all of the subpackages, except those listed by __all__.
Is it only possible if I rename the file? Or is there a __module__ variable to the file to define what's its name?
If you really want to import the file 'oldname.py' with the statement 'import newname', there is a trick that makes it possible: Import the module somewhere with the old name, then inject it into sys.modules with the new name. Subsequent import statements will also find it under the new name. Code sample:
# this is in file 'oldname.py'
...module code...
Usage:
# inject the 'oldname' module with a new name
import oldname
import sys
sys.modules['newname'] = oldname
Now you can everywhere your module with import newname.
You can change the name used for a module when importing by using as:
import foo as bar
print bar.baz
Yes, you should rename the file. Best would be after you have done that to remove the oldname.pyc and oldname.pyo compiled files (if present) from your system, otherwise the module will be importable under the old name too.
When you do import module_name the Python interpreter looks for a file module_name.extension in PYTHONPATH. So there's no chaging that name without changing name of the file. But of course you can do:
import module_name as new_module_name
or even
import module_name.submodule.subsubmodule as short_name
Useful eg. for writing DB code.
import sqlite3 as sql
sql.whatever..
And then to switch eg. sqlite3 to pysqlite you just change the import line
Every class has an __module__ property, although I believe changing this will not change the namespace of the Class.
If it is possible, it would probably involve using setattr to insert the methods or class into the desired module, although you run the risk of making your code very confusing to your future peers.
Your best bet is to rename the file.
Where would you like to have this __module__ variable, so your original script knows what to import? Modules are recognized by file names and looked in paths defined in sys.path variable.
So, you have to rename the file, then remove the oldname.pyc, just to make sure everything works right.
I had an issue like this with bsddb. I was forced to install the bsddb3 module but hundreds of scripts imported bsddb. Instead of changing the import in all of them, I extracted the bsddb3 egg, and created a soft link in the site-packages directory so that both "bsddb" and "bsddb3" were one in the same to python.
You can set the module via module attribute like below.
func.__module__ = module
You can even create a decorator to change the module names for specific functions in a file for example:
def set_module(module):
"""Decorator for overriding __module__ on a function or class.
Example usage::
#set_module('numpy')
def example():
pass
assert example.__module__ == 'numpy'
"""
def decorator(func):
if module is not None:
func.__module__ = module
return func
return
and then use
#set_module('my_module')
def some_func(...):
Pay attention since this decorator is for changing individual
module names for functions.
This example is taken from numpy source code: https://github.com/numpy/numpy/blob/0721406ede8b983b8689d8b70556499fc2aea28a/numpy/core/numeric.py#L289