I am trying to import modules dynamically in Python. Right now, I have a directory called 'modules' with two files inside; they are mod1.py and mod2.py. They are simple test functions to return time (ie. mod1.what_time('now') returns the current time).
From my main application, I can import as follows :
sys.path.append('/Users/dxg/import_test/modules')
import mod1
Then execute :
mod1.what_time('now')
and it works.
I am not always going to know what modules are available in the dirctory. I wanted to import as follows :
tree = []
tree = os.listdir('modules')
sys.path.append('/Users/dxg/import_test/modules')
for i in tree:
import i
However I get the error :
ImportError: No module named i
What am I missing?
The import instruction does not work with variable contents (as strings) (see extended explanation here), but with file names. If you want to import dynamically, you can use the importlib.import_module method:
import importlib
tree = os.listdir('modules')
...
for i in tree:
importlib.import_module(i)
Note:
You can not import from a directory where the modules are not included under Lib or the current directory like that (adding the directory to the path won't help, see previous link for why). The simplest solution would be to make this directory (modules) a package (just drop an empty __init__.py file there), and call importlib.import_module('..' + i, 'modules.subpkg') or use the __import__ method.
You might also review this question. It discusses a similar situation.
You can achieve something like what you are proposing, but it will involve some un-pythonic code. I do not recommend doing this:
dynamic_imports = dict()
for filename in tree:
name = filename.replace('.py', '')
dynamic_imports[name] = __import__(name)
Related
I've got one source code which is compiled into two QGis plugins, on this model:
SourceCode -> build.py -> * MyPlugin.zip
* MyPluginExtended.zip
Since the cleanest way to import modules in QGis is to import them from the plugins directory (i.e. the parent directory), all the import statements in the MyPlugin code look like:
import MyPlugin.foo
import MyPlugin.foo.bar
from MyPlugin import foo`
But the problem is, the same import statements will be duplicated from the source code in the MyPluginExtended code, causing ModuleNotFound errors.
The QGis system prevents me to put my final code in a subdirectory that I would have then added to the sys.path, like this:
[Plugins directory] / MyPluginExtended / MyPlugin / [my files]
So the only solutions I could think of were:
1. Override the import system to dynamically replacing MyPlugin in every search path by MyPluginExtended
I tried to setup a custom importer, like proposed in the importlib documentation, doing something like this in my __init__ file, but none of my attempts worked:
try:
import MyPluginExtended
# this is the extended plugin, imports need to be overriden
class MyPathFinder():
# when imports contains 'MyPlugin', change the search path from `path/to/MyPlugin` to `path/to/MyPluginextended`
sys.meta_path.append(MyPathFinder)
except ImportError:
# this is the basic plugin, nothing to do
pass
2. Manually create aliases in sys.modules, like this:
import MyPlugin
sys.modules['MyPluginExtended'] = MyPlugin
But I would then have to do it for every submodule, and I don't knwo how to discover them programatically, something like:
for module_fullname, module in available_submodules():
module_fullname = module_fullname.replace('MyPlugin/', 'MyPluginExtended/')
sys.modules[module_fullname] = module
There are several posts around this error I have already read, but I still don't get what I am doing wrong.
I put it into a minimal example:
Imagine I have a Doc.py, and the package Tools which includes Tool1.py and Tool2.py.
Doc.py:
from Tools import *
import sys
def __main__():
TOOL_REPORT("Tool1","Test")
def TOOL_REPORT(tool, path):
if(tool == 'Tool1'):
Tool1.REPORT(path)
elif(tool == 'Tool2'):
Tool2.REPORT(path)
else:
sys.stderr.write("This tool is not yet included in Doc. Please check TOOLS for more information.")
if __name__=="__main__": __main__()
Tool1.py:
def REPORT(path):
print("Tool1 "+path)
Tool2.py:
def REPORT(path):
print("Tool2 "+path)
If I run this, I always end up with this error:
File "Doc.py", line 15, in TOOL_REPORT
Tool1.REPORT(path)
NameError: global name 'Tool1' is not defined
I'd appreciate any hint to what is going wrong!
Your Tool1 and Tool2 submodules are not visible until explicitly imported somewhere.
You can import them in the Tools/__init__.py package file:
import Tool1, Tool2
at which point they become available for import from Tools.
Another option is to import the modules from your own code:
import Tools.Tool1, Tools.Tool2
from Tools import *
Only when explicitly imported are submodules also set as attributes of the package.
Python will treat any folder as a module when there is __init__.py file present in it. Otherwise it will just be another folder for python and not a module from which it can import things. So just add init.py file in your Tool folder (so it will become module in pythonic terms) and then you can import that module in other python scripts.
One more things for better practice instead of using
from Tools import *
Always provide the file name of library specifically which you want to import like in your case you should use it like this
from Tools import Tool1, Tool2
This will enhance the code readbility for others and for you too.
I have two codefiles in python, let's say mainfile.py and separatecode.py. I would like to run separatecode.py from within mainfile.py, referencing the specific directory where separatecode.py is stored.
So the pseudocode of what I am looking to do would be something like:
import C:\Users\Jack\Documents\MyFolder\separatecode.py
A number of questions discuss importing, but I can't find one that discusses importing a specific file you wrote in a particular directory. I would like to be able to use functions defined in separatecode.py and am looking for the equivalent of the source("separatecode.r") command in R if that helps.
Add that directory to your sys.path by doing:
import sys
sys.path.append('/directory/to/my/file')
Import the module as normal:
import separatecode
The code you used will not work. It will give a syntax error.
Look at the documentation for the import statement, especially the grammar. A module is one or more identifiers separated by dots.
You can make import separatecode work by adding C:\Users\Jack\Documents\MyFolder to the PYTHONPATH environment variable. This will make it available to all Python scripts. Or you can add that path to sys.path in mainfile.py before importing separatecode.
Try something like this:
import imp
foo = imp.load_source('Module_name', 'Path\To\module.py')
foo.MyClass()
you dont need the foo.MyClass() it was just an example to show that the module works like any other module
no you can import a module from anywhere using the path and its name and you can acsess all its functions etc
for anything else check out:
Python Imp
I hope this is what you were looking for
I have a program that creates a module called "cool" using file operations. I later say import cool and then uses the cool module that was created.
Here is my directory
project/
main.py
modules/
maker.py
cool/ #this folder and its contents was made by maker.py
__init__.py
coolm.py
If I want to make my program into the .exe format, this strategy will not work anymore. Does anyone know another technique?
Note: I cannot use exec to use the cool module..
Import your module when you need it like this:
coolmod = __import__('coolm')
coolm.someproperty
Alternatively you could try:
import importlib
coolmod = importlib.import_module('coolm', 'cool')
This allows you to specify the package name as a second argument.
I'm using python and virtualenv/pip. I have a module installed via pip called test_utils (it's django-test-utils). Inside one of my django apps, I want to import that module. However I also have another file test_utils.py in the same directory. If I go import test_utils, then it will import this local file.
Is it possible to make python use a non-local / non-relative / global import? I suppose I can just rename my test_utils.py, but I'm curious.
You can switch the search order by changing sys.path:
del sys.path[0]
sys.path.append('')
This puts the current directory after the system search path, so local files won't shadow standard modules.
My problem was even more elaborate:
importing a global/site-packages module from a file with the same name
Working on aero the pm recycler I wanted access to the pip api, in particular pip.commands.search.SearchCommand from my adapter class Pip in source file pip.py.
In this case trying to modify sys.path is useless, I even went as far as wiping sys.path completely and adding the folder .../site-packages/pip...egg/ as the only item in sys.path and no luck.
I would still get:
print pip.__package__
# 'aero.adapters'
I found two options that did eventually work for me, they should work equally well for you:
using __builtin__.__import__() the built-in function
global_pip = __import__('pip.commands.search', {}, {}, ['SearchCommand'], -1)
SearchCommand = global_pip.SearchCommand
Reading the documentation though, suggests using the following method instead.
using importlib.import_module() the __import__ conv wrapper.
The documentation explains that import_module() is a minor subset of functionality from Python 3.1 to help ease transitioning from 2.7 to 3.1
from importlib import import_module
SearchCommand = import_module('pip.commands.search').SearchCommand
Both options get the job done while import_module() definitely feels more Pythonic if you ask me, would you agree?
nJoy!
I was able to force python to import the global one with
from __future__ import absolute_import
at the beginning of the file (this is the default in python 3.0)
You could reset your sys.path:
import sys
first = sys.path[0]
sys.path = sys.path[1:]
import test_utils
sys.path = first + sys.path
The first entry of sys.path is "always" (as in "per default": See python docs) the current directory, so if you remove it you will do a global import.
Since my test_utils was in a django project, I was able to go from ..test_utils import ... to import the global one.
Though, in first place, I would always consider keeping the name of local file not matching with any global module name, an easy workaround, without modifying 'sys.path' can be to include global module in some other file and then import this global module from that file.
Remember, this file must be in some other folder then in the folder where file with name matching with global module is.
For example.
./project/root/workarounds/global_imports.py
import test_utils as tutil
and then in
./project/root/mycode/test_utils.py
from project.root.workarounds.global_imports import tutil
# tutil is global test_utils
# you can also do
from project.root.workarounds.global_imports import test_utils