I need to import and run a script from a string (path) which is located in another folder. The input needs to be completely dynamic. The code below works when the file is in the same folder but I can't seem to get it working when the file is located elsewhere.
main.py
path = 'bin\TestScript'
module = __import__(path)
my_class = getattr(module, '__main__')
instance = my_class(3,16)
print(instance)
TestScript.py
def __main__(a,b):
return(a*b)
Get the errror:
ImportError: No module named 'bin\\TestScript'
on windows os
You need to separate the directory from the module name and add that to the module search path. For example:
import os.path
import sys
path = 'bin\\TestScript'
mdir = os.path.dirname(path)
modname = os.path.basename(path)
sys.path.append(mdir)
module = __import__(modname)
my_class = getattr(module, '__main__')
instance = my_class(3,16)
print(instance)
An alternative is to make the directory "bin" a package.
Related
I define parameters for my programs in a python module for simplicity. These parameters are then loaded using import. Therefore, I have to make sure to always load from the working directory, and nowhere else (independent form the location of the executed script or available modules in python path).
I found two solutions. First modifying the path:
import sys
from os import getcwd
import_path = sys.path
sys.path = [str(getcwd(), ]
import xxx
sys.path = import_path
or using importlib
from pathlib import Path
from importlib.util import module_from_spec, spec_from_file_location
spec = spec_from_file_location('xxx', str(Path('.').expanduser()/'xxx.py'))
xxx = module_from_spec(spec)
spec.loader.exec_module(xxx)
Of course this can be wrapped into a context manager or a function respectively.
What would the pythonic way be to do this? Do those two approaches have advantages and disadvantages?
I checked How can I import a Python library located in the current working directory?
as well as Import python package from local directory into interpreter, but they lack the focus on robustness.
Local imports can be achieved by modifying the path. A context manager is a suitable solution:
import sys
from pathlib import Path
from contextlib import contextmanager
#contextmanager
def local_import(dir_=None):
"""Only import modules within `dir_` (default: cwd)."""
if dir_ is None:
dir_ = Path.cwd()
else:
dir_ = Path(dir_).absolute().resolve(strict=True)
import_path0 = sys.path[0]
sys.path[0] = str(dir_)
try:
yield
finally:
sys.path[0] = import_path0
Then the local import can be done using the standard import syntax
with local_import():
import xxx
This solution relys on the order, in which the paths are scanned, thus we temporarily replace sys.path[0]. We replace it, instead of prepending to avoid import conflicts with the script directory.
Note
You have to be careful to avoid name conflicts, as the import statement is used, modules with identical names will be imported only once. Thus if different modules with the same name exist in the working directory and in the original sys.path[0], only one of them will be imported. Thus, local_import should only be used for scripts, that only use the standard library or installed third party libraries, but not for scripts that import other scripts from the directory. For the unlikely case that you want to import different files with the same name, the following function can be used:
import uuid
from importlib.util import module_from_spec, spec_from_file_location
def import_file(file, content=None):
"""Try importing `file` as module avoiding name clashes.
If `content` is given `content = import_file('file.py', 'content')`
roughly corresponds to `from file import content`
else `file = import_file('file.py')`
roughly corresponds to `import file`.
Parameters
----------
file : str or Path
The Python file corresponding to the module.
content : str, optional
What to import from the module (optional).
"""
file = Path(file).expanduser().resolve(strict=True)
print(file)
spec = spec_from_file_location(file.stem + str(uuid.uuid4()), str(file))
module = module_from_spec(spec)
spec.loader.exec_module(module)
if content:
print(module)
return getattr(module, content)
else:
return module
My code starts with the following:
from module import *
How can I get a list of all submodules imported from this module.
For example module has:
module.submodule
module.submodule2
module.anothersubmodule
I want to get after from module import *:
[submodule, submodule2, anothersubmodule]
(not strings with name of submodules, but a list with all submodules themselves)
UPD: I understood that I asked about XY problem.
So here's what i'm trying to achieve:
I have a folder called modules that will have a bunch of scripts following the same pattern. They will all have a function like main(). In my main script i want to import them all and iterate like that:
for i in modules:
i.main(*some_args)
Use the pkgutil module. For example:
import pkgutil
import module
modualList = []
for importer, modname, ispkg in pkgutil.iter_modules(module.__path__):
modualList.append(modname)
What about importlib?
import importlib
import os
directory = './module'
for filename in os.listdir(directory):
filepath = os.path.join(directory, filename)
if not os.path.isfile(filepath):
continue
modulename = os.path.splitext(filename)[0]
args = [filename, filepath]
module = importlib.import_module('module.{}'.format(modulename))
module.main(*args)
With 3 differents Python files in ./modules, all like:
def main(*args):
print('module one: {}'.format(', '.join(args)))
It gives:
module three: three.py, ./module/three.py
module two: two.py, ./module/two.py
module one: one.py, ./module/one.py
So I've been trying to make a simple program that will dynamically import modules in a folder with a certain name. I cd with os to the directory and I run module = __import__(module_name) as I'm in a for loop with all of the files names described being iterated into the variable module_name.
My only problem is I get hit with a:
ImportError: No module named "module_name"
(saying the name of the variable I've given as a string). The file exists, it's in the directory mentioned and import works fine in the same directory. But normal even import doesn't work for modules in the cd directory. The code looks as follows. I'm sorry if this is an obvious question.
import os
class Book():
def __init__(self):
self.name = "Book of Imps"
self.moduleNames = []
# configure path
def initialize(self):
path = os.getcwd() + '/Imp-pit'
os.chdir(path)
cwd = os.walk(os.getcwd())
x, y, z = next(cwd)
# Build Modules
for name in z:
if name[:3] == 'Imp':
module_name = name[:len(name) - 3]
module = __import__(module_name)
def start_sim():
s = Book()
s.initialize()
if __name__ == '__main__':
start_sim()
I don't think the interpreter dynamically alters sys.path if you simply change the current directory with os.chdir. You'll manually have to insert the path variable into the sys.path list for this to work, i.e:
sys.path.insert(0, path)
Python generally searches sys.path when looking for modules, so it will find it if you specify it there.
An additional note; don't use __import__, rather use importlib.import_module. The interface is exactly the same but the second is generally advised in the documentation.
You should use the try-except concept, e.g.:
try:
import csv
except ImportError:
raise ImportError('<error message>')
If I did understand you correct then
try:
module = __import__(module_name)
except ImportError:
raise ImportError('<error message>')
When reading a python-based software, I feel confused about a line of python code: path = sys.modules[self.__class__.__module__].__file__.
I can guess it was trying to return the file name in class file, but I'm not very clear about the exact usage of this. I saved the related code segment into a file named test.py and I'm trying to test it by python test.py, but it does not print anything. How can I test this kind of file?
import os
import sys
class testloadfile:
def __init__(self, test_path=None):
if test_path is None:
path = sys.modules[self.__class__.__module__].__file__
# print path
path = os.path.abspath(os.path.join(path, os.pardir))
# print path
path = os.path.join(path, "test.r")
print(path)
test_path = path
print("r file loaded")
Classes in python have a __module__ attribute which contains the name of the module in which the class was defined. Additionally, each module contains a __file__ attribute which has the full path to the .py file.
He is trying to get the file path for the file in which the class was defined but he's not going the best way with doing it, ideally you could just index sys.modules by using __name__:
path = sys.modules[__name__].__file__
instead of going through the class (i.e self.__class__.__module__ == __name__). Do note here that if __name__ == "__main__" this will fail because the __main__ module does not have a __file__ attribute defined. You'll need to safeguard against it:
path = sys.modules[__name__].__file__ if __name__ == "__main__" else __file__
where if __name__ == "__main__" then __file__ will contain the path of the file being executed.
Next, add the usual clause in your script in order for it to initialize the object if the script is running as __main__:
if __name__ == "__main__":
testloadfile() # initialize
Now, if you call it as the __main__ script with:
python -m test.py
or if you import it, it will pickup the __file__ attribute, print it and then print the file name.
P.s: Fix the indentation in your final print.
Is there any way to get the current archive name in Python?
Something like
EggArchive.egg
Lib
---SomePythonFile.py
From SomePython.py, is there anyway to fetch the .egg name?
The variable __file__ contains the path to the current python file. So If you have a structure:
.
`- your.egg
`-your_module.py
And in your_module.py you have a function:
def func():
print(__file__)
The the code:
import sys
sys.path.append('/path/to/your.egg')
from your_module import func
func()
Will print out:
/path/to/your.egg/your_module.py
So basically you can manipulate the __file__ variable if you know where relatively your module is inside the egg file and get the full path to the egg.
To get the relative path to the egg in the script that's in the egg file you'll have to do:
def rel_to_egg(f):
my_dir = os.path.dirname(os.path.abspath(f))
current = my_dir
while not current.endswith('.egg'):
current = os.path.dirname(current)
return os.path.relpath(current, my_dir)
Now let's say __file__ == '/test/my.egg/some/dir/my_script.py':
>>> print(rel_to_egg(__file__))
'../..'