I have the following file structure:
A:
|_ a.py
|_ b.py
B:
|_ a.py
|_ b.py
I want to dynamically execute either A/b.py or B/b.py.
I am using the following code:
from importlib import import_module
path = '/home/username/test/' + module + '/'
if path not in sys.path:
sys.path.append(path)
script = import_module('b', 'Script')
myClass = getattr(script, 'Script')
run = myClass()
Doing this, if I run B/b.py and then A/b.py, it will execute B/b.py instead of A/b.py.
The first script to be run will be executed in the next round.
I need help in making sure the file in the directory I want is run only.
I'm making some assumption on what you want to accomplish here. Even if this is not exactly what you want, it might still push you the right direction: You got two different sub directories, A and B. These contain scripts of identical names a.py and b.py. Based on some condition, your script should call either A/a.py or A/a.py and then maybe B/b.py or B/b.py.
I would set up A and B as actual python modules, that is, create a __init__.py file in both folders. Then have a master-script which somehow determines which module to use..
# root_folder/master.py
import sys
import A
import B
master_script_name = sys.argv[0]
print("I'm the master script : " + str(master_script_name))
def choose_module_A_or_B(arg):
if arg == "A":
print(" You chose module A !")
return A
return B
module = choose_module_A_or_B("A")
module.b.print_locations()
Then,
# root_folder/A/__init__.py
from A import b
and,
# root_folder/A/b.py
import os
import sys
# how to obtain paths and script name:
folder = os.path.dirname(os.path.realpath(__file__))
script = __file__
parent = os.path.abspath(os.path.join(folder, os.pardir))
def print_locations():
print(" --> script : " + str(script))
print(" --> folder : " + str(folder))
print(" --> parent : " + str(parent))
Similarily ..
# root_folder/B/__init__.py
from B import b
and,
# root_folder/B/b.py
import os
import sys
# how to obtain paths and script name:
folder = os.path.dirname(os.path.realpath(__file__))
script = __file__
parent = os.path.abspath(os.path.join(folder, os.pardir))
def print_locations():
print(" --> script : " + str(script))
print(" --> folder : " + str(folder))
print(" --> parent : " + str(parent))
OUTPUT:
$ python master.py
I'm the master script : master.py
You chose module A !
--> script : A\b.py
--> folder : C:\dev\ScriptTesting\py\script_by_name\A
--> parent : C:\dev\ScriptTesting\py\script_by_name
I've read your other, similar question and come up with a solution without any pre-imports. This is (imho) highly un-pythonic, and may by all means be considered a dirty "hack". I highly recommend you to consider my other answer and just properly deal with the imports.
Your problem occurs because you're trashing the namespace, and all it holds dear. When you pollute the namespace with functions/methods with the same signature, there is absolutely no way for the Python-interpreter to distinguish them: it resolves to the one that was first imported.
However, as stated, there is a workaround: There is (currently) no way to unload a python module, but you may reload it, using the imp module. Essentially, it lets you clean up (redefine) the namespace. A complete, working example can be found at my repl.it
# root_folder/main.py
import sys
import imp
from importlib import import_module
def import_script(mod_dir, script):
sys.path.append(mod_dir)
mod = imp.reload(import_module(script, 'Script'))
sys.path.remove(mod_dir)
return mod
# input:
mod_dir = "A"
script = "b"
# import module/script.py
active_mod = import_script(mod_dir, script)
# use module/script.py
mod_name = active_mod.get_mod_name()
print(mod_name) # Prints "A : b.y"
# New input: different module/script.py
mod_dir = "C"
script = "b"
# import module/script.py
active_mod = import_script(mod_dir, script)
# use module/script.py
mod_name = active_mod.get_mod_name()
print(mod_name) # Prints "C : b.y"
when the modules look like below,
# root_folder/A/b.py
def get_mod_name():
return "A : b.py"
Do note that every import is doubled, since everytime you import a module (with possibly a duplicate name), it must also be reloaded to clean up the namespace. It is not enough to just del the module.
Related
I am using this approach to get the current directory of a file:
import pathlib
pathlib.Path(__file__).parent.resolve()
Is it possible to extract this string into a function, that will use the path of the caller? My attempt:
# a.py
import bb.b
print(bb.b.get_current_path(__file__)) # OK: path to a.py directory (.)
print(bb.b.get_current_path()) # WRONG: path to b.py directory (bb)
# bb/b.py
import pathlib
def get_current_path(file=__file__):
return pathlib.Path(file).parent.resolve()
Is it possible to avoid using __file__ in a.py?
You can use inspect module to inspect the stack. Though it satisfies the given condition
# bb/b.py
import inspect
import pathlib
stack = inspect.stack()
file_importer = stack[6][1]
def get_current_path(file=file_importer):
return pathlib.Path(file).parent.resolve()
I'm not sure if import will be always at the frame with the index 6. Search through code_context can be used instead:
file_importer = next(frameinfo[1] for frameinfo in stack
if frameinfo[4] and frameinfo[4][0].startswith('import'))
But this approach breaks the possibility to run b.py, so that exception handling of StopIteration is required with desired behaviour.
Both approaches return the direct importer of b.py, e.g., having a file c.py and calling it from a.py returns the path of c.py
# c.py
import bb.b
get_current_path = bb.b.get_current_path
# a.py
import bb.c
print(bb.c.get_current_path(__file__)) # OK: path to a.py directory (.)
print(bb.c.get_current_path()) # WRONG: path to c.py directory (bb)
Hence, depending on further conditions desired behaviour could be reached by processing inspect.stack().
For an example: there are 2 files
a.py
import b
...
b.py
print('???') # here I want to figure out which module called me
# this module could be imported from different modules/places not even from expected places.
# there is no certain purpose to use such information, just try to find out is it possible.
Question:
Is it possible to figure out name of module which make fisrt import on another module?
PS:
Import patching like here is not an option. b.py could be imported in 3rd party modules separately.
Add the below code in your b.py file then you will get the name of the file which is importing b.
import sys, os
def getCalledModuleName():
try:
sFile = os.path.abspath(sys.modules['__main__'].__file__)
names = sFile.split('\\')
length = len(names)
name = names[length-1]
except:
sFile = sys.executable
names = sFile.split('\\')
length = len(names)
name = names[length - 1]
return name
if __name__=='__main__':
print('Main Function Called')
else:
print('Imported as Module')
print(getCalledModuleName())
Problem
I have a folder structure like this:
- modules
- root
- abc
hello.py
__init__.py
- xyz
hi.py
__init__.py
blah.py
__init__.py
foo.py
bar.py
__init_.py
Here is the same thing in string format:
"modules",
"modues/__init__.py",
"modules/foo.py",
"modules/bar.py",
"modules/root",
"modules/root/__init__.py",
"modules/root/blah,py",
"modules/root/abc",
"modules/root/abc/__init__.py",
"modules/root/abc/hello.py",
"modules/root/xyz",
"modules/root/xyz/__init__.py",
"modules/root/xyz/hi.py"
I am trying to print out all the modules in the python import style format.
An example output would like this:
modules.foo
modules.bar
modules.root.blah
modules.root.abc.hello
modules.root.xyz.hi
How can I do this is in python(if possible without third party libraries) easily?
What I tried
Sample Code
import pkgutil
import modules
absolute_modules = []
def find_modules(module_path):
for package in pkgutil.walk_packages(module_path):
print(package)
if package.ispkg:
find_modules([package.name])
else:
absolute_modules.append(package.name)
if __name__ == "__main__":
find_modules(modules.__path__)
for module in absolute_modules:
print(module)
However, this code will only print out 'foo' and 'bar'. But not 'root' and it's sub packages. I'm also having trouble figuring out how to convert this to preserve it's absolute import style. The current code only gets the package/module name and not the actual absolute import.
This uses setuptools.find_packages (for the packages) and pkgutil.iter_modules for their submodules. Python2 is supported as well. No need for recursion, it's all handled by these two functions used together.
import sys
from setuptools import find_packages
from pkgutil import iter_modules
def find_modules(path):
modules = set()
for pkg in find_packages(path):
modules.add(pkg)
pkgpath = path + '/' + pkg.replace('.', '/')
if sys.version_info.major == 2 or (sys.version_info.major == 3 and sys.version_info.minor < 6):
for _, name, ispkg in iter_modules([pkgpath]):
if not ispkg:
modules.add(pkg + '.' + name)
else:
for info in iter_modules([pkgpath]):
if not info.ispkg:
modules.add(pkg + '.' + info.name)
return modules
So I finally figured out how to do this cleanly and get pkgutil to take care of all the edge case for you. This code was based off python's help() function which only displays top level modules and packages.
import importlib
import pkgutil
import sys
import modules
def find_abs_modules(module):
path_list = []
spec_list = []
for importer, modname, ispkg in pkgutil.walk_packages(module.__path__):
import_path = f"{module.__name__}.{modname}"
if ispkg:
spec = pkgutil._get_spec(importer, modname)
importlib._bootstrap._load(spec)
spec_list.append(spec)
else:
path_list.append(import_path)
for spec in spec_list:
del sys.modules[spec.name]
return path_list
if __name__ == "__main__":
print(sys.modules)
print(find_abs_modules(modules))
print(sys.modules)
This will work even for builtin packages.
The below code will give you the relative package module from the codes current working directory.
import os
import re
for root,dirname,filename in os.walk(os.getcwd()):
pth_build=""
if os.path.isfile(root+"/__init__.py"):
for i in filename:
if i <> "__init__.py" and i <> "__init__.pyc":
if i.split('.')[1] == "py":
slot = list(set(root.split('\\')) -set(os.getcwd().split('\\')))
pth_build = slot[0]
del slot[0]
for j in slot:
pth_build = pth_build+"."+j
print pth_build +"."+ i.split('.')[0]
This code will display:
modules.foo
modules.bar
modules.root.blah
modules.root.abc.hello
modules.root.xyz.hi
If you run it outside the modules folder.
This is not a problem with my code, but rather a general question about Python 3.
Say you had a game, which had 4 parts to it, and the first part (main.py) declares a variable
that say, part 2 needs to run itself. Would you be able to declare that variable, then import part2 (That needs the variable to run smoothly) and have the variable carry on from main.py to part2.py after importing part2.py into main.py.
If you want to use the variable once you can do this.
# part2.py
def scream():
print(sound)
# part1.py
import part2
if __name__=="__main__":
part2.sound = "Yoooo"
part2.scream()
#Output:
Yoooo
If you want to be able to change the variable later. You can create a property
Or simply do this:
# part2.py
# gvars is defined later
def scream():
print(gvars.sound)
# part1.py
import part2
class GameVariables:
pass
if __name__=="__main__":
gvars = GameVariables()
part2.gvars = gvars
gvars.sound = "Yooo"
part2.scream()
gvars.sound = "Whaa"
part2.scream()
#output
Yooo
Whaa
here is a simple example you can try out
file1.py
import sys
sys.argv = "Yellow" #probably not a really great Idea but should suffice
from file2 import sys as sys2
print "Imported argv from file2",sys2.argv
print "Same thing? ", sys is sys2
file2.py
import sys
print "My argv f2:",sys.argv
This question already has answers here:
How to import a module in Python with importlib.import_module
(3 answers)
Closed 8 years ago.
I have the following directory structure
+ code
|
--+ plugins
|
-- __init__.py
-- test_plugin.py (has a class TestPlugin)
-- another_test_plugin.py (has a class AnotherTestPlugin)
--+ load.py
--+ __init__.py
In load.py, I want to be able to initialize only those classes that the user specifies. For example, lets say I do something like
$ python load.py -c test_plugin # Should only import test_plugin.py and initialize an object of the TestPlugin class
I am having trouble trying to use the "imp" module to do it. It keeps on saying "No such file or directory". My understanding is that it is somehow not understanding the path properly. Can someone help me out with this?
ok, your problem is a path related problem. You expect that the script is being run in the same directory as where load.py is, where it is not the case.
what you have to do is something like:
import imp, os, plugins
path = os.path.dirname(plugins.__file__)
imp.load_source('TestPlugin', os.path.join(path, 'test_plugin.py')
where plugins is the module containing all your plugins (i.e. just the empty __init__.py), that will help you get the full path to your plugin modules' files.
Another solution, if you want a "plugins" discovery tool:
import imp, os
import glob
def load_plugins(path):
"""
Assuming `path` is the only directory in which you store your plugins,
and assuming each name follows the syntax:
plugin_file.py -> PluginFile
Please note that we don't import files starting with an underscore.
"""
plugins = {}
plugin_files = glob.glob(path + os.sep + r'[!_]*.py')
for plugin_path in plugin_files:
module_name, ext = os.path.splitext(plugin_path)
module_name = os.path.basename(module_name)
class_name = module_name.title().replace('_', '')
loaded_module = imp.load_source(class_name, plugin_path) # we import the plugin
plugins[module_name] = getattr(loaded_module, class_name)
return plugins
plugins = load_plugins(your_path_here)
plugin_name = sys.argv[3]
plugin = plugins.get(plugin_name)
if not plugin:
# manage a not existing plugin
else:
plugin_instance = plugin() # creates an instance of your plugin
This way, you can also specify different names by changing your keys, e.g., 'test_plugins' => 'tp'. You don't have to initialize your plugins, but you can still run this function whenever you want to load your plugins at runtime.
exec('import ' + sys.argv[2])
obj = test_plugin.TestPlugin()
Here sys.argv[2] is 'test_plugin' string from command line arguments.
EDIT: Another way to avoid using exec:
import importlib
mod = importlib.import_module(sys.argv[2])