I want to dynamically import a list of modules. I'm having a problem doing this. Python always yells out an ImportError and tells me my module doesn't exist.
First I get the list of module filenames and chop off the ".py" suffixes, like so:
viable_plugins = filter(is_plugin, os.listdir(plugin_dir))
viable_plugins = map(lambda name: name[:-3], viable_plugins)
Then I os.chdir to the plugins directory and map __import__ the entire thing, like so:
active_plugins = map(__import__, viable_plugins)
However, when I turn active_plugins into a list and try to access the modules within, Python will throw out an error, saying it cannot import the modules since they don't appear to be there.
What am I doing wrong?
Edit: By simply using the interactive interpreter, doing os.chdir and __import__(modulefilename) produces exactly what I need. Why isn't the above approach working, then? Am I doing something wrong with Python's more functional parts?
It says it can't do it, because even though you're changing your directory to where the modules are, that directory isn't on your import path.
What you need to do, instead of changing to the directory where the modules are located, is to insert that directory into sys.path.
import sys
sys.path.insert(0, directory_of_modules)
# do imports here.
Related
I'm currently trying to import another .py file that's within a different directory, but am having some problems when it comes to doing so. I've tried various methods, and am currently attempting the method of adding the directory to the sys.path list. If I print this list once adding the directory, either absolute or relative, it appears in this list. However, my IDE (VSCode) still flags up saying the file cannot be found.
I'm aware that if I use VSCode's feature to add this to their 'extraPaths' list, it then works. However, I'm trying to avoid a dependency on my VS settings, thus avoid using this feature where possible.
Here is my file:
import sys
sys.path.append(r'c:\Users\Kiana\Documents\MyStuff\Home\Python\Pocket-Pet\game_scripts')
import pet
for p in sys.path:
print( p )
The output of the print statements (when 'import pet' is commented out):
My project structure:
I have tried both 'insert' and 'append' methods, as well as absolute and relative paths. Any ideas? Cheers
have you tried to use os.path.join('C:/', 'code', 'my-library') ? I'm not a windows user so I can't test it by myself
I use to import classes of programs relative to my script path like this
home_dir = os.path.dirname(os.path.realpath(__file__))
sys.path.append(os.path.join(home_dir, "/../"))
from folder.program import MyClass
When writing throwaway scripts it's often needed to load a configuration file, image, or some such thing from the same directory as the script. Preferably this should continue to work correctly regardless of the directory the script is executed from, so we may not want to simply rely on the current working directory.
Something like this works fine if defined within the same file you're using it from:
from os.path import abspath, dirname, join
def prepend_script_directory(s):
here = dirname(abspath(__file__))
return join(here, s)
It's not desirable to copy-paste or rewrite this same function into every module, but there's a problem: if you move it into a separate library, and import as a function, __file__ is now referencing some other module and the results are incorrect.
We could perhaps use this instead, but it seems like the sys.argv may not be reliable either.
def prepend_script_directory(s):
here = dirname(abspath(sys.argv[0]))
return join(here, s)
How to write prepend_script_directory robustly and correctly?
I would personally just os.chdir into the script's directory whenever I execute it. It is just:
import os
os.chdir(os.path.split(__file__)[0])
However if you did want to refactor this thing into a library, you are in essence wanting a function that is aware of its caller's state. You thus have to make it
prepend_script_directory(__file__, blah)
If you just wanted to write
prepend_script_directory(blah)
you'd have to do cpython-specific tricks with stack frames:
import inspect
def getCallerModule():
# gets globals of module called from, and prints out __file__ global
print(inspect.currentframe().f_back.f_globals['__file__'])
I think the reason it doesn't smell right is that $PYTHONPATH (or sys.path) is the proper general mechanism to use.
You want pkg_resources
import pkg_resources
foo_fname = pkg_resources.resource_filename(__name__, "foo.txt")
I'm planning to have project with following structure:
./runme.py
./my_modules/__init__.py
./my_modules/global_imports.py
./my_modules/user_defined_functions.py
Idea is to store important variables in global_imports.py from where they will be imported into runme.py using from my_modules.global_imports import * (I know it is a bad practice import modules this way, but I promise there will be just few variables with not colliding names)
Four questions:
Two of the variables contained inside global_imports.py should be SCRIPT_PATH and SCRIPT_DIR. I've tried SCRIPT_PATH = os.path.realpath(__file__) and SCRIPT_DIR = os.path.dirname(SCRIPT_PATH) but it returns path (directory) for global_imports.py not for runme.py. How can I get path (directory) of runme.py?
Inside global_imports.py I will probably import modules such as os and sys. I also need to import those modules inside runme.py. Is this considered as problem, when modules are imported first from another module and later from main script or vice versa?
Is it possible to import variables from global_imports.py into user_defined_functions.py? I consider this as bad practice I'm just curious.
Is there better approach to separate project into modules?
Addressing your questions in order:
In the first variable SCRIPT_DIR you are getting the full path of the file global_imports.py, which would be something like this:
SCRIPT_PATH = '/home/../../my_project/my_modules/global_imports.py'
now in order to get the directory of runme.py, we should consider another variable:
SCRIPT_PATH_DIR = os.path.dirname(SCRIPT_PATH)
this will give us the path
SCRIPT_PATH_DIR = '/home/../../my_project/my_modules/'
now to get to its parent directory which contains runme.py we can get like this:
SCRIPT_DIR = os.path.abspath(os.path.join(SCRIPT_PATH_DIR, os.pardir))
Now, SCRIPT_DIR gives the path of runme.py i.e.:
SCRIPT_DIR = '/home/../../my_project/'
As per our project structure, runme.py should only conrtain an import to the main module and then command to run the app. So, it shouldn't contain any other imports. Still if you need to use a module then explicitly import it in the runme.py as one of the Zen of Python says 'Explicit is better than implicit'
Yes, it is possible, you can do it like this:
from .global_imports import variable_name
but in general you should have a separate config.py or settings.py file in '/../my_project/' directory which should contain all the settings and variables which you may need to use anywhere in the project.
This approach is good enough as far as I've seen. Your main project directory contains runme.py and other modules which are used inside the project. 'my_modules' is one of the modules I think. You can have more of such modules inside the project directory. A better approach is to have settings and configurations inside one of the modules(such as my_modules) only and to have other modules for the functionality.
Hope this helps, please comment if something is unclear.
I have two codefiles in python, let's say mainfile.py and separatecode.py. I would like to run separatecode.py from within mainfile.py, referencing the specific directory where separatecode.py is stored.
So the pseudocode of what I am looking to do would be something like:
import C:\Users\Jack\Documents\MyFolder\separatecode.py
A number of questions discuss importing, but I can't find one that discusses importing a specific file you wrote in a particular directory. I would like to be able to use functions defined in separatecode.py and am looking for the equivalent of the source("separatecode.r") command in R if that helps.
Add that directory to your sys.path by doing:
import sys
sys.path.append('/directory/to/my/file')
Import the module as normal:
import separatecode
The code you used will not work. It will give a syntax error.
Look at the documentation for the import statement, especially the grammar. A module is one or more identifiers separated by dots.
You can make import separatecode work by adding C:\Users\Jack\Documents\MyFolder to the PYTHONPATH environment variable. This will make it available to all Python scripts. Or you can add that path to sys.path in mainfile.py before importing separatecode.
Try something like this:
import imp
foo = imp.load_source('Module_name', 'Path\To\module.py')
foo.MyClass()
you dont need the foo.MyClass() it was just an example to show that the module works like any other module
no you can import a module from anywhere using the path and its name and you can acsess all its functions etc
for anything else check out:
Python Imp
I hope this is what you were looking for
I am trying to import one python script from another. I have a few common functions defined in one script and then lots of other scripts that want to import those functions. No classes, just functions.
The importing script needs to import from a relative path e.g. ../../SharedScripts/python/common.py
I then a have a few functions def f1(...) defined which I will call.
I found the imp module which seemed to be the right thing to use but I was unable to figure out the exact syntax that would work for my example.
Can someone suggest the correct code to use or the simplest approach if imp is not the right module?
SOLUTION from the answers below I was able to get this working...
projectKey = 'THOR'
# load the shared script relative to this script
sys.path.append(os.path.dirname(__file__) + '/../../SharedScripts/python')
import jira
jira.CheckJiraCommitMessage(sys.argv[1], sys.argv[2], projectKey)
Where I had an empty __init__.py and a jira.py in the SharedScripts/python directory with plain function definitions.
Why not adding ../../SharedScripts/python/ to the python path? Then you could use common.py like any other module:
import common
common.f1()
You can alternate the Python path through the system variable PYTHONPATH or by manipulating it directly from python: sys.path.append("../../SharedScripts/python/")
Please notice that it is probably wiser to work with absolute pathes... (The current directory of the app could change)
To get the absolute path could can call use the function os.path.abspath: os.path.abspath('../../SharedScripts/python/')
A possible way is to add the directory to the Python path before doing the import.
#!/usr/bin/env python
import sys
sys.path.append('../../SharedScripts/python')
import common