I've got one source code which is compiled into two QGis plugins, on this model:
SourceCode -> build.py -> * MyPlugin.zip
* MyPluginExtended.zip
Since the cleanest way to import modules in QGis is to import them from the plugins directory (i.e. the parent directory), all the import statements in the MyPlugin code look like:
import MyPlugin.foo
import MyPlugin.foo.bar
from MyPlugin import foo`
But the problem is, the same import statements will be duplicated from the source code in the MyPluginExtended code, causing ModuleNotFound errors.
The QGis system prevents me to put my final code in a subdirectory that I would have then added to the sys.path, like this:
[Plugins directory] / MyPluginExtended / MyPlugin / [my files]
So the only solutions I could think of were:
1. Override the import system to dynamically replacing MyPlugin in every search path by MyPluginExtended
I tried to setup a custom importer, like proposed in the importlib documentation, doing something like this in my __init__ file, but none of my attempts worked:
try:
import MyPluginExtended
# this is the extended plugin, imports need to be overriden
class MyPathFinder():
# when imports contains 'MyPlugin', change the search path from `path/to/MyPlugin` to `path/to/MyPluginextended`
sys.meta_path.append(MyPathFinder)
except ImportError:
# this is the basic plugin, nothing to do
pass
2. Manually create aliases in sys.modules, like this:
import MyPlugin
sys.modules['MyPluginExtended'] = MyPlugin
But I would then have to do it for every submodule, and I don't knwo how to discover them programatically, something like:
for module_fullname, module in available_submodules():
module_fullname = module_fullname.replace('MyPlugin/', 'MyPluginExtended/')
sys.modules[module_fullname] = module
Related
I'm trying to dynamically build out __all__ in a __init__.py that will bring classes from many files into one namespace AND be given as suggestions in my editor. I've got the first part working. What I can't get working, though, is the ability to have my editor auto-complete discover everything that's in __all__ (I also tried updating __dir__ and defining a dir()/__dir__() method to no success).
For example, my directory tree is this:
things/
__init__.py
one.py
two.py
In __init__.py, I have code that automatically discovers classes One and Two (that are in their so-named files). However, in VS Code, when I type from things import , none of the suggestions are One or Two in the same way as when I type from things.one import , and have One suggested to me. If I manually type it all out, everything works fine, but I would really like to have the auto-complete working.
If I define __all__ with static names, VS Code auto-complete works as expected.
I've been scouring the Internet on this question to no avail and wonder if anyone has any tips or thoughts on how to accomplish it.
Here is what I have in __init__.py:
"""Import all submodules so they are available from the top level"""
import pkgutil
import sys
from importlib import import_module
from pathlib import Path
__all__ = []
def _class_name(module_name):
"""Assume camelCase class name form snake-case filename."""
return "".join([x.title() for x in module_name.split("_")])
# Loop through all modules in this directory and add to namespace
for (_, mod_name, _) in pkgutil.iter_modules([Path(__file__).parent]):
# Ensure that module isn't already loaded
if mod_name not in sys.modules:
loaded_mod = import_module("." + mod_name, package=__name__)
# Load class from imported module
name = _class_name(mod_name)
loaded_class = getattr(loaded_mod, name, None)
if not loaded_class:
continue
# Add this class to the top-level namespace
setattr(sys.modules[__name__], name, loaded_class)
__all__.append(getattr(loaded_mod, name))
Here is my directory:
This is my init.py:
When I use shortcut ctrl+space, I can get the following tips(hi is belong to a.py while hello is belong to b.py):
I am trying to set up a library in python. I have created a setup.py file and in a folder under that I have a library folder, and then I tried to create an sample and test folder (for sample code and tests that I would include)
Folders:
- setup.py
- cvImageUtils # this is the library
- __init__.py # this includs ColorUtils
- ColorUtils.py # this includes a class called ColorUtils
- examples
- color.py # this is shown below
init.py in ColorUtils folder
from . import ColorUtils
ColorUtils.py
class ColorUtils:
def __...
Color.py
from cvImageUtils import ColorUtils
m1 = cv2.imread(os.path.join(image_folder, "bb.jpeg"), 1) # 1 = load color
cv2.imshow('grayscale', ColorUtils.convert_rbg_to_grayscale(m1))
At first, it said, unable to find the module, so I added to the top of the file the following based on another SO solution:
import sys
sys.path.append('../')
Now that seems broken to me already, but it did get me past the no module found, but now it says ColorUtils has no method convert_rbg_to_grayscale. So Then I had to change it to ColorUtils.ColorUtils.convert_rbg_to_grayscale
cv2.imshow('grayscale', ColorUtils.ColorUtils.convert_rbg_to_grayscale(m1))
How can I setup the folder so that it allows me to include the library without sys, and call it without declaring ColorUtils twice.
change your __init__.py:
from cvImageUtils.ColorUtils import ColorUtils
I don't think you'll need to import sys anymore, and you don't have import ColorUtils twice. but just like you have to instantiate an object, you should create a ColorUtils object.
my personal preference would be not creating a Class for Utils.
you might have done this already, but if you want to use a method straight from a class like you did in python, you might want to declare it static.
class ColorUtils:
#staticmethod
def util_method():
pass
then you can just do:
ColorUtils.util_method()
Update:
you can read more about relative/absolute import from here as well.
to fix your actual problem though, you can do:
color.py
remove your import sys and sys call from color.py
change: import cvImageUtils.ColorUtils as ct
to: from cvImageUtils.ColorUtils import *
remove all your ct reference instead just use the actual functions.
cvImageUtils/__init__.py
change: from . import ColorUtils
to __all__=['ColorUtils']
I was able to run color.py to get all the images printed out on screen.
a image.png was also generated locally as well.
Every directory that you want to expose in module search(we usually hide test.py) in python need a init.py file. That should be rule of thumb, by using sys module you can add the module to your "module search path".
After having init.py in your directories, you need to import packages/modules/funcitons you want to use:-
import cvImageUtils.ColorUtils.convert_rbg_to_grayscale
You can execute following code in python to see, what have included in your sys path(used by python to search for modules/packages)
import sys
sys.path
Look into below links for more detailed explations
https://www.programiz.com/python-programming/package
https://www.programiz.com/python-programming/modules#search
I spent some time researching this and I just cannot work this out in my head.
I run a program in its own directory home/program/core/main.py
In main.py I try and import a module called my_module.py thats located in a different directory, say home/program/modules/my_module.py
In main.py this is how I append to sys.path so the program can be run on anyone's machine (hopefully).
import os.path
import sys
# This should give the path to home/program
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..'))
# Which it does when checking with
print os.path.join(os.path.abspath(os.path.dirname(__file__), '..')
# So now sys.path knows the location of where modules directory is, it should work right?
import modules.my_module # <----RAISES ImportError WHY?
However if I simply do:
sys.path.append('home/program/modules')
import my_module
It all works fine. But this is not ideal as it now depends on the fact that the program must exist under home/program.
that's because modules isn't a valid python package, probably because it doesn't contain any __init__.py file (You cannot traverse directories with import without them being marked with __init__.py)
So either add an empty __init__.py file or just add the path up to modules so your first snippet is equivalent to the second one:
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..','modules'))
import my_module
note that you can also import the module by giving the full path to it, using advanced import features: How to import a module given the full path?
Although the answer can be found here, for convenience and completeness here is a quick solution:
import importlib
dirname, basename = os.path.split(pyfilepath) # pyfilepath: /my/path/mymodule.py
sys.path.append(dirname) # only directories should be added to PYTHONPATH
module_name = os.path.splitext(basename)[0] # /my/path/mymodule.py --> mymodule
module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")
Now you can directly use the namespace of the imported module, like this:
a = module.myvar
b = module.myfunc(a)
I am trying to import modules dynamically in Python. Right now, I have a directory called 'modules' with two files inside; they are mod1.py and mod2.py. They are simple test functions to return time (ie. mod1.what_time('now') returns the current time).
From my main application, I can import as follows :
sys.path.append('/Users/dxg/import_test/modules')
import mod1
Then execute :
mod1.what_time('now')
and it works.
I am not always going to know what modules are available in the dirctory. I wanted to import as follows :
tree = []
tree = os.listdir('modules')
sys.path.append('/Users/dxg/import_test/modules')
for i in tree:
import i
However I get the error :
ImportError: No module named i
What am I missing?
The import instruction does not work with variable contents (as strings) (see extended explanation here), but with file names. If you want to import dynamically, you can use the importlib.import_module method:
import importlib
tree = os.listdir('modules')
...
for i in tree:
importlib.import_module(i)
Note:
You can not import from a directory where the modules are not included under Lib or the current directory like that (adding the directory to the path won't help, see previous link for why). The simplest solution would be to make this directory (modules) a package (just drop an empty __init__.py file there), and call importlib.import_module('..' + i, 'modules.subpkg') or use the __import__ method.
You might also review this question. It discusses a similar situation.
You can achieve something like what you are proposing, but it will involve some un-pythonic code. I do not recommend doing this:
dynamic_imports = dict()
for filename in tree:
name = filename.replace('.py', '')
dynamic_imports[name] = __import__(name)
In python 2 I can create a module like this:
parent
->module
->__init__.py (init calls 'from file import ClassName')
file.py
->class ClassName(obj)
And this works. In python 3 I can do the same thing from the command interpreter and it works (edit: This worked because I was in the same directory running the interpreter). However if I create __ init __.py and do the same thing like this:
"""__init__.py"""
from file import ClassName
"""file.py"""
class ClassName(object): ...etc etc
I get ImportError: cannot import name 'ClassName', it doesn't see 'file' at all. It will do this as soon as I import the module even though I can import everything by referencing it directly (which I don't want to do as it's completely inconsistent with the rest of our codebase). What gives?
In python 3 all imports are absolute unless a relative path is given to perform the import from. You will either need to use an absolute or relative import.
Absolute import:
from parent.file import ClassName
Relative import:
from . file import ClassName
# look for the module file in same directory as the current module
Try import it this way:
from .file import ClassName
See here more info on "Guido's decision" on imports in python 3 and complete example on how to import in python 3.