In my project I have a generic class named Interface and many subclasses which inherit from Interface:
/interface_folder/
interface.py
interface_a.py
interface_b.py
interface_c.py
...
/test/
test.py
The Interface class has a method which lists all the available methods of all its subclasses : their names, docstring, input and output arguments. I did this using the inspect module on Interface.__subclasses__().
In test.py, if I want the Interface method using __subclasses__() to work, I need to import interface_a.py, interface_b.py, interface_c.py, etc.
I would like to automate these imports, so that I can add as many interface_x.py as I want in the folder without having to think of adding every time "from interface_folder.interface_d import Interface_d" at the beginning of test.py.
Is this possible?
If not, is there a solution to force python to build the __subclasses__()?
Here is an idea:
Create a new subfolder, say /interface_folder/child_interfrace_folder/.
Move all interface_x.py in the new subfolder.
Import all files of the new subdirectory in interface.py. To do this:
Create an __init__.py file in the new subdirectory and open it in your editor.
In the new __init__.py file, list all interface_x.py files and put them in a variable named __all__. Example:
From here:
# __init__.py
from os.path import dirname, basename, isfile, join
import glob
modules = glob.glob(join(dirname(__file__), "*.py"))
__all__ = [ basename(f)[:-3] for f in modules if isfile(f) and not f.endswith('__init__.py')]
You can now use from child_interfrace_folder import * in your interface.py file.
Related
Is there another way to write it other than doing it in two separate lines like this?
from os.path import isfile, isdir
from os import scandir # or import os.scandir
TL;DR You can't. You got only that method from os.path import <stuff> if you wanna import the functions/objects from inside of os.path.
When you import stuff using just import you can import only modules (module is a term for python file containing python objects). To import the objects inside of a module you use from to traverse upto that module, and then you use import <whateverObject> to import that object from that file (anything and everything is an object in python, as long as it's inside a .py file). The . you use is (generally) to traverse through the directories ('sub' packages?) inside of a package.
How does the interpreter know which directory to include in a package, or how to recognise a given directory as a package? It looks for an __init__.py file inside it. If it finds one, it is a package, and thus you can import it.
import is limited to accessing directories and modules at most, if used alone. When you use from <module_or_directory> import <objects>, the task of accessing directories and/or modules is handed over to the clause after from, and the clause after import takes over the task of looking for python objects inside of the module. You see, these are two distinct things - 1) accessing a file/directory, which comes under file system, and 2) accessing the contents of a file, (specifically, a .py file), which comes under python's domain - neatly separated in the from-import style of importing stuff.
In the case of the os module, os.path is another .pyi file (ntpath.pyi on Windows) that is alias-ed in the os.py file as path. Since it is a module, it goes in the clause after from in from os.path import isfile, isdir. Whereas scandir is a function in os module, hence it goes in the clause after import in from os import scandir.
I'm making a program that detects python files and reads their classes and functions for then to be called. How do I list the functions from another python file without importing it? and how do I call them?
Structure of the program:
Index.py (main py file)
/Content
/Modules
Modules.py
ChickenModule.py (module I want to get functions from)
Modules.py list all the python files from /Modules and stores them on a list.
Index.py calls the functions in Modules.py (Just to keep things less messy)
ChickenModule.py has a Class named ChickenStuff that prints "I'm a chicken" whenever it's self.(something) called.
The goal of the program is for the user to be able to put .py files in modules and when running index.py the functions and classes of said .py files will be listed.
There's a built in library called importlib that might be what you need. I'm assuming that you want classes listed. You'll need inspect for this.
import os
import os.path as path
import importlib
import inspect
# get a list of all files in Content/Modules
filelist = os.listdir("Content/Modules")
# import every file ending in .py
for fname in filelist:
if path.splitext(fname)[1] == 'py':
my_module = importlib.import_module(path.splitext(fname)[0]) # load the module
for _, obj in inspect.getmembers(my_module): # iterate through members
if isinstance(obj, type): # check if members is a class
print(obj)
I am trying to set up a library in python. I have created a setup.py file and in a folder under that I have a library folder, and then I tried to create an sample and test folder (for sample code and tests that I would include)
Folders:
- setup.py
- cvImageUtils # this is the library
- __init__.py # this includs ColorUtils
- ColorUtils.py # this includes a class called ColorUtils
- examples
- color.py # this is shown below
init.py in ColorUtils folder
from . import ColorUtils
ColorUtils.py
class ColorUtils:
def __...
Color.py
from cvImageUtils import ColorUtils
m1 = cv2.imread(os.path.join(image_folder, "bb.jpeg"), 1) # 1 = load color
cv2.imshow('grayscale', ColorUtils.convert_rbg_to_grayscale(m1))
At first, it said, unable to find the module, so I added to the top of the file the following based on another SO solution:
import sys
sys.path.append('../')
Now that seems broken to me already, but it did get me past the no module found, but now it says ColorUtils has no method convert_rbg_to_grayscale. So Then I had to change it to ColorUtils.ColorUtils.convert_rbg_to_grayscale
cv2.imshow('grayscale', ColorUtils.ColorUtils.convert_rbg_to_grayscale(m1))
How can I setup the folder so that it allows me to include the library without sys, and call it without declaring ColorUtils twice.
change your __init__.py:
from cvImageUtils.ColorUtils import ColorUtils
I don't think you'll need to import sys anymore, and you don't have import ColorUtils twice. but just like you have to instantiate an object, you should create a ColorUtils object.
my personal preference would be not creating a Class for Utils.
you might have done this already, but if you want to use a method straight from a class like you did in python, you might want to declare it static.
class ColorUtils:
#staticmethod
def util_method():
pass
then you can just do:
ColorUtils.util_method()
Update:
you can read more about relative/absolute import from here as well.
to fix your actual problem though, you can do:
color.py
remove your import sys and sys call from color.py
change: import cvImageUtils.ColorUtils as ct
to: from cvImageUtils.ColorUtils import *
remove all your ct reference instead just use the actual functions.
cvImageUtils/__init__.py
change: from . import ColorUtils
to __all__=['ColorUtils']
I was able to run color.py to get all the images printed out on screen.
a image.png was also generated locally as well.
Every directory that you want to expose in module search(we usually hide test.py) in python need a init.py file. That should be rule of thumb, by using sys module you can add the module to your "module search path".
After having init.py in your directories, you need to import packages/modules/funcitons you want to use:-
import cvImageUtils.ColorUtils.convert_rbg_to_grayscale
You can execute following code in python to see, what have included in your sys path(used by python to search for modules/packages)
import sys
sys.path
Look into below links for more detailed explations
https://www.programiz.com/python-programming/package
https://www.programiz.com/python-programming/modules#search
I spent some time researching this and I just cannot work this out in my head.
I run a program in its own directory home/program/core/main.py
In main.py I try and import a module called my_module.py thats located in a different directory, say home/program/modules/my_module.py
In main.py this is how I append to sys.path so the program can be run on anyone's machine (hopefully).
import os.path
import sys
# This should give the path to home/program
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..'))
# Which it does when checking with
print os.path.join(os.path.abspath(os.path.dirname(__file__), '..')
# So now sys.path knows the location of where modules directory is, it should work right?
import modules.my_module # <----RAISES ImportError WHY?
However if I simply do:
sys.path.append('home/program/modules')
import my_module
It all works fine. But this is not ideal as it now depends on the fact that the program must exist under home/program.
that's because modules isn't a valid python package, probably because it doesn't contain any __init__.py file (You cannot traverse directories with import without them being marked with __init__.py)
So either add an empty __init__.py file or just add the path up to modules so your first snippet is equivalent to the second one:
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..','modules'))
import my_module
note that you can also import the module by giving the full path to it, using advanced import features: How to import a module given the full path?
Although the answer can be found here, for convenience and completeness here is a quick solution:
import importlib
dirname, basename = os.path.split(pyfilepath) # pyfilepath: /my/path/mymodule.py
sys.path.append(dirname) # only directories should be added to PYTHONPATH
module_name = os.path.splitext(basename)[0] # /my/path/mymodule.py --> mymodule
module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")
Now you can directly use the namespace of the imported module, like this:
a = module.myvar
b = module.myfunc(a)
I have a file structure like this:
dir_a
__init__.py
mod_1.py
mod_2.py
dir_b
__init__.py
my_mod.py
I want to dynamically import every module in dir_a in my_mod.py, and in order to do that, I have the following inside the __init__.py of dir_a:
import os
import glob
__all__ = [os.path.basename(f)[:-3] for f in glob.glob(os.path.dirname(os.path.abspath(__file__)) + "/*.py")]
But when I do:
from dir_a import *
I get ImportError: No module named dir_a
I wonder what causes this, is it because the parent directory containing dir_a and dir_b has to be in PYTHONPATH? In addition, the above approach is dynamic only in the sense that my_mod.py picks up everything when it is run, but if a new module is added to dir_a during its run this new module won't be picked up. So is it possible to implement a truly dynamic importing mechanism?
The answer to the first part of your question is a trivial yes: you cannot import dir_a when the directory containing dir_a isn't in the search path.
For the second part, I don't believe that is possible in general - you could listen for 'file created' OS signals in dir_a, see whether the created file is a .py, and add it to dir_a.__all__ - but that is complicated, probably expensive, and doesn't retroactively add the new thing to the global namespace, since from foo import * only sees what is in foo.__all__ at import time. And changing that would be error-prone - it allows your namespace to change at any time based on an unpredictable external event. Say you do this:
from dir_a import *
bar = 5
And then a bar.py gets added to dir_a. You can't write a "truly dynamic importer" without considering that situation, and deciding what you want to have happen.