Suppose that I have a python module A in my libraries directory. How can I export the same library as module B, however use only the functions and the code parts I need automatically? For instance, I want to build a minimal version of openCV for Python for a laser tracking project and I am sure that the code I actually need is much less than the whole openCV library. I don't want to do this by hand, so is there any kind of application that automates this proceedure?
EDIT: I saw the comments and noted you are struggling with a space issue. In that case the code below is not what you are looking for. I think you will have to write the minimized version of the target module. Otherwise you will have to implement some kind of tool that enables you to find all dependences of functions you want to use automatically.
You have to choose if it's worth the effort.
class Importer:
def __init__(self, module_name, funcion_list):
self.__module = __import__(module_name)
self.__specific = funcion_list
self.initialize()
def initialize(self):
for elem in self.__specific:
setattr(self, elem, getattr(self.__module, elem))
Lets say you want only export the function listdir from module os. So, in your B module you can:
os_minimized = Importer("os", ("listdir",))
And then in other module you write:
from B import os_minimized
print(os_minimized.listdir("."))
Related
I want future adding on to be super easy for my current project and I'm trying to have it so I can just add a module to a package and have it imported and implemented easily.
Right now, I have the following code. Is there a better way to handle what I want accomplished? I also have a question about pkgutil.walk_packages. It returns modules that are not in the "selected" package (parent.package.path). Any tips on how to fix that?
import pkgutil
import importlib
sub_modules = []
for importer, modname, ispkg in pkgutil.walk_packages('parent.package.path'):
if 'parent.package.path.' not in str(modname):
continue
sub_modules.append(str(modname))
for m in sub_modules:
i = importlib.import_module(m)
i.handle_command()
My goal is to add things to parent.package.path and have them activated by running the handle_command() method in each module.
For instance, there are os.path.walk, os.walk and assuming another md.walk, and assume os is imported but md is not. I desire a function like
whereis('walk')
while can return os.path.walk, os.walk and md.walk.
Or if it's difficult to know there is a md.walk, how to get the imported os.path.walk and os.walk?
OK, that's was fun, here's updated solution. Some Python magic, solution for Python 2.7. Any improvements are welcome. It behaves like any other import - so be careful to wrap any of your executable code in if name == "__main__".
import inspect
import pkgutil
import sys
def whereis_in_globals(fname):
return [m for m in sys.modules.itervalues()
if module_has_function(m, fname)]
def whereis_in_locals(fname):
modules = (__import__(module_name, globals(), locals(), [fname], -1)
for _, module_name, _ in pkgutil.walk_packages(path="."))
return [m for m in modules if module_has_function(m, fname)]
def module_has_function(m, fname):
return hasattr(m, fname) and inspect.isfunction(getattr(m, fname))
if __name__ == "__main__":
# these should never raise AttributeError
for m in whereis_in_locals('walk'):
m.walk
for m in whereis_in_globals('walk'):
m.walk
Can you tell us more about your use case or larger goals? Getting this answered for non-imported modules is an interesting challenge, and I don't have an answer for the way you've phrased it but depending on your actual use case you may be interested in knowing that may IDEs (Integrated Development Environments) can sort of do what you're asking.
For example if you install PyCharm (available for free in community edition) you can type a full or partial function name (just a text string) and then have the IDE search the active project to see what you might be looking for. For example you could just type "walk" on a line and then (IIR) Ctrl-click on it to see what possible modules you might have matching functions. You can also search the other direction, seeing where in a project the function is used.
To search on a wide scale you might have to cheat a little and open your Python directory as a "project" but that's easy.
Again, you haven't provided a use case so my apologies if you are specifically looking for, e.g., an in-script way to get that result for later use in a program. If it's something you want an answer to for you, as a developer and as you work, you might look to the tools you're using to provide an answer.
I was wondering if there are any sort of python codeing etc that will displays the files imports/used locations in a python file?
Eg. TestA.py contains 3 files from 3 different directory
Import01 : /u/ext/TestA/UI
Import02 : /u/ext/TestA/src
Import03 : /user_data/setup/localImports
And hence, while executing the coding, it will displays the list of directories used in the python file?
I am asking as I am working on several (and maybe tons, in the future) scripts that are heavily involved in Maya, there are times in which when I located the path but they are the wrong ones (with same name) and is actually located in another path
Add this code to module
import inspect
frame = inspect.currentframe()
if frame and frame.f_back:
print('module "{}" is imported by "{}"'.format(__file__, frame.f_back.f_locals['__file__']))
If module_a.py contains the code above, and main.py imports it. the output is
module "/path/to/module_a.py" is imported by "/path/to/main.py"
As documented, this answer may not be an exact solution. Because if not supported, returns None.
CPython implementation detail: This function relies on Python stack frame support in the interpreter, which isn’t guaranteed to exist in all implementations of Python. If running in an implementation without Python stack frame support this function returns None.
At any point in when the code is running, you can determine the origin of a module by checking it's file attribute:
import sys
for name, each_mod in sys.modules.items():
try:
print name, each_mod.__file__
except AttributeError: # = built in module or dll
print "?"
To check the imports without running the code, you'd need do more complex analysis: Here's an example method that could probably be adapted to figure it out :http://www.tarind.com/depgraph.html
You could also create a custom ModuleFinder that printed out file sources as imports are processed. Something like this, which prints out name of py/pyc files when trying to load them.
import os
import sys
import imp
import ihooks
class ReportingFinder(object):
"""Find modules collected in a shelve archive."""
def __init__(self, path_entry):
self.path_entry = path_entry
if not os.path.isdir(path_entry):
raise ImportError
def find_module(self, fullname, path=None):
for suffix in (".py", ".pyc"):
test_path = os.path.join(self.path_entry, fullname + suffix)
print test_path
if os.path.exists(test_path):
print "attemnpting to load from %s" % test_path
return self
return None
def load_module(self, name):
stuff = imp.find_module(name)
return ihooks.FancyModuleLoader(verbose=1).load_module(name, stuff)
sys.path_hooks.insert(0, ReportingFinder)
HACK WARNING!!!! Please be aware this code is a quick diagnostic hack! Don't use it for production :) Among other flaws, it will print out py path names even if the code comes from the pyc, and it's dumb about packages -- I only provided it because it sounds like you're using single-file scripts rather than packages. It is handy for catching imported modules as they get loaded. It won't print out the names of zip files.
It sounds like the real problem is having too many competing paths: you should try to get down to as few as you can so that there are fewer suprises.
I am using the PyMOL molecular viewer as a subset of a larger program, and for ease of reading am breaking up my files like so...
### command1ClassFile.py
class command1Class():
def command1(self):
print "do action 1, this requires pymol functions"
### command2ClassFile.py
class command2Class():
def command2(self):
print "do action 2, this also requires pymol functions"
### mainModule.py
import command1ClassFile, command2ClassFile
class commandsClass(command1Class, command2Class):
pass
class guiClass(parentClass, commandsClass):
def onLeftClick(self):
self.command1()
def onRightClick(self):
self.command2()
# this imports the module as well as launching the program, unfortunately
import pymol
pymol.finish_launching()
I can't just add "import pymol" to the beginning of the other files, because that would launch the program multiple times. I can solve this by just using one .py file, but that leads to an excessively large source file.
I did not catch anyone's interest on the PyMOL mailing list, so I was hoping there was some other way around this. If not, is there a better way to break up code? I am used to being spoiled by header files in C++, and the architecture of Python projects is bit difficult for me to handle properly.
EDIT: For different cases, is using multiple inheritance across files and dummy compilation classes in this way a good way to structure python projects with complicated methods?
If I understand the question correctly, that's what if __name__ == '__main__' is for.
As I'm a novice, I started learning python by writing simple programs using python GUI .. everytime, when I try to clear the console I need to define the following few lines of code
import os
def cls():
os.system('cls')
print("Console Cleared")
and later I call it as
cls()
now I'm tired of writing the above code again and again(every time I open the gui window )..
so, I want to create a module called cls for future use to save my time ..
in the end I'll call it as follows
import cls
cls()
is there a way to do this ...
Thanks !
Well, a solution is to create a directory to add your modules and add this directory to the Python pat. For example, you can create a directory at, let us say, C:\mypymodules and put a cls.py file there with your function.
Now, let us add this directory to the Python path. Follow these instructions, just inserting C:\mypymodules in place of the directories mentioned there. Now, open a new command line window and try to import the module.
Another solution is to use distutils*. Instead of creating your own modules directory, in the same directory of your cls.py file create a file named setup.py with the following content:
from distutils.core import setup
setup(name='cls', py_modules=['cls'])
Then, just execute it:
> python setup.py install
This may install your module in the default Python library directories. Actually, this is the way (or, better yet, one of the ways) to pack, manage or python packages.
(I am not really using Windows, so some details may be broken. Nonetheless, I believe the general concepts are the same.)
* Some may argue about using setuptools but I think it is really overkill for your case. Anyway, if you want to learn more, see this question.
If you put your cls() function in a file called cls.py, and put that either in the same directory as the program that's calling it or else somewhere in your PYTHONPATH on your system, then you can use your function like this:
import cls
cls.cls()
This is because the cls() function is inside the cls module (the name of the module is determined by the file name).
If you want to just be able to do cls() then do this:
from cls import cls
cls()
You can spice it up a little bit by making your module cross platform. Here it is:
import os, sys
def cls():
if 'win' in sys.paltform:
os.system('cls')
print 'Console cleared'
else:
os.system('clear')
print 'Console cleared'
Now, you can use it in any other script using:
from cls import cls
cls()