Develop Reference

Set a module's class method as an attribute of that module from outside that module

The objective:
I have a package with submodules that I would like to be accessible in the most straightforward way possible. The submodules contain classes to take advantage of the class structure, but don't need to be initialized (as they contain static and class methods). So, ideally, I would like to access them as follows:
from myPackage.subModule import someMethod
print (someMethod)
from myPackage import subModule
print (subModule.someMethod)
import myPackage
print(myPackage.subModule.someMethod)
Here is the package structure:
myPackage ─┐
__init__.py
subModule
subModule2
etc.
Example of a typical submodule:
# submodule.py
class SomeClass():
someAttr = list(range(10))
#classmethod
def someMethod(cls):
pass
#staticmethod
def someMethod2():
pass
Here is the code I have in my '__init __.py': In order to achieve the above; it attempts to set attributes for each class at the package level, and the same for it's methods at the sub-module level.
# __init__.py
def import_submodules(package, filetypes=('py', 'pyc', 'pyd'), ignoreStartingWith='_'):
'''Import submodules to the given package, expose any classes at the package level
and their respective class methods at submodule level.
:Parameters:
package (str)(obj) = A python package.
filetypes (str)(tuple) = Filetype extension(s) to include.
ignoreStartingWith (str)(tuple) = Ignore submodules starting with given chars.
'''
if isinstance(package, str):
package = sys.modules[package]
if not package:
return
pkg_dir = os.path.dirname(os.path.abspath(package.__file__))
sys.path.append(pkg_dir) #append this dir to the system path.
for mod_name in os.listdir(pkg_dir):
if mod_name.startswith(ignoreStartingWith):
continue
elif os.path.isfile(os.path.join(pkg_dir, mod_name)):
mod_name, *mod_ext = mod_name.rsplit('.', 1)
if filetypes:
if not mod_ext or mod_ext[0] not in filetypes:
continue
mod = importlib.import_module(mod_name)
vars(package)[mod_name] = mod
classes = inspect.getmembers(mod, inspect.isclass)
for cls_name, clss in classes:
vars(package)[cls_name] = clss
methods = inspect.getmembers(clss, inspect.isfunction)
for method_name, method in methods:
vars(mod)[method_name] = method
del mod_name
import_submodules(__name__)
At issue is this line:
vars(mod)[method_name] = method
Which ultimately results in: (indicating that the attribute was not set)
from myPackage.subModule import someMethod
ImportError: cannot import name 'someMethod' from 'myPackage.subModule'
I am able to set the methods as attributes to the module within that module, but setting them from outside (ie. in the package __init __), isn't working as written. I understand this isn't ideal to begin with, but my current logic is; that the ease of use, outweighs any perceived issues with namespace pollution. I am, of course, always open to counter-arguments.

I just checked it on my machine.
Created a package myPackage with a module subModule that has a function someMethod.
I run a python shell with working directory in the same directory that the myPackage is in, and to get these 3 import statements to work:
from myPackage.subModule import someMethod
from myPackage import subModule
import myPackage
All I had to do was to create an __init__.py with this line in it:
from . import subModule

Found a nice "hacky" solution -
subModule.py:
class myClass:
#staticmethod
def someMethod():
print("I have a bad feeling about this")
myInstance = myClass()
someMethod = myInstance.someMethod
init.py is empty

Still scratching my head of why I am unable to do this from the package __init __, but this solution works with the caveat it has to be called at the end of each submodule. Perhaps someone, in the future, someone can chime in as to why this wasn't working when completely contained in the __init __.
def addMembers(module, ignoreStartingWith='_'):
'''Expose class members at module level.
:Parameters:
module (str)(obj) = A python module.
ignoreStartingWith (str)(tuple) = Ignore class members starting with given chars.
ex. call: addMembers(__name__)
'''
if isinstance(module, str):
module = sys.modules[module]
if not module:
return
classes = inspect.getmembers(module, inspect.isclass)
for cls_name, clss in classes:
cls_members = [(o, getattr(clss, o)) for o in dir(clss) if not o.startswith(ignoreStartingWith)]
for name, mem in cls_members:
vars(module)[name] = mem

This is the solution I ended up going with. It needs to be put at the end of each submodule of your package. But, it is simple and in addition to all the standard ways of importing, allows you to import a method directly:
def __getattr__(attr):
'''Attempt to get a class attribute.
:Parameters:
attr (str): A name of a class attribute.
:Return:
(obj) The attribute.
'''
try:
return getattr(Someclass, attr)
except AttributeError as error:
raise AttributeError(f'{__file__} in __getattr__\n\t{error} ({type(attr).__name__})')
from somePackage.someModule import someMethod

Failed to import defined modules in __init__.py

My directory looks like this :
- HttpExample:
- __init__.py
- DBConnection.py
- getLatlong.py
I want to import DBConnection and import getLatlong in __init__.py. There is no error in my __init__.py until I run it, I received :
System.Private.CoreLib: Exception while executing function: Functions.HttpExample. System.Private.CoreLib: Result: Failure
Exception: ModuleNotFoundError: No module named 'getLatlong'
I'm trying to use function in getLatlong to use the information input by user from __init__.py to getLatlong. Below is the code:
__init__.py :
from getLatlong import result
from DBConnection import blobService, container_name, account_key, file_path
def main(req: func.HttpRequest) -> func.HttpResponse:
logging.info('Python HTTP trigger function processed a request.')
section = req.params.get('section')
bound = req.params.get('bound')
km_location = req.params.get('km_location')
location = req.params.get('location')
if not section:
try:
req_body = req.get_json()
except ValueError:
pass
else:
section = req_body.get('section')
if section and bound and km_location:
result(section, km_location, bound, location).getResult() #HERE
return func.HttpResponse(f"Hello {section}, {bound}!")
#elif section is None or bound is None or km_location is None:
# return func.HttpResponse("All values are mandatory!")
I am also receiving compile error at getLatlong to import DBConnection to this class. The following values will pass to getLatlong.py. The code :
from DBConnection import blobService, container_name, account_key, file_path #Another import error here says : Unable to import DBConnection
class result:
def __init__(self, section, bound, km_location, location):
self.section = section
self.bound = bound
self.km_location = km_location
self.location = location
def getResult(self):
print(self.section)
print(self.bound)
print(self.km_location)
print(self.location)
I've tried every way to import these files before I lost my mind..

You get these errors, because Python does not know where to look for the files you want to import. Depending on which Python version you are using, I see three ways to solve this:
You could add HttpExample to your PYTHONPATH and than your imports should work as you have them currently.
Another way would be to use the sys module and append the path to HttpExample, e.g.
import sys
sys.path.append('PATH/TO/HttpExample')
But you would have to do this in all files, where you want to import something from the parent folder.
Or you use relative imports, which have been available since Python 2.5 (See PEP238). Those are only available in modules, but as you have your __init__.py file, it should work. For relative imports you are using dots . to tell Python where to look for the import. One dot . tells Python to look for the desired import in the parent folder. You could also use .. to go up two levels. But one level should be enough in your case.
So in your case changing your code to this, should solve your problem.
In __init.py__:
from .getLatlong import result
from .DBConnection import blobService, container_name, account_key, file_path
In getLangLong.py:
from .DBConnection import blobService, container_name, account_key, file_path

You could try from __app__.HttpExample import getLatlong.
There is a document about how to import module in the Shared Code folder. Check this doc:Folder structure.
It says Shared code should be kept in a separate folder in __app__. And in my test this could work for me.

how to import a python class's moudle?

I have this main.py, in it:
import uuid
class tools(object):
def generate_uuid(self):
return self.uuid.uuid4()
in my calling program callmain.py, I have
import main
result = main.tool.generate_uuid()
print ("result")
if I run my callmain.py: I get
"TypeError: generate_uuid() missing 1 required positional argument:
'self'
if I add self to the line
result = main.tool.generate_uuid(self): I get
NameError: name 'self' is not defined
How to fix this? thank for help.

because you should make a object from your class first. then call your sub function like this:
import main
result = tools()
result.generate_uuid()
print(result) # "result" is a string! you should just call result without any "".

If you want to use the module of a class you have to create an instance of that class first and call if from that instance, that way the self argument is passed a valid reference to an instance of that class. For example:
import main
tools_instance = main.tools()
result = tools_instance.generate_uuid()
The style of importing shown in your question looks like a package. In packages a folder of python files __init__.py can be arranged in a particular way, documented here Python Packages. An example from the docs
parent/
__init__.py
one/
__init__.py
two/
__init__.py
three/
__init__.py
So an package of the format
main/
__init__.py
tools/
__init__.py # Add function 'generate_uuid' in this file
Could be utilized as follow:
import main
result = main.tools.generate_uuid()
print(result)

How do I change the name of an imported library?

I am using jython with a third party application. The third party application has some builtin libraries foo. To do some (unit) testing we want to run some code outside of the application. Since foo is bound to the application we decided to write our own mock implementation.
However there is one issue, we implemented our mock class in python while their class is in java. Thus to use their code one would do import foo and foo is the mock class afterwards. However if we import the python module like this we get the module attached to the name, thus one has to write foo.foo to get to the class.
For convenience reason we would love to be able to write from ourlib.thirdparty import foo to bind foo to the foo-class. However we would like to avoid to import all the classes in ourlib.thirdparty directly, since the loading time for each file takes quite a while.
Is there any way to this in python? ( I did not get far with Import hooks I tried simply returning the class from load_module or overwriting what I write to sys.modules (I think both approaches are ugly, particularly the later))
edit:
ok: here is what the files in ourlib.thirdparty look like simplified(without magic):
foo.py:
try:
import foo
except ImportError:
class foo
....
Actually they look like this:
foo.py:
class foo
....
__init__.py in ourlib.thirdparty
import sys
import os.path
import imp
#TODO: 3.0 importlib.util abstract base classes could greatly simplify this code or make it prettier.
class Importer(object):
def __init__(self, path_entry):
if not path_entry.startswith(os.path.join(os.path.dirname(__file__), 'thirdparty')):
raise ImportError('Custom importer only for thirdparty objects')
self._importTuples = {}
def find_module(self, fullname):
module = fullname.rpartition('.')[2]
try:
if fullname not in self._importTuples:
fileObj, self._importTuples[fullname] = imp.find_module(module)
if isinstance(fileObj, file):
fileObj.close()
except:
print 'backup'
path = os.path.join(os.path.join(os.path.dirname(__file__), 'thirdparty'), module+'.py')
if not os.path.isfile(path):
return None
raise ImportError("Could not find dummy class for %s (%s)\n(searched:%s)" % (module, fullname, path))
self._importTuples[fullname] = path, ('.py', 'r', imp.PY_SOURCE)
return self
def load_module(self, fullname):
fp = None
python = False
print fullname
if self._importTuples[fullname][1][2] in (imp.PY_SOURCE, imp.PY_COMPILED, imp.PY_FROZEN):
fp = open( self._importTuples[fullname][0], self._importTuples[fullname][1][1])
python = True
try:
imp.load_module(fullname, fp, *self._importTuples[fullname])
finally:
if python:
module = fullname.rpartition('.')[2]
#setattr(sys.modules[fullname], module, getattr(sys.modules[fullname], module))
#sys.modules[fullname] = getattr(sys.modules[fullname], module)
if isinstance(fp, file):
fp.close()
return getattr(sys.modules[fullname], module)
sys.path_hooks.append(Importer)

As others have remarked, it is such a plain thing in Python that the import statement iself has a syntax for that:
from foo import foo as original_foo, for example -
or even import foo as module_foo
Interesting to note is that the import statemente binds a name to the imported module or object ont he local context - however, the dictionary sys.modules (on the moduels sys of course), is a live reference to all imported modules, using their names as a key. This mechanism plays a key role in avoding that Python re-reads and re-executes and already imported module , when running (that is, if various of yoru modules or sub-modules import the samefoo` module, it is just read once -- the subsequent imports use the reference stored in sys.modules).
And -- besides the "import...as" syntax, modules in Python are just another object: you can assign any other name to them in run time.
So, the following code would also work perfectly for you:
import foo
original_foo = foo
class foo(Mock):
...

Loading files into variables

I am trying to write a small function that gets a variable name, check if it exists, and if not loads it from a file (using pickle) to the global namespace.
I tried using this in a file:
import cPickle
#
# Load if neccesary
#
def loadfile(variable, filename):
if variable not in globals():
cmd = "%s = cPickle.load(file('%s','r'))" % (variable, filename)
print cmd
exec(cmd) in globals()
But it doesn't work - the variable don't get defined. What am I doing wrong?

Using 'globals' has the problem that it only works for the current module. Rather than passing 'globals' around, a better way is to use the 'setattr' builtin directly on a namespace. This means you can then reuse the function on instances as well as modules.
import cPickle
#
# Load if neccesary
#
def loadfile(variable, filename, namespace=None):
if module is None:
import __main__ as namespace
setattr(namespace, variable, cPickle.load(file(filename,'r')))
# From the main script just do:
loadfile('myvar','myfilename')
# To set the variable in module 'mymodule':
import mymodule
...
loadfile('myvar', 'myfilename', mymodule)
Be careful about the module name: the main script is always a module main. If you are running script.py and do 'import script' you'll get a separate copy of your code which is usually not what you want.

You could alway avoid exec entirely:
import cPickle
#
# Load if neccesary
#
def loadfile(variable, filename):
g=globals()
if variable not in g:
g[variable]=cPickle.load(file(filename,'r'))
EDIT: of course that only loads the globals into the current module's globals.
If you want to load the stuff into the globals of another module you'd be best to pass in them in as a parameter:
import cPickle
#
# Load if neccesary
#
def loadfile(variable, filename, g=None):
if g is None:
g=globals()
if variable not in g:
g[variable]=cPickle.load(file(filename,'r'))
# then in another module do this
loadfile('myvar','myfilename',globals())