Develop Reference

Import a Python module without adding it to the local namespace

What I'd like to do
I'd like to import a Python module without adding it to the local namespace.
In other words, I'd like to do this:
import foo
del foo
Is there a cleaner way to do this?
Why I want to do it
The short version is that importing foo has a side effect that I want, but I don't really want it in my namespace afterwards.
The long version is that I have a base class that uses __init_subclass__() to register its subclasses. So base.py looks like this:
class Base:
_subclasses = {}
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls._subclasses[cls.__name__] = cls
#classmethod
def get_subclass(cls, class_name):
return cls._subclasses[class_name]
And its subclasses are defined in separate files, e.g. foo_a.py:
from base import Base
class FooA(Base):
pass
and so on.
The net effect here is that if I do
from base import Base
print(f"Before import: {Base._subclasses}")
import foo_a
import foo_b
print(f"After import: {Base._subclasses}")
then I would see
Before import: {}
After import: {'FooA': <class 'foo_a.FooA'>, 'FooB': <class 'foo_b.FooB'>}
So I needed to import these modules for the side effect of adding a reference to Base._subclasses, but now that that's done, I don't need them in my namespace anymore because I'm just going to be using Base.get_subclass().
I know I could just leave them there, but this is going into an __init__.py so I'd like to tidy up that namespace.
del works perfectly fine, I'm just wondering if there's a cleaner or more idiomatic way to do this.

If you want to import a module without assigning the module object to a variable, you can use importlib.import_module and ignore the return value:
import importlib
importlib.import_module("foo")
Note that using importlib.import_module is preferable over using the __import__ builtin directly for simple usages. See the builtin documenation for details.

How do I expose a python class under its containing package without importing the entire package?

I have the following (toy) package structure
root/
- package1/
- __init__.py
- class_a.py
- class_b.py
- run.py
In both class_a.py and class_b.py I have a class definition that I want to expose to run.py. If I want to import them this way, I will have to use
from package1.class_a import ClassA # works but doesn't look nice
I don't like that this shows the class_a.py module, and would rather use the import style
from package1 import ClassA # what I want
This is also closer to what I see from larger libraries. I found a way to do this by importing the classes in the __init__.py file like so
from class_a import ClassA
from class_b import ClassB
This works fine if it wasn't for one downside: as soon as I import ClassA as I would like (see above), I also immediately 'import' ClassB as, as far as I know, the __init__.py will be run, importing ClassB. In my real scenario, this means I implicitly import a huge class that I use very situationally (which itself imports tensorflow), so I really want to avoid this somehow. Is there a way to create the nice looking imports without automatically importing everything in the package?

It is possible but require a rather low level customization: you will have to customize the class of your package (possible since Python 3.5). That way, you can declare a __getattr__ member that will be called when you ask for a missing attribute. At that moment, you know that you have to import the relevant module and extract the correct attribute.
The init.py file should contain (names can of course be changed):
import importlib
import sys
import types
class SpecialModule(types.ModuleType):
""" Customization of a module that is able to dynamically loads submodules.
It is expected to be a plain package (and to be declared in the __init__.py)
The special attribute is a dictionary attribute name -> relative module name.
The first time a name is requested, the corresponding module is loaded, and
the attribute is binded into the package
"""
special = {'ClassA': '.class_a', 'ClassB': '.class_b'}
def __getattr__(self, name):
if name in self.special:
m = importlib.import_module(self.special[name], __name__) # import submodule
o = getattr(m, name) # find the required member
setattr(sys.modules[__name__], name, o) # bind it into the package
return o
else:
raise AttributeError(f'module {__name__} has no attribute {name}')
sys.modules[__name__].__class__ = SpecialModule # customize the class of the package
You can now use it that way:
import package1
...
obj = package1.ClassA(...) # dynamically loads class_a on first call
The downside is that clever IDE that look at the declared member could choke on that and pretend that you are accessing an inexistant member because ClassA is not statically declared in package1/__init__.py. But all will be fine at run time.
As it is a low level customization, it is up to you do know whether it is worth it...
Since 3.7 you could also declare a __gettatr__(name) function directly at the module level.

Importing a module for all methods of an instance in Python3

I encountered a problem whilst importing a module in a Python3 class:
I am importing a module in the __init__ method of my class. It works fine in this method, however is there a way to use this module again in another method? I tried saving the module into a self. variable, still doesn't work.
Of course I could import it again inside the method, but I would rather import the module for all of my methods, since most of them need it.
I'll give you some example code below:
class Example(object)
def __init__(self):
import moduleName as module
module.function() # works perfectly
# Trying to save module for the whole instance:
self.module = module
def method(self):
module.function() # Does not recognize module
self.module.function() # Does not recognize attribute either
I'd be happy if someone could help me with this:)

I don't know if in python we can save a module in a variable. I attempt to declare the import at class level, but the code doesn't work.
I python the function are first-citizen so we can save function in a variable. You should save the function you need in variables when you are in __init__():
import module as mod
mod.function()
self.function = mod.function
After a while...
You can load a module dinamically, but to this you have to import the module importlib. This is the code:
import importlib
class MyClass:
def __init__(self):
self.module = importlib.import_module("module")
self.module.function()
def func(self):
self.module.function()
c = MyClass()
c.func()
There is also the imp module.

How to mock external module calls and test code in the class body

I want to store static information in a class, shared by all the instances. The information is something obtained by using another module, I only want to do this once, eg. let's say that this is what I have in mymodule.py:
import os
MyClass:
bus = os.environ.get('DBUS_SESSION_BUS_ADDRESS', None)
def __init__(self):
pass
How can I test this code and mock os.environ.get to make sure that the call is made correctly?
Since the execution happens at the time of the first import, I would need to reload the module in the test, but even so, I can't have os.environ.get mocked at the right time:
import unittest
from unittest import patch, MagicMock
import importlib
import mymodule
class TestMyClass(unittest.TestCase):
#patch('mymodule.os.environ', spec=['get'])
def test_class_init_patch(self, mock_env):
# Too early - the reload overrides the mock
importlib.reload(mymodule)
mock_env.get.assert_called_once_with('DBUS_SESSION_BUS_ADDRESS', None)
def test_class_init_mock(self):
importlib.reload(mymodule)
# Too late - the class body already executed
mymodule.MyClass.os.environ = MagickMock()
I have managed to come up with two alternatives, that make this testable:
Move the class initialization code into a class method and call it from the class body. I can test this class method in my unit test.
Move the class initialization into the __init__ method, guarded by a flag stored in a class variable so that it is only initialized once on the first instantiation.
While both of these should work just fine, it just feels more clean and straightforward to leave this in the class body if possible.

I have managed to figure out how to do this the right way (I hope)!
If you have imported only the module and not the class, function into your code's namespace, eg.:
like this: import os
and not like this: from os import geteuid
You can do this in your test to patch the os module directly:
import os
import unittest
from unittest import patch
import importlib
import mymodule
class TestMyClass(unittest.TestCase):
#patch('os.environ.get')
def test_class_init_patch(self, mock_env_get):
importlib.reload(mymodule)
mock_env_get.assert_called_once_with('DBUS_SESSION_BUS_ADDRESS', None)
This is described in the official documentation as well:
However, consider the alternative scenario where instead of from a import SomeClass module b does import a and some_function uses a.SomeClass. Both of these import forms are common. In this case the class we want to patch is being looked up in the module and so we have to patch a.SomeClass instead:
More details in the official documentation.
This way the module is being patched directly and reloading your own module doesn't affect the patching.
Once the test has run, the patching of the module is undone as usual, however, keep in mind, that the state of your class remains the same way as it was initialized while the external module was patched, so you might need to reload the module again before you run your other tests.
The easiest way to make sure, that you reclaim your "normal" class state, is to reload the module in your TestCase object's setUp method, eg.:
def setUp(self):
importlib.reload(mymodule)

How does Python handle imported subclasses?

I am working on a project that requires me to build several classes and subclasses in one file, and use them in a second file. I would like to ask how Python handles importing the first file into the second.
For instance, if i have a file my_classes.py:
class Myclass(object):
pass
class Mysubclass(myclass):
pass
will using the following code work:
from my_classes import Myclass
print Mysubclass
(where the print command is just an example of using Mysubclass), or do I need to import Mysubclass explicitly?
Thanks in advance!

This won't work. Python import statement doesn't care about subclasses. Actually, it doesn't care about anything. It does precisely what you tell it to do. "Explicit is better than implicit" is a popular saying in Python circles.
Here:
from my_classes import Myclass
You told Python to import only Myclass.
This will import both classes:
from my_classes import Myclass, Mysubclass
You can read how Python import works here.

Your subclass will not be available if you do it like this. You must import every object by itself.
Here a quick example
test_class.py
class MyClass(object):
def init(self):
print self.class
class MySubClass(MyClass):
def __init__(self):
print self.__class__
test_class_import.py
from test_class import MyClass
MyClass()
MySubClass()
##output##
<class 'test_class.MyClass'>
Traceback (most recent call last):
File "test_class2.py", line 4, in <module>
MySubClass()
NameError: name 'MySubClass' is not defined
but
from test_class import MyClass, MySubClass
MyClass()
MySubClass()
##output##
<class 'test_class.MyClass'>
<class 'test_class.MySubClass'>

Every module has a namespace.
A namespace is a mapping from variable names to values (Python objects).
The statement
import my_classes
makes the my_classes namespace accessible from the current module by placing the variable name my_classes in the current module's namespace. You can then access values from my_classes with the syntax
my_classes.variable
So, for example:
import my_classes
print my_classes.Mysubclass
print my_classes.MyClass
If that is too much typing, I suggest
import my_classes as MC
print MC.Mysubclass
print MC.MyClass
you could also do
from my_classes import Mysubclass, MyClass
but this form of import is discouraged by some Python experts.
from my_classes import Mysubclass, MyClass
loads the entire module my_class but only
places the variable names Mysubclass and MyClass in the current module's namespace. They point to the same values as do the variables of the same name in the my_classes namespace. You don't get access to anything else from the my_class module's namespace.