How to import nested package using the "as" shorthand?
This question is similar to importing a module in nested packages only the nesting is within the same .py file, not across folders.
In foo.py (All python files are in the same package, and are version 3.4):
class Foo:
class Bar:
...
I can access these subclasses in another .py file:
from . import foo
...
bar = foo.Foo.Bar()
What I would like to do:
from . import foo.Foo.Bar as Bar # DOES NOT WORK: "unresolved reference" error.
...
bar = Bar() # saves typing.
bar2 = Bar()
...
Is there a way to do this?
There is little point in nesting Python classes; there is no special meaning attached to doing so other than nesting the namespaces. There rarely is any need to do so. Just use modules instead if you need to produce additional namespaces.
You cannot directly import a nested class; you can only import module globals, so Foo in this case. You'd have to import the outer-most class and create a new reference:
from .foo import Foo
Bar = Foo.Bar
del Foo # remove the imported Foo class again from this module globals
The del Foo is entirely optional. The above does illustrate why you'd not want to nest classes to begin with.
Related
What I'd like to do
I'd like to import a Python module without adding it to the local namespace.
In other words, I'd like to do this:
import foo
del foo
Is there a cleaner way to do this?
Why I want to do it
The short version is that importing foo has a side effect that I want, but I don't really want it in my namespace afterwards.
The long version is that I have a base class that uses __init_subclass__() to register its subclasses. So base.py looks like this:
class Base:
_subclasses = {}
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls._subclasses[cls.__name__] = cls
#classmethod
def get_subclass(cls, class_name):
return cls._subclasses[class_name]
And its subclasses are defined in separate files, e.g. foo_a.py:
from base import Base
class FooA(Base):
pass
and so on.
The net effect here is that if I do
from base import Base
print(f"Before import: {Base._subclasses}")
import foo_a
import foo_b
print(f"After import: {Base._subclasses}")
then I would see
Before import: {}
After import: {'FooA': <class 'foo_a.FooA'>, 'FooB': <class 'foo_b.FooB'>}
So I needed to import these modules for the side effect of adding a reference to Base._subclasses, but now that that's done, I don't need them in my namespace anymore because I'm just going to be using Base.get_subclass().
I know I could just leave them there, but this is going into an __init__.py so I'd like to tidy up that namespace.
del works perfectly fine, I'm just wondering if there's a cleaner or more idiomatic way to do this.
If you want to import a module without assigning the module object to a variable, you can use importlib.import_module and ignore the return value:
import importlib
importlib.import_module("foo")
Note that using importlib.import_module is preferable over using the __import__ builtin directly for simple usages. See the builtin documenation for details.
I have the following (toy) package structure
root/
- package1/
- __init__.py
- class_a.py
- class_b.py
- run.py
In both class_a.py and class_b.py I have a class definition that I want to expose to run.py. If I want to import them this way, I will have to use
from package1.class_a import ClassA # works but doesn't look nice
I don't like that this shows the class_a.py module, and would rather use the import style
from package1 import ClassA # what I want
This is also closer to what I see from larger libraries. I found a way to do this by importing the classes in the __init__.py file like so
from class_a import ClassA
from class_b import ClassB
This works fine if it wasn't for one downside: as soon as I import ClassA as I would like (see above), I also immediately 'import' ClassB as, as far as I know, the __init__.py will be run, importing ClassB. In my real scenario, this means I implicitly import a huge class that I use very situationally (which itself imports tensorflow), so I really want to avoid this somehow. Is there a way to create the nice looking imports without automatically importing everything in the package?
It is possible but require a rather low level customization: you will have to customize the class of your package (possible since Python 3.5). That way, you can declare a __getattr__ member that will be called when you ask for a missing attribute. At that moment, you know that you have to import the relevant module and extract the correct attribute.
The init.py file should contain (names can of course be changed):
import importlib
import sys
import types
class SpecialModule(types.ModuleType):
""" Customization of a module that is able to dynamically loads submodules.
It is expected to be a plain package (and to be declared in the __init__.py)
The special attribute is a dictionary attribute name -> relative module name.
The first time a name is requested, the corresponding module is loaded, and
the attribute is binded into the package
"""
special = {'ClassA': '.class_a', 'ClassB': '.class_b'}
def __getattr__(self, name):
if name in self.special:
m = importlib.import_module(self.special[name], __name__) # import submodule
o = getattr(m, name) # find the required member
setattr(sys.modules[__name__], name, o) # bind it into the package
return o
else:
raise AttributeError(f'module {__name__} has no attribute {name}')
sys.modules[__name__].__class__ = SpecialModule # customize the class of the package
You can now use it that way:
import package1
...
obj = package1.ClassA(...) # dynamically loads class_a on first call
The downside is that clever IDE that look at the declared member could choke on that and pretend that you are accessing an inexistant member because ClassA is not statically declared in package1/__init__.py. But all will be fine at run time.
As it is a low level customization, it is up to you do know whether it is worth it...
Since 3.7 you could also declare a __gettatr__(name) function directly at the module level.
So I have a python 2.7 project with three modules. Two contain classes and one is a script. An example files structure is shown below
project/
__main__.py
__init__.py
- serial_connect/
ser_conn.py
__init__.py
- ui/
parse_file.py
__init__.py
ser_conn.py contains a class which handles all interaction with some RS-232 hardware device. It contains methods such as open(), close(), send_go() just basically everything required for this connection.
parse_file.py contains a class which has methods relating to parsing a file and getting text commands which are associated with serial commands.
e.g. if the text file contains the command "send_go" parse_file.py will parse this command and call Ser_Conn.send_go()
__main.py__ contains the main() function. in main() an instance of the Ser_Conn class is created and then passed to the Parse_File class as there exists only one instance of a serial connection in the program and it is required by both main() and the Parse_File class.
My question is as follows
In methods of Parse_File I call methods of the Ser_Conn instance, such as Ser_Conn.send_go() but parse_file.py does not complain about there being no
from serial_connect.ser_conn import Ser_Conn
There only exists a ser_conn import in __main__.py. why does this work?
In python, an instance carries all the "stuff" that it needs to do it's work along with it. At a high level, (and somewhat simplified) when you write:
qux.whatever
python looks at the object qux and then finds its whatever attribute (if it has one). Note that python doesn't care what type qux is, only that it has a whatever attribute. I could create multiple objects that satisfy this interface:
class Foo(object):
whatever = 'Hey Dawg!'
class Bar(object):
whatever = 'I satisfy the interface too!'
Also note that these objects could be defined anywhere. As long as you manage to get a reference to the object (e.g. it was passed into a function) you can use the object with all of it's attributes and methods.
maybe a more concrete example would help -- Say you have 4 modules, foo, bar, call_method and driver. foo, bar and call_method don't know anything about each other, but driver imports the other 3. Then you can do something like this:
# foo.py
class Foo(object):
def method(self):
return 'foo!'
# bar.py
class Bar(object):
def method(self):
return 'bar!'
# call_method.py
def go(object_with_method):
print(object_with_method.method())
# driver.py
import call_method
import foo
import bar
call_method.go(Foo()) # call the method on a foo instance
call_method.go(Bar()) # call the method on a bar instance
You're passing the connection instance to parse_file which means Python already knows the class and other details of that object. So that's why you don't need to import the class again in the parse_file code.
You only need to import something if you wish to use that something in that file.
When you created the object Python used the class to construct the object and that's sufficient. You can of course add that import line in parse_file but only if you actually need to use the class in that file (otherwise the import line it's very useful).
This might be a terribly simple one, but I don't know what's the "right" answer. Assume that I have a script
import utils
bar = 1
utils.foo()
print bar
Furthermore, the module utils is:
def foo():
bar = bar+1
As given above, I ,obviously, get:
UnboundLocalError: local variable 'bar' referenced before assignment
How can I use bar inside foo()? In my specific case, I don't really want to alter foo, but I do need to be able to use it and its state inside foo().
One workaround would be to pass bar to foo():
def foo(bar):
return bar+1
And replace the third line in the script: bar = utils.foo(bar).
However, this feels like a cumbersome solution; in particular if bar is a complex object.
I am interested in a best-practice approach the case described above.
Why don't you want to alter foo? If you import a module, you want to use its functionality. If the foo function is without parameters, then bar or other variables in it are used in the module utils itself. If you want to use a function with values that are not inside the module, then:
def foo(bar):
return bar+1
is totally acceptable.
EDIT:
// When you create class foo1, just set bar in the constructor.
class foo1:
def init(self, bar):
self.bar = bar
Image this situation:
import someModule
# now you want to use a function of this module
foo()
Maybe then there would be an error like: bar is not defined or whatever --> modules are not loosely coupled. Either make the function foo as you proposed with parameters (totally acceptable) or set the bar value via a constructor or setBar method.
I am interested in a best-practice approach the case described above
As you describe, bar is an argument to foo, and the best practice way to pass an argument to a function is to pass it as an argument to the function.
in utils.py:
def foo(bar):
return bar+1
And in your other script:
import utils
bar = 1
bar = utils.foo(bar)
print bar
This is the best practice approach. It follows the correct semantics. It is also testable:
import unittest
import utils
class MyTest(unittest.TestCase):
def setUp(self):
self.bar = 1
def test_bar(self):
self.assertEquals(2, utils.foo(self.bar))
Please consider the following Python modules excerpts:
foo.py:
class Foo:
(...)
bar.py:
import foo
foo = foo.Foo()
The variable foo, which was a module object, is overwritten with a Foo object.
I know that I can use other names for the object, e.g.:
foobar = foo.Foo()
but semantically it makes more sense in my code to have it called foo, since it will be the only instance.
(I tried to workaround this by dropping classes and using modules only, but I went back to using classes because using modules only had "robustness" problems.)
This is kind of a philosophical question, but what is the "right" way of handling this potential object/module names clash?
In my opinion there is nothing wrong with what you are currently doing, but to make it more clear for everyone reading the code I would suggest changing your code to something like the following:
import foo as foo_mod
foo = foo_mod.Foo()
Or alternatively:
from foo import Foo
foo = Foo()
This prevents the name clash so it will be more obvious that the variable foo in your module is not going to refer to the module of the same name.
I've also been favoring the following style nowadays:
import foo
my_foo = foo.Foo()
I prefer this because it keeps module names untouched, and those are are more global and sacred than local variables.
This pattern doesn't seem to bother peeps who use Flask + Celery,
from celery import Celery
def make_celery(app):
celery = Celery(
app.import_name,
backend=app.config['CELERY_RESULT_BACKEND'],
broker=app.config['CELERY_BROKER_URL']
)
...
Obviously, the correct way to create an instance of this class is stalk = Celery() (hehe)