Develop Reference

Automatic attribute copying from member class to parent class at class definition using ABC's __init_subclass__ in Python

I have this code:
from abc import ABC
class ConfigProto(ABC):
def __init__(self, config_dict):
self._config_dict = config_dict
def parse(self, cfg_store: dict) -> None:
# do something with the config dict and the cfg_store and setup attributes
...
class ConfigurableComponent(ABC):
class Config(ConfigProto):
...
def __init__(self, parsed_config: Config):
for key, value in parsed_config.__dict__.items():
setattr(self, key, value)
def __init_subclass__(cls, **kwargs):
"""
Adds relevant attributes from the member Config class to itself.
This is done for all subclasses of ConfigurableComponent, so that the attributes are not required to be
specified in two places (in the Config class and the component class).
If there are significant differences between the Config attributes and the component attributes, they can be
also specified in the component class, and then they will not be overwritten by the Config attributes.
"""
super().__init_subclass__(**kwargs)
config = getattr(cls, 'Config', None)
if config:
# copy non-callable, non-protected, non-private attributes from config to component class
conf_class_attributes = [attr_ for attr_ in dir(config)
if not attr_.startswith('_')
and not callable(getattr(config, attr_))]
for attr_name in conf_class_attributes:
# ignore private attributes, methods, and members already defined in the class
if not attr_name.startswith('_') \
and not callable(getattr(config, attr_name)) \
and not hasattr(cls, attr_name):
setattr(cls, attr_name, getattr(config, attr_name))
class ExampleComponent(ConfigurableComponent):
class Config(ConfigProto):
param1: int
param2: str
def parse(self, cfg_store: dict) -> None:
...
example_component = ExampleComponent(ExampleComponent.Config(config_dict={'param1': 1, 'param2': 'test'}))
assert hasattr(example_component, 'param1')
It does not work. When subclassing the parent class ConfigurableComponent(ABC), the __init_subclass__ is called, but the config class variable does not contain the attributes defined in ExampleComponent.Config despite it showing it is the correct type.
I expect the __init_subclass__ method to be called after the subclass is defined (including its members). Still, even though the members are here (there is a member named "Config" in the ExampleComponent class), they are not initialised - the Config class seems to be empty.
So far I think the reason is that the member class gets fully initialised only after the owner class gets instantiated into an object, but I am not sure and can't seem to find the details in the documentation.
Does anybody have an idea how to make this code work so that I can:
define attributes in the member class Config and get them added to the owning class automatically when subclassing the ConfigurableComponent class?

Python: creating a class instance via static method vs class method

Let's say I have a class and would like to implement a method which creates an instance of that class. What I have is 2 options:
static method,
class method.
An example:
class DummyClass:
def __init__(self, json):
self.dict = json
#staticmethod
def from_json_static(json):
return DummyClass(json)
#classmethod
def from_json_class(cls, json):
return cls(json)
Both of the methods work:
dummy_dict = {"dummy_var": 124}
dummy_instance = DummyClass({"test": "abc"})
dummy_instance_from_static = dummy_instance.from_json_static(dummy_dict)
print(dummy_instance_from_static.dict)
> {'dummy_var': 124}
dummy_instance_from_class = DummyClass.from_json_class(dummy_dict)
print(dummy_instance_from_class.dict)
> {'dummy_var': 124}
What I often see in codes of other people is the classmethod design instead of staticmethod. Why is this the case?
Or, rephrasing the question to possibly get a more comprehensive answer: what are the pros and cons of creating a class instance via classmethod vs staticmethod in Python?

Two big advantages of the #classmethod approach:
First, you don't hard-code the name. Given modern refactoring tools in IDEs, this isn't as big of a deal, but it is nice to not have your code break if you change the name of your Foo, class to Bar::
class Bar:
#statmicmethod
def make_me():
return Foo()
Another advantage (at least, you should understand the difference!) is how this behaves with inheritance:
class Foo:
#classmethod
def make_me_cm(cls):
return cls()
#staticmethod
def make_me_sm():
return Foo()
class Bar(Foo):
pass
print(Bar.make_me_cm()) # it's a Bar instance
print(Bar.make_me_sm()) # it's a Foo instance

Type hinting a class decorator that returns a subclass

I have a set of unrelated classes (some imported) which all have a common attribute (or property) a of type dict[str, Any].
Within a there should be another dict under the key "b", which I would like to expose on any of these classes as an attribute b to simplify inst.a.get("b", {})[some_key] to inst.b[some_key].
I have made the following subclass factory to work as a class decorator for local classes and a function for imported classes.
But so far I'm failing to type hint its cls argument and return value correctly.
from functools import wraps
def access_b(cls):
#wraps(cls, updated=())
class Wrapper(cls):
#property
def b(self) -> dict[str, bool]:
return self.a.get("b", {})
return Wrapper
MRE of my latest typing attemp (with mypy 0.971 errors):
from functools import wraps
from typing import Any, Protocol, TypeVar
class AProtocol(Protocol):
a: dict[str, Any]
class BProtocol(AProtocol, Protocol):
b: dict[str, bool]
T_a = TypeVar("T_a", bound=AProtocol)
T_b = TypeVar("T_b", bound=BProtocol)
def access_b(cls: type[T_a]) -> type[T_b]:
#wraps(cls, updated=())
class Wrapper(cls): # Variable "cls" is not valid as a type & Invalid base class "cls"
#property
def b(self) -> dict[str, bool]:
return self.a.get("b", {})
return Wrapper
#access_b
class Demo1:
"""Local class."""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo1 = Demo1({"b": {"allow_X": True}})
demo1.b["allow_X"] # "Demo1" has no attribute "b"
class Demo2:
"""Consider me an imported class."""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo2 = access_b(Demo2)({"b": {"allow_X": True}}) # Cannot instantiate type "Type[<nothing>]"
demo2.b["allow_X"]
I do not understand why cls is not valid as a type, even after reading https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases.
I understand I should probably not return a Protocol (I suspect that is the source of Type[<nothing>]), but I don't see how I could specify "returns the original type with an extension".
PS1. I have also tried with a decorator which adds b dynamically, still failed to type it...
PS2. ...and with a decorator which uses a mixin as per #DaniilFajnberg's answer, still failing.
References:
functools.wraps(cls, update=()) from https://stackoverflow.com/a/65470430/17676984

(Type) Variables as base classes?
This is actually a really interesting question and I am curious about what solutions other people come up with.
I read up a little on these two errors:
Variable "cls" is not valid as a type / Invalid base class "cls"
There seems to be an issue here with mypy that has been open for a long time now. There doesn't seem to be a workaround yet.
The problem, as I understand it, is that no matter how you annotate it, the function argument cls will always be a type variable and that is considered invalid as a base class. The reasoning is apparently that there is no way to make sure that the value of that variable isn't overwritten somewhere.
I honestly don't understand the intricacies well enough, but it is really strange to me that mypy seems to treat a class A defined via class A: ... different than a variable of Type[A] since the former should essentially just be syntactic sugar for this:
A = type('A', (object,), {})
There was also a related discussion in the mypy issue tracker. Again, hoping someone can shine some light onto this.
Adding a convenience property
In any case, from your example I gather that you are not dealing with foreign classes, but that you define them yourself. If that is the case, a Mix-in would be the simplest solution:
from typing import Any, Protocol
class AProtocol(Protocol):
a: dict[str, Any]
class MixinAccessB:
#property
def b(self: AProtocol) -> dict[str, bool]:
return self.a.get("b", {})
class SomeBase:
...
class OwnClass(MixinAccessB, SomeBase):
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
demo1 = OwnClass({"b": {"allow_X": True}})
print(demo1.b["allow_X"])
Output: True
No mypy issues in --strict mode.
Mixin with a foreign class
If you are dealing with foreign classes, you could still use the Mix-in and then use functools.update_wrapper like this:
from functools import update_wrapper
from typing import Any, Protocol
class AProtocol(Protocol):
a: dict[str, Any]
class MixinAccessB:
"""My mixin"""
#property
def b(self: AProtocol) -> dict[str, bool]:
return self.a.get("b", {})
class Foreign:
"""Foreign class documentation"""
def __init__(self, a: dict[str, Any]):
self.a = a.copy()
class MixedForeign(MixinAccessB, Foreign):
"""foo"""
pass
update_wrapper(MixedForeign, Foreign, updated=())
demo2 = MixedForeign({"b": {"allow_X": True}})
print(demo2.b["allow_X"])
print(f'{MixedForeign.__name__=} {MixedForeign.__doc__=}')
Output:
True
MixedForeign.__name__='Foreign' MixedForeign.__doc__='Foreign class documentation'
Also no mypy issues in --strict mode.
Note that you still need the AProtocol to make it clear that whatever self will be in that property follows that protocol, i.e. has an attribute a with the type dict[str, Any].
I hope I understood your requirements correctly and this at least provides a solution for your particular situation, even though I could not enlighten you on the type variable issue.

How class is constructed?

For the below code in python 3,
class Spam(object):
def __init__(self,name):
self.name = name
def bar(self):
print('Am Spam.bar')
metaclass for Spam is type and base class for Spam is object.
My understanding is,
the purpose of base class is to inherit the properties. Metaclass is to construct the given class definition, as shown below,
body= \
"""
def __init__(self, name):
self.name = name
def bar(self):
print('Am ', self.name)
"""
clsdict = type.__prepare__('type', 'Spam', (object,))
exec(body, globals(), clsdict)
Spam = type('Spam', (object,), clsdict)
s = Spam('xyz')
s.bar()
Code is tested here.
With the given syntax def __prepare__(metacls, name, bases) to use,
Does __prepare__() require passing 'type' as first argument?

type.__prepare__ is a bit special, in that it ignores all and any arguments passed to it and returns an empty dict.
>>> type.__prepare__()
{}
That said, you are not calling __prepare__ correctly. It is called with: the name of the class to be created, its bases and any keyword arguments the class is being created with. __prepare__ is called as metaclass.__prepare__(name, bases, **kwds) Thus,
class MyClass(SomeBase, arg="value", metaclass=MyMeta):
...
will have __prepare__ called as:
MyMeta.__prepare__("MyClass", (SomeBase,), arg="value")
However, most user defined meta classes define their __prepare__ as a classmethod meaning the metaclass is implicitly passed. Meaning your __prepare__ definition can look like:
#classmethod
def __prepare__(metaclass, name, bases, **kwargs):
...
But __prepare__ is still called in the same way as before. It is through the magic of descriptors that the metaclass argument is added.

class __init__ (not instance __init__)

Here's a very simple example of what I'm trying to get around:
class Test(object):
some_dict = {Test: True}
The problem is that I cannot refer to Test while it's still being defined
Normally, I'd just do this:
class Test(object):
some_dict = {}
def __init__(self):
if self.__class__.some_dict == {}:
self.__class__.some_dict = {Test: True}
But I never create an instance of this class. It's really just a container to hold a group of related functions and data (I have several of these classes, and I pass around references to them, so it is necessary for Test to be it's own class)
So my question is, how could I refer to Test while it's being defined, or is there something similar to __init__ that get's called as soon as the class is defined? If possible, I want self.some_dict = {Test: True} to remain inside the class definition. This is the only way I know how to do this so far:
class Test(object):
#classmethod
def class_init(cls):
cls.some_dict = {Test: True}
Test.class_init()

The class does in fact not exist while it is being defined. The way the class statement works is that the body of the statement is executed, as a block of code, in a separate namespace. At the end of the execution, that namespace is passed to the metaclass (such as type) and the metaclass creates the class using the namespace as the attributespace.
From your description, it does not sound necessary for Test to be a class. It sounds like it should be a module instead. some_dict is a global -- even if it's a class attribute, there's only one such attribute in your program, so it's not any better than having a global -- and any classmethods you have in the class can just be functions.
If you really want it to be a class, you have three options: set the dict after defining the class:
class Test:
some_dict = {}
Test.some_dict[Test] = True
Use a class decorator (in Python 2.6 or later):
def set_some_dict(cls):
cls.some_dict[cls] = True
#set_some_dict
class Test:
some_dict = {}
Or use a metaclass:
class SomeDictSetterType(type):
def __init__(self, name, bases, attrs):
self.some_dict[self] = True
super(SomeDictSetterType, self).__init__(name, bases, attrs)
class Test(object):
__metaclass__ = SomeDictSetterType
some_dict = {}

You could add the some_dict attribute after the main class definition.
class Test(object):
pass
Test.some_dict = {Test: True}

I've tried to use classes in this way in the past, and it gets ugly pretty quickly (for example, all the methods will need to be class methods or static methods, and you will probably realise eventually that you want to define certain special methods, for which you will have to start using metaclasses). It could make things a lot easier if you just use class instances instead - there aren't really any downsides.
A (weird-looking) alternative to what others have suggested: you could use __new__:
class Test(object):
def __new__(cls):
cls.some_dict = {cls: True}
Test()
You could even have __new__ return a reference to the class and use a decorator to call it:
def instantiate(cls):
return cls()
#instantiate
class Test(object):
def __new__(cls):
cls.some_dict = {cls: True}
return cls

You can also use a metaclass (a function here but there are other ways):
def Meta(name, bases, ns):
klass = type(name, bases, ns)
setattr(klass, 'some_dict', { klass: True })
return klass
class Test(object):
__metaclass__ = Meta
print Test.some_dict

Thomas's first example is very good, but here's a more Pythonic way of doing the same thing.
class Test:
x = {}
#classmethod
def init(cls):
# do whatever setup you need here
cls.x[cls] = True
Test.init()