I am setting a class property fields using a metaclass:
class MyMeta(type):
def __new__(mcs, name, bases, clsdict):
clsdict['fields'] = {k: v
for k, v in clsdict.items()
if <my_condition>}
return super(MyMeta, mcs).__new__(mcs, name, bases, clsdict)
class MyBaseClass(metaclass=MyMeta):
fields = {}
The following instantiation leads to expected results:
class SubClass(MyBaseClass):
param1 = 1 # meets <my_condition>
>>> SubClass.fields
{param1: 1}
But if I now subclass SubClass, fields is empty:
class SubSubClass(SubClass):
pass
>>> SubSubClass.fields
{}
How would I be able to update the classdict of all classes in inheritance hierarchy so that the fields variable would be updated from base classes?
You need to somehow keep the fields of the superclasses, for example by iterating over the "bases" and using their fields as starting point:
class MyMeta(type):
def __new__(mcs, name, bases, clsdict):
if 'fields' not in clsdict:
clsdict['fields'] = {}
# Initialize "fields" from base classes
for base in bases:
try:
clsdict['fields'].update(base.fields)
except AttributeError:
pass
# Fill in new fields (I included a "trivial" condition here, just use yours instead.)
clsdict['fields'].update({k: v for k, v in clsdict.items() if k.startswith('param')})
return super(MyMeta, mcs).__new__(mcs, name, bases, clsdict)
And it works for SubClass and SubSubClass:
>>> SubClass.fields
{'param1': 1}
>>> SubSubClass.fields
{'param1': 1}
I suggest turning fields into a property descriptor which fetches all the contents of _fields from parent classes. This way you can also more easily customize what happens when there are name conflicts, etc.
class MyMeta(type):
def __new__(mcs, name, bases, clsdict):
# change fields to _fields
clsdict['_fields'] = {k: v
for k, v in clsdict.items()
if <my_condition>}
return super(MyMeta, mcs).__new__(mcs, name, bases, clsdict)
#property
def fields(cls):
# reversed makes most recent key value override parent values
return {k:v
for c in reversed(cls.__mro__)
for k,v in getattr(c, '_fields', {}).items() }
Usage:
class MyBaseClass(metaclass=MyMeta):
fields = {}
class SubClass(MyBaseClass):
param1 = 1
>>> SubClass.fields
{param1: 1}
class SubSubClass(SubClass):
pass
>>> SubSubClass.fields
{param1: 1} # success
Now, usage of SomeChildClass.fields always refers to the metaclass property. The third argument to getattr allows classes with no _fields attribute (such as object) to fail silently.
Using a descriptor also has the advantage of preventing a child class from accidentally overriding the fields attribute:
>>> SubSubClass.fields = 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: can't set attribute
You could also create a setter, if desired, and use it in the __init__ method (i.e., go back to using fields instead of _fields) so that the entirety rest of the class is implementation agnostic:
#fields.setter
def fields(cls, mapping):
try:
cls._fields.update(**mapping)
except AttributeError:
cls._fields = dict(**mapping)
Related
I have a number of classes that inherit a common one. I need the parent class to keep track of a bunch of dependencies/relationships that are defined at the class level. Something like:
class Meta(type):
ALLDEPENDENCIES = {}
def __new__(meta, name, bases, attrs):
if "DEPENDENCIES" in attrs.keys():
for key, value in attrs.items():
if key == "DEPENDENCIES":
meta.ALLDEPENDENCIES.update(attrs["DEPENDENCIES"])
return type.__new__(meta, name, bases, attrs)
class DataTable(DataFrameWrapper, metaclass=Meta):
pass
class Foo(DataTable):
DEPENDENCIES = {"a":1}
class Bar(DataTable):
DEPENDENCIES = {"b":2}
So essentially, as I create new classes (Foo, Bar, Baz...) each of them has a dictionary. I need to merge the info from each dictionary. So I'm using the metaclass, as shown above. Each class as an DEPENDENCIES attribute, and I'm gathering all of those into the ALLDEPENDENCIES attribute defined in the metaclass.
If I do this, it seems to work alright:
import Foo, Bar
print(Foo.ALLDEPENDENCIES)
>> {"a":1, "b":2}
print(Bar.ALLDEPENDENCIES)
>> {"a":1, "b":2}
However, when working if obj instances, the ALLDEPENDENCIES attributes is missing:
f = Foo()
b = Bar()
print(f.ALLDEPENDENCIES)
print(b.ALLDEPENDENCIES)
Attribute error - there is no ALLDEPENDENCIES.
I thought that the class attribute defined in the metaclass would be accessible from self.myattribute in the instances, just like DEPENDENCIES is. What am I doing wrong?
Meta describes how to create class but not what class that will be.
Meta != Parent with inherited attributes
So you have to pass proper attributes into new class:
class Meta(type):
_a = {}
def __new__(meta, name, bases, attrs):
if "d" in attrs:
meta._a.update(attrs["d"])
attrs["a"] = meta._a
return type.__new__(meta, name, bases, attrs)
class Data:
pass
class DataTable(Data, metaclass=Meta):
pass
class Foo(DataTable):
d = {"a":1}
class Bar(DataTable):
d = {"b":2}
f = Foo()
print(Foo.a)
print(f.a)
{'a': 1, 'b': 2}
{'a': 1, 'b': 2}
Instance class attribute search does not go into the metaclass - just to the class. The metaclass could set ALLDEPENDANCIES in each new class, with a single line in its __new__, but if you want cleaner code, in the sense the dictionary is not aliased everywhere, you can just access the attribute through the class.
Using your code, as is:
Foo().__class__.ALLDEPENDANCIES
will work from anywhere (just as `type(Foo()).ALLDEPENDANCIES).
In order to set the attribute in the new classes, so that it will be visible in the newly created classes, an option is:
from types import MappingProxyType
class Meta(type):
ALLDEPENDANCIES = {}
ALLDEPSVIEW = MappingProxyType(ALLDEPENDANCIES)
def __new__(meta, name, bases, attrs):
if "DEPENDANCIES" in attrs.keys():
for key, value in attrs.items():
if key == "DEPENDANCIES":
meta.ALLDEPENDANCIES.update(attrs["DEPENDANCIES"])
new_cls = super().__new__(meta, name, bases, attrs)
new_cls.ALLDEPENDANCIES = meta.ALLDEPSVIEW
return new_cls
(Inserting the new attr in attrs before calling type.__new__ will also work)
Here I do two other extras: (1) call super().__new__ instead of hardcoding a call to type.__new__: this allows your metaclass to be composable with other metaclasses, which might be needed if one of your classes will cross with other metaclass (for example, if you are using abstract base classes from abc or collections.abc). And (2) using a MappingProxyType which is a "read only" dictionary view, and will stop acidental direct updates of the dict through classes or instances.
In my object's init, I would like to create object properties from an iterable. For example:
class MyClass(object):
def __init__(self, parameters):
attributes = ['name',
'memory',
'regressors',
'use_const']
for attr_name in attributes():
try:
attr_val = parameters[attr_name]
except KeyError:
raise Error("parameters must contain {}".format(attr_name))
setattr(self, attr_name, attr_val)
This lets me get the attributes that I want. However, what I lose compared to defining
#property
def name(self):
"""str: This class' name"""
return self._name
is that I don't get the docstrings for the properties now.
I'd like to have the docstrings for each property (for my auto-generated documentation), but I'd also like to use an iterable instead of having to define each property separately. For example, can I turn attributes into a dict with the docstring as a value, and set the attribute's docstring dynamically?
Can I have my cake and eat it too?
You can only set property objects on the class. You can do this in a loop, but this has to be done when building the class, not instances.
Simply produce property objects:
def set_property(cls, name, attr, docstring):
def getter(self):
return getattr(self, attr)
prop = property(getter, None, None, docstring)
setattr(cls, name, prop)
for name in attributes:
attr = '_' + name
docstring = "str: This class' {}".format(name)
set_property(SomeClass, name, attr, docstring)
I suppose i'm misunderstand how type inheritance work in python.
While i'm defining variable inside Parent class, any Child class inherited in parent
referencing same variable from parent.
class Parent(object):
store = dict()
class ChildA(Parent):
pass
class ChildB(Parent):
pass
ChildA.store['key1'] = 'val'
ChildB.store['key2'] = 'val'
print ChildB.store['key1'] == ChildA.store['key2']
What i'm trying to achieve is store dictionary instance to be created in every Child class inherited from Parent. So referencing ChildB.store['key1'] would raise KeyError
I have tried to use __new__ to create dictionary instance while type is creating:
class NewParent(object):
def __new__(cls, *args, **kwargs):
rv = super(NewParent,cls).__new__(cls, *args, **kwargs)
rv.store = dict()
return rv
But it's seems like __new__ running only before instantiating Child class, so referencing variable via type (e.g. Child.store is raising AttributeError)
So is there any way to achieve behavior i want?
You want to use a metaclass, which lets you initialize a class definition sort of like how a constructor lets you initalize an instance. For more details, see http://eli.thegreenplace.net/2011/08/14/python-metaclasses-by-example/.
Example:
#!/usr/bin/env python2
class ParentMeta(type):
def __new__(meta, name, bases, dct):
dct['store'] = dict()
return super(ParentMeta, meta).__new__(meta, name, bases, dct)
class Parent(object):
__metaclass__ = ParentMeta
class ChildA(Parent):
pass
class ChildB(Parent):
pass
ChildA.store['key1'] = 'val'
ChildB.store['key2'] = 'val'
print ChildB.store['key1'] == ChildA.store['key2']
will result in
Traceback (most recent call last):
File "test.py", line 20, in <module>
print ChildB.store['key1'] == ChildA.store['key2']
KeyError: 'key1'
I am trying to find a way to set dict values encapsulated into a class, for example using __getattr__ i can return the internal dict value, however the __setattr__ is called even when attributes exists, making my implementation ugly. The example below is simplified my actual class inherits from a Subject class (the subject part of the observer pattern)
i am trying to achieve something like this:
obj = Example()
obj.username = 'spidername' # all OK username is a key in the internal dict
# but company is not a key in the internal dict so
obj.company = 'ABC' # will raise AttributeError
and i am asking if there is a better way than the way i am doing below:
class Example(object):
def __init__(self, table=None):
self._fields = {}
self._table = table
def _set_fields(self):
"""
this method will be implemented by
subclasses and used to set fields names and values
i.e.
self._field['username'] = Field(default='unknown', is_primary=False)
"""
raise NotImplementedError
def __getattr__(self, name):
"""
great this method is only called when "name"
is not an attribute of this class
"""
if name in self._fields:
return self._fields[name].value
return None
def __setattr__(self, name, value):
"""
not so great, this method is called even for
attributes that exists in this class
is there a better way to do the following?
this can be in __init__, but its still ugly
"""
attribs = ['_fields', '_table']
if name in attribs:
super(Example, self).__setattr__(name, value)
else:
if name in self._fields:
self._fields[name].value = value
else:
raise AttributeError
EDIT: adjusted comment in code, added missin quotes
The problem is that the attributes don't exist when they are first assigned. In __init__, when you first assign a dict to _fields, _fields is not an attribute. It only becomes an existing attribute after its been assigned. You could use __slots__ if you know in advance what the attributes are, but my guess is that you don't. So my suggestion would be to insert these into the instance dict manually:
class Example(object):
def __init__(self, table=None):
self.__dict__['_fields'] = {}
self.__dict__['_table'] = table
...
def __setattr__(self, name, value):
if name in self._fields:
self._fields[name].value = value
else:
raise AttributeError
However, with this implementation, the only way you can add or change instance attributes later would be through __dict__. But I assume this is not likely.
FWIW, your overall goal can be achieved directly just by using __slots__:
>>> class Example(object):
__slots__ = ['username']
>>> obj = Example()
>>> obj.username = 'spiderman'
>>> obj.company = 'ABC'
Traceback (most recent call last):
File "<pyshell#18>", line 1, in <module>
obj.company = 'ABC'
AttributeError: 'Example' object has no attribute 'company'
Currently __setattr__ only works for instance. Is there any similar method for class? I am asking this question because I want to collect the list of defined attribute in order when user define it in class as below:
class CfgObj(object):
_fields = []
def __setattr__(self, name, value):
self._fields.append([name, value])
object.__setattr__(self, name, value)
class ACfg(CfgObj):
setting1 = Field(str, default='set1', desc='setting1 ...')
setting2 = Field(int, default=5, desc='setting2...')
I know the above code will not work as expected because the __setattr__ only called by instance as below:
acfg = ACfg()
acfg.c = 1
acfg._fields == [['c', 1]]
So, is there any equivalent __setattr__ for python class? The main purpose is to collect the define attribute in order when user define it in class.
Yes, but that's not how you want to do it.
class MC(type):
def __init__(cls, name, bases, dct):
print dct
super(MC, cls).__init__(name, bases, dct)
class C(object):
__metaclass__ = MC
foo = 42
If you define __setattr__() on the metaclass of a class, it will be called when setting attributes on the class, but only after creating the class:
>>> class Meta(type):
... def __setattr__(cls, name, value):
... print "%s=%r" % (name, value)
...
>>> class A(object):
... __metaclass__ = Meta
...
>>> A.a = 1
a=1
But it won't work at the time of class definition, so it's probably not what you want.
Getting the class attributes in the metaclass __init__() works, but you loose the order of definition (and multiple definitions as well).
What I would do to solve your problem - but not your question - is to set the timestamp of the field creation create a counter of Field objects and set the current value of the counter to the created one:
class Field(object):
count = 0
def __init__(self, value, default=None, desc=None):
self.value = value
self.default = default
self.desc = desc
# Here comes the magic
self.nth = Field.count
Field.count += 1
# self.created_at = time.time()
Then I would create a method for returning all fields ordered by its counter value:
class CfgObj(object):
def params(self):
ns = dir(self)
fs = [getattr(self, field)
for field in ns
if isinstance(getattr(self, field), Field)]
# fs = sorted(fs, key=lambda f: f.created_at)
fs = sorted(fs, key=lambda f: f.nth)
return fs
Its usage is intuitive:
class ACfg(CfgObj):
setting1 = Field(str, default='set1', desc='setting1 ...')
setting2 = Field(int, default=5, desc='setting2...')
print ACfg().params()
Clearly the fields are ordered by time of object creation, not field creation, but it can be enough for you. Is it?