Develop Reference

Python extend the __dict__ object with another dict

Let's say I have a class of Person and I want to assign new properties for each instance but I also want to keep track of said new properties, something like:
class Person:
def __init__(self, **kwargs):
self.props = {}
for arg in kwargs:
self.props[arg] = self.__dict__[arg] = kwargs[arg]
But for example, the following code would show why this doesn't gets me what I need:
person = Person(name='Tomer')
person.props['name'] = 'Michael'
print(person.name)
# >> 'Tomer'
How can I keep a reference to the added attributes with the option to edit their source?

The __dict__ object is the dictionary object of your class or instance. There is no need to directly manipulate that, because the class can handle setting the attributes itself. You can simply set the attributes directly without the need of an intermediate props:
class Person:
def __init__(self, **kwargs):
for k, v in kwargs.items():
setattr(self, k, v)
Then you can just say:
person = Person(name='Tomer')
print(person.name) # output: "Tomer"
person.name = 'Michael'
print(person.name) # output: "Michael"
I'm not sure if that is what you want, though.

Got my answer, you can just set the __getattr__ function on the Person class which would execute when access to a variable that is not in the class and parent class is accessed.

class P(dict):
def __init__(self, *k, **kwargs):
self.__dict__ = self
super().__init__(*k, **kwargs)
p = P(name = "me", age = 40)
>>> p['name'] == p.name == "me"
True
etc.

Ruby like DSL in Python

I'm currently writing my first bigger project in Python, and I'm now wondering how to define a class method so that you can execute it in the class body of a subclass of the class.
First to give some more context, a slacked down (I removed everything non essential for this question) example of how I'd do the thing I'm trying to do in Ruby:
If I define a class Item like this:
class Item
def initialize(data={})
#data = data
end
def self.define_field(name)
define_method("#{name}"){ instance_variable_get("#data")[name.to_s] }
define_method("#{name}=") do |value|
instance_variable_get("#data")[name.to_s] = value
end
end
end
I can use it like this:
class MyItem < Item
define_field("name")
end
item = MyItem.new
item.name = "World"
puts "Hello #{item.name}!"
Now so far I tried achieving something similar in Python, but I'm not happy with the result I've got so far:
class ItemField(object):
def __init__(self, name):
self.name = name
def __get__(self, item, owner=None):
return item.values[self.name]
def __set__(self, item, value):
item.values[self.name] = value
def __delete__(self, item):
del item.values[self.name]
class Item(object):
def __init__(self, data=None):
if data == None: data = {}
self.values = data
for field in type(self).fields:
self.values[field.name] = None
setattr(self, field.name, field)
#classmethod
def define_field(cls, name):
if not hasattr(cls, "fields"): cls.fields = []
cls.fields.append(ItemField(name, default))
Now I don't know how I can call define_field from withing a subclass's body. This is what I wished that it was possible:
class MyItem(Item):
define_field("name")
item = MyItem({"name": "World"})
puts "Hello {}!".format(item.name)
item.name = "reader"
puts "Hello {}!".format(item.name)
There's this similar question but none of the answers are really satisfying, somebody recommends caling the function with __func__() but I guess I can't do that, because I can't get a reference to the class from within its anonymous body (please correct me if I'm wrong about this.)
Somebody else pointed out that it's better to use a module level function for doing this which I also think would be the easiest way, however the main intention of me doing this is to make the implementation of subclasses clean and having to load that module function wouldn't be to nice either. (Also I'd have to do the function call outside the class body and I don't know but I think this is messy.)
So basically I think my approach is wrong, because Python wasn't designed to allow this kind of thing to be done. What would be the best way to achieve something as in the Ruby example with Python?
(If there's no better way I've already thought about just having a method in the subclass which returns an array of the parameters for the define_field method.)

Perhaps calling a class method isn't the right route here. I'm not quite up to speed on exactly how and when Python creates classes, but my guess is that the class object doesn't yet exist when you'd call the class method to create an attribute.
It looks like you want to create something like a record. First, note that Python allows you to add attributes to your user-created classes after creation:
class Foo(object):
pass
>>> foo = Foo()
>>> foo.x = 42
>>> foo.x
42
Maybe you want to constrain which attributes the user can set. Here's one way.
class Item(object):
def __init__(self):
if type(self) is Item:
raise NotImplementedError("Item must be subclassed.")
def __setattr__(self, name, value):
if name not in self.fields:
raise AttributeError("Invalid attribute name.")
else:
self.__dict__[name] = value
class MyItem(Item):
fields = ("foo", "bar", "baz")
So that:
>>> m = MyItem()
>>> m.foo = 42 # works
>>> m.bar = "hello" # works
>>> m.test = 12 # raises AttributeError
Lastly, the above allows you the user subclass Item without defining fields, like such:
class MyItem(Item):
pass
This will result in a cryptic attribute error saying that the attribute fields could not be found. You can require that the fields attribute be defined at the time of class creation by using metaclasses. Furthermore, you can abstract away the need for the user to specify the metaclass by inheriting from a superclass that you've written to use the metaclass:
class ItemMetaclass(type):
def __new__(cls, clsname, bases, dct):
if "fields" not in dct:
raise TypeError("Subclass must define 'fields'.")
return type.__new__(cls, clsname, bases, dct)
class Item(object):
__metaclass__ = ItemMetaclass
fields = None
def __init__(self):
if type(self) == Item:
raise NotImplementedError("Must subclass Type.")
def __setattr__(self, name, value):
if name in self.fields:
self.__dict__[name] = value
else:
raise AttributeError("The item has no such attribute.")
class MyItem(Item):
fields = ("one", "two", "three")

You're almost there! If I understand you correctly:
class Item(object):
def __init__(self, data=None):
fields = data or {}
for field, value in data.items():
if hasattr(self, field):
setattr(self, field, value)
#classmethod
def define_field(cls, name):
setattr(cls, name, None)
EDIT: As far as I know, it's not possible to access the class being defined while defining it. You can however call the method on the __init__ method:
class Something(Item):
def __init__(self):
type(self).define_field("name")
But then you're just reinventing the wheel.

When defining a class, you cannot reference the class itself inside its own definition block. So you have to call define_field(...) on MyItem after its definition. E.g.,
class MyItem(Item):
pass
MyItem.define_field("name")
item = MyItem({"name": "World"})
print("Hello {}!".format(item.name))
item.name = "reader"
print("Hello {}!".format(item.name))

counter part of __getattr__

I am trying to find a way to set dict values encapsulated into a class, for example using __getattr__ i can return the internal dict value, however the __setattr__ is called even when attributes exists, making my implementation ugly. The example below is simplified my actual class inherits from a Subject class (the subject part of the observer pattern)
i am trying to achieve something like this:
obj = Example()
obj.username = 'spidername' # all OK username is a key in the internal dict
# but company is not a key in the internal dict so
obj.company = 'ABC' # will raise AttributeError
and i am asking if there is a better way than the way i am doing below:
class Example(object):
def __init__(self, table=None):
self._fields = {}
self._table = table
def _set_fields(self):
"""
this method will be implemented by
subclasses and used to set fields names and values
i.e.
self._field['username'] = Field(default='unknown', is_primary=False)
"""
raise NotImplementedError
def __getattr__(self, name):
"""
great this method is only called when "name"
is not an attribute of this class
"""
if name in self._fields:
return self._fields[name].value
return None
def __setattr__(self, name, value):
"""
not so great, this method is called even for
attributes that exists in this class
is there a better way to do the following?
this can be in __init__, but its still ugly
"""
attribs = ['_fields', '_table']
if name in attribs:
super(Example, self).__setattr__(name, value)
else:
if name in self._fields:
self._fields[name].value = value
else:
raise AttributeError
EDIT: adjusted comment in code, added missin quotes

The problem is that the attributes don't exist when they are first assigned. In __init__, when you first assign a dict to _fields, _fields is not an attribute. It only becomes an existing attribute after its been assigned. You could use __slots__ if you know in advance what the attributes are, but my guess is that you don't. So my suggestion would be to insert these into the instance dict manually:
class Example(object):
def __init__(self, table=None):
self.__dict__['_fields'] = {}
self.__dict__['_table'] = table
...
def __setattr__(self, name, value):
if name in self._fields:
self._fields[name].value = value
else:
raise AttributeError
However, with this implementation, the only way you can add or change instance attributes later would be through __dict__. But I assume this is not likely.

FWIW, your overall goal can be achieved directly just by using __slots__:
>>> class Example(object):
__slots__ = ['username']
>>> obj = Example()
>>> obj.username = 'spiderman'
>>> obj.company = 'ABC'
Traceback (most recent call last):
File "<pyshell#18>", line 1, in <module>
obj.company = 'ABC'
AttributeError: 'Example' object has no attribute 'company'

Does python has methods similar to __setattr__ but for python class?

Currently __setattr__ only works for instance. Is there any similar method for class? I am asking this question because I want to collect the list of defined attribute in order when user define it in class as below:
class CfgObj(object):
_fields = []
def __setattr__(self, name, value):
self._fields.append([name, value])
object.__setattr__(self, name, value)
class ACfg(CfgObj):
setting1 = Field(str, default='set1', desc='setting1 ...')
setting2 = Field(int, default=5, desc='setting2...')
I know the above code will not work as expected because the __setattr__ only called by instance as below:
acfg = ACfg()
acfg.c = 1
acfg._fields == [['c', 1]]
So, is there any equivalent __setattr__ for python class? The main purpose is to collect the define attribute in order when user define it in class.

Yes, but that's not how you want to do it.
class MC(type):
def __init__(cls, name, bases, dct):
print dct
super(MC, cls).__init__(name, bases, dct)
class C(object):
__metaclass__ = MC
foo = 42

If you define __setattr__() on the metaclass of a class, it will be called when setting attributes on the class, but only after creating the class:
>>> class Meta(type):
... def __setattr__(cls, name, value):
... print "%s=%r" % (name, value)
...
>>> class A(object):
... __metaclass__ = Meta
...
>>> A.a = 1
a=1
But it won't work at the time of class definition, so it's probably not what you want.
Getting the class attributes in the metaclass __init__() works, but you loose the order of definition (and multiple definitions as well).

What I would do to solve your problem - but not your question - is to set the timestamp of the field creation create a counter of Field objects and set the current value of the counter to the created one:
class Field(object):
count = 0
def __init__(self, value, default=None, desc=None):
self.value = value
self.default = default
self.desc = desc
# Here comes the magic
self.nth = Field.count
Field.count += 1
# self.created_at = time.time()
Then I would create a method for returning all fields ordered by its counter value:
class CfgObj(object):
def params(self):
ns = dir(self)
fs = [getattr(self, field)
for field in ns
if isinstance(getattr(self, field), Field)]
# fs = sorted(fs, key=lambda f: f.created_at)
fs = sorted(fs, key=lambda f: f.nth)
return fs
Its usage is intuitive:
class ACfg(CfgObj):
setting1 = Field(str, default='set1', desc='setting1 ...')
setting2 = Field(int, default=5, desc='setting2...')
print ACfg().params()
Clearly the fields are ordered by time of object creation, not field creation, but it can be enough for you. Is it?

How to find out the nested class hierarchy?

I have a python code like this.
File named mymodule.py
class MyBase(object):
pass
File named data.py
from mymodule import MyBase
class A:
class NestA(MyBase):
pass
class NestB(MyBase):
pass
class B:
class NestA(MyBase):
pass
class NestB(MyBase):
pass
if I have a = A.NestA (not it is referring to a class, a is not the object of class NestA but the class itself) how do I find out what nested class hierarchy does a belong to? a.name gives me NestA so that is not a problem. I want to find out what outer class NestA is part of, i.e class A or class B. How do I do it?

You can do this with the inspect module:
import inspect
a = A.NestA
print a in [x[1] for x in inspect.getmembers(A, inspect.isclass)]
print a in [x[1] for x in inspect.getmembers(B, inspect.isclass)]
Result:
True
False
Addendum:
If you don't know anything about the classes in the module, you can backtrack and get the module.
# for each class in a's module...
for klass in inspect.getmembers(inspect.getmodule(a), inspect.isclass):
# see if a is in that class
if a in [x[1] for x in inspect.getmembers(klass[1], inspect.isclass)]:
print a, "is a member of", klass[0]
Result:
__main__.NestA is a member of A

You can use __qualname__ to get the nested class hierarchy,
A.NestA.__qualname__ == 'A.NestA'

You can do something like this with metaclass programming.
class SetOuterClassType(type):
def __init__(cls, name, bases, attrs):
for attrname, attrvalue in attrs.iteritems():
if getattr(attrvalue, '__set_outerclass__', False):
attrvalue.__outerclass__ = cls
class OuterClassSetter(object):
__metaclass__ = SetOuterClassType
class MyBase(object):
#classmethod
def fullname(cls):
if hasattr(cls,'__outerclass__'):
return '%s.%s' % (
cls.__outerclass__.__name__, cls.__name__ )
else:
return '%s' % cls.__name__
class A(OuterClassSetter):
class NestA(MyBase):
__set_outerclass__ = True
class NestB(MyBase):
__set_outerclass__ = True
class B(OuterClassSetter):
class NestA(MyBase):
__set_outerclass__ = True
class NestB(MyBase):
__set_outerclass__ = True
print A.NestA.fullname() # prints 'A.NestA'
print A.NestB.fullname() # prints 'A.NestB'
print B.NestA.fullname() # prints 'B.NestA'
print B.NestB.fullname() # prints 'B.NestB'