I am using a bunch class to transform a dict to an object.
class Bunch(object):
""" Transform a dict to an object """
def __init__(self, kwargs):
self.__dict__.update(kwargs)
The problem is , i have a key with a dot in its name({'test.this':True}).
So when i call:
spam = Bunch({'test.this':True})
dir(spam)
I have the attibute:
['__class__',
'__delattr__',
...
'__weakref__',
'test.this']
But i can't access it:
print(spam.test.this)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-7-ea63f60f74ca> in <module>()
----> 1 print(spam.test.this)
AttributeError: 'Bunch' object has no attribute 'test'
i got an AttributeError.
How can i access this attribute?
You can use getattr:
>>> getattr(spam, 'test.this')
True
Alternatively, you can get the value from the object's __dict__. Use vars to get spam's dict:
>>> vars(spam)['test.this']
True
Implement __getitem__(self, key):
class D():
def __init__(self, kwargs):
self.__dict__.update(kwargs)
def __getitem__(self, key):
return self.__dict__.get(key)
d = D({"foo": 1, "bar.baz": 2})
print(d["foo"])
print(d["bar.baz"])
Edit:
I don't recommend accessing d.__dict__ directly from a client of a D instance. Client code like this
d = D({"foo": 1, "bar.baz": 2})
print(d.__dict__.get("bar.baz"))
is trying to reach into the underpants of d and requires knowledge about implementation details of D.
A correct suggestion would be to avoid using dot in the variables.
And even if we use somehow, its better to get it using getattr.
getattr(spam, 'test.this')
If we are being stubborn by avoid standards so this may help.
class Objectify(object):
def __init__(self, obj):
for key in obj:
if isinstance(obj[key], dict):
self.__dict__.update(key=Objectify(obj[key]))
else:
self.__dict__.update(key=obj[key])
class Bunch(object):
""" Transform a dict to an object """
def __init__(self, obj, loop=False):
for key in obj:
if isinstance(obj[key], dict):
self.__dict__.update(key=Objectify(obj[key]))
else:
self.__dict__.update(key=obj[key])
spam1 = Bunch({'test': {'this': True}})
print(spam1.test.this)
spam2 = Bunch({'test': {'this': {'nested_this': True}}})
print(spam2.test.this.nested_this)
Not provided test.this as the key. You may want to create a nested dict iterating through the keys having dots.
Try spam["test.this"] or spam.get("test.this")
Related
For a project I'm working on, I want to be able to associate a name with an object. The way I would like to do it is to set the .name attribute of the object to the name I want. What I really need is a function that takes an instance of an object, and returns something that is identical in every way but with a .name attribute. The problem is that I don't know what type of data the object will be ahead of time, so I can't use subclassing for example
Every method I've tried has hit a problem. Trying to give it a .name attribute directly doesnt work, for example:
>>> cats = ['tabby', 'siamese']
>>> cats.name = 'cats'
Traceback (most recent call last):
File "<pyshell#197>", line 1, in <module>
cats.name = 'cats'
AttributeError: 'list' object has no attribute 'name'
Using setattr has the same problem.
I've tried creating a new class that on init copies all attributes from the instance and also has a .name attribute, but this doesn't work either. If I try:
class NamedThing:
def __init__(self, name, thing):
thing_dict = {#not all types have a .__dict__ method
name: getattr(thing, name) for name in dir(thing)
}
self.__dict__ = thing_dict
self.name = name
It copies over the dict without a problem, but for some reason unless I directly call the new methods, python fails to find them, so the object loses all of its functionality. For example:
>>> cats = ['tabby', 'siamese']
>>> named_thing_cats = NamedThing('cats', cats)
>>> named_thing_cats.__repr__()#directly calling .__repr__()
"['tabby', 'siamese']"
>>> repr(named_thing_cats)#for some reason python does not call the new repr method
'<__main__.NamedThing object at 0x0000022814C1A670>'
>>> hasattr(named_thing_cats, '__iter__')
True
>>> for cat in named_thing_cats:
print(cat)
Traceback (most recent call last):
File "<pyshell#215>", line 1, in <module>
for cat in named_thing_cats:
TypeError: 'NamedThing' object is not iterable
I've also tried setting the type and attributes by setting class directly:
class NamedThing:
def __init__(self, name, thing):
thing_dict = {#not all types have a .__dict__ method
name: getattr(thing, name) for name in dir(thing)
}
self.__class__ = type('NamedThing', (type(thing),), thing_dict)
self.name = name
But this runs into a problem depending on what type thing is:
>>> cats = ['tabby', 'siamese']
>>> named_thing_cats = NamedThing('cats', cats)
Traceback (most recent call last):
File "<pyshell#217>", line 1, in <module>
named_thing_cats = NamedThing('cats', cats)
File "C:/Users/61490/Documents/Python/HeirachicalDict/moduleanalyser.py", line 12, in __init__
self.__class__ = type('NamedThing', (type(thing),), thing_dict)
TypeError: __class__ assignment: 'NamedThing' object layout differs from 'NamedThing'
I'm really stuck, help would be great
What you want is called an object proxy. This is some pretty sophisticated stuff, as you're getting into the data model of python and manipulating some pretty fundamental dunder (double underscore) methods in interesting ways
class Proxy:
def __init__(self, proxied):
object.__setattr__(self, '_proxied', proxied)
def __getattribute__(self, name):
try:
return object.__getattribute__(self, name)
except AttributeError:
p = object.__getattribute__(self, '_proxied')
return getattr(p, name)
def __setattr__(self, name, value):
p = object.__getattribute__(self, '_proxied')
if hasattr(p, name):
setattr(p, name, value)
else:
setattr(self, name, value)
def __getitem__(self, key):
p = object.__getattribute__(self, '_proxied')
return p[key]
def __setitem__(self, key, value):
p = object.__getattribute__(self, '_proxied')
p[key] = value
def __delitem__(self, key):
p = object.__getattribute__(self, '_proxied')
del p[key]
The most obvious thing that's going on here is that internally this class has to use the object implementation of the dunders to avoid recursing infinitely. What this does is holds a reference to a proxied object, then if you try to get or set an attribute it will check the proxied object, if the proxied object has that attribute it uses it, otherwise it sets the attribute on itself. For indexing, like with a list, it just directly acts on the proxied object, since the Proxy itself doesn't allow indexing.
If you need to use this in production, there's a package called wrapt you should probably look at instead.
Why not just create a __iter__ magic method with yield from:
class NamedThing():
def __init__(self, name, thing):
self.thing = thing
self.name = name
def __iter__(self):
yield from self.thing
cats = ['tabby', 'siamese']
named_thing_cats = NamedThing('cats', cats)
for cat in named_thing_cats:
print(cat)
Output;
tabby
siamese
Does this work?
class Thingy(list):
def __init__(self, name, thing):
list.__init__(self, thing)
self.name = name
cats = Thingy('cats', ['tabby', 'siamese'])
print(cats.name) # shows 'cats'
for cat in cats:
print(cat) # shows tabby, siamese
Or you could do:
class Thingy:
def __init__(self, name, thing):
self.thing = thing
self.name = name
I came across this question of trying to implement a dictionary using the collections.abc MutableMapping because I was looking for something similar.
For context, I was looking to implement a dictionary that would out of convenience also act as a mutable object so if I write this for example d = CustomDictionary({'a': 4}) then d.a returns 4.
For reference, here is the code posted by Aaron Hall for this particular problem:
from collections.abc import MutableMapping
class D(MutableMapping):
'''
Mapping that works like both a dict and a mutable object, i.e.
d = D(foo='bar')
and
d.foo returns 'bar'
'''
# ``__init__`` method required to create instance from class.
def __init__(self, *args, **kwargs):
'''Use the object dict'''
self.__dict__.update(*args, **kwargs)
# The next five methods are requirements of the ABC.
def __setitem__(self, key, value):
self.__dict__[key] = value
def __getitem__(self, key):
return self.__dict__[key]
def __delitem__(self, key):
del self.__dict__[key]
def __iter__(self):
return iter(self.__dict__)
def __len__(self):
return len(self.__dict__)
# The final two methods aren't required, but nice for demo purposes:
def __str__(self):
'''returns simple dict representation of the mapping'''
return str(self.__dict__)
def __repr__(self):
'''echoes class, id, & reproducible representation in the REPL'''
return '{}, D({})'.format(super(D, self).__repr__(),
self.__dict__)
However I didn't consider the dangers of doing so. Namely, if I created this custom dictionary class then I would expect to have methods. But what if a method name clashes with a key with the same name? For example:
def doSomething(self):
""" A method of CustomDictionary"""
print("hey!")
d = CustomDictionary()
d['a'] = 3
d['doSomething'] = 4
d.doSomething()
would raise a TypeError: 'int' object is not callable since d.doSomething would return 4 which is not a callable function.
What do you think? How would I go about implementing methods for a custom dictionary class while avoiding this problem.
Unfortunately I couldn't comment on the post since I don't have enough reputation but I was hoping this question deserves its own post.
I have a class with __getitem__() function which is subscribable like a dictionary. However, when I try to pass it to a str.format() i get a TypeError. How can I use a class in python with the format() function?
>>> class C(object):
id=int()
name=str()
def __init__(self, id, name):
self.id=id
self.name=name
def __getitem__(self, key):
return getattr(self, key)
>>> d=dict(id=1, name='xyz')
>>> c=C(id=1, name='xyz')
>>>
>>> #Subscription works for both objects
>>> print(d['id'])
1
>>> print(c['id'])
1
>>>
>>> s='{id} {name}'
>>> #format() only works on dict()
>>> print(s.format(**d))
1 xyz
>>> print(s.format(**c))
Traceback (most recent call last):
File "<pyshell#13>", line 1, in <module>
print(s.format(**c))
TypeError: format() argument after ** must be a mapping, not C
As some of the comments mention you could inherit from dict, the reason it doesn't work is that:
If the syntax **expression appears in the function call, the expression must evaluate to a mapping, the contents of which are treated as additional keyword arguments. In the case of a keyword appearing in both expression and as an explicit keyword argument, a TypeError exception is raised.
For it to work you need to implement the Mapping ABC. Something along the lines of this:
from collections.abc import Mapping
class C(Mapping):
id=int()
name=str()
def __init__(self, id, name):
self.id = id
self.name = name
def __iter__(self):
for x in self.__dict__.keys():
yield x
def __len__(self):
return len(self.__dict__)
def __getitem__(self, key):
return self.__dict__[key]
This way you should just be able to use s = '{id}{name}'.format(**c)
rather than s = '{id}{name}'.format(**c.__dict__)
You can also use MutableMapping from collections.abc module if you want to be able to change your class variables like in a dictionary. MutableMapping would also require the implementation of __setitem__ and __delitem__
I extended dict in a simple way to directly access it's values with the d.key notation instead of d['key']:
class ddict(dict):
def __getattr__(self, item):
return self[item]
def __setattr__(self, key, value):
self[key] = value
Now when I try to pickle it, it will call __getattr__ to find __getstate__, which is neither present nor necessary. The same will happen upon unpickling with __setstate__:
>>> import pickle
>>> class ddict(dict):
... def __getattr__(self, item):
... return self[item]
... def __setattr__(self, key, value):
... self[key] = value
...
>>> pickle.dumps(ddict())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getattr__
KeyError: '__getstate__'
How do I have to modify the class ddict in order to be properly pickable?
The problem is not pickle but that your __getattr__ method breaks the expected contract by raising KeyError exceptions. You need to fix your __getattr__ method to raise AttributeError exceptions instead:
def __getattr__(self, item):
try:
return self[item]
except KeyError:
raise AttributeError(item)
Now pickle is given the expected signal for a missing __getstate__ customisation hook.
From the object.__getattr__ documentation:
This method should return the (computed) attribute value or raise an AttributeError exception.
(bold emphasis mine).
If you insist on keeping the KeyError, then at the very least you need to skip names that start and end with double underscores and raise an AttributeError just for those:
def __getattr__(self, item):
if isinstance(item, str) and item[:2] == item[-2:] == '__':
# skip non-existing dunder method lookups
raise AttributeError(item)
return self[item]
Note that you probably want to give your ddict() subclass an empty __slots__ tuple; you don't need the extra __dict__ attribute mapping on your instances, since you are diverting attributes to key-value pairs instead. That saves you a nice chunk of memory per instance.
Demo:
>>> import pickle
>>> class ddict(dict):
... __slots__ = ()
... def __getattr__(self, item):
... try:
... return self[item]
... except KeyError:
... raise AttributeError(item)
... def __setattr__(self, key, value):
... self[key] = value
...
>>> pickle.dumps(ddict())
b'\x80\x03c__main__\nddict\nq\x00)\x81q\x01.'
>>> type(pickle.loads(pickle.dumps(ddict())))
<class '__main__.ddict'>
>>> d = ddict()
>>> d.foo = 'bar'
>>> d.foo
'bar'
>>> pickle.loads(pickle.dumps(d))
{'foo': 'bar'}
That pickle tests for the __getstate__ method on the instance rather than on the class as is the norm for special methods, is a discussion for another day.
First of all, I think you may need to distinguish between instance attribute and class attribute.
In Python official document Chapter 11.1.4 about pickling, it says:
instances of such classes whose dict or the result of calling getstate() is picklable (see section The pickle protocol for details).
Therefore, the error message you're getting is when you try to pickle an instance of the class, but not the class itself - in fact, your class definition will just pickle fine.
Now for pickling an object of your class, the problem is that you need to call the parent class's serialization implementation first to properly set things up. The correct code is:
In [1]: import pickle
In [2]: class ddict(dict):
...:
...: def __getattr__(self, item):
...: super.__getattr__(self, item)
...: return self[item]
...:
...: def __setattr__(self, key, value):
...: super.__setattr__(self, key, value)
...: self[key] = value
...:
In [3]: d = ddict()
In [4]: d.name = "Sam"
In [5]: d
Out[5]: {'name': 'Sam'}
In [6]: pickle.dumps(d)
Out[6]: b'\x80\x03c__main__\nddict\nq\x00)\x81q\x01X\x04\x00\x00\x00nameq\x02X\x03\x00\x00\x00Samq\x03s}q\x04h\x02h\x03sb.'
This has been previously asked on Stack Overflow, but none of the answers seem to address exactly what I need to do. In my case, I want these dynamically-added properties to be a shortcut to store and read values from a database, so unfortunately it's not as easy as in this answer (where a lambda function was used) or this one (where values where stored in a dictionary): I must call other methods of the class.
This is my attempt:
import operator
class Foo(object):
def get_value(self, name):
# read and return value from database
return -1
def set_value(self, name, value):
# store value in database
pass
def add_metadata_property(name):
getter = operator.methodcaller('get_value', name)
setter = operator.methodcaller('set_value', name) # gets value at runtime
setattr(Foo, name, property(getter, setter))
add_metadata_property('spam')
f = Foo()
f.spam # works!
f.spam = 2
The last line, however, raises:
Traceback (most recent call last):
File "<stdin>", line 27, in <module>
TypeError: methodcaller expected 1 arguments, got 2
Any ideas on how to achieve this?
I don't know why you use operator.methodcaller here.
When you call f.spam=2, it will invoke setter.
setter = operator.methodcaller('set_value', name) means setter(r) = r.set_value(name). Make no sense in your case.
I suggest you write this way, using #classmethod:
class Foo(object):
#classmethod
def get_value(self, name):
# read and return value from database
return -1
#classmethod
def set_value(self, name, value):
# store value in database
pass
def add_metadata_property(name):
setattr(Foo, name, property(Foo.get_value, Foo.set_value))
add_metadata_property('spam')
f = Foo()
f.spam # works!
f.spam = 2
If this helped you, please confirm it as the answer. Thanks!
Modifying class template looks a bit odd for me. I would suggest to overload __getattr__() and __setattr__() methods in your case.
class Foo:
def __getattr__(self, name):
print('read and return value from database for ', name)
return 123
def __setattr__(self, name, value):
print('store value', value, 'for', name, 'in database')