In Python 3, I have a tree of lists and dicts that I get from another library. I'd like to instrument the dicts in that tree with objects containing more behavior (giving a richer model to the simple dict classes). I've tried replacing the class of these objects with a subclass of dict, but that is not allowed:
class MyClass(dict): pass
{}.__class__ = MyClass
That fails with TypeError: __class__ assignment: only for heap types.
So I'm instead trying to write a wrapper or adapter or delegate class:
class InstrumentedDict(object):
"""
Instrument an existing dictionary with additional
functionality, but always reference and mutate
the original dictionary.
>>> orig = {'a': 1, 'b': 2}
>>> inst = InstrumentedDict(orig)
>>> hasattr(inst, '__getitem__')
True
>>> inst.__getitem__('a')
1
>>> inst['a']
1
>>> inst['c'] = 3
>>> orig['c']
3
>>> inst.keys() == orig.keys()
True
"""
def __init__(self, orig):
self._orig = orig
def __getattribute__(self, name):
orig = super(InstrumentedDict, self).__getattribute__('_orig')
return orig.__getattribute__(name)
However, the doctests fail at inst['a'] with TypeError: 'InstrumentedDict' object is not subscriptable. Note, however, that it doesn't fail to invoke __hasattr__ or __getitem__.
I'm hoping to delegate all behavior to the underlying dictionary, and I'd like not to have to think about or explicitly delegate the whole signature of a dictionary.
It's important that whatever this class does should affect the underlying dict (rather than creating separate references to the values). Ideally, it should not impose or negate mutability on the underlying Mapping, but should mirror its behavior.
Is there a simple and elegant solution that meets the specified interface but doesn't require explicit mirroring of the signature (such as in this implementation)?
Edit: To clarify, I want to overlay behavior on existing dictionaries without creating new copies, such that if the instrumented copy is modified, so is the original.
At a risk of completely missing the point of your question...
Is there any reason to build a proxy instead of just subclassing dict? Something like:
class InstrumentedDict(dict):
""" Walks like a dict, talks like a dict... """
Edit after comment:
Ah, I see :) Makes sense...
Seems like UserDict is the answer, check this out:
from collections import UserDict
class InstrumentedDict(UserDict):
def __init__(self, data):
super(InstrumentedDict, self).__init__()
self.data = data
remote_dict = {"a": 1}
instr_dict = InstrumentedDict(remote_dict)
print(instr_dict) # {'a': 1}
instr_dict["b"] = 2
print(instr_dict) # {'a': 1, 'b': 2}
print(remote_dict) # {'a': 1, 'b': 2}
UserDict seems to be a relic from olden days when we couldn't subclass dict directly. But it's useful because it exposes data attribute. And that's pretty much all what the docs say: UserDict
Related
What is the difference between the two class definitions below,
class my_dict1(dict):
def __init__(self, data):
self = data.copy()
self.N = sum(self.values)
The above code results in AttributeError: 'dict' object has no attribute 'N', while the below code compiles
class my_dict2(dict):
def __init__(self, data):
for k, v in data.items():
self[k] = v
self.N = sum(self.values)
For example,
d = {'a': 3, 'b': 5}
a = my_dict1(d) # results in attribute error
b = my_dict2(d) # works fine
By assigning self itself to anything you assign self to a completely different instance than you were originally dealing with, making it no longer the "self". This instance will be of the broader type dict (because data is a dict), not of the narrower type my_dict1. You would need to do self["N"] in the first example for it to be interpreted without error, but note that even with this, in something like:
abc = mydict_1({})
abc will still not have the key "N" because a completely difference instance in __init__ was given a value for the key "N". This shows you that there's no reasonable scenario where you want to assign self itself to something else.
In regards to my_dict2, prefer composition over inheritance if you want to use a particular dict as a representation of your domain. This means having data as an instance field. See the related C# question Why not inherit from List?, the core answer is still the same. It comes down to whether you want to extend the dict mechanism vs. having a business object based on it.
Is there a way to set multiple attributes of the same object on a single line, similarly to the way one assigns values to multiple variables?
If I can write
a,b,c=1,2,3
I would like to have something like
someObject.(a,b,c)=1,2,3
Having the same effect as
someObject.a=1
someObject.b=2
someObject.c=3
def setattrs(_self, **kwargs):
for k,v in kwargs.items():
setattr(_self, k, v)
Use this function like this:
setattrs(obj,
a = 1,
b = 2,
#...
)
You can also define this function on class, but that would be less generic (i.e. apply only to that class instances).
Another answer mentions __dict__.update and it can be rewritten to get rid of quotes: obj.__dict__.update(a=1, b=2), however i would not recommend using this method: it doesn't work with properties and it might be hard to notice if you migrate from simple attributes to properties. Basically, __dict__ is "hidden" attribute, implementation detail, which you shouldn't use unless you really want to change implementation in some way.
First of all you can do the same with object attributes as with other variables:
someObject.a, someObject.b, someObject.c = 1,2,3
Not sure if that is already what you have in mind. If you want to update arbitrary attributes (e.g. in a function) then you could exploit the fact that Python objects store their attributes in a dict, which supports the update() method:
someObject.__dict__.update({"a":1, "b":2, "c":3})
However, this should only be done with care! If one would offer this in a function any attribute of someObject can be changed with that function and even new attributes can be inserted!
You can use map to update the attributes of an object with a dictionary of attributes and values.
attrs = {'a': 1, 'b': 2, 'c': 3}
map(lambda item: setattr(someObject, *item), attrs.iteritems())
In Python 3, you need to surround the map call with a list and convert iteritems to items like so.
list(map(lambda item: setattr(someObject, *item), attrs.items()))
attributes = ['a', 'b', 'c']
values = [1, 2, 3]
for attr, val in zip(attributes, values):
setattr(obj, attr, val)
How can I iterate over an object and assign all it properties to a list
From
a = []
class A(object):
def __init__(self):
self.myinstatt1 = 'one'
self.myinstatt2 = 'two'
to
a =['one','two']
Don't create a full fledged class if you just want to store a bunch of attributes and return a list so that your API can consume it. Use a namedtuple instead. Here is an example.
>>> import collections
>>> Point = collections.namedtuple('Point', ['x', 'y'])
>>> p = Point(1, 2)
>>> p
Point(x=1, y=2)
If your API just expects a sequence (not specifically a list), you can pass p directly. If it needs a list specifically, it is trivial to convert the Point object to a list.
>>> list(p)
[1, 2]
You can even subclass the newly created Point class and add more methods (documentation has details). If namedtuple doesn't meet your needs, consider sub-classing abc.Sequence Abstract Base Class or using it as a mixin.
One approach is to make your class behave like a list by implementing some or all of the container API. Depending on how the external API you're using works, you might only need to implement __iter__. If it needs more, you could always pass it list(a), which will build a list using an iterator.
Here's an example of how easy it can be to add an __iter__ method:
class A(object):
def __init__(self):
self.myAttr1 = "one"
self.myAttr2 = "two"
def __iter__(self):
yield self.myAttr1
yield self.myAttr2
I have a class that inherits from a dictionary in order to add some custom behavior - in this case it passes each key and value to a function for validation. In the example below, the 'validation' simply prints a message.
Assignment to the dictionary works as expected, printing messages whenever items are added to the dict. But when I try to use the custom dictionary type as the __dict__ attribute of a class, attribute assignments, which in turn puts keys/values into my custom dictionary class, somehow manages to insert values into the dictionary while completely bypassing __setitem__ (and the other methods I've defined that may add keys).
The custom dictionary:
from collections import MutableMapping
class ValidatedDict(dict):
"""A dictionary that passes each value it ends up storing through
a given validator function.
"""
def __init__(self, validator, *args, **kwargs):
self.__validator = validator
self.update(*args, **kwargs)
def __setitem__(self, key, value):
self.__validator(value)
self.__validator(key)
dict.__setitem__(self, key, value)
def copy(self): pass # snipped
def fromkeys(validator, seq, v = None): pass # snipped
setdefault = MutableMapping.setdefault
update = MutableMapping.update
def Validator(i): print "Validating:", i
Using it as the __dict__ attribute of a class yields behavior I don't understand.
>>> d = ValidatedDict(Validator)
>>> d["key"] = "value"
Validating: value
Validating: key
>>> class Foo(object): pass
...
>>> foo = Foo()
>>> foo.__dict__ = ValidatedDict(Validator)
>>> type(foo.__dict__)
<class '__main__.ValidatedDict'>
>>> foo.bar = 100 # Yields no message!
>>> foo.__dict__['odd'] = 99
Validating: 99
Validating: odd
>>> foo.__dict__
{'odd': 99, 'bar': 100}
Can someone explain why it doesn't behave the way I expect? Can it or can't it work the way I'm attempting?
This is an optimization. To support metamethods on __dict__, every single instance assignment would need to check the existance of the metamethod. This is a fundamental operation--every attribute lookup and assignment--so the extra couple branches needed to check this would become overhead for the whole language, for something that's more or less redundant with obj.__getattr__ and obj.__setattr__.
What are advantages of using UserDict class?
I mean, what I really get if instead of
class MyClass(object):
def __init__(self):
self.a = 0
self.b = 0
...
m = MyClass()
m.a = 5
m.b = 7
I will write the following:
class MyClass(UserDict):
def __init__(self):
UserDict.__init__(self)
self["a"] = 0
self["b"] = 0
...
m = MyClass()
m["a"] = 5
m["b"] = 7
Edit: If I understand right I can add new fields to an object in a runtime in both cases?
m.c = "Cool"
and
m["c"] = "Cool"
UserDict.UserDict has no substantial added value since Python 2.2, since, as #gs mention, you can now subclass dict directly -- it exists only for backwards compatibility with Python 2.1 and earlier, when builtin types could not be subclasses. Still, it was kept in Python 3 (now in its proper place in the collections module) since, as the docs now mention,
The need for this class has been
partially supplanted by the ability to
subclass directly from dict; however,
this class can be easier to work with
because the underlying dictionary is
accessible as an attribute.
UserDict.DictMixin, in Python 2, is quite handy -- as the docs say,
The module defines a mixin, DictMixin,
defining all dictionary methods for
classes that already have a minimum
mapping interface. This greatly
simplifies writing classes that need
to be substitutable for dictionaries
(such as the shelve module).
You subclass it, define some fundamental methods (at least __getitem__, which is sufficient for a read-only mapping without the ability to get keys or iterate; also keys if you need those abilities; possibly __setitem__, and you have a R/W mapping without the ability of removing items; add __delitem__ for full capability, and possibly override other methods for reasons of performance), and get a full-fledged implementation of dict's rich API (update, get, and so on). A great example of the Template Method design pattern.
In Python 3, DictMixin is gone; you can get almost the same functionality by relying on collections.MutableMapping instead (or just collections.Mapping for R/O mappings). It's a bit more elegant, though not QUITE as handy (see this issue, which was closed with "won't fix"; the short discussion is worth reading).
Subclassing the dict gives you all the features of a dict, like if x in dict:. You normally do this if you want to extend the features of the dict, creating an ordered dict for example.
BTW: In more recent Python versions you can subclass dict directly, you don't need UserDict.
It's tricky to overwrite dict correctly, while UserDict makes it easy. There was some discussion to remove it from Python3, but I believe it was kept for this reason. Example:
class MyDict(dict):
def __setitem__(self, key, value):
super().__setitem__(key, value * 10)
d = MyDict(a=1, b=2) # Oups MyDict.__setitem__ not called
d.update(c=3) # Oups MyDict.__setitem__ not called
d['d'] = 4 # Good!
print(d) # {'a': 1, 'b': 2, 'c': 3, 'd': 40}
UserDict inherit collections.abc.MutableMapping, so don't have those drawback:
class MyDict(collections.UserDict):
def __setitem__(self, key, value):
super().__setitem__(key, value * 10)
d = MyDict(a=1, b=2) # Good: MyDict.__setitem__ correctly called
d.update(c=3) # Good: MyDict.__setitem__ correctly called
d['d'] = 4 # Good
print(d) # {'a': 10, 'b': 20, 'c': 30, 'd': 40}
Well, as of 3.6 there are certainly some disadvantages, as I just found out. Namely, isinstance(o, dict) returns False.
from collections import UserDict
class MyClass(UserDict):
pass
data = MyClass(a=1,b=2)
print("a:", data.get("a"))
print("is it a dict?:", isinstance(data, dict))
Not a dict!
a: 1
is it a dict?: False
Change to class MyClass(dict): and isinstance returns True.
However... with UserDict you can step into its implementation.
(pdb-ing into functions/methods is an easy way to see exactly how they work)
#assumes UserDict
di = MyClass()
import pdb
#pdb will have work if your ancestor is UserDict, but not with dict
#since it is c-based
pdb.set_trace()
di["a"]= 1