Develop Reference

How to evaluate values in a dictionary?

I have a dictionary (d).
I want to use this dictionary with different values for a and b.
How can I update the dictionary's output without calling the whole dictionary again?
a=1
b=2
d = {'value_1': a+b , 'value_2':a-b}

Based on your feedback to my first answer, it sounds like what you want is something that behaves more-or-less like a spreadsheet. Below is how to implement one that is very dictionary-like. It's based on Raymond Hettinger's ActiveState recipe by that name with some modifications and extensions.
Note that except for the special case of keyword arguments passed when an instance of the class is created, the values in it should all be strings, not numerical values.
Also note that since it uses eval(), for security purposes it should only be used with input from trusted sources.
I think it's important to understand that although the Spreadsheet class presented below isn't technically a dictionary, it behaves a lot like (a subclass of) one and if used instead of a regular dictionary will give you the capabilities you want. See the description of mapping in the online documentation's glossary.
from collections.abc import MutableMapping
class SpreadSheet(MutableMapping):
def __init__(self, tools=None, **kwargs):
self._cells = {}
for key, value in kwargs.items():
self._cells[key] = value if isinstance(value, str) else str(value)
self._tools = {'__builtins__': None} # Prevent eval() from supplying.
if tools is not None:
self._tools.update(tools) # Add any caller-supplied functions.
def clear(self):
return self._cells.clear()
def copy(self):
return self._cells.copy()
def __contains__(self, key):
return key in self._cells
def __setitem__(self, key, formula):
self._cells[key] = formula
def __getitem__(self, key):
return eval(self._cells[key], self._tools, self)
def __len__(self):
return len(self._cells)
def __iter__(self):
return iter(self._cells)
def __delitem__(self, key):
del self._cells[key]
def getformula(self, key):
""" Return raw un-evaluated contents of cell. """
return self._cells[key]
def update(self, *args, **kwargs):
for k, v in dict(*args, **kwargs).items():
self[k] = v
Sample usage:
d = SpreadSheet(a=1, b=2)
d.update({'x': 'x1',
'x1': 'a+2',
'x2': 'b+1',
'x3': 'a+b'})
xx = d['x']
print(xx) # -> 3

You could do it by storing functions in a separate dictionary and then create new ones by evaluating it.
Here's a simple example illustrating what I'm suggesting:
funcs = {'value_1': lambda: a+b,
'value_2': lambda: a-b}
a=1
b=2
d = {k: v() for k, v in funcs.items()}
print(d) # -> {'value_1': 3, 'value_2': -1}
a=3
b=5
d = {k: v() for k, v in funcs.items()}
print(d) # -> {'value_1': 8, 'value_2': -2}

python class behaves like dictionary-or-list-data-like

In python3 console, input those:
>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=4, micro=3, releaselevel='final', serial=0)
>>> type(sys.version_info) # this is class type
<class 'sys.version_info'>
>>> sys.version_info[0:2] # ?? But it acts like a list-data-like
(3, 4)
My questions are:
How can a class act like dictionary-or-list-data-like?
May give an example to construct a class like this?
Is there some documentation about
this?

Python contains several methods for emulating container types such as dictionaries and lists.
In particular, consider the following class:
class MyDict(object):
def __getitem__(self, key):
# Called for getting obj[key]
def __setitem__(self, key, value):
# Called for setting obj[key] = value
If you write
obj = MyDict()
Then
obj[3]
will call the first method, and
obj[3] = 'foo'
will call the second method.
If you further want to support
len(obj)
then you just need to add the method
def __len__(self):
# Return here the logical length
Here is an example of a (very inefficient) dictionary implemented by a list
class MyDict(object):
def __init__(self, seq=None):
self._vals = list(seq) if seq is not None else []
def __getitem__(self, key):
return [v[1] for v in self._vals if v[0] == key][0]
def __setitem__(self, key, val):
self._vals = [v for v in self._vals if v[0] != key]
self._vals.append((key, val))
def __len__(self):
return len(self._vals)
You can use it pretty much like a regular dict:
obj = MyDict()
obj[2] = 'b'
>>> obj[2]
'b'

It's quite easy ... All you need to do is define a __getitem__ method that handles slicing or integer/string lookup. You can do pretty much whatever you want...
class Foo(object):
def __init__(self, bar, baz):
self.bar = bar
self.baz = baz
def __getitem__(self, ix):
return (self.bar, self.baz).__getitem__(ix)
Here's a cheat sheet of what will be passed to __getitem__ as ix in the following situations:
f[1] # f.__getitem__(1)
f[1:] # f.__getitem__(slice(1, None, None))
f[1:, 2] # f.__getitem__( (slice(1, None, None), 2) )
f[1, 2] # f.__getitem__( (1, 2) )
f[(1, 2)] # f.__getitem__( (1, 2) )
The trick (which can be slightly non-trivial) is simply writing __getitem__ so that it looks at the type of the object that was passed and then does the right thing. For my answer, I cheated by creating a tuple in my __getitem__ and then I called __getitem__ on the tuple (since it already does the right thing in all of the cases that I wanted to support)
Here's some example usage:
>>> f = Foo(1, 2)
>>> f[1]
2
>>> f[0]
1
>>> f[:]
(1, 2)
note that you don't typically need to even do this yourself. You can create a named tuple to do the job for you:
from collections import namedtuple
Foo = namedtuple('Foo', 'bar, baz')
And usage is pretty much the same:
>>> f = Foo(1, 2)
>>> f[1]
2
>>> f[0]
1
>>> f[:]
(1, 2)
The main difference here is that our namedtuple is immutable. Once created, we can't change it's members.

i think in python like ECMAScript (aka javascript) class is a dictionary or associative array(associative array). since you can add a property or method to your class at runtime.(see)
class A(object):
def __init__(self):
self.x = 0
a = A()
a.y=5
print a.y # 5
if you want write a class like that you can use __getitem__ and __setitem__ methods:
class A(object):
class B(object):
def __init__(self, x, y):
self.vals = (x, y)
def __getitem__(self, key):
return self.vals[key]
def __setitem__(self, key, val):
self.vals[key] = val
def __len__(self):
return len(self.__vals)
def __init__(self, x, y):
self.b = self.B(x,y)
a = A('foo','baz')
print type(a.b) # __main__.b __main__ because we run script straightly
print a.b[:] # ('foo', 'baz')

You can achieve the same behaviour by overriding getitem() and setitem() in your class.
class Example:
def __getitem__(self, index):
return index ** 2
>>> X = Example()
>>> X[2]
>>> 4
You can override setitem() too in your class for achieving the setter kind of thing.

Python make dictionary items accessible as object property

This is mostly syntactic sugar but I'd like to access the items of a dictionary as object properties.
Example:
class CoolThing():
def __init__(self):
self.CoolDict = {'a': 1, 'b': 2}
and I'd like to have
my_cool_thing.a # => 1
my_cool_thing.b # => 2
Edit: some code of a potential solution with a nested structure with dot notation: device.property.field
class Parameters():
def __init__(self, ids, devices):
self._ids = ids
self._devices = devices
for p in self._devices:
p = p[0]
if self.__dict__.get(p.device) is None:
self.__dict__[p.device] = SmartDict()
else:
if self.__dict__[p.device].get(p.property) is None:
self.__dict__[p.device][p.property] = SmartDict()
else:
if self.__dict__[p.device][p.property].get(p.field) is None:
self.__dict__[p.device][p.property][p.field] = ParameterData(p)
class SmartDict():
def __init__(self):
self.__dict__ = {}
def __getitem__(self, k):
return self.__dict__[k]
def __setitem__(self, k, v):
self.__dict__[k] = v
def get(self, k):
return self.__dict__.get(k)
def __len__(self):
return len(self.__dict__)

You want __getattr__ and __setattr__, though you'll have to roll your own class (I'm not aware of any builtins, though namedtuple might work if you don't need to change values much)
class AttrDict(dict):
def __getattr__(self, attr):
return self[attr]
def __setattr__(self, attr, value):
self[attr] = value
If you just want to access a sub-dictionary that way, you just change self to self.cool_dict
class CoolThing:
def __init__(self):
self.cool_dict = {'a': 1, 'b': 2}
def __getattr__(self, attr):
return self.cool_dict[attr]
def __setattr__(self, attr, value):
# Note, you'll have to do this for anything that you want to set
# in __init__.
if attr == 'cool_dict':
super().__setattr__(attr, value)
else:
self.cool_dict[attr] = value
Note that __getattr__ is used after any other lookups fail, but if you want to ensure that your function is called first, you can use __getattribute__
Also note that self.cool_dict does not exist on CoolThing until after __init__ is called. My initial version of this would throw a maximum recursion depth exceeded, because as you created the class it would go to set self.cool_dict in init, call __setattr__, which would try to get self.cool_dict so it could set [attr] = value on it. Naturally it can't find cool_dict yet, and so it will try to call __getattr__ again... which can't find cool_dict and round and round it goes.
Another option would be to use a class-level variable instead, but that's probably not at all what you want :)

CoolDict already exists, it's named __dict__:
>>> class CoolThing(object):
... def __init__(self):
... self.__dict__['a'] = 1
... self.__dict__['b'] = 2
...
>>> thing = CoolThing()
>>> thing.a
1
>>> thing.b
2
>>> thing.c = 3
>>> thing.__dict__
{'a': 1, 'b': 2, 'c': 3}

Memoization recipe that allows non-hashable arguments

The common memoization recipes (like this or these) use dict to store the cache, and therefore require that the function arguments be hashable.
I want the function to work with as many different argument types as possible, and certainly including dict, set, list. What is the best way to achieve that?
One approach I was considering is to wrap all non-hashable arguments into their hashable subclasses (i.e., define a subclass of dict that defines its own __hash__ function).
Alternatively, I was thinking to create a subclass of dict that relies on a different hash function than hash (it's not too hard to define a global my_hash function that recursively works on containers), and use this subclass to store the cache. But I don't think there's an easy way to achieve do that.
EDIT:
I think I will try the solution that I suggested for general hashing of python containers. With that, I should be able to wrap the tuple of (*args, **kwargs) into the automatically hashable class, and use the regular memoization.

Method 1 (splitting keys and values)
This is based on the idea that dictionaries are just zipped keys and values.
With this idea, we can make something like a dictionary to store keys (function arguments) and values (returned values from the function).
Not sure how slow it will be since it uses list.index. Maybe zipping would be faster?
class memoize:
def __init__(self, func):
self.func = func
self.known_keys = []
self.known_values = []
def __call__(self, *args, **kwargs):
key = (args, kwargs)
if key in self.known_keys:
i = self.known_keys.index(key)
return self.known_values[i]
else:
value = self.func(*args, **kwargs)
self.known_keys.append(key)
self.known_values.append(value)
return value
It works!:
>>> #memoize
... def whatever(unhashable):
... print(*unhashable) # Just to know when called for this example
... return 12345
...
>>> whatever([1, 2, 3, 4])
1 2 3 4
12345
>>> whatever([1, 2, 3, 4])
12345
>>> whatever({"a": "b", "c": "d"})
a c
12345
>>> whatever({"a": "b", "c": "d"})
12345
Method 2 (fake hashes)
class memoize:
def __init__(self, func):
self.func = func
self.known = {}
def __call__(self, *args, **kwargs):
key = give_fake_hash((args, kwargs))
try:
return self.known[key]
except KeyError:
value = self.func(*args, **kwargs)
self.known[key] = value
return value
def give_fake_hash(obj):
cls = type(obj)
name = "Hashable" + cls.__name__
def fake_hash(self):
return hash(repr(self))
t = type(name, (cls, ), {"__hash__": fake_hash})
return t(obj)
Method 2.5 (working for dicts)
import operator
class memoize:
def __init__(self, func):
self.func = func
self.known = {}
def __call__(self, *args, **kwargs):
key = give_fake_hash((args, kwargs))
try:
return self.known[key]
except KeyError:
value = self.func(*args, **kwargs)
self.known[key] = value
return value
def fake_hash(self):
return hash(repr(self))
class HashableTuple(tuple):
__hash__ = fake_hash
class RereprDict(dict):
def __repr__(self):
try:
self._cached_repr
except AttributeError:
self._cached_repr = repr(sorted(self.items(), key=operator.itemgetter(0)))
finally:
return self._cached_repr
__hash__ = fake_hash
def fix_args(args):
for elem in args:
if isinstance(elem, dict):
elem = RereprDict(elem)
yield elem
def give_fake_hash(tup):
args, kwargs = tup
args = tuple(fix_args(args))
kwargs = RereprDict(kwargs)
return HashableTuple((args, kwargs))

There's a reason dict/list/set/etc. are not hashable, and it is they are mutable.
That's one of the main reasons their immutable counterparts exist (frozendict/frozenset/tuple). (Well, tuple isn't exactly a frozen-list, but in practice it serves the purpose).
Therefor, for your purpose, use the immutable alternatives.
Here's a quick demonstration of why you shouldn't hash mutable objects. Bare in mind that hashing requires that a==b ==> hash(a)==hash(b).
#memoize
def f(x): ...
d1 = {'a':5}
d2 = {'a':99}
res1 = f(d1)
res2 = f(d2)
d1['a'] = 99
res3 = f(d1) # what should be returned? not well defined...

How to implement an ordered, default dict?

I would like to combine OrderedDict() and defaultdict() from collections in one object, which shall be an ordered, default dict.
Is this possible?

The following (using a modified version of this recipe) works for me:
from collections import OrderedDict, Callable
class DefaultOrderedDict(OrderedDict):
# Source: http://stackoverflow.com/a/6190500/562769
def __init__(self, default_factory=None, *a, **kw):
if (default_factory is not None and
not isinstance(default_factory, Callable)):
raise TypeError('first argument must be callable')
OrderedDict.__init__(self, *a, **kw)
self.default_factory = default_factory
def __getitem__(self, key):
try:
return OrderedDict.__getitem__(self, key)
except KeyError:
return self.__missing__(key)
def __missing__(self, key):
if self.default_factory is None:
raise KeyError(key)
self[key] = value = self.default_factory()
return value
def __reduce__(self):
if self.default_factory is None:
args = tuple()
else:
args = self.default_factory,
return type(self), args, None, None, self.items()
def copy(self):
return self.__copy__()
def __copy__(self):
return type(self)(self.default_factory, self)
def __deepcopy__(self, memo):
import copy
return type(self)(self.default_factory,
copy.deepcopy(self.items()))
def __repr__(self):
return 'OrderedDefaultDict(%s, %s)' % (self.default_factory,
OrderedDict.__repr__(self))

Here is another possibility, inspired by Raymond Hettinger's super() Considered Super, tested on Python 2.7.X and 3.4.X:
from collections import OrderedDict, defaultdict
class OrderedDefaultDict(OrderedDict, defaultdict):
def __init__(self, default_factory=None, *args, **kwargs):
#in python3 you can omit the args to super
super(OrderedDefaultDict, self).__init__(*args, **kwargs)
self.default_factory = default_factory
If you check out the class's MRO (aka, help(OrderedDefaultDict)), you'll see this:
class OrderedDefaultDict(collections.OrderedDict, collections.defaultdict)
| Method resolution order:
| OrderedDefaultDict
| collections.OrderedDict
| collections.defaultdict
| __builtin__.dict
| __builtin__.object
meaning that when an instance of OrderedDefaultDict is initialized, it defers to the OrderedDict's init, but this one in turn will call the defaultdict's methods before calling __builtin__.dict, which is precisely what we want.

If you want a simple solution that doesn't require a class, you can just use OrderedDict.setdefault(key, default=None) or OrderedDict.get(key, default=None). If you only get / set from a few places, say in a loop, you can easily just setdefault.
totals = collections.OrderedDict()
for i, x in some_generator():
totals[i] = totals.get(i, 0) + x
It is even easier for lists with setdefault:
agglomerate = collections.OrderedDict()
for i, x in some_generator():
agglomerate.setdefault(i, []).append(x)
But if you use it more than a few times, it is probably better to set up a class, like in the other answers.

Here's another solution to think about if your use case is simple like mine and you don't necessarily want to add the complexity of a DefaultOrderedDict class implementation to your code.
from collections import OrderedDict
keys = ['a', 'b', 'c']
items = [(key, None) for key in keys]
od = OrderedDict(items)
(None is my desired default value.)
Note that this solution won't work if one of your requirements is to dynamically insert new keys with the default value. A tradeoff of simplicity.
Update 3/13/17 - I learned of a convenience function for this use case. Same as above but you can omit the line items = ... and just:
od = OrderedDict.fromkeys(keys)
Output:
OrderedDict([('a', None), ('b', None), ('c', None)])
And if your keys are single characters, you can just pass one string:
OrderedDict.fromkeys('abc')
This has the same output as the two examples above.
You can also pass a default value as the second arg to OrderedDict.fromkeys(...).

Another simple approach would be to use dictionary get method
>>> from collections import OrderedDict
>>> d = OrderedDict()
>>> d['key'] = d.get('key', 0) + 1
>>> d['key'] = d.get('key', 0) + 1
>>> d
OrderedDict([('key', 2)])
>>>

A simpler version of #zeekay 's answer is:
from collections import OrderedDict
class OrderedDefaultListDict(OrderedDict): #name according to default
def __missing__(self, key):
self[key] = value = [] #change to whatever default you want
return value

A simple and elegant solution building on #NickBread.
Has a slightly different API to set the factory, but good defaults are always nice to have.
class OrderedDefaultDict(OrderedDict):
factory = list
def __missing__(self, key):
self[key] = value = self.factory()
return value

I created slightly fixed and more simplified version of the accepted answer, actual for python 3.7.
from collections import OrderedDict
from copy import copy, deepcopy
import pickle
from typing import Any, Callable
class DefaultOrderedDict(OrderedDict):
def __init__(
self,
default_factory: Callable[[], Any],
*args,
**kwargs,
):
super().__init__(*args, **kwargs)
self.default_factory = default_factory
def __getitem__(self, key):
try:
return super().__getitem__(key)
except KeyError:
return self.__missing__(key)
def __missing__(self, key):
self[key] = value = self.default_factory()
return value
def __reduce__(self):
return type(self), (self.default_factory, ), None, None, iter(self.items())
def copy(self):
return self.__copy__()
def __copy__(self):
return type(self)(self.default_factory, self)
def __deepcopy__(self, memo):
return type(self)(self.default_factory, deepcopy(tuple(self.items()), memo))
def __repr__(self):
return f'{self.__class__.__name__}({self.default_factory}, {OrderedDict(self).__repr__()})'
And, that may be even more important, provided some tests.
a = DefaultOrderedDict(list)
# testing default
assert a['key'] == []
a['key'].append(1)
assert a['key'] == [1, ]
# testing repr
assert repr(a) == "DefaultOrderedDict(<class 'list'>, OrderedDict([('key', [1])]))"
# testing copy
b = a.copy()
assert b['key'] is a['key']
c = copy(a)
assert c['key'] is a['key']
d = deepcopy(a)
assert d['key'] is not a['key']
assert d['key'] == a['key']
# testing pickle
saved = pickle.dumps(a)
restored = pickle.loads(saved)
assert restored is not a
assert restored == a
# testing order
a['second_key'] = [2, ]
a['key'] = [3, ]
assert list(a.items()) == [('key', [3, ]), ('second_key', [2, ])]

Inspired by other answers on this thread, you can use something like,
from collections import OrderedDict
class OrderedDefaultDict(OrderedDict):
def __missing__(self, key):
value = OrderedDefaultDict()
self[key] = value
return value
I would like to know if there're any downsides of initializing another object of the same class in the missing method.

i tested the default dict and discovered it's also sorted!
maybe it was just a coincidence but anyway you can use the sorted function:
sorted(s.items())
i think it's simpler