Develop Reference

How to increment a value (in defaultdict of defaultdicts)?

How to increment d['a']['b']['c'][1][2][3] if d is defaultdict of defaultdict without code dublication?
from collections import defaultdict
nested_dict_type = lambda: defaultdict(nested_dict_type)
nested_dict = nested_dict_type()
# incrementation
if type(nested_dict['a']['b']['c']['d'][1][2][3][4][5][6]) != int:
nested_dict['a']['b']['c']['d'][1][2][3][4][5][6] = 0
nested_dict['a']['b']['c']['d'][1][2][3][4][5][6] += 1 # ok, now it contains 1
Here we can see that we duplicated (in the code) a chain of keys 3 times.
Question: Is it possible to write a function inc that will take nested_dict['a']['b']...[6] and do the same job as above? So:
def inc(x):
if type(x) != int:
x = 0
x += 1
inc(nested_dict['a']['b']['c']['d'][1][2][3][4][5][6]) # ok, now it contains 1
Update (20 Aug 2018):
There is still no answer to the question. It's clear that there are options "how to do what I want", but the question is straightforward: there is "value", we pass it to a function, function modifies it. It looks that it's not possible.
Just a value, without any "additional keys", etc.
If it is so, can we make an answer more generic?
Notes:
What is defaultdict of defaultdicts - SO.
This question is not about "storing of integers in a defaultdict", so I'm not looking for a hierarchy of defaultdicts with an int type at the leaves.
Assume that type (int in the examples) is known in advance / can be even parametrized (including the ability to perform += operator) - the question is how to dereference the object, pass it for modification and store back in the context of defaultdict of defaultdicts.
Is the answer to this question related to the mutability? See example below:
Example:
def inc(x):
x += 1
d = {'a': int(0)}
inc(d['a'])
# d['a'] == 0, immutable
d = {'a': Int(0)}
inc(d['a'])
# d['a'] == 1, mutated
Where Int is:
class Int:
def __init__(self, value):
self.value = value
def __add__(self, v):
self.value += v
return self
def __repr__(self):
return str(self.value)

It's not exactly abut mutability, more about how assignment performs name binding.
When you do x = 0 in your inc function you bind a new object to the name x, and any connection between that name and the previous object bound to that name is lost. That doesn't depend on whether or not x is mutable.
But since x is an item in a mutable object we can achieve what you want by passing the parent mutable object to inc along with the key needed to access the desired item.
from collections import defaultdict
nested_dict_type = lambda: defaultdict(nested_dict_type)
nested_dict = nested_dict_type()
# incrementation
def inc(ref, key):
if not isinstance(ref[key], int):
ref[key] = 0
ref[key] += 1
d = nested_dict['a']['b']['c']['d'][1][2][3][4][5]
inc(d, 6)
print(d)
output
defaultdict(<function <lambda> at 0xb730553c>, {6: 1})
Now we aren't binding a new object, we're merely mutating an existing one, so the original d object gets updated correctly.
BTW, that deeply nested dict is a bit painful to work with. Maybe there's a better way to organize your data... But anyway, one thing that can be handy when working with deep nesting is to use lists or tuples of keys. Eg,
q = nested_dict
keys = 'a', 'b', 'c', 'd', 1, 2, 3, 4, 5
for k in keys:
q = q[k]
q now refers to nested_dict['a']['b']['c']['d'][1][2][3][4][5]

You can't have multiple default types with defaultdict. You have the following options:
Nested defaultdict of defaultdict objects indefinitely;
defaultdict of int objects, which likely won't suit your needs;
defaultdict of defaultdict down to a specific level with int defined for the last level, e.g. d = defaultdict(lambda: defaultdict(int)) for a single nesting;
Similar to (3), but for counting you can use collections.Counter instead, i.e. d = defaultdict(Counter).
I recommend the 3rd or 4th options if you are always going to go down to a set level. In other words, a scalar value will only be supplied at the nth level, where n is constant.
Otherwise, one manual option is to have a function perform the type-testing. In this case, try / except may be a good alternative. Here we also define a recursive algorithm to allow you to feed a list of keys rather than defining manual __getitem__ calls.
from collections import defaultdict
from functools import reduce
from operator import getitem
nested_dict_type = lambda: defaultdict(nested_dict_type)
d = nested_dict_type()
d[1][2] = 10
def inc(d_in, L):
try:
reduce(getitem, L[:-1], d_in)[L[-1]] += 1
except TypeError:
reduce(getitem, L[:-1], d_in)[L[-1]] = 1
inc(d, [1, 2])
inc(d, [1, 3])
print(d)
defaultdict({1: defaultdict({2: 11, 3: 1})})

python - dictionary with set of tuples as value is not created correctly

I have read this but it does not seem to work for me -
>>> dict = {}
>>> dict[ "a" ] = set(("1111","test1"))
>>> dict
{'a': set(['1111', 'test1'])}
>>> dict[ "a" ].add( ("2222","test2"))
>>> dict
{'a': set(['1111', 'test1', ('2222', 'test2')])}
I was expecting also the first one to look like a tuple - why is that splitted?
I want when doing :
>>> for key,val_set in dict.items() :
... for val in val_set :
... print val
...
1111
test1
('2222', 'test2')
to have 2 items printed not three.

set() takes the values from an iterable and adds each value separately. You passed in a tuple, which is an iterable with two values.
If you wanted to add the tuple itself, you need to wrap that in an iterable, like a list:
set([("1111","test1")])
In Python 3, you can instead use a set display to create a new set:
{("1111","test1")}
A set display takes elements separated by commas, and ("1111", "test1") is one tuple element here.
In the other post, the set was created with a set display (so {tuple1, tuple2, etc.}) and the answers used either an itertools function or a generator expression to produce the sequence of tuples for the set() function to process; a both produce an iterable type.

You have got your answer but I will advice you to use defaultdict instead. All these type of cases like this set([("1111","test1")]) are properly handled. You don't have to worry about initialize too.
Try This.
from collections import defaultdict
s = defaultdict(set)
s["a"].add(("pw", "sq"));
s["a"].add(("wq", "qq"));
s["a"].add(("aa", "bb"));
for k, v in s.items():
print (k)
for g in v:
print (g)
And please note this, I have assumed all your values opposite your keys are sets.
if this is not the case, go with the first one.

__getitem__ for a list vs a dict

The Dictionary __getitem__ method does not seem to work the same way as it does for List, and it is causing me headaches. Here is what I mean:
If I subclass list, I can overload __getitem__ as:
class myList(list):
def __getitem__(self,index):
if isinstance(index,int):
#do one thing
if isinstance(index,slice):
#do another thing
If I subclass dict, however, the __getitem__ does not expose index, but key instead as in:
class myDict(dict):
def __getitem__(self,key):
#Here I want to inspect the INDEX, but only have access to key!
So, my question is how can I intercept the index of a dict, instead of just the key?
Example use case:
a = myDict()
a['scalar'] = 1 # Create dictionary entry called 'scalar', and assign 1
a['vector_1'] = [1,2,3,4,5] # I want all subsequent vectors to be 5 long
a['vector_2'][[0,1,2]] = [1,2,3] # I want to intercept this and force vector_2 to be 5 long
print(a['vector_2'])
[1,2,3,0,0]
a['test'] # This should throw a KeyError
a['test'][[0,2,3]] # So should this

Dictionaries have no order; there is no index to pass in; this is why Python can use the same syntax ([..]) and the same magic method (__getitem__) for both lists and dictionaries.
When you index a dictionary on an integer like 0, the dictionary treats that like any other key:
>>> d = {'foo': 'bar', 0: 42}
>>> d.keys()
[0, 'foo']
>>> d[0]
42
>>> d['foo']
'bar'
Chained indexing applies to return values; the expression:
a['vector_2'][0, 1, 2]
is executed as:
_result = a['vector_2'] # via a.__getitem__('vector_2')
_result[0, 1, 2] # via _result.__getitem__((0, 1, 2))
so if you want values in your dictionary to behave in a certain way, you must return objects that support those operations.

Attach functions to array elements?

Is there a way to attach a function (same function) to all the elements of an array without looping through and attaching it one by one?
So like
# create function foo from some computation
foo # some def
# list
objects # list of objects
# attach same foo function to all elements of objects
# maybe using a decorator?
# loop through list to execute foo
for obj in objects:
obj.foo()
Let me explain this more:
Of course I can just assign the value of an object like
obj.attr = value
or for an object list:
for obj in objects:
obj.attr = value
What I am trying to avoid is setting the value of an attr on each single object, but rather applying a function on the entire list/array and each element would execute that function.

You could make a function to wrap it up:
def for_each(l, f):
for item in l:
f(item)
Then for a function foo you could do this:
for_each(objects, foo)
For a method foo you could do this:
for_each(objects, lambda item: item.foo())
Or this:
from operator import methodcaller
for_each(objects, methodcaller('foo'))
In Python 2, you can also use map:
map(foo, objects)
For Python 3, you'll have to wrap that in list(...). In either version, you can use list comprehensions:
[foo(item) for item in objects]
However, if you're calling the function just for its side effect rather than transforming the list somehow, I'd recommend against these last two ways as it's against the Zen of Python:
Explicit is better than implicit.
And frankly, one more line for a for loop isn't that much.

You can use map. It is generally used to create a second list, and will return that value, but you can just ignore it.
map(lambda x: x.foo(), objects)

use numpy vectorize! It will work perfectly for you!
from numpy import np
def fun(x):
#do something
array = np.array(your_list)
vectfun = np.vectorize(fun)
answer = vectfun(array)
So now answer will be a resulting array consisting of all the items in the previous list with the function applied to them!! Here is an example:
>>> your_list = [1,2,3,4,5]
>>> def fun(x):
... return x**x
...
>>> array = np.array((your_list))
>>> vectfun = np.vectorize(fun)
>>> answer = vectfun(array)
>>> answer
array([ 1, 4, 27, 256, 3125])

Assigning to a dict

Forgive me if this has been asked before. I did not know how to search for it.
I'm quite familiar with the following idiom:
def foo():
return [1,2,3]
[a,b,c] = foo()
(d,e,f) = foo()
wherein the values contained within the left hand side will be assigned based upon the values returned from the function on the right.
I also know you can do
def bar():
return {'a':1,'b':2,'c':3}
(one, two, three) = bar()
[four, five, six] = bar()
wherein the keys returned from the right hand side will be assigned to the containers on the left hand side.
However, I'm curious, is there a way to do the following in Python 2.6 or earlier:
{letterA:one, letterB:two, letterC:three} = bar()
and have it work in the same manner that it works for sequences to sequences? If not, why? Naively attempting to do this as I've written it will fail.

Dictionary items do not have an order, so while this works:
>>> def bar():
... return dict(a=1,b=2,c=3)
>>> bar()
{'a': 1, 'c': 3, 'b': 2}
>>> (lettera,one),(letterb,two),(letterc,three) = bar().items()
>>> lettera,one,letterb,two,letterc,three
('a', 1, 'c', 3, 'b', 2)
You can see that you can't necessarily predict how the variables will be assigned. You could use collections.OrderedDict in Python 3 to control this.

If you modify bar() to return a dict (as suggested by #mikerobi), you might want to still preserve keyed items that are in your existing dict. In this case, use update:
mydict = {}
mydict['existing_key'] = 100
def bar_that_says_dict():
return { 'new_key': 101 }
mydict.update(bar_that_says_dict())
print mydict
This should output a dict with both existing_key and new_key. If mydict had a key of new_key, then the update would overwrite it with the value returned from bar_that_says_dict.

No, if you can not change bar function, you could create a dict from the output pretty easily.
This is the most compact solution. But I would prefer to modify the bar function to return a dict.
dict(zip(['one', 'two', 'three'], bar()))