Develop Reference

python dictionary getter with default value not behaving as expected [duplicate]

I am trying to provide a function as the default argument for the dictionary's get function, like this
def run():
print "RUNNING"
test = {'store':1}
test.get('store', run())
However, when this is run, it displays the following output:
RUNNING
1
so my question is, as the title says, is there a way to provide a callable as the default value for the get method without it being called if the key exists?

Another option, assuming you don't intend to store falsy values in your dictionary:
test.get('store') or run()
In python, the or operator does not evaluate arguments that are not needed (it short-circuits)
If you do need to support falsy values, then you can use get_or_run(test, 'store', run) where:
def get_or_run(d, k, f):
sentinel = object() # guaranteed not to be in d
v = d.get(k, sentinel)
return f() if v is sentinel else v

See the discussion in the answers and comments of dict.get() method returns a pointer. You have to break it into two steps.
Your options are:
Use a defaultdict with the callable if you always want that value as the default, and want to store it in the dict.
Use a conditional expression:
item = test['store'] if 'store' in test else run()
Use try / except:
try:
item = test['store']
except KeyError:
item = run()
Use get:
item = test.get('store')
if item is None:
item = run()
And variations on those themes.
glglgl shows a way to subclass defaultdict, you can also just subclass dict for some situations:
def run():
print "RUNNING"
return 1
class dict_nokeyerror(dict):
def __missing__(self, key):
return run()
test = dict_nokeyerror()
print test['a']
# RUNNING
# 1
Subclassing only really makes sense if you always want the dict to have some nonstandard behavior; if you generally want it to behave like a normal dict and just want a lazy get in one place, use one of my methods 2-4.

I suppose you want to have the callable applied only if the key does not exist.
There are several approaches to do so.
One would be to use a defaultdict, which calls run() if key is missing.
from collections import defaultdict
def run():
print "RUNNING"
test = {'store':1}
test.get('store', run())
test = defaultdict(run, store=1) # provides a value for store
test['store'] # gets 1
test['runthatstuff'] # gets None
Another, rather ugly one, one would be to only save callables in the dict which return the apropriate value.
test = {'store': lambda:1}
test.get('store', run)() # -> 1
test.get('runrun', run)() # -> None, prints "RUNNING".
If you want to have the return value depend on the missing key, you have to subclass defaultdict:
class mydefaultdict(defaultdict):
def __missing__(self, key):
val = self[key] = self.default_factory(key)
return val
d = mydefaultdict(lambda k: k*k)
d[10] # yields 100
#mydefaultdict # decorators are fine
def d2(key):
return -key
d2[5] # yields -5
And if you want not to add this value to the dict for the next call, you have a
def __missing__(self, key): return self.default_factory(key)
instead which calls the default factory every time a key: value pair was not explicitly added.

If you only know what the callable is likely to be at he get call site you could subclass dict something like this
class MyDict(dict):
def get_callable(self,key,func,*args,**kwargs):
'''Like ordinary get but uses a callable to
generate the default value'''
if key not in self:
val = func(*args,**kwargs)
else:
val = self[key]
return val
This can then be used like so:-
>>> d = MyDict()
>>> d.get_callable(1,complex,2,3)
(2+3j)
>>> d[1] = 2
>>> d.get_callable(1,complex,2,3)
2
>>> def run(): print "run"
>>> repr(d.get_callable(1,run))
'2'
>>> repr(d.get_callable(2,run))
run
'None'
This is probably most useful when the callable is expensive to compute.

I have a util directory in my project with qt.py, general.py, geom.py, etc. In general.py I have a bunch of python tools like the one you need:
# Use whenever you need a lambda default
def dictGet(dict_, key, default):
if key not in dict_:
return default()
return dict_[key]
Add *args, **kwargs if you want to support calling default more than once with differing args:
def dictGet(dict_, key, default, *args, **kwargs):
if key not in dict_:
return default(*args, **kwargs)
return dict_[key]

Here's what I use:
def lazy_get(d, k, f):
return d[k] if k in d else f(k)
The fallback function f takes the key as an argument, e.g.
lazy_get({'a': 13}, 'a', lambda k: k) # --> 13
lazy_get({'a': 13}, 'b', lambda k: k) # --> 'b'
You would obviously use a more meaningful fallback function, but this illustrates the flexibility of lazy_get.
Here's what the function looks like with type annotation:
from typing import Callable, Mapping, TypeVar
K = TypeVar('K')
V = TypeVar('V')
def lazy_get(d: Mapping[K, V], k: K, f: Callable[[K], V]) -> V:
return d[k] if k in d else f(k)

How to create `if in` function check?

I have been investigating and found out that using if in is the fastest compare to ->
benchmark
and I have been trying to create a function where I can pass arguments on what path I want the if in will follow e.g.
def main():
d = {"foo": "spam"}
if "bar" in d:
if "eggs" in d["bar"]:
d["bar"]["eggs"]
else:
{}
else:
{}
But instead of having a long code, I was trying to do a function where I can pass argument e.g. get_path(json_data, 'foo', 'eggs') which would try to do something similar to the code above and return if value found else return empty.
My question is how can I create a function where we can pass argument to do the if in checks and return the value if it's found?

You could pass your keys as tuple/list:
def main(data, keys):
for k in keys:
if k not in data:
return {}
data = data[k]
return data
d = {"foo": "spam", "bar": {"eggs": "HAM!"}}
print(main(d, ('bar', 'eggs')))
Out:
HAM!

This is a nice little problem that has a fairly easy solution as long as everything is dicts:
def get_path(data, *path):
node = data
for step in path:
if step in node:
node = node[step]
else:
return {} # None might be more appropriate here
return node
note that it won't work quite right if you encounter a list along the way: although lists support [] and they support in, in means something different to them ("is this value found", rather than "is this key found"), so the test generally won't succeed.

Python dict with default value based on key

I need a dictionary that is automatically filled with a default value for each accessed key that is missing. I've found defaultdict and some other ways to achieve this, but the problem in my case is that I want the default value for each key to be specific to the key itself.
For example, with defaultdict I can achieve something like this:
from collections import defaultdict
d = defaultdict(lambda: 5)
> d[1] = 3
> d[1]
> 3
> d[2]
> 5
But what if I need the default value for each accessed missing key to be for example key + 5? Something like:
from collections import defaultdict
d = defaultdict(lambda key: key + 5) # <-- This does not work as defaultdict expects lambda function to be without any parameters
> d[1] = 3
> d[1]
> 3
> d[2]
> 7 <- Calculated from accessed key + 5 (2+5)
> d[5]
> 10 <- Calculated from accessed key + 5 (5+5)
Is there a clean, builtin way to achieve what I need? I know that I can create a subclass of dict and implement this specific functionality at __getitem__ level, but I want to avoid that if possible.
I couldn't find a solution in other answers, so sorry if it is still a duplicate.

I don't think there is a builtin way of doing this. However, instead of subclassing dict and change getitem, you can subclass defaultdict itself to tell __missing__() to call the default_factory with an argument (the key), rather than without args. It is still a custom class, not a builtin, but it should be quite efficient still.
from collections import defaultdict
class DefaultDict(defaultdict):
def __missing__(self, key):
return self.default_factory(key)
Then:
d = DefaultDict(lambda key: key + 5)
d[2]
# 7

You can also use a function like this
dic = {}
DEFAULT_VALUE = 5
def dict_get(item):
try:
return [dic[item]]
except:
dic[int(item)] = DEFAULT_VALUE + int(item)
return DEFAULT_VALUE + int(item)
print(dict_get(10))

Here is how to do it without creating any class. You can use dict.setdefault:
d = {}
d_default_factory = lambda key: key + 5
d.setdefault(2, d_default_factory(2))
# 7
Optionally, you could save the default factory as an entry in the dictionary, like so:
d = {'default_factory': lambda key: key + 5}
d.setdefault(2, d['default_factory'](2))
# 7

Check if a specific Key and a value exist in a dictionary

I am trying to determine if a specific key and value pair exist in a dictionary; however, if I use the contains or has-key method, it only checks for the key. I need it to check both the key and the specific value. Some background:
We have a total of 4 dictionaries: one for A, B, CompareList, and ChangeList. Once A is initialized, I put A's contents into CompareList (I would compare them directly; but A and B are double hash tables. And I've tried all of the methods here; but none of them work for me). So once we put A into CompareList, I compare it with the ObjectAttributes dictionary in B to see if anything changed. So for example, B may have the key,value pairs shape:circle and fill:no. If CompareList had shape:circle and fill:yes, then I want only fill:yes to be ChangeList. The problem lies in the "if attributes.getName() not in self.CompareList:" line. Here is the code; I am running it on Python 2.7.8. Thanks in advance for any help!!
class ObjectSemanticNetwork:
def __init__(self):
self.ObjectNames = {}
self.ObjectAttributes = {}
def setName(self, name):
self.ObjectNames[name] = self.ObjectAttributes
def setData(self, name, attribute):
self.ObjectAttributes[name] = attribute
def checkData(self, key):
print(key)
for key, value in self.ObjectAttributes.iteritems():
print(key)
print(value)
print("\n")
class Agent:
(self):
self.CompareList = {}
self.ChangeListAB = {}
self.ChangeListCD = {}
def addToCompareList(self, name, value):
self.CompareList[name] = value
def addToChangeListAB(self, name, value):
self.ChangeListAB[name] = value
def addToChangeListCD(self, name, value):
self.ChangeListCD[name] = value
def CheckList(self, List, ListName):
print '-------------------------',ListName,'--------------------------------'
for key, value in List.iteritems():
print(key)
print(value)
def Solve(self,problem):
OSNAB = ObjectSemanticNetwork()
for object in problem.getFigures().get("A").getObjects():
for attributes in object.getAttributes():
self.addToCompareList(attributes.getName(), attributes.getValue())
OSNAB.ObjectNames["A"] = OSNAB.setData(attributes.getName(), attributes.getValue())
#OSNAB.checkData("A")
self.CheckList(self.CompareList,"CompareList")
for object in problem.getFigures().get("B").getObjects():
for attributes in object.getAttributes():
if attributes.getName() not in self.CompareList:
self.addToChangeListAB(attributes.getName(), attributes.getValue())
OSNAB.ObjectNames["B"] = OSNAB.setData(attributes.getName(), attributes.getValue())
# OSNAB.checkData("B")
self.CheckList(self.ChangeListAB,"ChangeList")
OSNCD = ObjectSemanticNetwork()
for object in problem.getFigures().get("C").getObjects():
for attributes in object.getAttributes():
OSNCD.ObjectNames["C"] = OSNCD.setData(attributes.getName(), attributes.getValue())
# OSNCD.checkData("C")
for object in problem.getFigures().get("1").getObjects():
for attributes in object.getAttributes():
OSNCD.ObjectNames["D"] = OSNCD.setData(attributes.getName(), attributes.getValue())
# OSNCD.checkData("D")
return "6"

Use
if key in d and d[key] == value:
Or (only in Python 3)
if (key, value) in d.items():
In Python 3 d.items() returns a Dictionary view object, which supports fast membership testing. In Python 2 d.items() returns a list, which is both slow to create and slow to to test membership. Python 2.7 is a special case where you can use d.viewitems() and get the same thing that you get with d.items() in Python 3.
Edit: In a comment you indicate that for performance reasons you prefer checkKeyValuePairExistence over key in d and d[key] == value. Below are some timings showing that checkKeyValuePairExistence is always slower (by about 2x on my system when the key-value pair is present 16x when it is not). I also tested larger and smaller dictionaries and found little variation in the timings.
>>> import random
>>> from timeit import timeit
>>> def checkKeyValuePairExistence(dic, key, value):
... try:
... return dic[key] == value
... except KeyError:
... return False
...
>>> d = {random.randint(0, 100000):random.randint(0, 100000) for i in range(1000)}
>>> setup = 'from __main__ import k, d, v, checkKeyValuePairExistence'
>>> test_try_except = 'checkKeyValuePairExistence(d, k, v)'
>>> test_k_in_d_and = 'k in d and d[k] == v'
>>> k, v = random.choice(d.items()) # to test if found
>>> timeit(test_try_except, setup=setup)
0.1984054392365806
>>> timeit(test_k_in_d_and, setup=setup)
0.10442071140778353
>>> k = -1 # test if not found
>>> timeit(test_try_except, setup=setup)
1.2896073903002616
>>> timeit(test_k_in_d_and, setup=setup)
0.07827843747497809

How about this function:
def checkKeyValuePairExistence(dic, key, value):
try:
return dic[key] == value
except KeyError:
return False
If you are using another type of dictionary other then the one python offers (I'm sorry, I couldnt understand from your post if you are using it or not) then let me know and i'll try to give your another solution

Why not just do this:
a = {1:'a', 2:'b'}
b = (1, 'a')
print b in a.iteritems() # prints True

how to program functions with alternative return value signatures in python? (next() for alternative iterators)

e.g. so that these would both work - is it possible?
(val,VAL2) = func(args)
val = func(args)
Where val is not a tuple
For example I'd like these to work for my custom object something
for item in something:
do_item(item) #where again item - is not a tuple
for (item,key) in something:
do_more(key,item)
I thought that I need to implement next() function in two different ways...
edit: as follows from the answers below, this should not really be done.

If you mean, can the function act differently based on the return types the caller is expecting, the answer is no (bar seriously nasty bytecode inspection). In this case, you should provide two different iterators on your object, and write something like:
for item in something: # Default iterator: returns non-tuple objects
do_something(item)
for (item,key) in something.iter_pairs(): # iter_pairs returns different iterator
do_something_else(item, key)
eg. see the dictionary object, which uses this pattern. for key in mydict iterates over the dictionary keys. for k,v in mydict.iteritems() iterates over (key, value) pairs.
[Edit] Just in case anyone wants to see what I mean by "seriously nasty bytecode inspection", here's a quick implementation:
import inspect, opcode
def num_expected_results():
"""Return the number of items the caller is expecting in a tuple.
Returns None if a single value is expected, rather than a tuple.
"""
f = inspect.currentframe(2)
code = map(ord, f.f_code.co_code)
pos = f.f_lasti
if code[pos] == opcode.opmap['GET_ITER']: pos += 1 # Skip this and the FOR_ITER
if code[pos] > opcode.EXTENDED_ARG: pos +=5
elif code[pos] > opcode.HAVE_ARGUMENT: pos +=3
else: pos += 1
if code[pos] == opcode.opmap['UNPACK_SEQUENCE']:
return code[pos+1] + (code[pos+2] << 8)
return None
Usable something like:
class MagicDict(dict):
def __iter__(self):
if num_expected_results() == 2:
for k,v in self.iteritems():
yield k,v
else:
for k in self.iterkeys():
yield k
d=MagicDict(foo=1, bar=2)
print "Keys:"
for key in d:
print " ", key
print "Values"
for k,v in d:
print " ",k,v
Disclaimer: This is incredibly hacky, insanely bad practice, and will cause other programmers to hunt you down and kill you if they ever see it in real code. Only works on cpython (if that). Never use this in production code (or for that matter, probably any code).

Have you tried that? It works.
def myfunction(data):
datalen = len(data)
result1 = data[:datalen/2]
result2 = data[datalen/2:]
return result1, result2
a, b = myfunction('stuff')
print a
print b
c = myfunction('other stuff')
print c
In fact there is no such thing as "return signature". All functions return a single object. It seems that you are returning more than one, but in fact you wrap them into a container tuple object.

Yes it's doable:
def a(b):
if b < 5:
return ("o", "k")
else:
return "ko"
and the result:
>>> b = a(4)
>>> b
('o', 'k')
>>> b = a(6)
>>> b
'ko'
I think the thing after is to be careful when you will use the values returned...

>>> def func(a,b):
return (a,b)
>>> x = func(1,2)
>>> x
(1, 2)
>>> (y,z) = func(1,2)
>>> y
1
>>> z
2
That doesn't really answer your question. The real answer is that the left side of the assignment doesn't affect the returned type of the function and can't be used to distinguish between functions with different return types. As noted in other answers, the function can return different types from different return statements but it doesn't know what's on the other side of the equals sign.
In the case of this function, it returns a tuple. If you assign it to x, x has the value of the tuple. (y, z) on the left side of the assignment is "tuple unpacking". The tuple returned by func() is unpacked into y and z.

Update:
Given the example use case, I'd write different generators to handle the cases:
class Something(object):
def __init__(self):
self.d = {'a' : 1,
'b' : 2,
'c' : 3}
def items(self):
for i in self.d.values():
yield i
def items_keys(self):
for k,i in self.d.items():
yield i,k
something = Something()
for item in something.items():
....: print item
....:
1
3
2
for item,key in something.items_keys():
....: print key, " : ", item
....:
a : 1
b : 2
c : 3
Or
You can return a tuple:
In [1]: def func(n):
...: return (n, n+1)
...:
In [2]: a,b = func(1)
In [3]: a
Out[3]: 1
In [4]: b
Out[4]: 2
In [5]: x = func(1)
In [6]: x
Out[6]: (1, 2)

Yes, both would work. In the first example, val1 and val2 would have the two values. In the second example, val would have a tuple. You can try this in your python interpreter:
>>> def foo():
... return ( 1, 2 )
...
>>> x = foo()
>>> (y,z) = foo()
>>> x
(1, 2)
>>> y
1
>>> z
2

It's possible only if you're happy for val to be a 2-item tuple (or if args need not be the same in the two cases). The former is what would happen if the function just ended with something like return 23, 45. Here's an example of the latter idea:
def weirdfunc(how_many_returns):
assert 1 <= how_many_returns <= 4
return 'fee fie foo fum'.split()[:how_many_returns]
var1, var2 = weirdfunc(2) # var1 gets 'fee', var2 gets 'fie'
var, = weirdfunc(1) # var gets 'fee'

This is asking for major confusion. Instead you can follow dict with separate keys, values, items, etc. methods, or you can use a convention of naming unused variables with a single underscore. Examples:
for k in mydict.keys(): pass
for k, v in mydict.items(): pass
for a, b in myobj.foo(): pass
for a, _ in myobj.foo(): pass
for _, b in myobj.foo(): pass
for _, _, _, d in [("even", "multiple", "underscores", "works")]:
print(d)
for item in something: # or something.keys(), etc.
do_item(item)
for item, key in something.items():
do_more(key, item)
If this doesn't fit your function, you should refactor it as two or more functions, because it's clearly trying to fulfill two or more different goals.