I have allowed = ['someoption','someother'],
,and default = 'default_value'
v comes from user input.
then when I do :
v = v if v.lower() in allowed else default, it works as expected,
v is taking values only if user specified option listed in allowed list, else is set to default_value.
However, I wanted to isolate validation into function by defining it like:
def validate(value, rules, default)
value if value.lower() in rules else default
, but now when I do validate(v, allowed, default), and try to enter value not in allowed list, instead of getting v hold the default_value, I get whatever I entered.
So I expected for v to be passed as reference and being changed, but that did not happened. What should I do, to get the expected result?
Simple solution
First, small correction to your validate function:
def validate(value, rules, default)
return value if value.lower() in rules else default
and then simply assign the value to your result
v = "something"
allowed = ["someoption", "someother"]
default = "default_value"
v = validate(v, allowed, default)
Just keep it simple.
Regarding "pass by reference" - Python does not have this concept. If you pass into a function immutable type of value (like a string), it goes in the way which could be called "by value". If you pass in mutable object (like list), it goes in the way which could be called "by reference". There is no way, you would instruct Python to change how this is done.
ugly_validate with changing result "in place"
If you would really insist on modifying the result being passed in, this (ugly) way could work:
>>> def ugly_validate(value_lst, rules, default):
... if value_lst[0].lower() not in rules:
... value_lst[0] = default
...
>>> v = "something"
>>> allowed = ["someoption", "someother"]
>>> default = "default_value"
>>> v_lst = [v]
>>> ugly_validate(v_lst, allowed, default)
>>> v_lst
['default_value']
>>> v = "someoption"
>>> v_lst = [v]
>>> ugly_validate(v_lst, allowed, default)
>>> v_lst
['someoption']
Alternative solution using "identity dictionary" of allowed values
>>> allowed_dct = {key: key for key in allowed}
>>> allowed_dct
{'someoption': 'someoption', 'someother': 'someother'}
>>> res = allowed_dct.get(v, default)
>>> res
'someoption'
>>> v = "unknown"
>>> res = allowed_dct.get(v, default)
>>> res
'default_value'
Using dedicated class
>>> class AllowedValues():
... def __init__(self, allowed_values, default):
... self.allowed_values = allowed_values
... self.default = default
... def get(self, value):
... if value.lower() in self.allowed_values:
... return value
... else:
... return self.default
...
>>> allowed
['someoption', 'someother']
>>> judge = AllowedValues(allowed, default)
>>> v = "someoption"
>>> v = judge.get(v)
>>> v
'someoption'
>>> v = "unknonw"
>>> v = judge.get(v)
>>> v
'default_value'
In Python everything is passed by asssigment. In your case, the value name is assigned the same object that v is pointing to but as soon as you change value, since what it is pointing to (a string) is immutable, it actually atarts poonting to a new string object and no longer to what v Is pointing to.
The best way to do what you want to do is for the validate function to return the new value (if it's not in rules.
Alternatively (but not recommended), if you make v point to a list containing the user-entered string, then since lists are mutable, the change to the list inside the validate function will also be "visible" outside of that function.
Related
I am trying to provide a function as the default argument for the dictionary's get function, like this
def run():
print "RUNNING"
test = {'store':1}
test.get('store', run())
However, when this is run, it displays the following output:
RUNNING
1
so my question is, as the title says, is there a way to provide a callable as the default value for the get method without it being called if the key exists?
Another option, assuming you don't intend to store falsy values in your dictionary:
test.get('store') or run()
In python, the or operator does not evaluate arguments that are not needed (it short-circuits)
If you do need to support falsy values, then you can use get_or_run(test, 'store', run) where:
def get_or_run(d, k, f):
sentinel = object() # guaranteed not to be in d
v = d.get(k, sentinel)
return f() if v is sentinel else v
See the discussion in the answers and comments of dict.get() method returns a pointer. You have to break it into two steps.
Your options are:
Use a defaultdict with the callable if you always want that value as the default, and want to store it in the dict.
Use a conditional expression:
item = test['store'] if 'store' in test else run()
Use try / except:
try:
item = test['store']
except KeyError:
item = run()
Use get:
item = test.get('store')
if item is None:
item = run()
And variations on those themes.
glglgl shows a way to subclass defaultdict, you can also just subclass dict for some situations:
def run():
print "RUNNING"
return 1
class dict_nokeyerror(dict):
def __missing__(self, key):
return run()
test = dict_nokeyerror()
print test['a']
# RUNNING
# 1
Subclassing only really makes sense if you always want the dict to have some nonstandard behavior; if you generally want it to behave like a normal dict and just want a lazy get in one place, use one of my methods 2-4.
I suppose you want to have the callable applied only if the key does not exist.
There are several approaches to do so.
One would be to use a defaultdict, which calls run() if key is missing.
from collections import defaultdict
def run():
print "RUNNING"
test = {'store':1}
test.get('store', run())
test = defaultdict(run, store=1) # provides a value for store
test['store'] # gets 1
test['runthatstuff'] # gets None
Another, rather ugly one, one would be to only save callables in the dict which return the apropriate value.
test = {'store': lambda:1}
test.get('store', run)() # -> 1
test.get('runrun', run)() # -> None, prints "RUNNING".
If you want to have the return value depend on the missing key, you have to subclass defaultdict:
class mydefaultdict(defaultdict):
def __missing__(self, key):
val = self[key] = self.default_factory(key)
return val
d = mydefaultdict(lambda k: k*k)
d[10] # yields 100
#mydefaultdict # decorators are fine
def d2(key):
return -key
d2[5] # yields -5
And if you want not to add this value to the dict for the next call, you have a
def __missing__(self, key): return self.default_factory(key)
instead which calls the default factory every time a key: value pair was not explicitly added.
If you only know what the callable is likely to be at he get call site you could subclass dict something like this
class MyDict(dict):
def get_callable(self,key,func,*args,**kwargs):
'''Like ordinary get but uses a callable to
generate the default value'''
if key not in self:
val = func(*args,**kwargs)
else:
val = self[key]
return val
This can then be used like so:-
>>> d = MyDict()
>>> d.get_callable(1,complex,2,3)
(2+3j)
>>> d[1] = 2
>>> d.get_callable(1,complex,2,3)
2
>>> def run(): print "run"
>>> repr(d.get_callable(1,run))
'2'
>>> repr(d.get_callable(2,run))
run
'None'
This is probably most useful when the callable is expensive to compute.
I have a util directory in my project with qt.py, general.py, geom.py, etc. In general.py I have a bunch of python tools like the one you need:
# Use whenever you need a lambda default
def dictGet(dict_, key, default):
if key not in dict_:
return default()
return dict_[key]
Add *args, **kwargs if you want to support calling default more than once with differing args:
def dictGet(dict_, key, default, *args, **kwargs):
if key not in dict_:
return default(*args, **kwargs)
return dict_[key]
Here's what I use:
def lazy_get(d, k, f):
return d[k] if k in d else f(k)
The fallback function f takes the key as an argument, e.g.
lazy_get({'a': 13}, 'a', lambda k: k) # --> 13
lazy_get({'a': 13}, 'b', lambda k: k) # --> 'b'
You would obviously use a more meaningful fallback function, but this illustrates the flexibility of lazy_get.
Here's what the function looks like with type annotation:
from typing import Callable, Mapping, TypeVar
K = TypeVar('K')
V = TypeVar('V')
def lazy_get(d: Mapping[K, V], k: K, f: Callable[[K], V]) -> V:
return d[k] if k in d else f(k)
My question comes from this page, while I would like to create a pointer(-like thing) for an list element. The element is a primitive value (string) so I have to create a FooWrapper class as that page says.
I know that by setting __repr__ one can directly access this value.
class FooWrapper(object):
def __init__(self, value):
self.value = value
def __repr__(self):
return repr(self.value)
>>> bar=FooWrapper('ABC')
>>> bar
'ABC'
>>> bar=FooWrapper(3)
>>> bar
3
Now I can use it as an reference of string:
>>> L=[3,5,6,9]
>>> L[1]=FooWrapper('ABC')
>>> L
[3,'ABC',6,9]
>>> this=L[1]
>>> this.value='BCD'
>>> print(L)
[3,'BCD',6,9]
So now I have a pointer-like this for the list element L[1].
However it is still inconvenient since I must use this.value='BCD' to change its value. While there exists a __repr__ method to make this directly return this.value, is there any similar method to make this='BCD' to do this.value='BCD' ? I know this changes the rule of binding.. but anyway, is it possible?
I would also appreciate if there is a better solution for a list element pointer.
Thank you in advance:)
I'm not sure exactly what you are trying to do, but you could do something like:
class FooWrapper(object):
def __init__(self, value):
self.value = value
def __repr__(self):
return 'FooWrapper(' + repr(self.value) + ')'
def __str__(self):
return str(self.value)
def __call__(self,value):
self.value = value
Here I got rid of your idea of using __repr__ to hide FooWrapper since I think it a bad idea to hide from the programmer what is happening at the REPL. Instead -- I used __str__ so that when you print the object you will print the wrapped value. The __call__ functions as a default method, which doesn't change the meaning of = but is sort of what you want:
>>> vals = [1,2,3]
>>> vals[1] = FooWrapper("Bob")
>>> vals
[1, FooWrapper('Bob'), 3]
>>> for x in vals: print(x)
1
Bob
3
>>> this = vals[1]
>>> this(10)
>>> vals
[1, FooWrapper(10), 3]
However, I think it misleading to refer to this as a pointer. It is just a wrapper object, and is almost certain to make dealing with the wrapped object inconvenient.
On Edit: The following is more of a pointer to a list. It allows you to create something like a pointer object with __call__ used to dereference the pointer (when no argument is passed to it) or to mutate the list (when a value is passed to __call__). It also implements a form of p++ called (pp) with wrap-around (though the wrap-around part could of course be dropped):
class ListPointer(object):
def __init__(self, myList,i=0):
self.myList = myList
self.i = i % len(self.myList)
def __repr__(self):
return 'ListPointer(' + repr(self.myList) + ',' + str(self.i) + ')'
def __str__(self):
return str(self.myList[self.i])
def __call__(self,*value):
if len(value) == 0:
return self.myList[self.i]
else:
self.myList[self.i] = value[0]
def pp(self):
self.i = (self.i + 1) % len(self.myList)
Used like this:
>>> vals = ['a','b','c']
>>> this = ListPointer(vals)
>>> this()
'a'
>>> this('d')
>>> vals
['d', 'b', 'c']
>>> this.pp()
>>> this()
'b'
>>> print(this)
b
I think that this is a more transparent way of getting something which acts like a list pointer. It doesn't require the thing pointed to to be wrapped in anything.
The __repr__ method can get a string however it wants to. Let's say it says return repr(self.value) + 'here'. If you say this = '4here', what should be affected? Should self.value be assigned to 4 or 4here? What if this had another attribute called key and __repr__ did return repr(self.key) + repr(self.value)? When you did this = '4here', would it assign self.key to the whole string, assign self.value to the whole string, or assign self.key to 4 and self.value to here? What if the string is completely made up by the method? If it says return 'here', what should this = '4here' do?
In short, you can't.
Self-Answer based on John Coleman's idea:
class ListPointer(object):
def __init__(self,list,index):
self.value=list[index]
list[index]=self
def __repr__(self):
return self.value
def __call__(self,value):
self.value=value
>>> foo=[2,3,4,5]
>>> this=ListPointer(foo,2)
>>> this
4
>>> foo
[2,3,4,5]
>>> this('ABC')
>>> foo
[2,3,'ABC',5]
>>> type(foo(2))
<class '__main__.ListPointer'>
ListPointer object accepts a list and an index, stores the list[index] in self.value and then substitutes the list element with self. Similarly a pp method can also be achieved, while the former element should be restored and the next element be substituted with self object. By directly referring foo[2] one also gets this object, which is what I want. (Perhaps this should be called as a reference rather than a pointer..)
I am trying to determine if a specific key and value pair exist in a dictionary; however, if I use the contains or has-key method, it only checks for the key. I need it to check both the key and the specific value. Some background:
We have a total of 4 dictionaries: one for A, B, CompareList, and ChangeList. Once A is initialized, I put A's contents into CompareList (I would compare them directly; but A and B are double hash tables. And I've tried all of the methods here; but none of them work for me). So once we put A into CompareList, I compare it with the ObjectAttributes dictionary in B to see if anything changed. So for example, B may have the key,value pairs shape:circle and fill:no. If CompareList had shape:circle and fill:yes, then I want only fill:yes to be ChangeList. The problem lies in the "if attributes.getName() not in self.CompareList:" line. Here is the code; I am running it on Python 2.7.8. Thanks in advance for any help!!
class ObjectSemanticNetwork:
def __init__(self):
self.ObjectNames = {}
self.ObjectAttributes = {}
def setName(self, name):
self.ObjectNames[name] = self.ObjectAttributes
def setData(self, name, attribute):
self.ObjectAttributes[name] = attribute
def checkData(self, key):
print(key)
for key, value in self.ObjectAttributes.iteritems():
print(key)
print(value)
print("\n")
class Agent:
(self):
self.CompareList = {}
self.ChangeListAB = {}
self.ChangeListCD = {}
def addToCompareList(self, name, value):
self.CompareList[name] = value
def addToChangeListAB(self, name, value):
self.ChangeListAB[name] = value
def addToChangeListCD(self, name, value):
self.ChangeListCD[name] = value
def CheckList(self, List, ListName):
print '-------------------------',ListName,'--------------------------------'
for key, value in List.iteritems():
print(key)
print(value)
def Solve(self,problem):
OSNAB = ObjectSemanticNetwork()
for object in problem.getFigures().get("A").getObjects():
for attributes in object.getAttributes():
self.addToCompareList(attributes.getName(), attributes.getValue())
OSNAB.ObjectNames["A"] = OSNAB.setData(attributes.getName(), attributes.getValue())
#OSNAB.checkData("A")
self.CheckList(self.CompareList,"CompareList")
for object in problem.getFigures().get("B").getObjects():
for attributes in object.getAttributes():
if attributes.getName() not in self.CompareList:
self.addToChangeListAB(attributes.getName(), attributes.getValue())
OSNAB.ObjectNames["B"] = OSNAB.setData(attributes.getName(), attributes.getValue())
# OSNAB.checkData("B")
self.CheckList(self.ChangeListAB,"ChangeList")
OSNCD = ObjectSemanticNetwork()
for object in problem.getFigures().get("C").getObjects():
for attributes in object.getAttributes():
OSNCD.ObjectNames["C"] = OSNCD.setData(attributes.getName(), attributes.getValue())
# OSNCD.checkData("C")
for object in problem.getFigures().get("1").getObjects():
for attributes in object.getAttributes():
OSNCD.ObjectNames["D"] = OSNCD.setData(attributes.getName(), attributes.getValue())
# OSNCD.checkData("D")
return "6"
Use
if key in d and d[key] == value:
Or (only in Python 3)
if (key, value) in d.items():
In Python 3 d.items() returns a Dictionary view object, which supports fast membership testing. In Python 2 d.items() returns a list, which is both slow to create and slow to to test membership. Python 2.7 is a special case where you can use d.viewitems() and get the same thing that you get with d.items() in Python 3.
Edit: In a comment you indicate that for performance reasons you prefer checkKeyValuePairExistence over key in d and d[key] == value. Below are some timings showing that checkKeyValuePairExistence is always slower (by about 2x on my system when the key-value pair is present 16x when it is not). I also tested larger and smaller dictionaries and found little variation in the timings.
>>> import random
>>> from timeit import timeit
>>> def checkKeyValuePairExistence(dic, key, value):
... try:
... return dic[key] == value
... except KeyError:
... return False
...
>>> d = {random.randint(0, 100000):random.randint(0, 100000) for i in range(1000)}
>>> setup = 'from __main__ import k, d, v, checkKeyValuePairExistence'
>>> test_try_except = 'checkKeyValuePairExistence(d, k, v)'
>>> test_k_in_d_and = 'k in d and d[k] == v'
>>> k, v = random.choice(d.items()) # to test if found
>>> timeit(test_try_except, setup=setup)
0.1984054392365806
>>> timeit(test_k_in_d_and, setup=setup)
0.10442071140778353
>>> k = -1 # test if not found
>>> timeit(test_try_except, setup=setup)
1.2896073903002616
>>> timeit(test_k_in_d_and, setup=setup)
0.07827843747497809
How about this function:
def checkKeyValuePairExistence(dic, key, value):
try:
return dic[key] == value
except KeyError:
return False
If you are using another type of dictionary other then the one python offers (I'm sorry, I couldnt understand from your post if you are using it or not) then let me know and i'll try to give your another solution
Why not just do this:
a = {1:'a', 2:'b'}
b = (1, 'a')
print b in a.iteritems() # prints True
One minor annoyance with dict.setdefault is that it always evaluates its second argument (when given, of course), even when the first the first argument is already a key in the dictionary.
For example:
import random
def noisy_default():
ret = random.randint(0, 10000000)
print 'noisy_default: returning %d' % ret
return ret
d = dict()
print d.setdefault(1, noisy_default())
print d.setdefault(1, noisy_default())
This produces ouptut like the following:
noisy_default: returning 4063267
4063267
noisy_default: returning 628989
4063267
As the last line confirms, the second execution of noisy_default is unnecessary, since by this point the key 1 is already present in d (with value 4063267).
Is it possible to implement a subclass of dict whose setdefault method evaluates its second argument lazily?
EDIT:
Below is an implementation inspired by BrenBarn's comment and Pavel Anossov's answer. While at it, I went ahead and implemented a lazy version of get as well, since the underlying idea is essentially the same.
class LazyDict(dict):
def get(self, key, thunk=None):
return (self[key] if key in self else
thunk() if callable(thunk) else
thunk)
def setdefault(self, key, thunk=None):
return (self[key] if key in self else
dict.setdefault(self, key,
thunk() if callable(thunk) else
thunk))
Now, the snippet
d = LazyDict()
print d.setdefault(1, noisy_default)
print d.setdefault(1, noisy_default)
produces output like this:
noisy_default: returning 5025427
5025427
5025427
Notice that the second argument to d.setdefault above is now a callable, not a function call.
When the second argument to LazyDict.get or LazyDict.setdefault is not a callable, they behave the same way as the corresponding dict methods.
If one wants to pass a callable as the default value itself (i.e., not meant to be called), or if the callable to be called requires arguments, prepend lambda: to the appropriate argument. E.g.:
d1.setdefault('div', lambda: div_callback)
d2.setdefault('foo', lambda: bar('frobozz'))
Those who don't like the idea of overriding get and setdefault, and/or the resulting need to test for callability, etc., can use this version instead:
class LazyButHonestDict(dict):
def lazyget(self, key, thunk=lambda: None):
return self[key] if key in self else thunk()
def lazysetdefault(self, key, thunk=lambda: None):
return (self[key] if key in self else
self.setdefault(key, thunk()))
This can be accomplished with defaultdict, too. It is instantiated with a callable which is then called when a nonexisting element is accessed.
from collections import defaultdict
d = defaultdict(noisy_default)
d[1] # noise
d[1] # no noise
The caveat with defaultdict is that the callable gets no arguments, so you can not derive the default value from the key as you could with dict.setdefault. This can be mitigated by overriding __missing__ in a subclass:
from collections import defaultdict
class defaultdict2(defaultdict):
def __missing__(self, key):
value = self.default_factory(key)
self[key] = value
return value
def noisy_default_with_key(key):
print key
return key + 1
d = defaultdict2(noisy_default_with_key)
d[1] # prints 1, sets 2, returns 2
d[1] # does not print anything, does not set anything, returns 2
For more information, see the collections module.
You can do that in a one-liner using a ternary operator:
value = cache[key] if key in cache else cache.setdefault(key, func(key))
If you are sure that the cache will never store falsy values, you can simplify it a little bit:
value = cache.get(key) or cache.setdefault(key, func(key))
No, evaluation of arguments happens before the call. You can implement a setdefault-like function that takes a callable as its second argument and calls it only if it is needed.
There seems to be no one-liner that doesn't require an extra class or extra lookups. For the record, here is a easy (even not concise) way of achieving that without either of them.
try:
value = dct[key]
except KeyError:
value = noisy_default()
dct[key] = value
return value
Consider the following:
p1=1;
p2=5;
p3=7;
highest=max(p1,p2,p3).
The max function would return 7. I am looking to create a similar function, which would return "p3". I have created a small function (by simple comparisons) for the above example, shown below. however I am having trouble when the number of arguments go up.
def highest(p1,p2,p3):
if (p1>p2) and (p1>p3):
return "p1"
if (p2>p1) and (p2>p3):
return "p2"
if (p3>p1) and (p3>p1):
return "p3"
Is there a simpler way to do this>
Update: Paul Hankin pointed out that max() took a key function, which I didn't know. So:
>>> def argmax(**kw):
... return max(kw, key=kw.get)
...
>>> argmax(foo=3, bar=5, frotz=1, kaka=-3)
'bar'
Other solutions for completeness:
In Python 2.7 and 3.x you can use dictionary comprehensions.
>>> def argmax(**kw):
... wk = {v:k for k,v in kw.items()}
... return wk[max(wk)]
...
>>> argmax(foo=3, bar=5, frotz=1, kaka=-3)
'bar'
Dictionary comprehensions are neat. :)
In earlier versions of Python you can do this:
>>> def argmax(**kw):
... wk = dict([(v,k) for k,v in kw.items()])
... return wk[max(wk)]
...
>>> argmax(foo=3, bar=5, frotz=1, kaka=-3)
'bar'
Which will work in anything after Python 2.2 or so.
There is no way to get the name of the variable that had the highest value in the caller (because it might be a number or a complex expression), but by using keyword arguments exclusively, you can get the name of the parameter. Something like this:
def argmax(**kwargs):
mx = -1e+400 # overflows to -Inf
amx = None
for k, v in kwargs.iteritems():
if v > mx:
mx = v
amx = k
return amx
Works like this:
>>> argmax(a=1,b=2,c=3)
'c'
but the catch is, it doesn't work if any of the arguments is positional:
>>> argmax(1,2,3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: argmax() takes exactly 0 arguments (3 given)
Depending on what you're doing, this construct might be more useful: (hat tip to http://lemire.me/blog/archives/2008/12/17/fast-argmax-in-python/ )
>>> a = [9,99,999]
>>> a.index(max(a))
2
The only way to keep this anything close to extensible is to take a list as argument and return the index of its highest element. You can put a p in front and begin counting at 1 if you really want to.
def highest(x):
return 'p' + repr(x.index(max(x))+1)
Obviously it dose not handle variable length arguments. If you want variable length argument then that's a different issue. If you have 10 arguments then just add them in the definition and it will return the correct argument name (not necessarily started with 'p'). But the catch is the number of arguments (3 or 5 or 10 or whatever else) is not variable. You need to know how many arguments you require.
def highest(p1,p2,p3,p4,p5):
d = locals()
keys = d.keys()
max_key = keys[0]
max_val = d[max_key]
for i in range(1,len(keys)):
key = keys[i]
val = d[key]
if val > max_val:
max_val = val
max_key = key
return max_key
print highest(3,2,5,10,1)
print highest(1,5,2,2,3)
print highest(5,2,5,1,11)
print highest(3,2,1,1,2)