Function _mget_search definition will be as,
def _mget_search(query, with_scores=False, count=False, record_term=True,
**kwargs):
This function called with keyword arguments.
_mget_search(**search_kwargs)
Function parameter search_kwargs contains as,
{'count': False, 'min_score': 20, 'end': 49, 'with_scores': False, 'start': 0, 'query': u'ouuuuu', 'max_score': inf, 'record_term': True}
I am getting this error -
_mget_search() got multiple values for keyword argument 'query'
I am unable to understand why it is happening and how to correct it.
**kwargs allows you to pass keyworded variable length of arguments to a function. You should use **kwargs if you want to handle named arguments in a function. Here is an example to get you going with it:
def greet_me(**kwargs):
if kwargs is not None:
for key, value in kwargs.iteritems():
print "%s == %s" %(key,value)
>>> greet_me(name="yasoob")
name == yasoob
It's because you are unpacking arguments you are trying to pass. Try to use:
_mget_search(search_kwargs)
EDIT
Let's dive more into this problem. We'll define two functions that and see how they behave when passing various arguments.
In [1]: def fun1(a, b, c):
...: print('Passed arguments: a={} b={} c={}'.format(a, b, c))
...:
In [2]: def fun2(*args):
...: print('Passed args: {}'.format(args))
...:
In [3]: fun1(1, 2, 3)
Passed arguments: a=1 b=2 c=3
In [4]: fun2(1, 2, 3)
Passed args: (1, 2, 3)
In [5]: fun2([1, 2, 3])
Passed args: ([1, 2, 3],)
In [6]: fun2(*[1, 2, 3])
Passed args: (1, 2, 3)
In [7]: fun1(*[1, 2, 3])
Passed arguments: a=1 b=2 c=3
In 3rd call we passed 3 arguments separately which is the same as if we used 7th call where an unpacked list was called. Compare it with situations when fun2 was called.
I believe the issue is because query is defined as a positional arg, so when the dictionary you pass with ** is unpacked, the first entry (regardless of its name) is being used for query, then query also appears in the dictionary, hence the multiple values for query.
Try fixing by making query a named arg:
def _mget_search(query='', with_scores=False, count=False, record_term=True, **kwargs):
Alternatively, don't include query in the dictionary when you pass it.
I would bet that something somewhere is also passing a parameter to that function, in addition to the search_kwargs. If it's part of a callback function, there's a good chance the callback is calling it with some parameters and those are getting assigned to the first parameter query.
The only way that I can see you getting that error is if you called your function with a parameter and kwargs, like so:
assuming:
def _mget_search(query, with_scores=False, count=False, record_term=True,
**kwargs):
then the way to get that error message is:
>>> search_kwargs= {'query': u'ouuuuu', 'count': False, 'min_score': 20, 'end': 49, 'with_scores': False, 'start': 0, 'max_score': inf, 'record_term': True}
>>> _mget_search(some_random_parameter, **search_kwargs)
Here's what I used to try and recreate the error.
def func(a, b=2, **kwargs):
return '{}{}{}'.format(a, b, kwargs)
Only the last one succeeded in getting that error.
>>> func(**{'a':2, 'b': 3, 'other': 3})
"23{'other': 3}"
>>> func(7, **{'stuff': 5})
"72{'stuff': 5}"
>>> func(2, **{'b': 7})
'27{}'
>>> func(**{'b': 3, 'other': 3, 'a': 5})
"53{'other': 3}"
>>> func(2, **{'a': 5})
TypeError: func() got multiple values for argument 'a'
Related
I'm trying to learn the different special function parameters in Python and get somewhat confused about them.
Consider this function bar:
def bar(arg, default_arg = 99, *pos_vari_arg, **kywd_vari_arg):
print(arg, default_arg, pos_vari_arg, kywd_vari_arg)
When I call it by
bar(0, 1, 2, 3, 4, foo=5, test=6)
It prints
0 1 (2, 3, 4) {'foo': 5, 'test': 6}
However, I actually want the default_arg to just be 99 and pass 1,2,3,4 to pos_vari_arg. To achieve this, I tried to access default_arg as a keyword parameter instead:
bar(0, 1, 2, 3, 4, foo=5, test=6, default_arg = 99)
But that would simply give me a #TypeError: bar() got multiple values for argument 'default_arg', because Python is not sure whether my default_arg = 99 should go to the parameter default_arg or the dictionary kywd_vari_arg.
I also tried to force default_arg to be a keyword-only parameter by inserting a *,:
def bar(arg, *, default_arg = 99, *pos_vari_arg, **kywd_vari_arg):
print(arg, default_arg, pos_vari_arg, kywd_vari_arg)
But Python throws an error:
File "<string>", line 1
def bar(arg, * , default_arg = 99, *pos_vari_arg, **kywd_vari_arg):
^
SyntaxError: invalid syntax
After some research, I found that *vari_arg must be a positional parameter and thus it cannot be placed after the *, and hence the syntax error. I also found that all parameters after *vari_arg would be forced to be keyword-only parameters, so I do not need the *, to do this job. Instead, I can just swap the places of the parameters:
def bar(arg, *vari_arg, default_arg = 99, **vari_kwarg):
print(arg, default_arg, vari_arg, vari_kwarg)
Now, bar(0, 1, 2, 3, 4, foo=5, test=6) will print 0 99 (1, 2, 3, 4) {'foo': 5, 'test': 6}, which is what I intended to achieve originally.
Question:
Is this the only way to achieve this effect?
Are all my understanding correct? Especially sentence in italic.
What is the best practice to follow if a function has standard
parameters, parameters with default values, and variadic parameters?
Thank you!
I have a question about passing arguments in a function.
In python when you want to unpack an iterable obj on the right-hand side, you have to do it within some context, like a tuple or a set or etc.
For example, you can not say:
a, b = *(1, 2)
and you should say:
a, b = (*(1, 2),) // a=1, b=2
or you can say:
a, b = {*(1, 2)} //a=1, b=2 or a=2, b=1
Am I rigth?
But when you want to unpack iterable and then pass it as arguments in a function, you do not need any context at all and you just unpack your iterable object.
For example:
def f(param1, param2):
pass
f(*(1, 2))
and you do not need to use some kind of context like before. For example, you do not say:
f({*(1, 2)}) // it will be f({1, 2})
I think we don't use {} or any other context in this case because we were looking for pass 2 values as arguments in our f function. Thus, I assume we have to say f(*(1, 2)) not f({*(1, 2)}) .
If I was right, Could you please explain more about how f(*(1, 2)) is worked without using any context under the hood?
f(*(1, 2)) does have context! The context is the list (and dictionary) of function arguments itself. In essence, in a function call f(...) the stuff between the parentheses can be seen as a hybrid between a tuple and a dictionary literal (only using x=4 instead of 'x': 4), which then gets passed to the function:
>>> def f(*args, **kwargs):
... print(args, kwargs)
>>> l = [10, 11]; d = {'a': 12, 'b': 13}
>>> f(1, *l, 2, 3, x=4, y=5, **d, z=6)
(1, 10, 11, 2, 3) {'x': 4, 'y': 5, 'a': 12, 'b': 13, 'z': 6}
Viewed like this it makes perfect sense that you can unpack sequences and dictionaries into this context.
I have a function which takes two parameters and perform a binary operation:
def foo(arg1,arg2):
return operation(arg1,arg2)
I need to generalize this function such that if three args are passed it returns operation(arg1,operation(arg2,arg3)), if four are provided operation(arg1,operation(arg2,operation(arg3,arg4))) and so on. Is it possible to do that in python?
You can do this using the *args form of declaring a function; check if the length of the arguments is 2 and if so return the value of the operation, otherwise return the value of the operation of the first argument with foo of the remaining arguments:
def operation(arg1, arg2):
return arg1 + arg2
def foo(*args):
if (len(args) == 2):
return operation(*args)
return operation(args[0], foo(*args[1:]))
print(foo(1, 3))
print(foo(2, 3, 5))
print(foo(1, 2, 3, 4, 5, 6, 7))
Output:
4
10
28
Note you may also want to check if 0 or 1 arguments are passed to prevent "index out of range" errors. For 1 argument you could just return the input value e.g.
if (len(args) == 1):
return args[0]
As pointed out by #wallefan in the comments, there is a standard library function for this: functools.reduce. You can use that like this:
from functools import reduce
print(reduce(operation, (1, 3)))
print(reduce(operation, (2, 3, 5)))
print(reduce(operation, (1, 2, 3, 4, 5, 6, 7)))
The output is the same as the foo function above.
Yes, and in fact it's built into the standard library: https://docs.python.org/3/library/functools.html#functools.reduce
import functools
def operation(a,b):
return a + b
# returns 15
functools.reduce(operation, (1, 2, 3, 4, 5))
If you'd like, you can combine this with varargs mentioned in Nick's answer:
import functools
def operation(a,b):
return a + b
def foo(*args):
return functools.reduce(operation, args)
# returns 15
foo(1,2,3,4,5)
I have a list of parameters such as this:
import numpy as np
param1 = np.arange(0., 1., 0.01)
param2 = np.arange(10., 8000., 100.)
...
I also have a function foo defined with a list of keyword arguments arg1, arg2, ... and their default values:
def foo(arg1=default1, arg2=default2, ...)
What I need to do is call this function, changing one of those default values (one by one) with the arguments and values from my list of parameters like so:
foo(arg1=param1[0])
foo(arg1=param1[1])
...
foo(arg2=param2[0])
foo(arg2=param2[0])
The best way that I thought of was to create a dictionary of all parameters, and then iterate over keys and values and create a new temporary dictionary out of it and then call the function:
all_params = {'arg1':param1, 'arg2':param2, ...}
for key, value_list in all_params.items():
for value in value_list:
tmp_dict = {key:value}
foo(**tmp_dict)
But I have a feeling that 1) I'm iterating in a non-Pythonic way, 2) and that there is obviously a much better way to solve this problem.
EDIT: streamlined the nested loops a bit according to #Sebastian's suggestion.
This is relatively simple in my opinion.
def foo(a=0, b=0, c=0):
return a * b + c
args1 = [1, 2]
args2 = [3, 4, 5]
args3 = [6, 7]
args = [args1, args2, args3]
d = {}
for n, a in enumerate(args): # Enumerate through all of the parameters.
for val in a: # For each parameter, iterate through all of the desired arguments.
a = [0, 0, 0] # default_args
a[n] = val # Insert the relavent argument into the correct parameter location.
d[tuple(a)] = foo(*a) # Call the function and unpack all of the arguments.
# This dictionary holds the function arguments as keys the returned values for those arguments.
>>> d
{(0, 0, 6): 6,
(0, 0, 7): 7,
(0, 3, 0): 0,
(0, 4, 0): 0,
(0, 5, 0): 0,
(1, 0, 0): 0,
(2, 0, 0): 0}
1) I'm iterating in a non-Pythonic way
"Pythonic" is subjective.
2) That there is obviously a much better way to solve this
problem.
Not so, what you're currently doing is the only possible scenario considering you've to pass them by keyword and that you've to pass them one at a time.
As an improvement you might consider passing all your arguments at the same time.
MVCE:
First, define your function and dictionary:
In [687]: def foo(a, b, c):
...: print(a, b, c)
...:
In [688]: dict_ = {'a': [1, 2, 3], 'b' : [4, 5, 6], 'c' : [7, 8, 9]}
Convert to dict of iters:
In [689]: dict_ = {k : iter(v) for k, v in dict_.items()}
Run your loop:
In [690]: while True:
...: try:
...: foo(**{k : next(v) for k, v in dict_.items()})
...: except StopIteration:
...: break
...:
1 4 7
2 5 8
3 6 9
You can simplify the iterating slightly, so that you don't need to access all_params[key] again, like this:
for key, param in all_params.items():
for value in param:
I have two class instances which have list attributes, e.g.:
foo.range = [1,2]
bar.range = [3,4]
I also have a function which takes multiple arguments:
def permutations(*args):
return list(itertools.product(arg.range for arg in args))
I want permutations(foo, bar) to return all permutations of those two (or more) lists (i.e. [(1,3), (1,4) …]) but I actually get [([1, 2],), ([3, 4],)]
Could someone please help me to understand where am I going wrong and how to achieve the result I'm hoping for?
itertools.product expects the iterables as individual arguments(itertools.product(*iterables[, repeat])), so you need to unpack the items to it using the the splat(*) operator:
>>> def permutations(*args):
return list(itertools.product(*(arg.range for arg in args)))
>>> permutations(foo, bar)
[(1, 3), (1, 4), (2, 3), (2, 4)]
Consider this simple function, it will collect all positional arguments passed to it in args:
>>> def func(*args):
print args
print list(args[0])
...
When we pass it a generator expression it is collected as one individual item in the args tuple:
>>> func(x for x in range(5))
(<generator object <genexpr> at 0x7fda2ab4e780>,)
[0, 1, 2, 3, 4]
To fix this we need to unpack our generator expression, also as somethine like *x for x in range(5) is not a valid syntax in Python, we need to add extra parenthesis around the generator expression:
>>> func(*(x for x in range(5)))
(0, 1, 2, 3, 4)
# Failed as expected for args[0]
Traceback (most recent call last):
File "<ipython-input-215-13af4340dad1>", line 1, in <module>
func(*(x for x in range(5)))
File "<ipython-input-213-08941e2a3c06>", line 3, in func
print list(args[0])
TypeError: 'int' object is not iterable