I understand that how * and ** operators generally work. In the following code [ taken from django's source ]
def curry(_curried_func, *args, **kwargs):
def _curried(*moreargs, **morekwargs):
return _curried_func(*(args + moreargs), **dict(kwargs, **morekwargs))
return _curried
I get how the args + moreargs part work - the [non - keyword ] arguments passed initially to the function curry and the arguments passed to the curried function returned by curry are combined. What I don't get is how the **dict(kwargs, **morekwargs) works. Could someone please explain that bit?
dict(kwargs, **morekwargs) is a trick (that Guido dislikes) for combining 2 dictionaries into 1.
>>> d = {'foo':'bar'}
>>> kwargs = {'bar':'baz'}
>>> dict(d,**kwargs)
{'foo': 'bar', 'bar': 'baz'}
So, the curried function takes all kwargs passed to curry and adds them to the additional kwargs passed to the _curried function to create a super dictionary. that super dictionary gets unpacked and sent on to the _curried_func.
It effectively builds a union of two dicts, kwargs and morekwargs. Example:
>>> kwargs = {"ham": 1, "spam": 2}
>>> morekwargs = {"spam": 3, "eggs": 2}
>>> dict(kwargs, **morekwargs)
{'eggs': 2, 'ham': 1, 'spam': 3}
This works because the dict constructor takes keyword arguments, so it's the same as
>>> dict(kwargs, spam=3, eggs=2)
{'eggs': 2, 'ham': 1, 'spam': 3}
Finally, the resulting dict is fed as keyword arguments to _curried_func by means of **.
Related
Im trying to create a function that take unknown number of arguments (dictionaries) to merge them in one. Here is my sketch:
weight = {"sara": 60, "nick": 79, "sem": 78, "ida": 56, "kasia": 58, "slava": 95}
height = { "a" : 1, "b": 2, "c":3 }
width = {"u": "long", "q": 55, "qw": "erre", 30: "34"}
a = {10:20, 20:"a"}
def merge(**dict):
new_dict = {}
for x in dict:
for a, b in x.items():
new_dict[a] = b
return new_dict
print(merge(weight, height, width, a))
And I got error:
TypeError: merge() takes 0 positional arguments but 4 were given
Why?
First note: dict is a bad name for an argument as it is already the name of a type.
When you use ** in the argument list for a function it is slurping up any keyword arguments you haven't explicitly listed. Similarly, a parameter with a single * is slurping up any extra positional arguments not explicitly named.
Consider:
>>> def foo(bar, **baz): return (bar, baz)
...
>>> foo(42, wooble=42)
(42, {'wooble': 42})
>>> foo(bar=42, wooble=42)
(42, {'wooble': 42})
>>> foo(bar=42, wooble=42, hello="world")
(42, {'wooble': 42, 'hello': 'world'})
>>>
If you wish to take any number of dictionaries as arguments, you'd use:
def merge(*dicts):
...
As dicts will now slurp up any number of dictionaries passed in.
Change def merge(**dict): to def merge(*dict): and it is working. Avoid naming it dict since it is a keyword in python.
If you want to pass a list of dicts as a single argument you have to do this:
def foo(*dicts)
Anyway you SHOULDN'T name it *dict, since you are overwriting the dict class.
In Python you can pass all the arguments as a list with the * operator...
def foo(*args)
...and as a dict with the ** operator
def bar(**kwargs)
For example:
>>> foo(1, 2, 3, 4) # args is accessible as a list [1, 2, 3, 4]
>>> bar(arg1='hello', arg2='world') # kwargs is accessible as a dict {'arg1'='hello', 'arg2'='world'}
In your case you can edit the prototype of you function this way:
def merge(*dicts):
I am looking for the best way to combine a function with a dictionary that contains more items than the function's inputs
basic **kwarg unpacking fails in this case:
def foo(a,b):
return a + b
d = {'a':1,
'b':2,
'c':3}
foo(**d)
--> TypeError: foo() got an unexpected keyword argument 'c'
After some research I came up with the following approach:
import inspect
# utilities
def get_input_names(function):
'''get arguments names from function'''
return inspect.getargspec(function)[0]
def filter_dict(dict_,keys):
return {k:dict_[k] for k in keys}
def combine(function,dict_):
'''combine a function with a dictionary that may contain more items than the function's inputs '''
filtered_dict = filter_dict(dict_,get_input_names(function))
return function(**filtered_dict)
# examples
def foo(a,b):
return a + b
d = {'a':1,
'b':2,
'c':3}
print combine(foo,d)
--> 3
My question is: is this a good way of dealing with this problem, or is there a better practice or is there a mechanism in the language that I'm missing perhaps?
How about making a decorator that would filter allowed keyword arguments only:
import inspect
def get_input_names(function):
'''get arguments names from function'''
return inspect.getargspec(function)[0]
def filter_dict(dict_,keys):
return {k:dict_[k] for k in keys}
def filter_kwargs(func):
def func_wrapper(**kwargs):
return func(**filter_dict(kwargs, get_input_names(func)))
return func_wrapper
#filter_kwargs
def foo(a,b):
return a + b
d = {'a':1,
'b':2,
'c':3}
print(foo(**d))
What is nice about this decorator is that it is generic and reusable. And you would not need to change the way you call and use your target functions.
All of these answers are wrong.
It is not possible to do what you are asking, because the function might be declared like this:
def foo(**kwargs):
a = kwargs.pop('a')
b = kwargs.pop('b')
if kwargs:
raise TypeError('Unexpected arguments: %r' % kwargs)
Now, why on earth would anyone write that?
Because they don't know all of the arguments ahead of time. Here's a more realistic case:
def __init__(self, **kwargs):
for name in self.known_arguments():
value = kwargs.pop(name, default)
self.do_something(name, value)
super().__init__(**kwargs) # The superclass does not take any arguments
And here is some real-world code which actually does this.
You might ask why we need the last line. Why pass arguments to a superclass that doesn't take any? Cooperative multiple inheritance. If my class gets an argument it does not recognize, it should not swallow that argument, nor should it error out. It should pass the argument up the chain so that another class I might not know about can handle it. And if nobody handles it, then object.__init__() will provide an appropriate error message. Unfortunately, the other answers will not handle that gracefully. They will see **kwargs and either pass no arguments or pass all of them, which are both incorrect.
The bottom line: There is no general way to discover whether a function call is legal without actually making that function call. inspect is a crude approximation, and entirely falls apart in the face of variadic functions. Variadic does not mean "pass whatever you like"; it means "the rules are too complex to express in a signature." As a result, while it may be possible in many cases to do what you're trying to do, there will always be situations where there is no correct answer.
Your problem lies with the way you defined your function, it should be defined like this -
def foo(**kwargs):
And then inside the function you can iterate over the number of arguments sent to the function like so -
if kwargs is not None:
for key, value in kwargs.iteritems():
do something
You can find more info about using **kwargs in this post -
http://pythontips.com/2013/08/04/args-and-kwargs-in-python-explained/
You can also use a decorator function to filter out those keyword arguments that are not allowed in you function. Of you use the signature function new in 3.3 to return your function Signature
from inspect import signature
from functools import wraps
def decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
sig = signature(func)
result = func(*[kwargs[param] for param in sig.parameters])
return result
return wrapper
From Python 3.0 you can use getargspec which is deprecated since version 3.0
import inspect
def decorator(func):
#wraps(func)
def wrapper(*args, **kwargs):
argspec = inspect.getargspec(func).args
result = func(*[kwargs[param] for param in argspec])
return result
return wrapper
To apply your decorate an existing function you need to pass your function as argument to your decorator:
Demo:
>>> def foo(a, b):
... return a + b
...
>>> foo = decorator(foo)
>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> foo(**d)
3
To apply your decorator to a newly function simply use #
>>> #decorator
... def foo(a, b):
... return a + b
...
>>> foo(**d)
3
You can also define your function using arbitrary keywords arguments **kwargs.
>>> def foo(**kwargs):
... if 'a' in kwargs and 'b' in kwargs:
... return kwargs['a'] + kwargs['b']
...
>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> foo(**d)
3
I would do something like this:
def combine(function, dictionary):
return function(**{key:value for key, value in dictionary.items()
if key in inspect.getargspec(function)[0]}
)
Use:
>>> def this(a, b, c=5):
... print(a, b, c)
...
>>> combine(this, {'a': 4, 'b': 6, 'c': 6, 'd': 8})
4 6 6
>>> combine(this, {'a': 6, 'b': 5, 'd': 8})
6 5 5
This is still modifying the original function, but you can create a kwargs bitbucket at the end of the argument list:
def foo(a, b, **kwargs):
return a + b
foo(**{
'a': 5,
'b': 8,
'c': '🐘'
}) # 13
Description
Say I have the following function, which makes a call to another function:
def f1(arg1, arg2, arg3):
f2(...)
The arguments of f1 and f2 are the same, or f2 might look like this:
def f2(**kwargs)
pass # whatever
The client code is going to define and call f1, and it is required that the signature of f1 explicitly defines all arguments, and thus no **kwargs is allowed for f1.
So, to make a call to f2 from inside f1 I have to do this:
def f1(arg1, arg2, arg3):
f2(arg1, arg2, arg3)
Question
Is there a way I can pass arguments to f2 without explicitly writing them? Ideally, I think it should look like this:
def f1(arg1, arg2, arg3):
kwargs = <Some Magic Here>
f2(**kwargs)
Any magic?
UPDATE
Possible Solution
Is there a way I can combine locals() and inspect.getargspec() to aggregate my **kwargs?
In General
Well, you can create kwargs as a dictionary of all the arguments that f2() accepts and pass it. Though I do not see any benefit from that, using -
def f1(arg1, arg2, arg3):
f2(arg1=arg1, arg2=arg2, arg3=arg3)
Looks fine to me , and would be easier than building the dictionary and calling it as **kwargs.
Anyway the way to do it is -
>>> def a(a,b,c):
... kwargs = {'a':a , 'b':b , 'c':c}
... d(**kwargs)
...
>>> def d(**kwargs):
... print(kwargs)
...
>>> a(1,2,3)
{'c': 3, 'a': 1, 'b': 2}
For your use case
The problem is that f1 is going to be defined by the client, the processing of argument is common for all, so I want to hide the processing details, so that the client passes all the arguments to the implementation. Furthermore, I want to ease the definition and automatically pass all arguments and not specify them explicitly.
locals() inside a function returns you the copy of the local variables in the function at that time as a dictionary. If as in your question if the definition of f1() and f2() are same you can use locals() , by calling it at the start of the function before any other code. Example -
>>> def a(a,b,c):
... lcl = locals()
... print(lcl)
... d(**lcl)
... e = 123
... print(locals())
...
>>> def d(**kwargs):
... print(kwargs)
...
>>> a(1,2,3)
{'c': 3, 'a': 1, 'b': 2}
{'c': 3, 'a': 1, 'b': 2}
{'c': 3, 'a': 1, 'e': 123, 'lcl': {...}, 'b': 2}
What you want here is passing the arguments of the parent function to an enclosing function.So fist I must say that you can not use local() because local is contain the local variables which is contain the argument and all local variables that you have define in your function buddy, but if you just want to get all the arguments when they are dynamic and like your example, I suggest to use *args in parents function :
def f1(*args):
But there is a point here, since you want to use **kwargs and it is for collecting the arbitrarily keyword arguments you need to initial names of your argument.
For example you can use a dict comprehension like following :
def f1(*args):
kwargs = {'arg{}'.format(i):j for i,j in enumerate(args,1)}
f2(**kwargs)
Also if you are sure that there is no local variable in your function buddy specially before defining the enclosing function you can use locals :
Example :
>>> globsl_a=8
>>> def fun(arg1,arg2,arg3):
... print locals()
...
>>> fun(5,[3,4],'text')
{'arg1': 5, 'arg2': [3, 4], 'arg3': 'text'}
>>>
Here is how I solved my problem based on Anand S Kumar's answer as well as Alex Martelli's answer to another question:
def f2():
kwargs = inspect.getouterframes(inspect.currentframe(), 2)[1][0].f_locals
print(kwargs)
def f1(arg1, arg2, arg3)
f2() # this should be called right at the first line
>>> f1(1, 2, 3)
{'arg1': 1, 'arg2': 2, 'arg3': 3}
It seems that many aspects of python are just duplicates of functionality. Is there some difference beyond the redundancy I am seeing in kwargs and dict within Python?
There is a difference in argument unpacking (where many people use kwargs) and passing dict as one of the arguments:
Using argument unpacking:
# Prepare function
def test(**kwargs):
return kwargs
# Invoke function
>>> test(a=10, b=20)
{'a':10,'b':20}
Passing a dict as an argument:
# Prepare function
def test(my_dict):
return my_dict
# Invoke function
>>> test(dict(a=10, b=20))
{'a':10,'b':20}
The differences are mostly:
readability (you can simply pass keyword arguments even if they weren't explicitly defined),
flexibility (you can support some keyword arguments explicitly and the rest using **kwargs),
argument unpacking helps you avoid unexpected changes to the object "containing" the arguments (which is less important, as Python in general assumes developers know what they are doing, which is a different topic),
It is right that in most cases you can just interchange dicts and **kwargs.
For example:
my_dict = {'a': 5, 'b': 6}
def printer1(adict):
return adict
def printer2(**kwargs):
return kwargs
#evaluate:
>>> printer1(my_dict)
{'a': 5, 'b': 6}
>>> printer2(**my_dict)
{'a': 5, 'b': 6}
However with kwargs you have more flexibility if you combine it with other arguments:
def printer3(a, b=0, **kwargs):
return a,b,kwargs
#evaluate
>>> printer3(**my_dict)
(5, 6, {})
Python seems to have an inconsistency in what kind of keys it will accept for dicts. Or, put another way, it allows certain kinds of keys in one way of defining dicts, but not in others:
>>> d = {1:"one",2:2}
>>> d[1]
'one'
>>> e = dict(1="one",2=2)
File "<stdin>", line 1
SyntaxError: keyword can't be an expression
Is the {...} notation more fundamental, and dict(...) just syntactic sugar? Is it because there is simply no way for Python to parse dict(1="one")?
I'm curious...
This is not a dict issue, but an artifact of Python syntax: keyword arguments must be valid identifiers, and 1 and 2 are not.
When you want to use anything that is not a string following Python identifier rules as a key, use the {} syntax. The constructor keyword argument syntax is just there for convenience in some special cases.
dict is a function call, and function keywords must be identifiers.
As other answer have stated, dict is a function call. It has three syntactic forms.
The form:
dict(**kwargs) -> new dictionary initialized with the name=value pairs
in the keyword argument list. For example: dict(one=1, two=2)
The keys (or name as used in this case) must be valid Python identifiers, and ints are not valid.
The limitation is not only the function dict You can demonstrate it like so:
>>> def f(**kw): pass
...
>>> f(one=1) # this is OK
>>> f(1=one) # this is not
File "<stdin>", line 1
SyntaxError: keyword can't be an expression
However, there are two other syntactic forms of you can use.
There is:
dict(iterable) -> new dictionary initialized as if via:
d = {}
for k, v in iterable:
d[k] = v
Example:
>>> dict([(1,'one'),(2,2)])
{1: 'one', 2: 2}
And from a mapping:
dict(mapping) -> new dictionary initialized from a mapping object's
(key, value) pairs
Example:
>>> dict({1:'one',2:2})
{1: 'one', 2: 2}
While that may not seem like much (a dict from a dict literal) keep in mind that Counter and defaultdict are mappings and this is how you would covert one of those to a dict:
>>> from collections import Counter
>>> Counter('aaaaabbbcdeffff')
Counter({'a': 5, 'f': 4, 'b': 3, 'c': 1, 'e': 1, 'd': 1})
>>> dict(Counter('aaaaabbbcdeffff'))
{'a': 5, 'c': 1, 'b': 3, 'e': 1, 'd': 1, 'f': 4}
If you read the documentation, you will learn that the dict = {stringA = 1, stringB = 2} notation is valid when the keys are simple strings:
When the keys are simple strings, it is sometimes easier to specify
pairs using keyword arguments:
>>>
>>> dict(sape=4139, guido=4127, jack=4098)
{'sape': 4139, 'jack': 4098, 'guido': 4127}
Since integers (or other numbers) are not valid keyword arguments, the dict = {1 = 2, 3 = 4} will fail as any call to a function would if you passed an argument to it while naming it with a number:
>>> def test(**kwargs):
... for arg in kwargs:
... print arg, kwargs[arg]
...
>>> test(a=2,b=3)
a 2
b 3
>>> test(1=2, 3=4)
File "<stdin>", line 1
SyntaxError: keyword can't be an expression