def accept(**kwargs):
pass
If I defined accept and I expect it be called by passing a param which is dict. Are the asterisks necessary for all dict params?
What if I do things like:
def accept(dict):
pass
dict = {...}
accept(dict)
Specifically speaking, I would like to implement a update method for a class, which keeps a dict as a container. Just like the dict.update method, it takes a dict as a param and modify the content of the container. In this specific case should I use the kwargs or not?
** in function parameter collects all keyword arguments as a dictionary.
>>> def accept(**kwargs): # isinstance(kwargs, dict) == True
... pass
...
Call using keyword arguments:
>>> accept(a=1, b=2)
Call using ** operator:
>>> d = {'a': 1, 'b': 2}
>>> accept(**d)
>>> accept(d)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: accept() takes exactly 0 arguments (1 given)
See Python tutorial - Keyword argument and Unpacking Argument Lists.
BTW, don't use dict as variable name. It shadows builtin function/type dict.
See f below. The function f has two parameters, a positional one called name and a keyword argument message. They are local variable in the frame of the function call.
When you do f("John", **{"foo": "123", "message": "Hello World"}), the function f will unpack the dictionary into local variable by its key/value pair. In the end you have three local varaibles: name, foo=123, and message=Hello World.
The purpose of **kwargs, double asterisks is for uknown keyword arguments.
Contrast this:
def f(name, message=None):
if message:
return name + message
return name
Here I am telling user if you ever want to call f, you can pass a keyword argument message. This is the only kwarg I will ever accept and expect to receive if there is such one. If you try f("John", foo="Hello world") you will get unexpected keyword argument.
**kwargs is useful if you don't know ahead of time what keyword arguments you want to receive (very common for dispatching to lower-level functions/methods), then you use it.
def f(name, message=None, **kwargs):
if message:
return name + message
return name
In the second example, you can do f("John", **{"foo": "Hello Foo"}) while omitting message. You can also do f("John", **{"foo": "Hello Foo", "message": "Hello Message"}).
Can I ignore it?
As you see yes you can ignore it. Here f("John", **{"foo": "Hello Foo", "message": "Hello Message"}) I still only use message and ignore everything else.
Don't use **kwargs unless you need to be careless about the inputs.
What if my input is a dictionary?
If your function simply takes the dictionary and modifies the dictionary, NOT using individaul key, then just pass the dictionary. There is no reason to make a dictionary item into variables.
But here are two main usages of **kwargs.
Supposed I have a class and I want to create attributes on the fly. I can use setattr to set class attributes from input.
class Foo(object):
def __init__(**kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
If I do Foo(**{"a": 1, "b": 2}) I will get Foo.a and Foo.b at the end.
This is particularly useful when you have to deal with legacy code. However, there is a big security concern. Imagine you own a MongoDB instance and this is a container for writing into a database. Imagine this dict is a request form object from user. The user can shovel ANYTHING and you simply save it in the database like that? That's a security hole. Make sure you validate (use a loop).
The second common usage of kwargs is that you don't know things ahead of times which I have covered (it's actually sort of the first common usage anyway).
If you want to pass a dictionary as input to a function, you can simply do it like this
def my_function1(input_dict):
print input_dict
d = {"Month": 1, "Year": 2}
my_function1(d) # {'Month': 1, 'Year': 2}
This is straight forward. Lets see the **kwargs method. kwargs stands for keyword arguments. So, you need to actually pass the parameters as key-value pairs, like this
def my_function2(**kwargs):
print kwargs
my_function2(Month = 1, Year = 2)
But if you have a dictionary and if you want to pass that as a parameter to my_function2, it can be done with unpacking, like this
my_function2(**d)
Related
For this function
def eat_dog(name, should_digest=True):
print "ate dog named %s. Digested, too? %" % (name, str(should_digest))
I want to, external to the function, read its arguments and any default values attached. So for this specific example, I want to know that name has no default value (i.e. that it is a required argument) and that True is the default value for should_digest.
I'm aware of inspect.getargspec(), which does give me information about arguments and default values, but I see no connection between the two:
ArgSpec(args=['name', 'should_digest'], varargs=None, keywords=None, defaults=(True,))
From this output how can I tell that True (in the defaults tuple) is the default value for should_digest?
Additionally, I'm aware of the "ask for forgiveness" model of approaching a problem, but unfortunately output from that error won't tell me the name of the missing argument:
>>> eat_dog()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: eat_dog() takes at least 1 argument (0 given)
To give context (why I want to do this), I'm exposing functions in a module over a JSON API. If the caller omits certain function arguments, I want to return a specific error that names the specific function argument that was omitted. If a client omits an argument, but there's a default provided in the function signature, I want to use that default.
Python3.x
In a python3.x world, you should probably use a Signature object:
import inspect
def get_default_args(func):
signature = inspect.signature(func)
return {
k: v.default
for k, v in signature.parameters.items()
if v.default is not inspect.Parameter.empty
}
Python2.x (old answer)
The args/defaults can be combined as:
import inspect
a = inspect.getargspec(eat_dog)
zip(a.args[-len(a.defaults):],a.defaults)
Here a.args[-len(a.defaults):] are the arguments with defaults values and obviously a.defaults are the corresponding default values.
You could even pass the output of zip to the dict constructor and create a mapping suitable for keyword unpacking.
looking at the docs, this solution will only work on python2.6 or newer since I assume that inspect.getargspec returns a named tuple. Earlier versions returned a regular tuple, but it would be very easy to modify accordingly. Here's a version which works with older (and newer) versions:
import inspect
def get_default_args(func):
"""
returns a dictionary of arg_name:default_values for the input function
"""
args, varargs, keywords, defaults = inspect.getargspec(func)
return dict(zip(args[-len(defaults):], defaults))
Come to think of it:
return dict(zip(reversed(args), reversed(defaults)))
would also work and may be more intuitive to some people.
Depending on exactly what you need, you might not need the inspect module since you can check the __defaults__ attribute of the function:
>>> eat_dog.__defaults__
(True,)
>>> eat_dog.__code__.co_argcount
2
>>> eat_dog.__code__.co_varnames
('name', 'should_digest')
>>>
>>> eat_dog.__kwdefaults__
>>> eat_dog.__code__.co_kwonlyargcount
0
You can use inspect module with its getargspec function:
inspect.getargspec(func)
Get the names and default values of a Python function’s arguments. A tuple of four things is returned: (args, varargs, keywords, defaults). args is a list of the argument names (it may contain nested lists). varargs and keywords are the names of the * and ** arguments or None. defaults is a tuple of default argument values or None if there are no default arguments; if this tuple has n elements, they correspond to the last n elements listed in args.
See mgilson's answer for exact code on how to retrieve argument names and their default values.
To those looking for a version to grab a specific default parameter with mgilson's answer.
value = signature(my_func).parameters['param_name'].default
Here's a full working version, done in Python 3.8.2
from inspect import signature
def my_func(a, b, c, param_name='apple'):
pass
value = signature(my_func).parameters['param_name'].default
print(value == 'apple') # True
to take care of keyword-only args (and because defaults and kwonlydefaults can be None):
spec = inspect.getfullargspec(func)
defaults = dict(zip(spec.args[::-1], (spec.defaults or ())[::-1]))
defaults.update(spec.kwonlydefaults or {})
You can get this via some of the __dunder__ vars as mentioned by other posts. Putting that into a simple helper function can get you a dictionary of default values.
.__code__.co_varnames: A tuple of all input variables
.__defaults__: A tuple of the default values
It is worth noting that this tuple only incudes the default provided variables which must always be positioned last in the function arguments
You can use these two items to match the last n variables in the .__code__.co_varnames with all the items in the .__defaults__
EDIT Thanks to #griloHBG - Added if statement to prevent exceptions when no defaults are specified.
def my_fn(a, b=2, c='a'):
pass
def get_defaults(fn):
if fn.__defaults__==None:
return {}
return dict(zip(
fn.__code__.co_varnames[-len(fn.__defaults__):],
fn.__defaults__
))
print(get_defaults(my_fn))
Should give:
{'b': 2, 'c': 'a'}
In python, all the arguments with default value come after the arguments without default value. So the mapping should start from the end till you exhaust the default value list. Hence the logic:
dict(zip(reversed(args), reversed(defaults)))
gives the correctly mapped defaults.
I have python3 class function defined like below:
class class1:
def funcOne(self, reqvar1, reqVar2, optVar1=default1, optVar2=default2, optVar3="server.domain", optVar4="defaultUser", optVar5="<default_Flags>"):
It gets called (I want to call it like this rather) in the main program like:
argsIn=argparser.parse_args()
classInst=class1()
classInst.funcOne(5, 12, argsIn.inVal1, argsIn.inVal2, argsIn.inVal3, argsIn.inVal4, argsIn.inVal5)
args.inVal[1-5] are optional on the command line when running. If they don't get supplied I want the class function to use the defaults, if they do get supplied then they would use the supplied values.
Currently if they are not supplied on the command line, inVal[1-5] are passed as 'None' which overwrite the actual default values.
The class function is maintained separately and they manage the defaults. Putting them into my script (for example in the argparser options) is not appropriate.
Is there a way to easily work with this situation that doesn't resort to:
if args.inVal1 and not args.inVal2...
if not args.inVal1 and args.inVal2 and not args.inVal3...
as the number of combinations gets large.
It seems like it should be simple, but I am not connecting something here.
Thank you for the help.
If you create a dictionary that contains the optional variable names, you can pass that dictionary to the function call. I only commented out the argparser for testing.
class class1:
def funcOne(self, reqvar1, reqVar2, optVar1='default1', optVar2='default2', optVar3="server.domain", optVar4="defaultUser", optVar5="<default_Flags>"):
print(optVar1)
print(optVar2)
print(optVar3)
print(optVar4)
print(optVar5)
#argsIn=argparser.parse_args()
optionalArgs = {'optVar1': 'TestingVar1', #args.inVal1,
'optVar2': None, #args.inVal2,
'optVar3': 'TestingVar3', #args.inVal3,
'optVar4': None, #args.inVal4,
'optVar5': None} #args.inVal5}
optionalArgsClean = {k:v for k, v in optionalArgs.items() if v is not None}
classInst=class1()
classInst.funcOne(5, 12, **optionalArgsClean)
Running the above code produces:
TestingVar1
default2
TestingVar3
defaultUser
<default_Flags>
I get some strange behaviour of colletions.defaultdict:
import collections
c1 = collections.defaultdict(str)
c1['new'] # Works!
c2 = collections.defaultdict(default_factory=str)
c2['new'] # Raises KeyError...
Why raises c2 a KeyError?
Sometimes I like naming parameter because I think it increases readability.
First I thought maybe python does not allow me to pass the parameter by naming it and puts my default_factory parameter to the kwargs, so I checked:
def func(first, **kwargs):
print(first)
print(kwargs)
func(first='one', second='two')
This outputs:
one
{'second': 'two'}
So this is not the case.
The default_factory parameter of the defaultdict constructor is positional-only and doesn't really have a name. If you try to pass it by name, you are just passing a completely unrelated keyword argument. Since keyword arguments to the defaultdict constructor are interpreted as its initial contents, your dict starts out having a single key "default_factory" whose value is the str type object.
To understand how this works, imagine a function like this:
def func(*args, **kwds):
(default_factory,) = args
for k, v in kwds.items():
print(k, v) # do something with keys and values
If the documentation of this function were to name the positional argument default_factory, that might be a correct description of its meaning, but it would be misleading if it implied that one could pass it as a keyword argument.
Some built-in functions are like that because it is very easy to define positional-only arguments in CPython C code. With defaultdict, it is by design, to allow literally any string key to be used as the part of initial content, without having an exception for a key that happens to be named default_factory.
I want a dict or tuple I can sort based on attributes of the objects I'm using as arguments for *arg. The way I've been trying to do it just gives me AttributeErrors, which leads me to believe I'm doing it weird.
def function(*arg):
items = {}
for thing in arg:
items.update({thing.name:thing})
while True:
for thing in items:
## lots of other code here, basically just a game loop.
## Problem is that the 'turn order' is based on whatever
## Python decides the order of arguments is inside "items".
## I'd like to be able to sort the dict based on each object's
## attributes (ie, highest 'thing.speed' goes first in the while loop)
The problem is when I try to sort "items" based on an attribute of the objects I put into function(), it gives me "AttributeError: 'str' object has no attribute 'attribute'". Which leads me to believe I'm either unpacking *arg in a lousy way, or I'm trying to do something the wrong way.
while True:
for thing in sorted(items, key=attrgetter('attribute')):
...doesn't work either, keeps telling me I'm trying to manipulate a 'str' object. What am I not doing here?
arg already is a tuple you can sort by an attribute of each item:
def function(*args):
for thing in sorted(args, key=attrgetter('attribute')):
When you iterate over a dict, as sorted is doing, you just get the keys, not the values. So, if you want to use a dict, you need to do:
def function(*args):
# or use a dict comprehension on 2.7+
items = dict((thing.name, thing) for thing in args)
# or just items.values on 3+
for thing in sorted(items.itervalues(), key=attrgetter('attribute')):
to actually sort the args by an attribute. If you want the keys of the dict available as well (not necessary here because the key is also an attribute of the item), use something like:
for name, thing in sorted(items.iteritems(), key=lambda item: item[1].attribute):
Your items is a dict, you can't properly sort a dict. When you try to use it as an iterable, it silently returns its keys list, which is a list of strings. And you don't use your arg after creating a dict.
If you don't need dict lookup, as you just iterate through it, you can replace dict with list of 2-tuples (thing.name, thing), sort it by any attribute and iterate through it. You can also use collections.OrderedDict from Python 2.7 (it exists as a separate ordereddict package for earlier versions) if you really want both dict lookup and ordering.
{edit} Thanks to agf, I understood the problem. So, what I wrote below is a good answer in itself, but not when related to the question above... I let it here for the trace.
Looking to the answers, I may have not understood the question. But here's my understanding: as args is a tuple of arguments you give to your function, it's likely that none of these arguments is an object with a name attribute. But, looking to the errors you report, you're giving string arguments.
Maybe some illustration will help my description:
>>> # defining a function using name attribute
>>> def f(*args):
... for arg in args:
... print arg.name
>>> # defining an object with a name attribute
>>> class o(object):
... def __init__(self, name):
... self.name = name
>>> # now applying the function on the previous object, and on a string
>>> f( o('arg 1'), 'arg 2' )
arg 1
Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
f(o('arg 1'), 'ets')
File "<pyshell#3>", line 3, in f
print arg.name
AttributeError: 'str' object has no attribute 'name'
This is failing as strings have no such attribute.
For me, in your code, there is a mistake: you're trying to use attribute name on your inputs, without ever verifying that they have such an attribute. Maybe you should test with hasattr first:
>>> if hasattr(arg, 'name'):
... print arg.name
... else:
... print arg
or with some inspection on the input, to verify if it's an instance of a given class, known to have the requested attribute.
is there any way to define a mandatory *args (arbitrary arguments) in a method of a class?
class foo():
def __init__(self,*args):
....
Given this, you can create an object from the foo-class without any arguments x=foo(), i.e. its optional. Any ideas how to change it to a non-optional or "give me at least one arguments"-thing?
Another questions concerns the list unpacking with x=foo(*list) --> Is there a way to recognize the list as a list and unpack the list automatically, so that you don´t have to use the * in a function/method call?
Thx
*args is meant to receive all arguments that are not already taken by normal arguments.
Usually if you need an argument to be mandatory, you just use a normal argument:
>>> def foo(arg, *rest):
... print arg, rest
...
>>> foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() takes at least 1 argument (0 given)
If you think it is more elegant to gather all arguments in a tuple, you have to handle the error case yourself:
>>> def foo(*args):
... if len(args) < 1:
... raise TypeError('foo() takes at least 1 argument (%i given)' % len(args))
...
>>> foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in foo
TypeError: foo() takes at least 1 argument (0 given)
But as you can (or should) see, from the signature of that function it is not clear how many arguments to that function are mandatory to anyone who uses that function. You should either avoid this altogether or at least document it very well.
There are other problems as well: if you give on argument to foo() that is iterable (like a string), you will not get the intended result.
Responding to your comment below, your first approach was the right one: take a list.
def scrape(urls):
for url in urls:
do_something(url)
The caller simply has to pass a list with only one element: scrape(['localhost']).
Even better would probably be to take only one URL and let the caller iterate over a list. In that case the caller could parallelize the operations if she ever wants to.
As to your second question1: either you function takes a list as an argument or it doesn't. Either it makes sense in your program to pass around lists or it doesn't.
I guess, I'm not entirely sure what you are asking there, but then again your whole question sounds like you found a shiny new tool and now you want to use it everywhere, regardless of whether it makes sense or not.
1 please don't ask more than one question at once!
Either test the length of the resultant tuple, or put one or more normal arguments before it.
No.
For "give me at least one argument," just check the len() of the tuple you receive and throw an exception if you don't get at least one. Here I am using the fact that empty tuples are "falsy" to do that implicitly:
def __init__(self, *args):
if not args:
raise TypeError("__init__ takes at least 2 arguments (1 given)")
For "auto-unpacking," you will also need to test for this and perform it yourself. One method might be:
if len(args) == 1 and not isinstance(args[0], basestring) and iter(args[0]):
args = args[0]
The iter() will always be true, but if what you pass it is not iterable, it will raise an exception. If you want to provide a friendlier error message, you could catch it and raise something else.
Another alternative would be to write your method so that it recursively iterates over all elemets of args and all subcontainers within it; then it doesn't matter.
Or, you know, just have the caller pass in an iterable to begin with, and don't mess with *args. If the caller wants to pass in a single item, there is simple syntax to turn it into a list: foo([item])