Related
This question already has answers here:
What does ** (double star/asterisk) and * (star/asterisk) do for parameters?
(25 answers)
Closed 9 years ago.
Does * have a special meaning in Python as it does in C? I saw a function like this in the Python Cookbook:
def get(self, *a, **kw)
Would you please explain it to me or point out where I can find an answer (Google interprets the * as wild card character and thus I cannot find a satisfactory answer).
See Function Definitions in the Language Reference.
If the form *identifier is
present, it is initialized to a tuple
receiving any excess positional
parameters, defaulting to the empty
tuple. If the form **identifier is
present, it is initialized to a new
dictionary receiving any excess
keyword arguments, defaulting to a new
empty dictionary.
Also, see Function Calls.
Assuming that one knows what positional and keyword arguments are, here are some examples:
Example 1:
# Excess keyword argument (python 2) example:
def foo(a, b, c, **args):
print "a = %s" % (a,)
print "b = %s" % (b,)
print "c = %s" % (c,)
print args
foo(a="testa", d="excess", c="testc", b="testb", k="another_excess")
As you can see in the above example, we only have parameters a, b, c in the signature of the foo function. Since d and k are not present, they are put into the args dictionary. The output of the program is:
a = testa
b = testb
c = testc
{'k': 'another_excess', 'd': 'excess'}
Example 2:
# Excess positional argument (python 2) example:
def foo(a, b, c, *args):
print "a = %s" % (a,)
print "b = %s" % (b,)
print "c = %s" % (c,)
print args
foo("testa", "testb", "testc", "excess", "another_excess")
Here, since we're testing positional arguments, the excess ones have to be on the end, and *args packs them into a tuple, so the output of this program is:
a = testa
b = testb
c = testc
('excess', 'another_excess')
You can also unpack a dictionary or a tuple into arguments of a function:
def foo(a,b,c,**args):
print "a=%s" % (a,)
print "b=%s" % (b,)
print "c=%s" % (c,)
print "args=%s" % (args,)
argdict = dict(a="testa", b="testb", c="testc", excessarg="string")
foo(**argdict)
Prints:
a=testa
b=testb
c=testc
args={'excessarg': 'string'}
And
def foo(a,b,c,*args):
print "a=%s" % (a,)
print "b=%s" % (b,)
print "c=%s" % (c,)
print "args=%s" % (args,)
argtuple = ("testa","testb","testc","excess")
foo(*argtuple)
Prints:
a=testa
b=testb
c=testc
args=('excess',)
I only have one thing to add that wasn't clear from the other answers (for completeness's sake).
You may also use the stars when calling the function. For example, say you have code like this:
>>> def foo(*args):
... print(args)
...
>>> l = [1,2,3,4,5]
You can pass the list l into foo like so...
>>> foo(*l)
(1, 2, 3, 4, 5)
You can do the same for dictionaries...
>>> def foo(**argd):
... print(argd)
...
>>> d = {'a' : 'b', 'c' : 'd'}
>>> foo(**d)
{'a': 'b', 'c': 'd'}
All of the above answers were perfectly clear and complete, but just for the record I'd like to confirm that the meaning of * and ** in python has absolutely no similarity with the meaning of similar-looking operators in C.
They are called the argument-unpacking and keyword-argument-unpacking operators.
A single star means that the variable 'a' will be a tuple of extra parameters that were supplied to the function. The double star means the variable 'kw' will be a variable-size dictionary of extra parameters that were supplied with keywords.
Although the actual behavior is spec'd out, it still sometimes can be very non-intuitive. Writing some sample functions and calling them with various parameter styles may help you understand what is allowed and what the results are.
def f0(a)
def f1(*a)
def f2(**a)
def f3(*a, **b)
etc...
I find * useful when writing a function that takes another callback function as a parameter:
def some_function(parm1, parm2, callback, *callback_args):
a = 1
b = 2
...
callback(a, b, *callback_args)
...
That way, callers can pass in arbitrary extra parameters that will be passed through to their callback function. The nice thing is that the callback function can use normal function parameters. That is, it doesn't need to use the * syntax at all. Here's an example:
def my_callback_function(a, b, x, y, z):
...
x = 5
y = 6
z = 7
some_function('parm1', 'parm2', my_callback_function, x, y, z)
Of course, closures provide another way of doing the same thing without requiring you to pass x, y, and z through some_function() and into my_callback_function().
For example:
def func(a):
# how to get the name "x"
x = 1
func(x)
If I use inspect module I can get the stack frame object:
import inspect
def func(a):
print inspect.stack()
out:
[
(<frame object at 0x7fb973c7a988>, 'ts.py', 9, 'func', [' stack = inspect.stack()\n'], 0)
(<frame object at 0x7fb973d74c20>, 'ts.py', 18, '<module>', ['func(x)\n'], 0)
]
or use inspect.currentframe() I can get the current frame.
But I can only get the name "a" by inspect the function object.
Use inspect.stack I can get the call stack:"['func(x)\n']",
How can I get the name of actual parameters("x" here) when call the func by parsing the "['func(x)\n']"?
If I call func(x) then I get "x"
if I call func(y) then I get "y"
Edit:
A example:
def func(a):
# Add 1 to acttual parameter
...
x = 1
func(x)
print x # x expected to 2
y = 2
func(y)
print y # y expected to 3
Looking at your comment explanations, the reason you are trying to do this is:
Because I want to impelment Reference Parameters like in C++
You can't do that. In python, everything is passed by value, but that value is a reference to the object you pass. Now, if that object is mutable (like a list), you can modify it, but if it's immutable, a new object is created and set. In your code example, x is an int which is immutable, so if you want to change the value of x it, just return its new value from the function you are invoking:
def func(a):
b = 5 * a
# ... some math
return b
x = 1
x = func(x)
So what if you want to return multiple values? Use a tuple!
def func(a, b):
a, b = 5 * b, 6 * a
# ...
return b, a
a, b = 2, 3
a, b = func(a, b)
That is feasible, up to a point - but nonetheless,useless in any kind of "real code".
You can use it for backyard magic tricks:
Just retrieve the calling stack_frame like you are doing,
then loop linearly through the variables available in the calling frame for
one that references the same object you got as a parameter. The variables are in the
f_locals and f_globals dictionaries which are attributes of the frame object.
Of course, the parameter passed may not be in a variable name to start with:
it might be a constant in the caller, or it might be inside a dict or a list.
Either way, as #Nasser put it in the answer: it is not the way Python works. Objects exist in memory, and variable names, in each scope, just point to those objects. The names themselves are meaningless otherwise.
import inspect
from itertools import chain
def magic(x):
# note that in Python2, 'currentframe' could get a parameter
# to indicate which frame you want on the stack. Not so in Python3
fr = inspect.currentframe().f_back
for var_name, value in chain(fr.f_locals.items(), fr.f_globals.items()):
if value is x:
print ("The variable name in the caller context is {}".format(var_name))
def caler():
y = "My constant"
magic(y)
use lists as references
def func(a):
a[0] = a[0] + 1
x = [1]
func(x)
print x[0] # x expected to 2
y = [2]
func(y)
print y[0] # y expected to 3
This is my final solution. Use ast to parse the function statement and get the args.:
import inspect
import re
import ast
def func(a,b,c):
stack = inspect.stack()
stack_pre = stack[1]
function_call_string = ';'.join( stack_pre[4] ).strip('\n')
ret = re.search(r'(%s[ ]*\(.*\))' % func.func_name, function_call_string)
striped_function_call_string = ret.group()
parser = ast.parse(striped_function_call_string)
frame = inspect.currentframe()
for arg in parser.body[0].value.args:
iter_frame = frame.f_back
while iter_frame:
if arg.id in iter_frame.f_locals:
iter_frame.f_locals[arg.id] += 1
break
iter_frame = iter_frame.f_back
x=1;y=2;z=3;func(x,y,z)
print x,y,z
Out:
2 3 4
I am refactoring a piece of code, and I have run into the following problem. I have a huge parameter list, which now I want to pass as kwargs. The code is like this:
def f(a, b, c, ...):
print a
...
f(a, b, c, ...)
I am refactoring it to:
data = dict(a='aaa', b='bbb', c='ccc', ...)
f(**data)
Which means I have to do:
def f(**kwargs):
print kwargs['a']
...
But this is a pita. I would like to keep:
def f(**kwargs):
# Do some magic here to make the kwargs directly accessible
print a
...
Is there any straightforward way of making the arguments in the kwargs dict directly accessible, maybe by using some helper class / library?
There are some ways - but you can also wrap your function like this:
def f(**kwargs):
arg_order = ['a', 'b', 'c', ...]
args = [kwargs.get(arg, None) for arg in arg_order]
def _f(a, b, c, ...):
# The main code of your function
return _f(*args)
Sample:
def f(**kwargs):
arg_order = ['a', 'b', 'c']
args = [kwargs.get(arg, None) for arg in arg_order]
def _f(a, b, c):
print a, b, c
return _f(*args)
data = dict(a='aaa', b='bbb', c='ccc')
f(**data)
Output:
>>>
aaa bbb ccc
Well, you can update locals manually, but the documentation specifically warns against it.
for key, value in kwargs.iteritems(): #Python 2.7 here
locals()[key] = value
The other option is using exec, which though usually frowned on, is at least guaranteed to work correctly.
for key, value in kwargs.iteritems(): #Python 2.7 here
exec('{} = value'.format(key)
Though I wouldn't ever admit to anyone that you actually did either of these.
Inside the function:
for k, v in kwargs.iteritems():
locals()[k] = v
Another possible method (Based on your comment here) is to use an Attribute Dictionary:
class AttributeDict(dict):
def __getattr__(self, attr):
return self[attr] if attr in self.keys() else None
def __setattr__(self, attr, value):
self[attr] = value
To be used like this:
def f(**kwargs):
kwargs = AttributeDict(kwargs)
Sample:
def f(**kwargs):
kwargs = AttributeDict(kwargs)
print kwargs.a, kwargs.b, kwargs.c
data = dict(a='aaa', b='bbb', c='ccc')
f(**data)
Output:
>>>
aaa bbb ccc
Note: You can always name it x if you want a shorter variable name, then access would just be x.a
I have had to refactor my code in similar cases and I share your dislike of using kwargs['a'], which just feels a bit awkward. Sometimes I use a bunch object instead of a dictionary, which allows you to access fields by attribute access (params.a) instead of a dictionary lookup. It saves you only 3 characters of typing for every time you use a parameter, but I find it much better looking since you do not need quotes around the parameter name. There are various recipes around to implement one, see e.g. these ones.
So instead of using a dict like in your case, you would use it like
In [1]: class Bunch:
...: def __init__(self, **kwds):
...: self.__dict__.update(kwds)
In [2]: params = Bunch(a = 'aaa', b = 'bbb')
In [3]: def f(p):
...: print p.a
...:
In [4]: f(params)
aaa
I know this is not a direct answer to your question, but it is just an alternative for using kwargs.
It seems to me that you're doing redundand job, and easiest solution wouldb leave as it is. Those a, b, c are keyword arguments as well as positional, so you can call your function the way you like:
>>> def f(a, b, c=3):
... print a, b, c
With all keyword args
>>> f(**{'a':1, 'b':2})
1 2 3
With mix of positional and keyword args
>>> f(5, **{'b':4})
5 4 3
And get proper error in case of wrong keyword args
>>> f(**{'d':4, 'a':1, 'b':2})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() got an unexpected keyword argument 'd'
Say I have a Python function that returns multiple values in a tuple:
def func():
return 1, 2
Is there a nice way to ignore one of the results rather than just assigning to a temporary variable? Say if I was only interested in the first value, is there a better way than this:
x, temp = func()
You can use x = func()[0] to return the first value, x = func()[1] to return the second, and so on.
If you want to get multiple values at a time, use something like x, y = func()[2:4].
One common convention is to use a "_" as a variable name for the elements of the tuple you wish to ignore. For instance:
def f():
return 1, 2, 3
_, _, x = f()
If you're using Python 3, you can you use the star before a variable (on the left side of an assignment) to have it be a list in unpacking.
# Example 1: a is 1 and b is [2, 3]
a, *b = [1, 2, 3]
# Example 2: a is 1, b is [2, 3], and c is 4
a, *b, c = [1, 2, 3, 4]
# Example 3: b is [1, 2] and c is 3
*b, c = [1, 2, 3]
# Example 4: a is 1 and b is []
a, *b = [1]
The common practice is to use the dummy variable _ (single underscore), as many have indicated here before.
However, to avoid collisions with other uses of that variable name (see this response) it might be a better practice to use __ (double underscore) instead as a throwaway variable, as pointed by ncoghlan. E.g.:
x, __ = func()
Remember, when you return more than one item, you're really returning a tuple. So you can do things like this:
def func():
return 1, 2
print func()[0] # prints 1
print func()[1] # prints 2
The best solution probably is to name things instead of returning meaningless tuples (unless there is some logic behind the order of the returned items). You can for example use a dictionary:
def func():
return {'lat': 1, 'lng': 2}
latitude = func()['lat']
You could even use namedtuple if you want to add extra information about what you are returning (it's not just a dictionary, it's a pair of coordinates):
from collections import namedtuple
Coordinates = namedtuple('Coordinates', ['lat', 'lng'])
def func():
return Coordinates(lat=1, lng=2)
latitude = func().lat
If the objects within your dictionary/tuple are strongly tied together then it may be a good idea to even define a class for it. That way you'll also be able to define more complex operations. A natural question that follows is: When should I be using classes in Python?
Most recent versions of python (≥ 3.7) have dataclasses which you can use to define classes with very few lines of code:
from dataclasses import dataclass
#dataclass
class Coordinates:
lat: float = 0
lng: float = 0
def func():
return Coordinates(lat=1, lng=2)
latitude = func().lat
The primary advantage of dataclasses over namedtuple is that its easier to extend, but there are other differences. Note that by default, dataclasses are mutable, but you can use #dataclass(frozen=True) instead of #dataclass to force them being immutable.
Here is a video that might help you pick the right data class for your use case.
Three simple choices.
Obvious
x, _ = func()
x, junk = func()
Hideous
x = func()[0]
And there are ways to do this with a decorator.
def val0( aFunc ):
def pick0( *args, **kw ):
return aFunc(*args,**kw)[0]
return pick0
func0= val0(func)
This seems like the best choice to me:
val1, val2, ignored1, ignored2 = some_function()
It's not cryptic or ugly (like the func()[index] method), and clearly states your purpose.
If this is a function that you use all the time but always discard the second argument, I would argue that it is less messy to create an alias for the function without the second return value using lambda.
def func():
return 1, 2
func_ = lambda: func()[0]
func_() # Prints 1
This is not a direct answer to the question. Rather it answers this question: "How do I choose a specific function output from many possible options?".
If you are able to write the function (ie, it is not in a library you cannot modify), then add an input argument that indicates what you want out of the function. Make it a named argument with a default value so in the "common case" you don't even have to specify it.
def fancy_function( arg1, arg2, return_type=1 ):
ret_val = None
if( 1 == return_type ):
ret_val = arg1 + arg2
elif( 2 == return_type ):
ret_val = [ arg1, arg2, arg1 * arg2 ]
else:
ret_val = ( arg1, arg2, arg1 + arg2, arg1 * arg2 )
return( ret_val )
This method gives the function "advanced warning" regarding the desired output. Consequently it can skip unneeded processing and only do the work necessary to get your desired output. Also because Python does dynamic typing, the return type can change. Notice how the example returns a scalar, a list or a tuple... whatever you like!
When you have many output from a function and you don't want to call it multiple times, I think the clearest way for selecting the results would be :
results = fct()
a,b = [results[i] for i in list_of_index]
As a minimum working example, also demonstrating that the function is called only once :
def fct(a):
b=a*2
c=a+2
d=a+b
e=b*2
f=a*a
print("fct called")
return[a,b,c,d,e,f]
results=fct(3)
> fct called
x,y = [results[i] for i in [1,4]]
And the values are as expected :
results
> [3,6,5,9,12,9]
x
> 6
y
> 12
For convenience, Python list indexes can also be used :
x,y = [results[i] for i in [0,-2]]
Returns : a = 3 and b = 12
It is possible to ignore every variable except the first with less syntax if you like. If we take your example,
# The function you are calling.
def func():
return 1, 2
# You seem to only be interested in the first output.
x, temp = func()
I have found the following to works,
x, *_ = func()
This approach "unpacks" with * all other variables into a "throwaway" variable _. This has the benefit of assigning the one variable you want and ignoring all variables behind it.
However, in many cases you may want an output that is not the first output of the function. In these cases, it is probably best to indicate this by using the func()[i] where i is the index location of the output you desire. In your case,
# i == 0 because of zero-index.
x = func()[0]
As a side note, if you want to get fancy in Python 3, you could do something like this,
# This works the other way around.
*_, y = func()
Your function only outputs two potential variables, so this does not look too powerful until you have a case like this,
def func():
return 1, 2, 3, 4
# I only want the first and last.
x, *_, d = func()
e.g. so that these would both work - is it possible?
(val,VAL2) = func(args)
val = func(args)
Where val is not a tuple
For example I'd like these to work for my custom object something
for item in something:
do_item(item) #where again item - is not a tuple
for (item,key) in something:
do_more(key,item)
I thought that I need to implement next() function in two different ways...
edit: as follows from the answers below, this should not really be done.
If you mean, can the function act differently based on the return types the caller is expecting, the answer is no (bar seriously nasty bytecode inspection). In this case, you should provide two different iterators on your object, and write something like:
for item in something: # Default iterator: returns non-tuple objects
do_something(item)
for (item,key) in something.iter_pairs(): # iter_pairs returns different iterator
do_something_else(item, key)
eg. see the dictionary object, which uses this pattern. for key in mydict iterates over the dictionary keys. for k,v in mydict.iteritems() iterates over (key, value) pairs.
[Edit] Just in case anyone wants to see what I mean by "seriously nasty bytecode inspection", here's a quick implementation:
import inspect, opcode
def num_expected_results():
"""Return the number of items the caller is expecting in a tuple.
Returns None if a single value is expected, rather than a tuple.
"""
f = inspect.currentframe(2)
code = map(ord, f.f_code.co_code)
pos = f.f_lasti
if code[pos] == opcode.opmap['GET_ITER']: pos += 1 # Skip this and the FOR_ITER
if code[pos] > opcode.EXTENDED_ARG: pos +=5
elif code[pos] > opcode.HAVE_ARGUMENT: pos +=3
else: pos += 1
if code[pos] == opcode.opmap['UNPACK_SEQUENCE']:
return code[pos+1] + (code[pos+2] << 8)
return None
Usable something like:
class MagicDict(dict):
def __iter__(self):
if num_expected_results() == 2:
for k,v in self.iteritems():
yield k,v
else:
for k in self.iterkeys():
yield k
d=MagicDict(foo=1, bar=2)
print "Keys:"
for key in d:
print " ", key
print "Values"
for k,v in d:
print " ",k,v
Disclaimer: This is incredibly hacky, insanely bad practice, and will cause other programmers to hunt you down and kill you if they ever see it in real code. Only works on cpython (if that). Never use this in production code (or for that matter, probably any code).
Have you tried that? It works.
def myfunction(data):
datalen = len(data)
result1 = data[:datalen/2]
result2 = data[datalen/2:]
return result1, result2
a, b = myfunction('stuff')
print a
print b
c = myfunction('other stuff')
print c
In fact there is no such thing as "return signature". All functions return a single object. It seems that you are returning more than one, but in fact you wrap them into a container tuple object.
Yes it's doable:
def a(b):
if b < 5:
return ("o", "k")
else:
return "ko"
and the result:
>>> b = a(4)
>>> b
('o', 'k')
>>> b = a(6)
>>> b
'ko'
I think the thing after is to be careful when you will use the values returned...
>>> def func(a,b):
return (a,b)
>>> x = func(1,2)
>>> x
(1, 2)
>>> (y,z) = func(1,2)
>>> y
1
>>> z
2
That doesn't really answer your question. The real answer is that the left side of the assignment doesn't affect the returned type of the function and can't be used to distinguish between functions with different return types. As noted in other answers, the function can return different types from different return statements but it doesn't know what's on the other side of the equals sign.
In the case of this function, it returns a tuple. If you assign it to x, x has the value of the tuple. (y, z) on the left side of the assignment is "tuple unpacking". The tuple returned by func() is unpacked into y and z.
Update:
Given the example use case, I'd write different generators to handle the cases:
class Something(object):
def __init__(self):
self.d = {'a' : 1,
'b' : 2,
'c' : 3}
def items(self):
for i in self.d.values():
yield i
def items_keys(self):
for k,i in self.d.items():
yield i,k
something = Something()
for item in something.items():
....: print item
....:
1
3
2
for item,key in something.items_keys():
....: print key, " : ", item
....:
a : 1
b : 2
c : 3
Or
You can return a tuple:
In [1]: def func(n):
...: return (n, n+1)
...:
In [2]: a,b = func(1)
In [3]: a
Out[3]: 1
In [4]: b
Out[4]: 2
In [5]: x = func(1)
In [6]: x
Out[6]: (1, 2)
Yes, both would work. In the first example, val1 and val2 would have the two values. In the second example, val would have a tuple. You can try this in your python interpreter:
>>> def foo():
... return ( 1, 2 )
...
>>> x = foo()
>>> (y,z) = foo()
>>> x
(1, 2)
>>> y
1
>>> z
2
It's possible only if you're happy for val to be a 2-item tuple (or if args need not be the same in the two cases). The former is what would happen if the function just ended with something like return 23, 45. Here's an example of the latter idea:
def weirdfunc(how_many_returns):
assert 1 <= how_many_returns <= 4
return 'fee fie foo fum'.split()[:how_many_returns]
var1, var2 = weirdfunc(2) # var1 gets 'fee', var2 gets 'fie'
var, = weirdfunc(1) # var gets 'fee'
This is asking for major confusion. Instead you can follow dict with separate keys, values, items, etc. methods, or you can use a convention of naming unused variables with a single underscore. Examples:
for k in mydict.keys(): pass
for k, v in mydict.items(): pass
for a, b in myobj.foo(): pass
for a, _ in myobj.foo(): pass
for _, b in myobj.foo(): pass
for _, _, _, d in [("even", "multiple", "underscores", "works")]:
print(d)
for item in something: # or something.keys(), etc.
do_item(item)
for item, key in something.items():
do_more(key, item)
If this doesn't fit your function, you should refactor it as two or more functions, because it's clearly trying to fulfill two or more different goals.