I have started to learn python, and I would like to ask you about something which I considered a little magic in this language.
I would like to note that before learning python I worked with PHP and there I haven't noticed that.
What's going on - I have noticed that some call constructors or methods in Python are in this form.
object.call(variable1 = value1, variable2 = value2)
For example, in FLask:
app.run(debug=True, threaded=True)
Is any reason for this convention? Or is there some semantical reason outgoing from the language fundamentals? I haven't seen something like that in PHP as often as in Python and because I'm really surprised. I'm really curious if there is some magic or it's only convention to read code easier.
These are called keyword arguments, and they're usually used to make the call more readable.
They can also be used to pass the arguments in a different order from the declared parameters, or to skip over some default parameters but pass arguments to others, or because the function requires keyword arguments… but readability is the core reason for their existence.
Consider this:
app.run(True, False)
Do you have any idea what those two arguments mean? Even if you can guess that the only two reasonable arguments are threading and debugging flags, how can you guess which one comes first? The only way you can do it is to figure out what type app is, and check the app.run method's docstring or definition.
But here:
app.run(debug=True, threaded=False)
It's obvious what it means.
It's worth reading the FAQ What is the difference between arguments and parameters?, and the other tutorial sections near the one linked above. Then you can read the reference on Function definitions for full details on parameters and Calls for full details on arguments, and finally the inspect module documentation on kinds of parameters.
This blog post attempts to summarize everything in those references so you don't have to read your way through the whole mess. The examples at the end should also serve to show why mixing up arguments and parameters in general, keyword arguments and default parameters, argument unpacking and variable parameters, etc. will lead you astray.
Specifying arguments by keyword often creates less risk of error than specifying arguments solely by position. Consider this function to compute loan payments:
def pmt(principal, interest, term):
return **something**;
When one tries to compute the amortization of their house purchase, it might be invoked thus:
payment = pmt(100000, 4.2, 360)
But it is difficult to see which of those values should be associated with which parameter. Without checking the documentation, we might think it should have been:
payment = pmt(360, 4.2, 100000)
Using keyword parameters, the call becomes self-documenting:
payment = pmt(principal=100000, interest=4.2, term=360)
Additionally, keyword parameters allow you to change the order of the parameters at the call site, and everything still works correctly:
# Equivalent to previous example
payment = pmt(term=360, interest=4.2, principal=100000)
See http://docs.python.org/2/tutorial/controlflow.html#keyword-arguments for more information.
They are arguments passed by keywords. There is no semantical difference between keyword arguments and positional arguments.
They are often used like "options", and provide a much more readable syntax for this circumstance. Think of this:
>>> sorted([2,-1,3], key=lambda x: x**2, reverse=True)
[3, 2, -1]
Versus(python2):
>>> sorted([2,-1,3], None, lambda x: x**2, True)
[3, 2, -1]
In this second example can you tell what's the meaning of None or True?
Note that in keyword only arguments, i.e. arguments that you can only specify using this syntax, were introduced in python3. In python2 any argument can be specified by position(except when using **kwargs but that's another issue).
There is no "magic".
A function can take:
Positional arguments (args)
Keyworded arguments (kwargs)
Always is this order.
Try this:
def foo(*args, **kwargs):
print args
print kwargs
foo(1,2,3,4,a=8,b=12)
Output:
(1, 2, 3, 4)
{'a': 8, 'b': 12}
Python stores the positional arguments in a tuple, which has to be immutable, and the keyworded ones in a dictionary.
The main utility of the convention is that it allows for setting certain inputs when there may be some defaults in between. It's particularly useful when a function has many parameters, most of which work fine with their defaults, but a few need to be set to other values for the function to work as desired.
example:
def foo(i1, i2=1, i3=3, i4=5):
# does something
foo(1,2,3,4)
foo(1,2,i4=3)
foo(1,i2=3)
foo(0,i3=1,i2=3,i4=5)
Related
Implementing some Neural Network with tensorflow, I've faced a method which parameters have took my attention. I'm talking about tf.nn.sigmoid_cross_entropy_with_logits (Documentation here).
The first parameter it receives as first parameter _sentinel=None which, according to the documentation:
_sentinel: Used to prevent positional parameters. Internal, do not use.
I understand that by having this parameter, next ones have to be named instead of positional is this one don't have to be used, but my question is. In which cases does prevent positional parameters have some benefit? What is their main goal to use this? Because I could also run
tf.nn.sigmoid_cross_entropy_with_logits(None, my_labels, my_logits)
being all arguments positional. Anyway, I want to clarify that my question is not focused in TensorFlow, it's just the example that I have found.
Positional parameters couple the caller and receiver on the order of the parameters. It makes refactoring the order of the reciver's parameters more difficult.
For example, if I have
def foo(a, b, c):
do_stuff(a,b,c)
and I decide, for reasons, perhaps I want to make a partial function or whatever, that it would be better to have
def foo(b, a, c):
do_stuff(a,b,c)
But now I have callers in the wild and it would be very rude to change my contract, so I'm stuck.
Sandi Metz in Practical Object-Oriented Design in Ruby also addresses this. (I know this is python, but oop is oop)
When the code [is changed to use keyword arguments], it lost its dependency
on argument order but it gained a dependency on the names of the keys
in the [keyword arguments]. This change is healthy. The new dependency is
more stable than the old, and thus this code faces less risk of being
forced to change. Additionally, and perhaps unexpectedly, the [keywords]
provides one new, secondary benefit: The key names in the hash furnish
explicit documentation about the arguments. This is a byproduct of
using a hash but the fact that it is unintentional makes it no less
useful. Future maintainers of this code will be grateful for the
information.
Keyword arguments are also nice if you have a lot of parameters. Order is easy to get wrong. It may also make a nicer API in the opinion of the authors.
PEP-3102 also addresses this, but I find the rationale unsatisfying from the perspective of "why would I choose to design something like this"
The current Python function-calling paradigm allows arguments to be
specified either by position or by keyword. An argument can be filled
in either explicitly by name, or implicitly by position.
There are often cases where it is desirable for a function to take a
variable number of arguments. The Python language supports this using
the 'varargs' syntax (*name), which specifies that any 'left over'
arguments be passed into the varargs parameter as a tuple.
One limitation on this is that currently, all of the regular argument
slots must be filled before the vararg slot can be.
This is not always desirable. One can easily envision a function which
takes a variable number of arguments, but also takes one or more
'options' in the form of keyword arguments. Currently, the only way to
do this is to define both a varargs argument, and a 'keywords'
argument (**kwargs), and then manually extract the desired keywords
from the dictionary.
What is the use for keyword only parameters:
For some function, it is impossible to do otherwise (ex: print(a, b, end=''))
It prevents you from making silly mistakes, consider the following example:
# if it wasn't made with kw-only parameters, this would return 3
>>> sorted(3, 1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: sorted expected 1 arguments, got 2
>>> sorted((1,2), reverse=True)
[2, 1]
It allows you to change things later:
# if
def sorted(iterable, reverse=False)
# becomes
def sorted(iterable, key=None, reverse=False)
# you can guarantee backwards compatibility
First, a caveat that I can't know the intention of the person who wrote that. However, I can offer reason why “prevent positional parameters” might be desirable.
It's often important that a parameter be keyword-only, that is, it must be used only by name. The parameter is not conceptually an input to the function's purpose; it's more a modifier (change the behaviour in this way), or an external resource (here is the log file to emit your messages to), etc.
For that reason, Python 3 now allows you to define, in the signature of the function, specific parameters as keyword-only parameters. The change is documented in PEP 3102 Keyword-only arguments along with rationale.
def func(*v, **k): pass
func(**{'a': 1}, *(1, 2, 3)) # ERROR...
func(a=1, *(1, 2, 3)) # OK...
I don't get why this first doesn't work, instead the second does.
Someone could just tell "because the implementation says so", but I would like also to know why the implementation says so. Couldn't the implementation translate the first to the second in order to make this call work?
first, the second line works only if the a parameter is the last (the forth). for example, the following code will not work:
def func(a,b,c,d):
pass
func(a=1, *(1, 2, 3))
And to you question, I think the problem with the first line:
func(**{'a': 1}, *(1, 2, 3))
is that you can have multiple values for the same parameter this way, if the dictionary contains one of the first 3 parameters.
the difference is that the dictionary can be a variable and have different value on different runs so this code always considered illegal
Passing positional arugments and keyword arguments like this is part of the language's syntax, not a run-time feature. That is, *(1,2,3) does not simply create a special object that is passed to the function when it is called and "somehow" used to assign values to the defined parameters. As such, allowing this kind of flexibility in the ordering would come at the cost of greatly complicating the parser for no real benefit.
I am new to Python and higher level languages in general, so I was wondering if it is looked down upon if I have a function that takes in a lot of arguments, and how to better architect my code to prevent this.
For example what this function is essentially doing is printing each location of a string in a file.
def scan(fpin,base,string,pNo,print_to_file,dumpfile_path,quiet):
This function is being called from the main function, which is basically parsing the command line arguments and passing the data to the scan function. I have thought of creating a class containing all of these arguments and passing it to scan,but there will only be one instance of this data, so wouldn't that be pointless?
Named arguments are your friends. For things that act like semi-optional configuration options with reasonable defaults, give the parameters the defaults, and only pass them (as named arguments) for non-default situations. If there are a lot of parameters without reasonable defaults, then you may want to name all of them when you call the function.
Consider the built-in function sorted. It takes up to four arguments. Is the reverse parameter before or after cmp? What should I pass in as key if I want the default behavor? Answer: Hell if I can remember. I call sorted(A, reverse=True) and it does what I'd expect.
Incidentally, if I had a ton of "config"-style arguments that I was passing into every call to scan, and only changing (say, fpin and string) each time, I might be inclined to put all the other argumentsinto a dictionary, and then pass it to the function with **kwargs syntax. That's a little more advanced. See the manual for details. (Note that this is NOT the same as declaring the function as taking **kwargs. The function definition is the same, the only difference is what calls to it look like.)
No, there's really nothing wrong with it. If you have N different arguments (things that control the execution of your function), you have to pass them somehow - how you actually do that is just user preference if you ask me.
However... if you find yourself doing something like this, though:
func('somestring', a=A, b=B, c=C)
func('something else', a=A, b=B)
func('something third', a=A, c=C, d=D)
etc. where A,B,C are really configurations for lots of different things, then you should start looking into a class. A class does many things, but it does also create context. Instead, then you can do something like:
cf = myclass(a=A, b=B, c=C, d=D)
cf.func('somestring')
cf.func('something else')
cf.func('something third')
etc.
when I try the following, I get error in the function definition itself.
>>> def mymap(*seq,pad=None):
File "<stdin>", line 1
def mymap(*seq,pad=None):
SyntaxError: invalid syntax
I am trying to give default value for the parameter pad.
but then, I tried this and it works (for wrong reason):
>>> def mymap(pad=None,*seq):
... print seq
>>> mymap([1,2,3],[4,5,6])
([4, 5, 6],)
[(4,)]
>>>
It is not printing the tuple of seq which should be ([1,2,3],[4,5,6]).
What you really want here is for pad to be a keyword-only parameter.
One of the major changes in 3.0 was designing function parameters and arguments to, among other things, provide exactly the feature you're looking for. In fact, your first version works exactly as you'd hope it to. But if you want to stick with 2.x, you don't get any of the new features in 3.x.
So, you have no choice but to fake it:
def mymap(*seq, **kwargs):
pad = kwargs.pop('pad', None)
if kwargs:
raise TypeError("mymap() got an unexpected keyword argument '{}'".format(kwargs.keys()[0]))
print seq
This does very nearly the exact same thing as the equivalent Python 3 code. Of course it's more complicated, a lot less clear, slower, and opaque to introspection… all of which is exactly why keyword-only parameters were added in 3.0. But if you need it, and you can't use Python 3.x, this is what you have to do. (Just be glad you're not trying to write C-API functions.)
You seem to be mixing up two entirely independent things: a parameter having a default value (which I assume is what you mean by the term "default parameter", which doesn't really mean anything), and being keyword-only.
You've already got a parameter with a default value, in your second version. You can see that easily: call mymap(), and it succeeds, with pad getting its default value of None (and seq being empty).
But you want it to also be a keyword-only parameter, so it doesn't steal the first positional argument. And that's the part you can't do in 2.x.
If you want to know more, PEP 3012 explains the keyword-only feature, and Arguments and parameters attempts to gather all the relevant documentation links, and summarize how everything fits together.
It looks like you are inputting the args [1,2,3] and [4,5,6] as two seperate arguments. They are not in a tuple.
mymap([1,2,3], [4,5,6]) << in this example, from your code, the [1,2,3] is being passed in for pad and the [4,5,6] is being passed in for seq. That's why printing seq results in [4,5,6]
Also, named arguments like pad, must come before *args or **kwargs.
For example:
mymap(None, [1,2,3],[4,5,6]) #passing None for pad
prints:
([1,2,3], [4,5,6])
While I'm going through Python code and seeing functions called, I notice things like
functionCall(argument='something')
or
someclass.functionCall(argument='something')
I've played around with it and noticed you have to name that variable with the same name as the one in the scope of the function or class function itself. Is that just a convention (for useful naming) or is there more to it that I'm missing?
Those are just standard keyword arguments.
They are mainly useful when calling functions that usually assume default values, but the user is interested in passing a custom value without affecting the other defaults.
For example:
def foo(a='a', b='b', c='c'):
print a + b + c
Then:
>>> foo()
'abc'
>>> foo(c='x') # don't know or care about a and b's value
'abx'
This is called a keyword argument. Keyword arguments have a couple distinct advantages:
They can be specified in any order by the caller.
They have default values, which means the caller doesn't have to provide a value for each argument if the defaults are sensible.
They can make code more readable. For example, foo(width=100, height=200) is easier to understand than foo(100, 200). It is also easier to use, because you don't have to remember whether the function takes width,height or height,width.
It is good to have named arguments as arguments can be specified in any order by using named arguments.
Even required arguments (like object, which has no default value) can be named, and named arguments can appear in any order.
Also see This
Python's argument passing mechanisms are extremely flexible.
cons: too many arguments to a function. Solutions: split into multiple functions, pass some args together in a dictionary or object.
cons: bad variable names. Solution: give variables more descriptive names.
Or remember the correct order .. :)
class xyz:
def __init__ (self, a='1', b='2'):
print a,b
xyz(b=3,a=4)
xyz(a=5,b=6)
>>4 3
>>5 6
these are keyword parameters. They enable you to pass parameters in any order (useful for skipping optional parameters)
Please note though, calling functions that way comes with a little bit of overhead:
def test(a,b,c):
return a+b+c
def t1():
return test(1,2,3)
def t2():
return test(a=1, b=2, c=3)
timeit(t1)
0.23918700218200684
timeit(t2)
0.2716050148010254