I am trying to understand the use of *args and **kwds when creating subclasses in Python.
I want to understand why this code behaves the way it does. If I leave out the *args and **kwds in a call to super().__init__, I get some strange argument unpacking.
Here is my test case:
class Animal(object):
def __init__(self, moves, num_legs):
self.moves = moves
self.num_legs = num_legs
def describe(self):
print "Moves :{} , num_legs : {}".format(self.moves, self.num_legs)
class Snake(Animal):
def __init__(self, poisonous, *args, **kwds):
self.poisonous = poisonous
print "I am poisonous:{}".format(self.poisonous)
# This next line is key. You have to use *args , **kwds.
# But here I have deliberately used the incorrect form,
# `args` and `kwds`, and am suprised at what it does.
super(Snake, self).__init__(args, kwds)
Now, when I create instances of the Snake subclass, which contains the erroneous call to super(…).__init__ (where I use args and kwds instead of *args and **kwds), I get some interesting “argument unpacking”.
s1 = Snake(False, moves=True, num_legs=0)
s2 = Snake(poisonous=False, moves=True, num_legs=1)
s3 = Snake(False, True, 3)
s1.describe()
s2.describe()
s3.describe()
What I get is:
Moves :() , num_legs : {'moves': True, 'num_legs': 0}
Moves :() , num_legs : {'moves': True, 'num_legs': 1}
Moves :(True, 3) , num_legs : {}
So why is it that in s1 and s2, __init__ assumes that moves = True and num_legs = 0 or 1 are keyword arguments, and sets the num_legs to a dict?
In s3, it unpacks both of the variables to moves (in class Animal) as a tuple.
I stumbled into this as I was trying to understand argument unpacking. Sorry in advance—I don't know how to frame this question any better.
In Snake.__init__, args is a tuple of all positional arguments after poisonous and kwds is a dict of all the keyword arguments apart from poisonous. By calling
super(Snake,self).__init__(args,kwds)
you assign args to moves and kwds to num_legs in Animal.__init__. That’s exactly what you are seeing in your output.
The first two calls don’t have any positional arguments apart from poisonous, so args and consequential moves is an empty tuple. The third call has no keyword arguments, so kwds and consequential num_legs is an empty dict.
In short: def __init__(self,poisonous,*args,**kwds): means: capture positional arguments in a tuple args and keyword arguments in a dictionary kwds. Similarly, super(Snake,self).__init__(*args, **kwds) means: unpack the tuple args and the dictionary kwds into arguments so that they're passed separately to __init__.
If you don't use the * and ** then you're passing args and kwds as they are, which means you're getting a tuple and a dictionary.
As you've said, you'd need to write:
super(Snake,self).__init__(*args, **kwds)
to properly pack / unpack the arguments. In your current code you're not packing / unpacking the arguments so it sets num_legs to a dictionary as that's what kwds is at the moment.
If you don't give the arguments names then they're positional arguments. Hence Snake(False,True,3) are all positional arguments.
If you do give the arguments names then they're keyword arguments: Snake(poisonous=False,moves=True,num_legs=1).
In the first case you're combining both one positional argument and two keyword arguments: Snake(False,moves=True,num_legs=0).
A variability is nicer and more intuitive than this Snake(False, True, 3):
Snake("Python", constrictor=True, poisonous=False)
Animal("Snail") # Snail has a foot but no leg. Defaults are good for it.
# Cobra eat other snakes, including poisonous, fast attacks, snake fights.
Snake("Indian cobra", moves=True, poisonous=True)
Animal("Myriapod", num_legs=750) # Changes for an idividual after every molting.
Oh, really exciting question about Python, not only about programming. :)
It is a good idea to have the most individual parameters on the first places, that are common for all subclasses, like the universal "self" itself is. The next very common is a name like in this example.
If you believe that your classes will be never modified and they will be used everytimes with all implemented parameters and you will never make a mistake in the correct order, you need not any variability. You can continue to use fixed positional parameters as you are used. This assumption is frequently not fulfilled. Tomorrow will nobody remember what should be the first False and the second True without seeing it together with keywords.
If you need to call your class with fixed positional parameters by Snake(False, True, 3) you can not use **kwds for any of these parameters.
A)
Let we now expect it that your example Snake(False, True, 3) is a required test case. Then you can't use **kwds for anything of your positional parameters (poisonous, moves, num_legs). You have only these four possibilities of implementation __init__ header: (none good enough)
# the most fragile solution - easy extensible, not easy to observe the order
class Snake(Animal):
def __init__(self, *args):
self.poisonous = args.pop[0]
# or better ...pop[-1] that allows adding new parameters to the end
super(Snake,self).__init__(*args)
# now is args undefined if ancestors could eat parts from it but
# everything is in self
# the most naive solution - easy readable, not easy extensible because not DRY
class Snake(Animal):
def __init__(self, poisonous, moves, num_legs):
self.poisonous = poisonous
super(Snake,self).__init__(moves, num_legs)
# anythig between them combines disadvantages of both previous
class Snake(Animal):
def __init__(self, poisonous, *args):
self.poisonous = poisonous
super(Snake,self).__init__(*args)
class Snake(Animal):
def __init__(self, poisonous, moves, *args):
self.poisonous = poisonous
super(Snake,self).__init__(moves, *args)
.
B)
Keyword parameters are more robust because some their errors can be automatically reported.
Expect that you redefine Animal to increase its variablility:
class Animal(object):
def __init__(self,name, moves=True, num_legs=None):
self.name = name
self.moves = moves
self.num_legs = num_legs
# The recommended Snail !
class Snake(Animal):
def __init__(self, *args, **kwds):
"""Snake: Implements.. (Docs important, otherwise real keywords not seen in help)
kwds: (only what defined here)
poisonous: Bla bla. default=True
constrictor: Bla bla bla. default=False
"""
# A copy of kwds can be created, if manipulation with original is prohibited.
self.poisonous = kwds.pop('poisonous', True) # default: poisonous snake
self.constrictor = kwds.pop('constrictor', False)
# OK. This reports error if some keyword is misspelled and will not be consumed.
super(Snake,self).__init__(*args, **kwds)
# This Snake is more readable, but its descendants would be more complicated,
# otherwise is possible: "TypeError: got multiple values for keyword argument 'xy'".
class Snake(Animal):
def __init__(self, name, poisonous=True, constrictor=False, *args, **kwds):
self.poisonous = poisonous
self.constrictor = constrictor
super(Snake,self).__init__(name, *args, **kwds)
Now you have a big variability and the order of keyword arguments is not important.
Related
This question already has an answer here:
Python Multiple Inheritance: Argument passing (**kwargs) and super()
(1 answer)
Closed 1 year ago.
I am newbie , I have such a question , in this code
class One:
def __init__(self, x, **kwargs):
super().__init__(**kwargs)
self.x = x**2
class Two:
def __init__(self, y, **kwargs):
super().__init__(**kwargs)
self.y = y**3
class Three(One, Two):
def __init__(self, **kwargs):
super().__init__(**kwargs)
print(self.y, self.x)
Three(x=1, y=2)
I understood how **kwargs works , but cant understand why need to call parents init function in
classes One and Two . In my case in class One and class Two as I see **kwargs is just empty
dict. Why need I super().init(**kwargs) , if kwargs is just empty dict
I tried another ways too , without kwargs
class Three(One, Two):
def __init__(self, x,y):
One.__init__(self,x)
Two.__init__(self,y)
print(self.y, self.x)
This way is easier to Undertstand , But Can anyone Explain how it works with **kwargs . Thanks
Based on your code, you could just do:
class Three(One, Two):
pass
Unless you really have something specific to implement in Three at instanciation time
However, you need to leave **kwargs on your 2 parent classes as their __init__ will receive all the non-defined keyword arguments passed to Three.__init__
Advice: use a debugger, like PyCharm/VSCode to visualize what happens when the instance gets created, that really nice to observe how the code runs.
super is just a way to let Python decide in which order the parent classes method will be called (MRO)
When python executes Three(x=1, y=2) it actually sends to Three.init 3 things:
self: instance being instanciated
a tuple of positional args (commonly named args)
a dict of keyword args (commonly named kwargs)
In your case, as you only used keyword arguments, args will be an empty tuple and kwargs will be {'x': 1, 'y': 2}
Then:
One.__init__(x, **kwargs) will take x and leave y inside kwargs
Two.__init__(y, **kwargs) will take y and leave x inside kwargs
I have the follow init function who receives a lot of args to run the class (the args are default values if the user don't input anything or can be a value inputed by the user). What's the most elegant way to reduce the number of variables (not show a lot of args in the init) without lose readability? Use the *args function (like def__init__(self, *args))?
class World(object):
def __init__(self, grid_size=(GRID_WIDTH, GRID_HEIGHT),
cell_size=(CELL_WIDTH, CELL_HEIGHT),
obstacles_position= OBSTACLES,
recharge_position= RECHARGE_ZONE,
treadmill_position= TREADMILL_ZONE,
workers_positions= WORKERS_POS,
delivery_positions= DELIVERY_ZONE):
# some code bellow
def main():
# init some libraries
world = worldGrid()
# Do a while loop with the input variables from the world class
if __name__ = '__main__':
main()
Obs: I'm using Python 3+
In my opinion, you should probably stick with all of the function parameters in the function header (as you currently have it). This makes your code more readable, allows Python to tell you which arguments you may have omitted, plays nicely with Python's built-in help() method, allows third-party IDE code hinting, etc., etc...
If you really want to shorten the function header, you could use *args and **kwargs which will take any variadic arguments, e.g.:
def func(self, *args, **kwargs):
print("args:", args)
print("kwargs:", kwargs)
Usage would look like this:
>>> func(1, 2, 3, one="one", two="two")
args: (2, 3)
kwargs: {'one': 'one', 'two': 'two'}
Therefore, you could theoretically refactor your class to look something like below. This code doesn't handle default values or any error checking at all -- it just sets any keyword-arguments as attributes on the class itself:
class World(object):
def __init__(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
And usage:
>>> w = World(one=1, two=2, three=3)
>>> w.one
1
>>> w.two
2
>>> w.three
3
In python we can do this:
def myFun1(one = '1', two = '2'):
...
Then we can call the function and pass the arguments by their name:
myFun1(two = 'two', one = 'one')
Also, we can do this:
def myFun2(**kwargs):
print kwargs.get('one', 'nothing here')
myFun2(one='one')
So I was wondering if it is possible to combine both methods like:
def myFun3(name, lname, **other_info):
...
myFun3(lname='Someone', name='myName', city='cityName', otherInfo='blah')
In general what combinations can we do?
Thanks and sorry for my silly question.
The general idea is:
def func(arg1, arg2, ..., kwarg1=default, kwarg2=default, ..., *args, **kwargs):
...
You can use as many of those as you want. The * and ** will 'soak up' any remaining values not otherwise accounted for.
Positional arguments (provided without defaults) can't be given by keyword, and non-default arguments can't follow default arguments.
Note Python 3 also adds the ability to specify keyword-only arguments by having them after *:
def func(arg1, arg2, *args, kwonlyarg=default):
...
You can also use * alone (def func(a1, a2, *, kw=d):) which means that no arguments are captured, but anything after is keyword-only.
So, if you are in 3.x, you could produce the behaviour you want with:
def myFun3(*, name, lname, **other_info):
...
Which would allow calling with name and lname as keyword-only.
Note this is an unusual interface, which may be annoying to the user - I would only use it in very specific use cases.
In 2.x, you would need to manually make this by parsing **kwargs.
You can add your named arguments along with kwargs. If the keys are available in the calling function It will taken to your named argument otherwise it will be taken by the kwargs dictionary.
def add(a=1, b=2,**c):
res = a+b
for items in c:
res = res + c[items]
print(res)
add(2,3)
5
add(b=4, a =3)
7
add(a =1,b=2,c=3,d=4)
10
It's possible at least for Python 2.7. Keyword arguments get assigned to positional parameters by name, so you can do
In [34]: def func(name, lname, **kwargs):
print 'name='+name, 'lname='+lname
print kwargs
....:
In [35]: func(lname='lname_val', name='name_val', city='cityName', otherInfo='blah')
name=name_val lname=lname_val
{'city': 'cityName', 'otherInfo': 'blah'}
Official docs state it that way:
"If keyword arguments are present, they are first converted to positional arguments, as follows. First, a list of unfilled slots is created for the formal parameters. If there are N positional arguments, they are placed in the first N slots. Next, for each keyword argument, the identifier is used to determine the corresponding slot (if the identifier is the same as the first formal parameter name, the first slot is used, and so on). If the slot is already filled, a TypeError exception is raised. Otherwise, the value of the argument is placed in the slot, filling it (even if the expression is None, it fills the slot)."
https://docs.python.org/2/reference/expressions.html#calls
If I have a class like this:
class foo(object):
def __init__(self, a, b=None, c=None, d=None):
print a, b, c, d
and a derived class like this:
class bar(foo):
def __init__(self, *args, **kwargs):
if "c" in kwargs:
kwargs['c'] = 'else' # CHANGE C IFF IT IS PRESENT
super(bar, self).__init__(*args, **kwargs)
when someone calls this constructor, they could do it like this:
bar('a','b','c','d')
or they could call it like this:
bar('a', c='something')
In the second case, my constructor works as planned, but in the case of the first call c sneaks thru in the args array. This looks like I would have to watch the length of the args array as well as kwargs, and that seems brittle to the point of unusable. Is there anything you can do to make this situation better, other than just enumerate the arguments from foo in bar? (A somewhat brittle practice itself, but easier to recognize).
How about populating kwargs with args?
class bar(foo):
def __init__(self, *args, **kwargs):
for name, value in zip(['a', 'b', 'c', 'd'], args): # <---
kwargs[name] = value # <---
args = () # <---
if "c" in kwargs:
kwargs['c'] = 'else'
super(bar, self).__init__(*args, **kwargs)
UPDATE
Alternative that use inspect.getcallargs:
import inspect
class bar(foo):
def __init__(self, *args, **kwargs):
kwargs = inspect.getcallargs(foo.__init__, self, *args, **kwargs)
kwargs.pop('self')
if 'c' in kwargs:
kwargs['c'] = 'else'
super(bar, self).__init__(**kwargs)
This is more brittle than you think. Have you considered the situation where someone passes a keyword argument not present in foo.__init__'s argument list (e.g. bar('a', f='something'))? My recommendation would be requiring bar to take keyword arguments only and then filter the keys not present in foo.__init__'s argument list (which you can determine via introspection using inspect.getargspec [or related functions for newer versions of Python, particularly the Signature and Parameter objects starting in 3.3] if the arguments may change). That in itself has its own form of brittleness, of course, as a programmer using bar would need to know the relevant argument names for foo's constructor, but depending on what bar is being used for they may need to know what arguments foo takes anyway, and when someone knows that they usually know what the names of the arguments are as well.
Now that I'm looking at inspect again, I realize that there's another method you could use: inspect.getcallargs, which might be more useful for you. With this, you could do, say, inspect.getcallargs(super(bar, self).__init__, *[self, 1], **{'c':3, 'd':4}) and obtain the following dict: {'a': 1, 'self': <bar object>, 'b': None, 'c': 3, 'd': 4}. Then you can modify that dictionary and supply it as super(bar, self).__init__(**fixed_dict) or something of that sort. You'd still have the issue with keyword arguments not present in foo.__init__'s argument list, though (getcallargs raises the same errors foo.__init__ will when passed invalid arguments).
In python we can do this:
def myFun1(one = '1', two = '2'):
...
Then we can call the function and pass the arguments by their name:
myFun1(two = 'two', one = 'one')
Also, we can do this:
def myFun2(**kwargs):
print kwargs.get('one', 'nothing here')
myFun2(one='one')
So I was wondering if it is possible to combine both methods like:
def myFun3(name, lname, **other_info):
...
myFun3(lname='Someone', name='myName', city='cityName', otherInfo='blah')
In general what combinations can we do?
Thanks and sorry for my silly question.
The general idea is:
def func(arg1, arg2, ..., kwarg1=default, kwarg2=default, ..., *args, **kwargs):
...
You can use as many of those as you want. The * and ** will 'soak up' any remaining values not otherwise accounted for.
Positional arguments (provided without defaults) can't be given by keyword, and non-default arguments can't follow default arguments.
Note Python 3 also adds the ability to specify keyword-only arguments by having them after *:
def func(arg1, arg2, *args, kwonlyarg=default):
...
You can also use * alone (def func(a1, a2, *, kw=d):) which means that no arguments are captured, but anything after is keyword-only.
So, if you are in 3.x, you could produce the behaviour you want with:
def myFun3(*, name, lname, **other_info):
...
Which would allow calling with name and lname as keyword-only.
Note this is an unusual interface, which may be annoying to the user - I would only use it in very specific use cases.
In 2.x, you would need to manually make this by parsing **kwargs.
You can add your named arguments along with kwargs. If the keys are available in the calling function It will taken to your named argument otherwise it will be taken by the kwargs dictionary.
def add(a=1, b=2,**c):
res = a+b
for items in c:
res = res + c[items]
print(res)
add(2,3)
5
add(b=4, a =3)
7
add(a =1,b=2,c=3,d=4)
10
It's possible at least for Python 2.7. Keyword arguments get assigned to positional parameters by name, so you can do
In [34]: def func(name, lname, **kwargs):
print 'name='+name, 'lname='+lname
print kwargs
....:
In [35]: func(lname='lname_val', name='name_val', city='cityName', otherInfo='blah')
name=name_val lname=lname_val
{'city': 'cityName', 'otherInfo': 'blah'}
Official docs state it that way:
"If keyword arguments are present, they are first converted to positional arguments, as follows. First, a list of unfilled slots is created for the formal parameters. If there are N positional arguments, they are placed in the first N slots. Next, for each keyword argument, the identifier is used to determine the corresponding slot (if the identifier is the same as the first formal parameter name, the first slot is used, and so on). If the slot is already filled, a TypeError exception is raised. Otherwise, the value of the argument is placed in the slot, filling it (even if the expression is None, it fills the slot)."
https://docs.python.org/2/reference/expressions.html#calls