In [55]: a = 5
In [56]: b = 6
In [57]: (a, b) = (b, a)
In [58]: a
Out[58]: 6
In [59]: b
Out[59]: 5
How does this swapping of values of a and b work internally? Its definitely not using a temp variable.
Python separates the right-hand side expression from the left-hand side assignment. First the right-hand side is evaluated, and the result is stored on the stack, and then the left-hand side names are assigned using opcodes that take values from the stack again.
For tuple assignments with 2 or 3 items, Python just uses the stack directly:
>>> import dis
>>> def foo(a, b):
... a, b = b, a
...
>>> dis.dis(foo)
2 0 LOAD_FAST 1 (b)
3 LOAD_FAST 0 (a)
6 ROT_TWO
7 STORE_FAST 0 (a)
10 STORE_FAST 1 (b)
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
After the two LOAD_FAST opcodes (which push a value from a variable onto the stack), the top of stack holds [a, b]. The ROT_TWO opcode swaps the top two positions on the stack so the stack now has [b, a] at the top. The two STORE_FAST opcodes then takes those two values and store them in the names on the left-hand side of the assignment. The first STORE_FAST pops a value of the top of the stack and puts it into a, the next pops again, storing the value in b. The rotation is needed because Python guarantees that assignments in a target list on the left-hand side are done from left to right.
For a 3-name assignment, ROT_THREE followed by ROT_TWO is executed to reverse the top three items on the stack.
For longer left-hand-side assignments, an explicit tuple is built:
>>> def bar(a, b, c, d):
... d, c, b, a = a, b, c, d
...
>>> dis.dis(bar)
2 0 LOAD_FAST 0 (a)
3 LOAD_FAST 1 (b)
6 LOAD_FAST 2 (c)
9 LOAD_FAST 3 (d)
12 BUILD_TUPLE 4
15 UNPACK_SEQUENCE 4
18 STORE_FAST 3 (d)
21 STORE_FAST 2 (c)
24 STORE_FAST 1 (b)
27 STORE_FAST 0 (a)
30 LOAD_CONST 0 (None)
33 RETURN_VALUE
Here the stack with [d, c, b, a] is used to build a tuple (in reverse order, BUILD_TUPLE pops from the stack again, pushing the resulting tuple onto the stack), and then UNPACK_SEQUENCE pops the tuple from the stack again, pushes all elements back from the tuple back onto the stack again for the STORE_FAST operations.
The latter may seem like a wasteful operation, but the right-hand side of an assignment may be something entirely different, a function call that produces a tuple perhaps, so the Python interpreter makes no assumptions and uses the UNPACK_SEQUENCE opcode always. It does so even for the two and three-name assignment operations, but a later (peephole) optimization step replaces a BUILD_TUPLE / UNPACK_SEQUENCE combination with 2 or 3 arguments with the above ROT_TWO and ROT_THREE opcodes for efficiency.
Related
I tried in the snippet below:
a, b = a[b] = {}, 5
print('a={0},b={1}'.format(a,b))
The IDE spits out the follows:
a={5: ({...}, 5)},b=5
I have tried S3DEV's advice and execute:
from dis import dis
dis('a, b = a[b] = {}, 5')
And it gives me the follows:
1 0 BUILD_MAP 0
2 LOAD_CONST 0 (5)
4 BUILD_TUPLE 2
6 DUP_TOP
8 UNPACK_SEQUENCE 2
10 STORE_NAME 0 (a)
12 STORE_NAME 1 (b)
14 LOAD_NAME 0 (a)
16 LOAD_NAME 1 (b)
18 STORE_SUBSCR
20 LOAD_CONST 1 (None)
22 RETURN_VALUE
But I still cannot understand why a[b] = a, 5 happened in the step 18 STORE_SUBSCR. Any further explanation?
This is an assignment statement with multiple target_list:s, for which case the docs say that the statement "assigns the single resulting object to each of the target lists, from left to right." Within each target_list, assignments also proceed left to right.
Thus, the statement is equivalent to
a = {}
b = 5
a[b] = a, 5
The reason that the last assignment is a[b]=a,5 and not a,b={},5 is that the value ({}, 5) is only evaluated once, so it's the same dict that gets used throughout. First, a is set to refer to that dict, then the dict — through a — is modified to refer to itself.
EDIT: Perhaps it is clearer to say that the statement is equivalent to
temp1 = {}
temp2 = 5
a = temp1
b = temp2
a[b] = temp1, temp2
Right before the last step, a and temp1 refer to the same object, which thus becomes self-referring after the last step.
This is not code I want to see in production. :)
In [55]: a = 5
In [56]: b = 6
In [57]: (a, b) = (b, a)
In [58]: a
Out[58]: 6
In [59]: b
Out[59]: 5
How does this swapping of values of a and b work internally? Its definitely not using a temp variable.
Python separates the right-hand side expression from the left-hand side assignment. First the right-hand side is evaluated, and the result is stored on the stack, and then the left-hand side names are assigned using opcodes that take values from the stack again.
For tuple assignments with 2 or 3 items, Python just uses the stack directly:
>>> import dis
>>> def foo(a, b):
... a, b = b, a
...
>>> dis.dis(foo)
2 0 LOAD_FAST 1 (b)
3 LOAD_FAST 0 (a)
6 ROT_TWO
7 STORE_FAST 0 (a)
10 STORE_FAST 1 (b)
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
After the two LOAD_FAST opcodes (which push a value from a variable onto the stack), the top of stack holds [a, b]. The ROT_TWO opcode swaps the top two positions on the stack so the stack now has [b, a] at the top. The two STORE_FAST opcodes then takes those two values and store them in the names on the left-hand side of the assignment. The first STORE_FAST pops a value of the top of the stack and puts it into a, the next pops again, storing the value in b. The rotation is needed because Python guarantees that assignments in a target list on the left-hand side are done from left to right.
For a 3-name assignment, ROT_THREE followed by ROT_TWO is executed to reverse the top three items on the stack.
For longer left-hand-side assignments, an explicit tuple is built:
>>> def bar(a, b, c, d):
... d, c, b, a = a, b, c, d
...
>>> dis.dis(bar)
2 0 LOAD_FAST 0 (a)
3 LOAD_FAST 1 (b)
6 LOAD_FAST 2 (c)
9 LOAD_FAST 3 (d)
12 BUILD_TUPLE 4
15 UNPACK_SEQUENCE 4
18 STORE_FAST 3 (d)
21 STORE_FAST 2 (c)
24 STORE_FAST 1 (b)
27 STORE_FAST 0 (a)
30 LOAD_CONST 0 (None)
33 RETURN_VALUE
Here the stack with [d, c, b, a] is used to build a tuple (in reverse order, BUILD_TUPLE pops from the stack again, pushing the resulting tuple onto the stack), and then UNPACK_SEQUENCE pops the tuple from the stack again, pushes all elements back from the tuple back onto the stack again for the STORE_FAST operations.
The latter may seem like a wasteful operation, but the right-hand side of an assignment may be something entirely different, a function call that produces a tuple perhaps, so the Python interpreter makes no assumptions and uses the UNPACK_SEQUENCE opcode always. It does so even for the two and three-name assignment operations, but a later (peephole) optimization step replaces a BUILD_TUPLE / UNPACK_SEQUENCE combination with 2 or 3 arguments with the above ROT_TWO and ROT_THREE opcodes for efficiency.
Suppose I have two lists, A = [1,2,3,4] and B = [4,5,6]
I would like a list which includes the elements from both A and B. (I don't care if A itself gets altered).
A couple things I could do, and my understanding of them (please tell me if I am wrong):
A.extend(B) (elements of B get added in to A; A itself is altered)
C = A + B (makes a brand new object C, which contains the contents of A and B in it.)
I wanted to understand which is more efficient, so I was wondering if someone can someone please tell me if my assumptions below are incorrect.
In the case of A.extend(B), I'm assuming python only has to do 3 list add operations (the 3 elements of B, each of which it appends to A). However, in doing A + B, doesn't python have to iterate through both lists A and B, in that case doing 7 list add operations? (i.e., it has to make a new list, go through A and put all the elements in it, and then go through B and put all the elements in it).
Am I misunderstanding how the interpreter handles these things, or what these operations do in python?
Below is the bytecode analysis of both operations. There are no major performance difference between two. The only difference is that the .extend way involves a CALL_FUNCTION, which is slightly more expensive in Python than the BINARY_ADD.
But this should not be a problem unless of are working on huge data operations.
>>> import dis
>>> a = [1,2,3,4]
>>> b = [4,5,6]
>>> def f1(a,b):
... a.extend(b)
>>> def f2(a,b):
... c = a+ b
>>> dis.dis(f1)
2 0 LOAD_FAST 0 (a)
3 LOAD_ATTR 0 (extend)
6 LOAD_FAST 1 (b)
9 CALL_FUNCTION 1
12 POP_TOP
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
>>> dis.dis(f2)
2 0 LOAD_FAST 0 (a)
3 LOAD_FAST 1 (b)
6 BINARY_ADD
7 STORE_FAST 2 (c)
10 LOAD_CONST 0 (None)
13 RETURN_VALUE
Disclaimer: I'm new to programming, but new to Python. This may be a pretty basic question.
I have the following block of code:
for x in range(0, 100):
y = 1 + 1;
Is the calculation of 1 + 1 in the second line executed 100 times?
I have two suspicions why it might not:
1) The compiler sees 1 + 1 as a constant value, and thus compiles this line into y = 2;.
2) The compiler sees that y is only set and never referenced, so it omits this line of code.
Are either/both of these correct, or does it actually get executed each iteration over the loop?
Option 1 is executed; the CPython compiler simplifies mathematical expressions with constants in the peephole optimiser.
Python will not eliminate the loop body however.
You can introspect what Python produces by looking at the bytecode; use the dis module to take a look:
>>> import dis
>>> def f():
... for x in range(100):
... y = 1 + 1
...
>>> dis.dis(f)
2 0 SETUP_LOOP 26 (to 29)
3 LOAD_GLOBAL 0 (range)
6 LOAD_CONST 1 (100)
9 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
12 GET_ITER
>> 13 FOR_ITER 12 (to 28)
16 STORE_FAST 0 (x)
3 19 LOAD_CONST 3 (2)
22 STORE_FAST 1 (y)
25 JUMP_ABSOLUTE 13
>> 28 POP_BLOCK
>> 29 LOAD_CONST 0 (None)
32 RETURN_VALUE
The bytecode at position 19, LOAD_CONST loads the value 2 to store in y.
You can see the constants associated with the code object in the co_consts attribute of a code object; for functions you can find that object under the __code__ attribute:
>>> f.__code__.co_consts
(None, 100, 1, 2)
None is the default return value for any function, 100 the literal passed to the range() call, 1 the original literal, left in place by the peephole optimiser and 2 is the result of the optimisation.
The work is done in peephole.c, in the fold_binops_on_constants() function:
/* Replace LOAD_CONST c1. LOAD_CONST c2 BINOP
with LOAD_CONST binop(c1,c2)
The consts table must still be in list form so that the
new constant can be appended.
Called with codestr pointing to the first LOAD_CONST.
Abandons the transformation if the folding fails (i.e. 1+'a').
If the new constant is a sequence, only folds when the size
is below a threshold value. That keeps pyc files from
becoming large in the presence of code like: (None,)*1000.
*/
Take into account that Python is a highly dynamic language, such optimisations can only be applied to literals and constants that you cannot later dynamically replace.
In [55]: a = 5
In [56]: b = 6
In [57]: (a, b) = (b, a)
In [58]: a
Out[58]: 6
In [59]: b
Out[59]: 5
How does this swapping of values of a and b work internally? Its definitely not using a temp variable.
Python separates the right-hand side expression from the left-hand side assignment. First the right-hand side is evaluated, and the result is stored on the stack, and then the left-hand side names are assigned using opcodes that take values from the stack again.
For tuple assignments with 2 or 3 items, Python just uses the stack directly:
>>> import dis
>>> def foo(a, b):
... a, b = b, a
...
>>> dis.dis(foo)
2 0 LOAD_FAST 1 (b)
3 LOAD_FAST 0 (a)
6 ROT_TWO
7 STORE_FAST 0 (a)
10 STORE_FAST 1 (b)
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
After the two LOAD_FAST opcodes (which push a value from a variable onto the stack), the top of stack holds [a, b]. The ROT_TWO opcode swaps the top two positions on the stack so the stack now has [b, a] at the top. The two STORE_FAST opcodes then takes those two values and store them in the names on the left-hand side of the assignment. The first STORE_FAST pops a value of the top of the stack and puts it into a, the next pops again, storing the value in b. The rotation is needed because Python guarantees that assignments in a target list on the left-hand side are done from left to right.
For a 3-name assignment, ROT_THREE followed by ROT_TWO is executed to reverse the top three items on the stack.
For longer left-hand-side assignments, an explicit tuple is built:
>>> def bar(a, b, c, d):
... d, c, b, a = a, b, c, d
...
>>> dis.dis(bar)
2 0 LOAD_FAST 0 (a)
3 LOAD_FAST 1 (b)
6 LOAD_FAST 2 (c)
9 LOAD_FAST 3 (d)
12 BUILD_TUPLE 4
15 UNPACK_SEQUENCE 4
18 STORE_FAST 3 (d)
21 STORE_FAST 2 (c)
24 STORE_FAST 1 (b)
27 STORE_FAST 0 (a)
30 LOAD_CONST 0 (None)
33 RETURN_VALUE
Here the stack with [d, c, b, a] is used to build a tuple (in reverse order, BUILD_TUPLE pops from the stack again, pushing the resulting tuple onto the stack), and then UNPACK_SEQUENCE pops the tuple from the stack again, pushes all elements back from the tuple back onto the stack again for the STORE_FAST operations.
The latter may seem like a wasteful operation, but the right-hand side of an assignment may be something entirely different, a function call that produces a tuple perhaps, so the Python interpreter makes no assumptions and uses the UNPACK_SEQUENCE opcode always. It does so even for the two and three-name assignment operations, but a later (peephole) optimization step replaces a BUILD_TUPLE / UNPACK_SEQUENCE combination with 2 or 3 arguments with the above ROT_TWO and ROT_THREE opcodes for efficiency.