This question already has an answer here:
Python Assignment Operator Precedence - (a, b) = a[b] = {}, 5
(1 answer)
Closed 4 years ago.
I found the assignment a = a[1:] = [2] in an article. I tried it in python3 and python2; it all works, but I don't understand how it works. = here is not like in C; C processes = by right to left. How does python process the = operator?
Per the language docs on assignment:
An assignment statement evaluates the expression list (remember that this can be a single expression or a comma-separated list, the latter yielding a tuple) and assigns the single resulting object to each of the target lists, from left to right.
In this case, a = a[1:] = [2] has an expression list [2], and two "target lists", a and a[1:], where a is the left-most "target list".
You can see how this behaves by looking at the disassembly:
>>> import dis
>>> dis.dis('a = a[1:] = [2]')
1 0 LOAD_CONST 0 (2)
2 BUILD_LIST 1
4 DUP_TOP
6 STORE_NAME 0 (a)
8 LOAD_NAME 0 (a)
10 LOAD_CONST 1 (1)
12 LOAD_CONST 2 (None)
14 BUILD_SLICE 2
16 STORE_SUBSCR
18 LOAD_CONST 2 (None)
20 RETURN_VALUE
(The last two lines of the disassembly can be ignored, dis is making a function wrapper to disassemble the string)
The important part to note is that when you do x = y = some_val, some_val is loaded on the stack (in this case by the LOAD_CONST and BUILD_LIST), then the stack entry is duplicated and assigned, from left to right, to the targets given.
So when you do:
a = a[1:] = [2]
it makes two references to a brand new list containing 2, and the first action is a STORE one of these references to a. Next, it stores the second reference to a[1:], but since the slice assignment mutates a itself, it has to load a again, which gets the list just stored. Luckily, list is resilient against self-slice-assignment, or we'd have issues (it would be forever reading the value it just added to add to the end until we ran out of memory and crashed); as is, it behaves as a copy of [2] was assigned to replace any and all elements from index one onwards.
The end result is equivalent to if you'd done:
_ = [2]
a = _
a[1:] = _
but it avoids the use of the _ name.
To be clear, the disassembly annotated:
Make list [2]:
1 0 LOAD_CONST 0 (2)
2 BUILD_LIST 1
Make a copy of the reference to [2]:
4 DUP_TOP
Perform store to a:
6 STORE_NAME 0 (a)
Perform store to a[1:]:
8 LOAD_NAME 0 (a)
10 LOAD_CONST 1 (1)
12 LOAD_CONST 2 (None)
14 BUILD_SLICE 2
16 STORE_SUBSCR
The way I understand such assignments is that this is equivalent to
temp = [2]
a = temp
a[1:] = temp
The resulting value of [2, 2] is consistent with this interpretation.
Related
This question already has answers here:
"is" operator behaves unexpectedly with integers
(11 answers)
Closed 6 months ago.
I have this code
l = 40000
m = 40000
print(type(l),type(m))
if l is m:
print("same")
else:
print("nope")
Taking reference from here I was hoping id to be different since the value are not falling in range of (-2,256). Please let me know if I am missing out on something
Myself using Python-3.8.3(32bit) on windows Platform
This is due to a simple optimization at the bytecode compilation stage; if the same constant appears in the same code more than once, it is created only once, and each use of the constant will refer to the same instance.
We can investigate the bytecode with the dis module:
>>> import dis
>>> dis.dis('l = 40000\nm=40000')
1 0 LOAD_CONST 0 (40000)
2 STORE_NAME 0 (l)
2 4 LOAD_CONST 0 (40000)
6 STORE_NAME 1 (m)
8 LOAD_CONST 1 (None)
10 RETURN_VALUE
>>> dis.dis('l = 40000\nm=40001')
1 0 LOAD_CONST 0 (40000)
2 STORE_NAME 0 (l)
2 4 LOAD_CONST 1 (40001)
6 STORE_NAME 1 (m)
8 LOAD_CONST 2 (None)
10 RETURN_VALUE
Note that in the first case where both constants are 40000, the two LOAD_CONST operations both load constant #0, but in the second case, 40000 is constant #0 and 40001 is constant #1.
Also, you will usually get different results if you do this on separate lines in the REPL, since each line in the REPL is compiled and executed as a separate code object, so they cannot share constants:
>>> l = 50000
>>> m = 50000
>>> id(l)
140545755966480
>>> id(m)
140545755966448
But if you do both in one line in the REPL, the same instance of the constant is used again, because it's just one line compiled to one code object, so the constant can be shared:
>>> p = 60000; q = 60000
>>> id(p)
140545755966576
>>> id(q)
140545755966576
I tried in the snippet below:
a, b = a[b] = {}, 5
print('a={0},b={1}'.format(a,b))
The IDE spits out the follows:
a={5: ({...}, 5)},b=5
I have tried S3DEV's advice and execute:
from dis import dis
dis('a, b = a[b] = {}, 5')
And it gives me the follows:
1 0 BUILD_MAP 0
2 LOAD_CONST 0 (5)
4 BUILD_TUPLE 2
6 DUP_TOP
8 UNPACK_SEQUENCE 2
10 STORE_NAME 0 (a)
12 STORE_NAME 1 (b)
14 LOAD_NAME 0 (a)
16 LOAD_NAME 1 (b)
18 STORE_SUBSCR
20 LOAD_CONST 1 (None)
22 RETURN_VALUE
But I still cannot understand why a[b] = a, 5 happened in the step 18 STORE_SUBSCR. Any further explanation?
This is an assignment statement with multiple target_list:s, for which case the docs say that the statement "assigns the single resulting object to each of the target lists, from left to right." Within each target_list, assignments also proceed left to right.
Thus, the statement is equivalent to
a = {}
b = 5
a[b] = a, 5
The reason that the last assignment is a[b]=a,5 and not a,b={},5 is that the value ({}, 5) is only evaluated once, so it's the same dict that gets used throughout. First, a is set to refer to that dict, then the dict — through a — is modified to refer to itself.
EDIT: Perhaps it is clearer to say that the statement is equivalent to
temp1 = {}
temp2 = 5
a = temp1
b = temp2
a[b] = temp1, temp2
Right before the last step, a and temp1 refer to the same object, which thus becomes self-referring after the last step.
This is not code I want to see in production. :)
Suppose I have two lists, A = [1,2,3,4] and B = [4,5,6]
I would like a list which includes the elements from both A and B. (I don't care if A itself gets altered).
A couple things I could do, and my understanding of them (please tell me if I am wrong):
A.extend(B) (elements of B get added in to A; A itself is altered)
C = A + B (makes a brand new object C, which contains the contents of A and B in it.)
I wanted to understand which is more efficient, so I was wondering if someone can someone please tell me if my assumptions below are incorrect.
In the case of A.extend(B), I'm assuming python only has to do 3 list add operations (the 3 elements of B, each of which it appends to A). However, in doing A + B, doesn't python have to iterate through both lists A and B, in that case doing 7 list add operations? (i.e., it has to make a new list, go through A and put all the elements in it, and then go through B and put all the elements in it).
Am I misunderstanding how the interpreter handles these things, or what these operations do in python?
Below is the bytecode analysis of both operations. There are no major performance difference between two. The only difference is that the .extend way involves a CALL_FUNCTION, which is slightly more expensive in Python than the BINARY_ADD.
But this should not be a problem unless of are working on huge data operations.
>>> import dis
>>> a = [1,2,3,4]
>>> b = [4,5,6]
>>> def f1(a,b):
... a.extend(b)
>>> def f2(a,b):
... c = a+ b
>>> dis.dis(f1)
2 0 LOAD_FAST 0 (a)
3 LOAD_ATTR 0 (extend)
6 LOAD_FAST 1 (b)
9 CALL_FUNCTION 1
12 POP_TOP
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
>>> dis.dis(f2)
2 0 LOAD_FAST 0 (a)
3 LOAD_FAST 1 (b)
6 BINARY_ADD
7 STORE_FAST 2 (c)
10 LOAD_CONST 0 (None)
13 RETURN_VALUE
Disclaimer: I'm new to programming, but new to Python. This may be a pretty basic question.
I have the following block of code:
for x in range(0, 100):
y = 1 + 1;
Is the calculation of 1 + 1 in the second line executed 100 times?
I have two suspicions why it might not:
1) The compiler sees 1 + 1 as a constant value, and thus compiles this line into y = 2;.
2) The compiler sees that y is only set and never referenced, so it omits this line of code.
Are either/both of these correct, or does it actually get executed each iteration over the loop?
Option 1 is executed; the CPython compiler simplifies mathematical expressions with constants in the peephole optimiser.
Python will not eliminate the loop body however.
You can introspect what Python produces by looking at the bytecode; use the dis module to take a look:
>>> import dis
>>> def f():
... for x in range(100):
... y = 1 + 1
...
>>> dis.dis(f)
2 0 SETUP_LOOP 26 (to 29)
3 LOAD_GLOBAL 0 (range)
6 LOAD_CONST 1 (100)
9 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
12 GET_ITER
>> 13 FOR_ITER 12 (to 28)
16 STORE_FAST 0 (x)
3 19 LOAD_CONST 3 (2)
22 STORE_FAST 1 (y)
25 JUMP_ABSOLUTE 13
>> 28 POP_BLOCK
>> 29 LOAD_CONST 0 (None)
32 RETURN_VALUE
The bytecode at position 19, LOAD_CONST loads the value 2 to store in y.
You can see the constants associated with the code object in the co_consts attribute of a code object; for functions you can find that object under the __code__ attribute:
>>> f.__code__.co_consts
(None, 100, 1, 2)
None is the default return value for any function, 100 the literal passed to the range() call, 1 the original literal, left in place by the peephole optimiser and 2 is the result of the optimisation.
The work is done in peephole.c, in the fold_binops_on_constants() function:
/* Replace LOAD_CONST c1. LOAD_CONST c2 BINOP
with LOAD_CONST binop(c1,c2)
The consts table must still be in list form so that the
new constant can be appended.
Called with codestr pointing to the first LOAD_CONST.
Abandons the transformation if the folding fails (i.e. 1+'a').
If the new constant is a sequence, only folds when the size
is below a threshold value. That keeps pyc files from
becoming large in the presence of code like: (None,)*1000.
*/
Take into account that Python is a highly dynamic language, such optimisations can only be applied to literals and constants that you cannot later dynamically replace.
I tried mutating a list by swapping a common element between the list and another reference list with the first element. The implementation is as shown below:
>>> L = [1,2,3,4,5,6,7,8,9]
>>> A = [3]
>>> L[0], L[L.index(A[0])] = L[L.index(A[0])], L[0] #want to swap 3 with 1
>>> L
[1,2,3,4,5,6,7,8,9,] #List L was not mutated
The list was not mutated as I anticipated. But when I modify the implementation as shown below, it worked:
>>> L = [1,2,3,4,5,6,7,8,9]
>>> A = [3]
>>> i = L.index(A[0])
>>> L[0], L[i] = L[i], L[0]
>>> L
[3,2,1,4,5,6,7,8,9,] #Now list mutated as desired even though L[i] and L[L.index(A[0])] evaluate to same value.
My question is, why couldn't the first assignment mutate the list? I thought of it but my brain coudnt explain it.
Although in Python, the right-hand side is evaluated first when doing multiple assignments, the left-hand assigment targets, if they have expressions in them, are evaluated one by one when assigning.
If instead, they'd be evaluated as assignment targets first as you appear to expect, this would of course work.
This is documented in the assignment statements section:
An assignment statement evaluates the expression list (remember that this can be a single expression or a comma-separated list, the latter yielding a tuple) and assigns the single resulting object to each of the target lists, from left to right.
and
If the target list is a comma-separated list of targets: The object must be an iterable with the same number of items as there are targets in the target list, and the items are assigned, from left to right, to the corresponding targets.
Emphasis mine. The left-to-right is crucial here. L[0] is assigned to before the L[L.index(3)] is assigned to.
The documentation then describes in detail what happens to a subscription target such as L[0] and L[L.index(3)]:
If the target is a subscription: The primary expression in the reference is evaluated. It should yield either a mutable sequence object (such as a list) or a mapping object (such as a dictionary). Next, the subscript expression is evaluated.
Again, emphasis mine; the subscript expression is evaluated separately, and since the target list is evaluated from left-to-right, that evaluation takes place after the previous assignment to L[0].
You can see this by disassembling the python code:
>>> import dis
>>> def f(L):
... L[0], L[2] = L[2], L[0]
...
>>> def g(L):
... L[0], L[L.index(3)] = L[L.index(3)], L[0]
...
>>> dis.dis(f)
2 0 LOAD_FAST 0 (L) # L[2]
3 LOAD_CONST 1 (2)
6 BINARY_SUBSCR
7 LOAD_FAST 0 (L) # L[0]
10 LOAD_CONST 2 (0)
13 BINARY_SUBSCR
14 ROT_TWO
15 LOAD_FAST 0 (L) # Store in L[0]
18 LOAD_CONST 2 (0)
21 STORE_SUBSCR
22 LOAD_FAST 0 (L) # Store in L[2]
25 LOAD_CONST 1 (2)
28 STORE_SUBSCR
29 LOAD_CONST 0 (None)
32 RETURN_VALUE
>>> dis.dis(g)
2 0 LOAD_FAST 0 (L) # L[L.index(3)]
3 LOAD_FAST 0 (L)
6 LOAD_ATTR 0 (index)
9 LOAD_CONST 1 (3)
12 CALL_FUNCTION 1
15 BINARY_SUBSCR
16 LOAD_FAST 0 (L) # L[0]
19 LOAD_CONST 2 (0)
22 BINARY_SUBSCR
23 ROT_TWO
24 LOAD_FAST 0 (L) # Store in L[0]
27 LOAD_CONST 2 (0)
30 STORE_SUBSCR
31 LOAD_FAST 0 (L) # Store in L[L.index(3)]
34 LOAD_FAST 0 (L)
37 LOAD_ATTR 0 (index)
40 LOAD_CONST 1 (3)
43 CALL_FUNCTION 1
46 STORE_SUBSCR
47 LOAD_CONST 0 (None)
50 RETURN_VALUE
The storing operation first stores L[0] = 3, so the next call to L.index(3) returns 0 and the 1 is thus stored right back in position 0!
The following does work:
L[L.index(3)], L[0] = L[0], L[L.index(3)]
because now the L.index(3) lookup is done first. However, it's best to store the result of an .index() call in a temporary variable as not calling .index() twice is going to be more efficient in any case.
The problem is that the two are not equivalent. The first example is akin to doing:
>>> L = [1,2,3,4,5,6,7,8,9]
>>> A = [3]
>>> i = L.index(A[0])
>>> L[0] = L[i]
>>> i = L.index(A[0])
>>> L[i] = L[0]
This means that you end up swapping, then finding the element you just swapped and swapping back.
The reason you are confused is that you are thinking of the tuple assignment as Python doing both things at the same time - this isn't how the execution is performed, it is performed in an order, which changes the outcome.
It's worth noting that even if it did work, it would be a sub-optimal way of doing this. list.index() isn't a particularly fast operation, so doing it twice for no reason isn't a great idea.