This question already has answers here:
How do I clone a list so that it doesn't change unexpectedly after assignment?
(24 answers)
Closed 15 days ago.
I come from java world where I expect following things
int a = valueassignedbyfunction();
int b = a;
a = a + 1;
after this a is 1 greater than b. But in python the b automatically gets incremented by one once the a = a + 1 operation is done because this b is referencing to the same object as a does. How can I copy only the value of a and assign it to a new object called b?
Thanks!
Assuming integers, I cannot reproduce your issue:
>>> a = 1
>>> b = a
>>> a += 1
>>> a
2
>>> b
1
If we assume objects instead:
class Test(object):
... def __init__(self, v):
... self.v = v
...
>>> a = Test(1)
>>> b = a.v
>>> a.v += 1
>>> print a.v, b
2 1
# No issues so far
# Let's copy the object instead
>>> b = a
>>> a.v += 1
>>> print a.v, b.v
3 3
# Ah, there we go
# Using user252462's suggestion
>>> from copy import deepcopy
>>> b = deepcopy(a)
>>> a.v += 1
>>> print a.v, b.v
4 3
I think the main confusion here is the following: In Java, a line like
int i = 5;
allocates memory for an integer and associates the name i with this memory location. You can somehow identify the name i with this memory location and its type and call the whole thing "the integer variable i".
In Python, the line
i = 5
evaluates the expression on the right hand side, which will yield a Python object (in this case, the expression is really simple and will yield the integer object 5). The assignment statement makes the name i point to that object, but the relation between the name and the object is a completely different one than in Java. Names are always just references to objects, and there may be many names referencing the same object or no name at all.
This documentation might help out: http://docs.python.org/library/copy.html
You can use the copy library to deepcopy objects:
import copy
b = copy.deepcopy(a)
I'm not sure what you're seeing here.
>>> a = 1
>>> b = a
>>> a = a + 1
>>> b
1
>>> a
2
>>> a is b
False
Python Integers are immutable, the + operation assigns creates a new object with value a+1. There are some weird reference issues with integers (http://distilledb.com/blog/archives/date/2009/06/18/python-gotcha-integer-equality.page), but you should get the same thing you expected in Java
How about just doing
a = 1
b = a*1
Related
If I do the following:
v = [0,0,0,0]
v2 = v
v[0]=5
print(v2)
the change to list v changes list v2.
a=5
b=a
a=6
print(b)
If I do this on the other hand, changing a doesnt change b. Whats the difference here? When I print id(a) and id(b) they give the same number, so shouldn't they be referencing the same object and change just like the list does?
Your original question is really asking about two different things.
The first is this, with some annotation - you ask about lists. Lists are objects that are mutable sequences, and in your code, v and v2 always refer to the same list. You can modify the contents of the list using any referrer to it and those changes are visible to anything that refers to it.
v = [0,0,0,0] # create a list object and assign reference to v
v2 = v # assign v2 to be the reference to list v also refers to
v[0]=5 # modify first list element
print(v2) # print the list v2 refers to, which is the same list v refers to
In the second piece of code you show, you're changing what a variable refers to, rather than changing the underlying value of an object.
a=5 # Assign a to be a reference to 5
b=a # Assign b to be a reference to the thing a refers to
a=6 # Re-assign a to refer to 6, a now refers to a different object than b
print(b) # b still refers to 5
And you pointed out the use of id. I will also introduce sys.getrefcount() which lets you see what the reference count for any particular object is, so we can see that, say, v's referred-to list has multiple things referring to it.
import sys
v = [0,0,0,0]
v2 = v
v[0]=5
print(f"id(v) = {id(v)}")
print(f"id(v2) = {id(v2)}")
# this shows 3 instead of 2 because getrefcount(x)
# increases refcount of x by 1
print(f"v referrers: {sys.getrefcount(v)}")
del v2 # remove v2 as a reference
# and this will show 2 because v2 doesn't
# exist/doesn't refer to it anymore
print(f"v referrers: {sys.getrefcount(v)}")
a = 5
b = 5
print(f"id(a) = {id(a)}")
print(f"id(b) = {id(b)}")
# you're reassigning a here, so the id will change
# but you didn't change what b refers to
a = 6
print(f"id(a) = {id(a)}")
print(f"id(b) = {id(b)}")
And the output of this would look something like this...
id(v) = 4366332480
id(v2) = 4366332480
v referrers: 3
v referrers: 2
id(a) = 4365582704
id(b) = 4365582704
id(a) = 4365582736
id(b) = 4365582704
And as I mentioned - CPython does some special stuff for numbers in the range of [-5, 256] and keeps a static list of them in memory, so id on any integer in that range should return the same value for any referrers to them.
In an unpacking assignment statement, can the assigned object inspect the number of variables it is being assigned to?
class MyObject:
def __iter__(self):
n = some_diabolical_hack()
print(f"yielding {n} vals")
return iter(["potato"]*n)
Something like:
>>> x, y = MyObject()
yielding 2 vals
>>> a, b, c = MyObject()
yielding 3 vals
In the more general case, can it introspect the "shape" of the target_list being used in an assignment?
>>> first, *blob, d[k], (x, y), L[3:7], obj.attr, last = MyObject()
unpacking to <_ast.Tuple object at 0xcafef00d>
Example potential use case: an improved MagicMock() which doesn't need to be pre-configured with a fixed iteration length when being used to patch out some object on the right hand side of an assignment statement.
You could use the traceback module:
import traceback
def diabolically_invoke_traceback():
call = traceback.extract_stack()[-2]
print call[3]
unpackers = call[3].split('=')[0].split(',')
print len (unpackers)
return range(len(unpackers))
In [63]: a, b, c = diabolically_invoke_traceback()
a, b, c = diabolically_invoke_traceback()
3
In [64]: a
Out[64]: 0
In [65]: b
Out[65]: 1
In [66]: c
Out[66]: 2
(Disclaimer: I don't recommend using diabolical techniques in production-quality code. Everything in this answer might not work on a different computer from mine, or a different Python version from mine, or on a non-CPython distribution, and it might not work tomorrow morning.)
Perhaps you could do this by inspecting the calling frame's bytecode. If I'm reading the bytecode guide correctly, multiple assignment is handled by the instructions UNPACK_SEQUENCE or UNPACK_EX, depending on whether the target list has a starred name. Both of these instructions provide information about the shape of the target list in their arguments.
You could write your diabolical function to climb the frame hierarchy until it finds the calling frame, and inspect the bytecode instruction that occurs after the FUNCTION_CALL that represents the right-hand-side of the assignment. (this is assuming that your call to MyObject() is the only thing on the right side of the statement). Then you can extract the target list size from the instruction's argument and return it.
import inspect
import dis
import itertools
def diabolically_retrieve_target_list_size():
#one f_back takes us to `get_diabolically_sized_list`'s frame. A second one takes us to the frame of the caller of `get_diabolically_sized_list`.
frame = inspect.currentframe().f_back.f_back
#explicitly delete frame when we're done with it to avoid reference cycles.
try:
#get the bytecode instruction that immediately follows the CALL_FUNCTION that is executing right now
bytecode_idx = frame.f_lasti // 2
unresolved_bytecodes = itertools.islice(dis.get_instructions(frame.f_code), bytecode_idx+1, bytecode_idx+3)
next_bytecode = next(unresolved_bytecodes)
if next_bytecode.opname == "UNPACK_SEQUENCE": #simple multiple assignment, like `a,b,c = ...`
return next_bytecode.arg
elif next_bytecode.opname == "EXTENDED_ARG": #multiple assignment with splat, like `a, *b, c = ...`
next_bytecode = next(unresolved_bytecodes)
if next_bytecode.opname != "UNPACK_EX":
raise Exception(f"Expected UNPACK_EX after EXTENDED_ARG, got {next_bytecode.opname} instead")
args_before_star = next_bytecode.arg % 256
args_after_star = next_bytecode.arg >> 8
return args_before_star + args_after_star
elif next_bytecode.opname in ("STORE_FAST", "STORE_NAME"): #single assignment, like `a = ...`
return 1
else:
raise Exception(f"Unrecognized bytecode: {frame.f_lasti} {next_bytecode.opname}")
finally:
del frame
def get_diabolically_sized_list():
count = diabolically_retrieve_target_list_size()
return list(range(count))
a,b,c = get_diabolically_sized_list()
print(a,b,c)
d,e,f,g,h,i = get_diabolically_sized_list()
print(d,e,f,g,h,i)
j, *k, l = get_diabolically_sized_list()
print(j,k,l)
x = get_diabolically_sized_list()
print(x)
Result:
0 1 2
0 1 2 3 4 5
0 [] 1
[0]
I am not sure the proper way to phrase this question. I would like to assign/bind some arithmetic (with references/pointers to other "subVariables") to a variable and have the value of the variable update if any of the contributing "subVariables" are updated.
>>> a = 1
>>> b = 2
>>> c = a + b
>>> c
3
>>> a = 2
>>> c
3
In the ideal situation c would have a value of 4 at the end of this code sample.
--
Additional information: I am generating the arithmetic randomly and would like to be able to nest these variables (ex: d = a + c would be the same as d = a + (a + b) where d would reflect any changes that happen to a or b)
What you want isn't possible with immutable built-in types like int. Because a and b are bound to immutable types (int), even if some hypothetical class preserved references to its inputs, those inputs never change (a can be rebound to a new value, but a would no longer have anything to do with the value it was previously bound to, and the class that aliased the old value of a would remain unchanged; it doesn't preserve a direct tie to a itself).
The only way this could possibly work is if a and b were of a mutable type where the contents could be updated or reassigned, and they had an overload of __add__/__radd__ that produced yet another class that stored references to instances of said mutable type and/or instances of itself (to allow the nested case). Either way, implementing such a suite of classes is way beyond the scope of a simple answer; I'd strongly recommend finding a different solution to your problem.
Make c as a function which returns a+b value
Simple numbers in Python are immutable references. You can't do it that directly.
You could create objects with that kind of behavior in various ways. Here's an approximation using simple function calls.
>>> a = 1
>>> b = 2
>>> def c():
return a + b
>>> c()
3
>>> a = 2
>>> c()
4
You can avoid the () at the cost of a dot by using __getattribute__
>>> class CallsAttrs:
def __getattribute__(self, attr):
return object.__getattribute__(self, attr)()
>>> Q = CallsAttrs()
>>> a = 1
>>> b = 2
>>> Q.c = lambda: a + b
>>> Q.c
3
>>> a = 2
>>> Q.c
4
And, of course, the lambdas can get the variables from Q too.
>>> Q.a = lambda: 1
>>> Q.b = lambda: 2
>>> Q.c = lambda: Q.a + Q.b
>>> Q.c
3
>>> Q.a = lambda: 40
>>> Q.c
42
You could also override the globals dict to work this way,
>>> class ThunkDict(dict):
def __getitem__(self, key):
return super().__getitem__(key)()
>>> exec("""
a = lambda: 1
b = lambda: 2
c = lambda: a + b
print(c)
a = lambda: -10
print(c)
""", ThunkDict())
3
-8
but it's not as interactive if you have to use exec.
I just realized that the pythonic swap doesn't always work.
def swap(x,y):
x,y = y,x
a = 1
b = 2
swap(a,b)
print b
result: 2
Why does the pythonic way to swap variables not work in this case? Do I need temporary a variable?
In the first line of the function definition
def swap(x,y):
x and y are so called formal parameters.
When you call
swap(a, b)
you are passing in a and b as actual arguments.
What happens now is that Python creates new names for the actual arguments you have passed in.
Inside the function body x is now a new name for the integer object in memory which also has the name a. y is now a new name for the integer object in memory which also goes by the name b. This is why Python is neither call-by-value nor call-by-reference, but best described as call-by-assignment.
Contrary to popular belief, calling functions and assignment works exactly the same way for mutable and immutable parameters. You will see the same behavior you are observing for mutable values:
>>> def swap(x,y):
... x,y = y,x
...
>>> a = [1]
>>> b = [2]
>>> swap(a,b)
>>> a
[1]
>>> b
[2]
So once you understand how assignments work in Python, you will also have understood how function calls work.
To make an example: If you did not write a function, your code would have been equivalent to:
a = 1
b = 2
x = a
y = b
x,y = y,x
del x
del y
In line 3, you are creating a new name x for the integer object which also goes by the name a. Line 4 follows the same logic.
In line 5, you are creating the tuple (y, x) with the value (2, 1) on the right hand side, which is then unpacked, i.e. the name x is reassigned to the value 2 and the name y is reassigned to the value 1:
>>> a = 1
>>> b = 2
>>> x = a
>>> y = b
>>> x,y = y,x
>>> x
2
>>> y
1
The important thing to note here is that a and b never stopped to be names for the value 1 and 2 respectively:
>>> a
1
>>> b
2
The last two lines just unbind the names x and y which is roughly equivalent to what happens in your function once you exit its scope. Note that the names a and b are still bound after unbinding x and y.
>>> a
1
>>> b
2
You never returned and assigned the results. Otherwise, as Joran said, you are only creating local variables in the function. For example:
def swap(a, b):
print "swap called"
return (b,a)
a = 1
b = 2
print a,b
a,b = swap(a,b)
print a,b
Results in the following being displayed:
1 2
swap called
2 1
You need to return something; the swapped vars
Is there a way to assign references in python?
For example, in php i can do this:
$a = 10;
$b = &$a;
$a = 20;
echo $a." ".$b; // 20, 20
how can i do same thing in python?
In python, if you're doing this with non-primitive types, it acts exactly like you want: assigning is done using references. That's why, when you run the following:
>>> a = {'key' : 'value'}
>>> b = a
>>> b['key'] = 'new-value'
>>> print a['key']
you get 'new-value'.
Strictly saying, if you do the following:
>>> a = 5
>>> b = a
>>> print id(a) == id(b)
you'll get True.
But! Because of primitive types are immutable, you cant change the value of variable b itself. You are just able create a new variable with a new value, based on b. For example, if you do the following:
>>> print id(b)
>>> b = b + 1
>>> print id(b)
you'll get two different values.
This means that Python created a new variable, computed its value basing on b's value and then gave this new variable the name b. This concerns all of the immutable types. Connecting two previous examples together:
>>> a = 5
>>> b = a
>>> print id(a)==id(b)
True
>>> b += 1
>>> print id(b)==id(a)
False
So, when you assign in Python, you always assign reference. But some types cannot be changed, so when you do some changes, you actually create a new variable with another reference.
In Python, everything is by default a reference. So when you do something like:
x=[1,2,3]
y=x
x[1]=-1
print y
It prints [1,-1,3].
The reason this does not work when you do
x=1
y=x
x=-1
print y
is that ints are immutable. They cannot be changed. Think about it, does a number really ever change? When you assign a new value to x, you are assigning a new value - not changing the old one. So y still points to the old one. Other immutable types (e.g. strings and tuples) behave in the same way.