First let me say that I thought that the way data storage worked in python is that everything is an object so there is no need for such things as pointers. If you pass an piece of data into a function then that function has the real piece of data. When you exit out of that function it the data passed in could have been modified.
Now I was working with lists and I thought that if I put the same piece of data on two lists then modifying it in one place would modify it in the other.
How can I have one piece of data that is on two, or more, different lists? I would want to change this data in one place and then have the other change.
For example take the following:
p = 9
d = []
f = []
d.append(p)
f.append(p)
print 'd',d
print 'f',f
p = 3
print 'd',d
print 'f',f
When this is run the output is:
d [9]
f [9]
d [9]
f [9]
I would like the second set of data to be 3 but it doesn't seem to work. So where in my thought process did I go wrong? Is there an implicit copy operation when putting data onto a list?
First of all, integers (really, number objects) are immutable. There is no "modifying" this "data".
Second of all, there is also in Python the notion of name binding. When you do:
p = 9
Two things happen: first, a number object (9) is created; second, it is then bound to the name p.
When you later do:
p = 3
It does not, as you think, "modify" the immutable number object created earlier. What this simply does is, again, two things: first, a new number object (3) is created; second, it is then bound to the name p.
Pictorally:
(1a) 9
(1b) p --> 9
(2a) p --> 9
3
(2b) 9
p --> 3
Your second assignment doesn't change the value of the object bound to p, it rebinds p to a whole new object. d and f still contain references to the old object, which still has the same value.
Related
What I have:
A dataframe as the following
I.D Score
1 11 26
3 12 26
5 13 26
6 14 25
multiply a million rows....
What I am trying to do:
1) Pass both columns to a function called Points and create a list called players. credits to #jezrael
players = [Points(int(a),b) for a, b in zip(score['I.D'],score['Score'])]
2) Pass the list to another basic function, that simply uploads the whole list
upload_to_scoreboard(players)
What I get
MemoryError
I believe that passing the whole list consisting of million rows to the next function causes it to run out of memory. how can I modify the code to pass the list to the upload_to_scoreboard without causing memory error?
Use generator instead of list.
To do so replace [] by () to surround your list comprehension.
players = [Points(int(a),b) for a, b in zip(score['I.D'],score['Score'])] loads the entire list in memory. To avoid it you can use a generator. Generators are kind of list but with the difference that every item is computed and loaded in memory only when it is itered, and it is unloaded once you no longer use it.
However in your case, the upload_to_score_board function must accept a such data structure and don't use list specificities.
To know more about generators vs list, read this : https://medium.freecodecamp.org/python-list-comprehensions-vs-generator-expressions-cef70ccb49db#741d
This question already has answers here:
List assignment to other list
(3 answers)
Closed 5 years ago.
I'm trying to understand (on the level of principle) the difference between assignment between (say) integers variables and list variables.
Integer case:
a=6
b=a
print(b) #prints 6
a=7
print(b) #prints 6
That makes sense to me with this logic: in the original b=a, b was given the same value as a (6), not identified with a. So if I change the value of a, the value of b does not change: b is not the same thing as a.
List case:
L=[1,2]
M = L
print(M)
L.append(6)
print(M)
This can make sense with this logic: in M=L I'm forcing M to literally be the same object as L is, I'm identifying it with L. So if L changes, so does M.
What doesn't make sense (to me) is why I need different logic in different cases. (Probably this has to do with integers being "immutable" and lists "mutable" but I don't see how that bears on this.) Can someone point me to a explanation? What is the principle behind the difference in behaviors? (I'm not looking so much for how the technical difference in implementation of integers and lists leads to the difference, but for the reason things were designed this way. What's the logic?)
Every name is a variable holding a reference to some object.
What happens in the first case is that
a = 6
b = a # (a and b point to the same object)
But here, you are changing what a points to:
a = 7
Compare this to the second/list situation, where you actually call the method on the first object. You didn't update the reference as you did in the case with the integers.
L = [1,2]
M = L # here you introduce a new name, referencing the same object as L.
L.append(6) # update that same object.
print(M) # you print that same object
You don't have different logic in different cases here. Lists and integers work in exactly the same way as far as assignment is concerned. If, in your second snippet, to assigned a different list to L in the penultimate line, the two variables would be unrelated.
But lists have an additional capability, which integers and strings don't have, which is that you can modify them. That's all you're seeing here.
Well M = L gets what L currently is and after it prints M you then append 6 to L but this will only effect L because M had only received what L previously was, but if you did M = L again after the append it would print the updated version of the list. Basically if you get a xvariable set to a yvariable and then that yvariable updates the xvariable will not update because you will have to update the xvariable again but this is usually this happens by it self if a loop is being used
I'm trying to accomplish something with Python but I can't find info about this at all on internet so here is my problem.
Having this sample code:
a = 5
b = [a]
I need b[0] to be a pointer reference to a. Currently if I do
b[0] = 7
a.var will still be 5, and if I set value to a b[0], a keeps the same.
I'd like to know if there is any way in Python to store an Object in an array without copying this, so if I modify one I also modify another.
EDIT ONE
I want to clarify a bit better what I'm trying to accomplish so maybe I can find the real answer.
I have a custom class like
class Foo(object):
def __init__(self):
self.var = 7
I have also a list like this:
myList = [Foo()]
My real problem is here:
In some place of a code, I need to do the following assignment:
self.foo = myList[0]
Then when I try to access this foo var, I notice this is never updated from the original instance. So basically that is what I need.
Edit Two
All answers were right. Using id(var) on every step of my assignments I could verify the Object was the same all the time, so the error must be somewhere else. Thanks to everyone for all the help and I hope this post can help someone else in a future.
You already are storing the object, namely the integer 5. What you are not storing is the name-value binding between the name a and the value 5. Your problem is that the operation b[0] = 7 does not modify the object 5; it only modifies the binding between b[0] and its value (from 5 to 7), leaving the values themselves (5 and 7) untouched. In fact, you cannot modify the value 5, since 5 is an integer, and integers in Python are immutable.
Another way to say that is that b[0] = 7 does not modify the object referred to by b[0]; instead, it modifies the object referred to by b. b is a list, and lists are objects that internally keep track of what objects their indices point to. When you do b[0] = 7, this tells b "make your 0 index point at this new object". It does not "notify" 5 (the old value) or 7 (the new value) that they are now pointed to by b[0]. Objects in Python do not know what kinds of references point at them; they only know what they point at (if anything).
What you are asking for is not a way to store an object, but to store a "variable" --- that is, a binding between a name in the local namespace and a value. The answer is that there isn't, in general, a way to do this in Python. There's no construct that will allow you do b = something(a) so that when you later do b[0] = 8, a will change.
If you want to be able to modify the name-value binding, you need to change a, by making it some kind of object whose contents can be changed, instead of just making it a plain variable. For instance, if you do:
a = [5]
b = a
b[0] = 7
Then you will find a has also changed. The same will be true for other kinds of objects as well: if you set a to some other kind of object whose properties can be altered, and then you alter its properties in-place, those changes will show up no matter how you reference the object.
You may find it helpful to read up on how Python variables work, for instance in this article and this one.
Whether you can do this comes down to whether your list elements are immutable or mutable types. Immutable types can't be changed, so when you re-assign a variable that is immutable it's actually creating a new object and labeling it with the same label. Mutable types can be changed, so it's possible to change the underlying object without reassigning it.
For example, say that a is a list, which is a mutable type.
a = [5]
b = [a]
a[0] = 3
print(b)
# [[3]]
a.append('hi')
print(b)
# [[3, 'hi']]
Your test code actually works the way you want it, because objects are mutable.
class Object():
pass
a = Object()
a.var = 5
b = [a]
b[0].var = 7
print(a.var)
# 7
Your other example code should still work, for all the same reasons. When I assign a = l[0], it isn't copying anything, it's just creating an additional reference a to the object that l[0] references.
l = [Object() for _ in range(5)]
l[0].var = 5
a = l[0]
l[0].var = 7
print(a.var)
# 7
You write
a = 5
b = [a]
I need b[0] to be a pointer reference to a.
You can't. You can make a point to a mutable object and change the value for that object:
class Object(object):
'''an object'''
a = Object()
a.value = 5
b = a
b.value = 7
and now a.value returns:
7
Similarly, but with a list this time:
a = [5]
b = a
b[0] = 7
and now a[0] is:
7
This question already has answers here:
How to modify list entries during for loop?
(10 answers)
Closed 5 months ago.
When I try this code:
bar = [1,2,3]
print(bar)
for foo in bar:
print(id(foo))
foo = 0
print(id(foo))
print(bar)
I get this result:
[1, 2, 3]
5169664
5169676
5169652
5169676
5169640
5169676
[1, 2, 3]
I expected the end result to be [0,0,0] and that id would return identical values for each iteration. Why does it behave like this? How can I elegantly assign back to the elements of the list, without using enumerate or range(len(bar))?
See also: How to change variables fed into a for loop in list form
First of all, you cannot reassign a loop variable—well, you can, but that won’t change the list you are iterating over. So setting foo = 0 will not change the list, but only the local variable foo (which happens to contain the value for the iteration at the begin of each iteration).
Next thing, small numbers, like 0 and 1 are internally kept in a pool of small integer objects (This is a CPython implementation detail, doesn’t have to be the case!) That’s why the ID is the same for foo after you assign 0 to it. The id is basically the id of that integer object 0 in the pool.
If you want to change your list while iterating over it, you will unfortunately have to access the elements by index. So if you want to keep the output the same, but have [0, 0, 0] at the end, you will have to iterate over the indexes:
for i in range(len(bar)):
print id(bar[i])
bar[i] = 0
print id(bar[i])
print bar
Otherwise, it’s not really possible, because as soon as you store a list’s element in a variable, you have a separate reference to it that is unlinked to the one stored in the list. And as most of those objects are immutable and you create a new object when assigning a new value to a variable, you won’t get the list’s reference to update.
Yes, the output you got is the ordinary Python behavior. Assigning a new value to foo will change foo's id, and not change the values stored in bar.
If you just want a list of zeroes, you can do:
bar = [0] * len(bar)
If you want to do some more complicated logic, where the new assignment depends on the old value, you can use a list comprehension:
bar = [x * 2 for x in bar]
Or you can use map:
def double(x):
return x * 2
bar = map(double, bar)
you actually didnt change the list at all.
the first thing for loop did was to assign bar[0] to foo(equivalent to foo = bar[0]). foo is just an reference to 1. Then you assign another onject 0 to foo. This changed the reference of foo to 0. But you didnt change bar[0]. Remember, foo as a variable, references bar[0], but assign another value/object to foo doesn't affect bar[0] at all.
bar = [0 for x in bar]
Long answer : foo is just a local name, rebinding does not impact the list. Python variables are really just key:value pairs, not symbolic names for memory locations.
I was working with queue in python when I had a error in code even while the code looked very perfect to me but latter when I changed assignment style all of sudden the code started working. The code looked some what like this before.
x=y=Queue()
x.put("a")
x.put("b")
print y.get()
later i changed to this and it started working
x=Queue()
y=Queue()
x.put("a")
x.put("b")
print y.get(10)
why do both code work differently?
Variables in Python are references, or names, not like variables in C etc.
This code:
x=y=Queue()
means "allow the name y to reference an object in memory made by calling Queue(), and allow the name x to reference the object that y is pointing to." This means both variables refer to the same object - as you can verify with id(x) == id(y).
This code:
x=Queue()
y=Queue()
means "allow the name x to reference one object made by Queue(), and allow the name y to reference another object made by Queue()". In this case, id(x) == id(y) is False
This can often bite you:
a = [1,2,3,4,5]
b = a
b.append(6)
print(a)
# [1,2,3,4,5,6] even though we didn't seem to do anything to a!
To get around this, do import copy; b = a.copy(); instead of b = a.
However, this behaviour doesn't occur to immutable objects like integers:
a = 7
a += 1
This doesn't go to the object that a is referencing and change it by adding one, instead it dereferences a from the object 7, and references it to the object representing the previous value of a + 1 (that is to say, 8). This is unlike actions performed on mutable objects, like lists in the previous example - appending to a list does change the object that the variable is referencing.
So we can do this:
a = 7
b = a
a += 1
print(a)
# 8
print(b)
# 7
Your first code is actually equivalent to:
y=Queue()
x=y
x.put("a")
x.put("b")
print y.get()
Which is different than your second example, because Python handles object by reference. After x=y both variables x and y refer to the same object. In your second example you have two independent queues.