Develop Reference

How do Python Variables Reference Class Instances and Objects? [duplicate]

I've been learning Python for a few months, and also know very little C, and I was wondering if anyone could clear this doubt up for me:
Is a variable the name, the value, or the memory location?
For example:
x = 5
Is the variable x, the value of x, or the location of x in memory?
I'm looking for a clear explanation of what a variable is. I've already looked at Wikipedia's page on variables and this question, but neither were too clear to me. If this is a duplicate question, a link to the right answer would be great.
Thanks!

There are several things going on with the statement x=5:
an int object with the value of 5 is created (or found if it already exists);
the name x is created (or disassociated with the last object 'x' labeled);
the reference count to the new (or found) int object is increased by 1;
the name x is associated with the object with the value '5' created (or found).
Since int objects are immutable, they may be interned for efficiency. String objects are more likely to be interned.
Here are some examples:
>>> x=5 # discussed
>>> id(x) # the 'id' which in cPython is the memory address.
140246146681256
>>> y=x # now two names, 'x' and 'y' associated with that object
>>> id(y)
140246146681256 # same object
>>> z=5 # no guaranteed, likely the same object referred to by 'x' and 'y'
>>> id(z)
140246146681256 # id is the same! The object labeled 'x' was found and labeled 'z'
>>> del x # ref count to object '140246146681256' decreased by 1
>>> del y # same
>>> z
5
>>> id(z)
140246146681256 # same object but the names ''x' and 'y' no longer label it
The best way to think of variables names, like 'x' in x=5 is as a label.
It is possible to have objects with no label, such as the 5 individual string objects created in this list comprehension:
>>> li=[str(x) for x in range(5)]
>>> li
['0', '1', '2', '3', '4']
You can then create objects that match the value of those same 5 objects independently:
>>> li2=list('012345') # longer list; completely different construction
>>> li2
['0', '1', '2', '3', '4', '5']
You can get their individual memory addresses (in cPython) or a unique id address:
>>> [id(x) for x in li]
[4373138488, 4372558792, 4372696960, 4373139288, 4373139368]
>>> [id(x) for x in li2]
[4373138488, 4372558792, 4372696960, 4373139288, 4373139368, 4372696720]
Note that the two independently created lists of anonymous object are the same (for the first 5 in this case). I purposely used strings because they are more likely to be interred...
So think of it this way: two different processes are happening with x=5:
the object is created (or found if it is immutable, interned and exists) and
the object is labeled.

There is a space somewhere in a memory where the values are stored. These memory is then located through address which is usually represented by hexadecimal number, or sometimes binary number. So, these numbers are hard to remember for programmer, so they used the term variable to refer to that memory location in a easy way, or you can say higher abstraction. So, x is just a name that refers to some location in the memory which contains value of 5. That's all!

A variable is a mapping of a name to associated storage and serves as a way to refer to that storage location in your program. When you assign to a variable and perform operations on it, you're populating and altering that storage. When you reassign a variable, you can point it to a different storage location -- you are remapping that name.

In Python variables are best considered as names.
In the case of numbers, numbers are immutable so reassigning a variable will always make the variable point to a different number without changing the number.
x = 1
y = x
x += 1 # y is still 1
The difference is clearer with mutable objects, where you can change a value OR make the variable refer to a different value
x = [1,2,3]
y = x
x.append(4) # I mutated x's value, so y is now [1,2,3,4]
x = [] # I made x point to a different list, so y is still [1,2,3,4]
Recommended reading:
http://web.archive.org/web/20180411011411/http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html
http://me.veekun.com/blog/2012/05/23/python-faq-passing/

First Of All
The formal definition of "Memory Model" is a description of the behavior of multi-threaded data access, see Memory_model.
Data Model
According to your description, what you want to express is actually "Data Model", see Python Data Model.
Let me try to summarize:
In Python, data was represented by object.
Every object has three key concepts: identity, a type and a value.
identity: value never changed to identify an object, use id() function to check the id of an object. In CPython, id is object's memory address.
type: object's type, like int/ string/ list or so. never changed. type() to check.
value: the object's value. For example, for a string object, the value is "some string", and it is immutable.
Variable is just a symbol to refer to the object.
Your Question
what happens when typing: x=5?
an object with type int, value 5 was created(or reused if there already has).
a variable named x refer to the object. The object's reference counter increase 1.
That's all.

Comparing object ids of two numpy arrays

I have been using numpy for quite a while but I stumbled upon one thing that I didn't understand fully:
a = np.ones(20)
b = np.zeros(10)
print(id(a)==id(b)) # prints False
print(id(a), id(b)) # prints (4591424976, 4590843504)
print(id(a[0])==id(b[0])) # prints True
print(id(a[0]), id(b[0])) # prints (4588947064, 4588947064)
print(id(a[0])) # 4588947184
print(id(b[0])) # 4588947280
Can someone please explain the behavior observed in last four print statements? Also, I was aware of the fact that id gives you unique object id actually allocated in the memory but every time I run the last two print statements, I got different id values. Is this the expected behavior?

The short answer is that you should forget about relying on id to try and gain deep insight into the workings of python. Its output is affected by CPython implementation details, peephole optimizations and memory reuse. More often than not id is a red herring. This is especially true with numpy.
In your specific case only a and b exist as python objects. When you take an element, a[0], you instantiate a new python object, a scalar of type numpy.float64. These are new python objects and are thus given a new id, unless the interpreter realizes that you're trying to use this object twice (this is probably what's happening in your middle example, although I do find it surprising that two numpy.float64 objects with different values are given the same id. But the weird magic goes away if you assign a[0] and b[0] to proper names first, so this is probably due to some optimization). It could also happen that memory addresses get reused by the interpreter, giving you ids that have appeared before.
Just to see how pointless id is with numpy, even trivial views are new python objects with new ids, even though for all intents and purposes they are as good as the original:
>>> arr = np.arange(3)
>>> id(arr)
140649669302992
>>> id(arr[...])
140649669667056
And here's an example for id reuse in an interactive shell:
>>> id(np.arange(3))
139775926634896
>>> id(np.arange(3))
139775926672480
>>> id(np.arange(3))
139775926634896
Surely there's no such thing as int interning for numpy arrays, so the above is only due to the interpreter reusing ids. The fact that id returns a memory address is again just a CPython implementation detail. Forget about id.
The only thing you might want to use with numpy is numpy.may_share_memory and numpy.shares_memory.

It's important to note that everything in Python is an object, even numbers and Classes.
You have taken 2 numpy array object and each of contains same values i.e 0.
When you say:
print('id of 0 =',id(0))
a = 0
print('id of a =',id(a))
b = a
print('id of b =',id(b))
c = 0.0
print('id of c =',id(c))
The answer you get something like (your case it's different):
id of 0 = 140472391630016
id of a = 140472391630016
id of b = 140472391630016
id of c = 140472372786520
Hence, integer 0 has a unique id. The id of the integer 0 remains constant during the lifetime. Similar is the case for float 0.0 and other objects.
So in your case a[0] or b[0] object id of zero will remain same until or unless it is alive because both contains 0 as object value.
Each time you print a[0] or b[0] in different line you return it's different identity of object because you triggering it at different line hence different lifetime.
You can try:
print(id(a)==id(b))
print(id(a),id(b))
print(id(a[0])==id(b[0]))
print(id(a[0]),id(b[0]))
The output will be:
False
2566443478752 2566448028528
True
2566447961120 2566447961120
Note that second line will return to you 2 different identity of object of numpy array type because both are different list.

Why does this assigned object share the same memory space as the original object?

In python I came across this strange phenomena while working with itertools groupby module.
In python, variable assignment means assigning the new variable its own memory instead of a pointer to the original memory (from my understanding if this is incorrect please let me know):
y = 7
x = y
y = 9
x will still be 7
Yet when I was working with groupby module, I was using this module to group items that had the same key into one group. I wanted two groups since reiterating over the original group was useless as the memory would have already been modified. Example:
for key, group in groupby(rows, lambda x: x[0]):
data = [thing[1] for thing in group] #accesses 1st attribute of element
data2 = [thing[2] for thing in group] # would yield [] as group is empty
So I tried this instead:
for key, group in groupby(rows, lambda x: x[0]):
#create a copy of group to reiterate over
toup = group
print toup #<itertools._grouper object at 0x1039a8850>
print group #<itertools._grouper object at 0x1039a8850>
data = [thing[1] for thing in group] #accesses 1st attribute of element
data2 = [thing[2] for thing in toup]
data2 should access the 2nd item but yields [] since they both share the same memory
My question is why does this happen? Shouldn't assigning group to toup means toup would have a copy of groups memory at a different hex address location?
Also what can I do to circumvent this problem so I don't have to write two groupby iterations?

You state:
In python, variable assignment means assigning the new variable its
own memory instead of a pointer to the original memory (from my
understanding if this is incorrect please let me know):
That is incorrect. Python names can have aspects that (at time) are like C variables and can also have aspects that (at times) are like C pointers. To try and say they are like one or the other is just confusing. Don't. Consider them as unique and idiomatic to Python.
Python 'variables' should more be thought of as names. More than one may refer to the same memory location even if you did not intend them to.
Example:
>>> y=7
>>> x=7
>>> x is y
True
>>> id(x)
140316099265400
>>> id(y)
140316099265400
And (due to interning, the following may be true. See PEP 237 regarding interning of short ints, but this is an implementation detail):
>>> x=9
>>> y=5+4
>>> x is y
True
The Python is operator returns True if the two are the same objects by comparing their memory address. The id function returns that address.
Consider as a final example:
>>> li1=[1,2,3]
>>> li2=[1,2,3]
>>> li1==li2
True
>>> li1 is li2
False
Even though li1 == li2, they have to be separate lists otherwise both would change if you change one, as in this example:
>>> li1=[1,2,3]
>>> li2=li1
>>> li1.append(4)
>>> li2
[1, 2, 3, 4]
>>> li1==li2
True
>>> li1 is li2
True
(Be sure to understand another classic mistake all Python programers will make sooner or later. This is caused by multiple references to a single mutable object and then expecting one reference to act like a single object.)
As jonrsharpe pointed out in the comments, read Ned Batchelders excellent Facts and myths about Python Names and Values or How to Think Like a Pythonista for more detailed overview.

In python, variable assignment means assigning the new variable its own memory instead of a pointer to the original memory
Python has mutable (e.g. lists, iterators, just about everything) and immutable objects (e.g. integers and strings). Assignment does not copy the object in either case. With immutable objects, all operations on them result in a new instance, so you won't run into the problem of "modifying" an integer or a string like you do with mutable types.
My question is why does this happen? Shouldn't assigning group to toup means toup would have a copy of groups memory at a different hex address location?
Both variables will point to the same object. When you iterate over one and exhaust the iterator, iterating over the second variable will give you an empty sequence.

Python Objects Types

I've been learning python and i've got a doubt and i'm not sure if what i'm thinking is the correct. As you know Python is an OOP Language and all objects have an id, type and a value.
However there's a concept that i'm not sure if i understood well. It's the mutable and immutable objects.
I know there are some objects that are mutables like arrays, lists and there are others that are immutable like strings, ints, tuples etc.
Basically the main diference is that the immutable can't change its value. For example if i've got an int var:
x = 1
its value is always the same.
After that line of code if i create another var x but now = 2, it's another object, because they have different id's, right? But now, how can i access a var by id, for example my first x var?
Hope you can help.
Thanks! :)

But now, how can i access a var by id, for example my first x var?
You can't. When you do:
x = 1
x = 2
when python executes the x = 2 the 1 object you assigned to x doesn't have any references that can reach it and thus you cannot access it in any way. There's no such a thing as get_object_from_id that allows you to reach any object given its id.
When an object doesn't have any reference to it, it is effectively unreachable. When the garbage collector will be run the object will be destroyed (actually, in CPython most of the objects are destroyed immediately, but there isn't any guarantee about when an object will be destroyed).
As Martijn pointed out, the id of an object is just an implementation detail and it may be reused by the interpreter when an object is destroyed, hence, even from a theoretical point of view, a function such as get_object_from_id can't be implemented in any sensible manner.
Also, assignments have nothing to do with mutable vs immutable types.
You can see the difference between mutable and immutable types when you perform operations on them. For example:
some_list.append(1)
Adds 1 to some_list (modifying it), without creating a new list object, while:
a = 1
a += 2
Here a is not modified by increasing its value to 3, but a new integer object of value 3 is assigned to a.
You can use the id to check that the as are different objects:
>>> a = 1
>>> id(a)
25889896
>>> a += 2
>>> id(a)
25889848
Note that relying on id is a really poor choice. For example:
>>> a = (1, 2)
>>> b = (1, 2)
>>> a is b # is compare the ids just like: id(a) == id(b)
False
>>> a[0] is b[0]
True
As you can see the interpreter decided to re-use the same object 1 for two different tuples. This is an optimization used for small integers, that works because integers are immutable.
Bottom line: id can only tell you if two things aren't the same thing at a certain moment in time. Any other use is just wrong.

This seems like more of a question about scope,.. for which I would recommend reading this.
http://docs.python.org/release/1.5.1p1/tut/scopes.html
say we have the following
x = 1
def PrintStuff(x):
print (x)
PrintStuff(2)
>>> 2
In the function PrintStuff, x is a local variable.
Outside the function PrintStuff lies a global variable x
To get the global x, there are multiple options, but for right now lets just use the globals() function, which returns a dictionary containing all global variables.
x = 1
def PrintStuff(x):
print( x)
print( globals()["x"])
PrintStuff(2)
>>> 2
>>> 1
Similarly we could use the locals() function to get the local x
def PrintStuff(x):
print( locals()["x"])
PrintStuff(2)
>>> 2

What is the id( ) function used for?

I read the Python 2 docs and noticed the id() function:
Return the “identity” of an object. This is an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value.
CPython implementation detail: This is the address of the object in memory.
So, I experimented by using id() with a list:
>>> list = [1,2,3]
>>> id(list[0])
31186196
>>> id(list[1])
31907092 // increased by 896
>>> id(list[2])
31907080 // decreased by 12
What is the integer returned from the function? Is it synonymous to memory addresses in C? If so, why doesn't the integer correspond to the size of the data type?
When is id() used in practice?

Your post asks several questions:
What is the number returned from the function?
It is "an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime." (Python Standard Library - Built-in Functions) A unique number. Nothing more, and nothing less. Think of it as a social-security number or employee id number for Python objects.
Is it the same with memory addresses in C?
Conceptually, yes, in that they are both guaranteed to be unique in their universe during their lifetime. And in one particular implementation of Python, it actually is the memory address of the corresponding C object.
If yes, why doesn't the number increase instantly by the size of the data type (I assume that it would be int)?
Because a list is not an array, and a list element is a reference, not an object.
When do we really use id( ) function?
Hardly ever. You can test if two references are the same by comparing their ids, but the is operator has always been the recommended way of doing that. id( ) is only really useful in debugging situations.

That's the identity of the location of the object in memory...
This example might help you understand the concept a little more.
foo = 1
bar = foo
baz = bar
fii = 1
print id(foo)
print id(bar)
print id(baz)
print id(fii)
> 1532352
> 1532352
> 1532352
> 1532352
These all point to the same location in memory, which is why their values are the same. In the example, 1 is only stored once, and anything else pointing to 1 will reference that memory location.

Rob's answer (most voted above) is correct. I would like to add that in some situations using IDs is useful as it allows for comparison of objects and finding which objects refer to your objects.
The later usually helps you for example to debug strange bugs where mutable objects are passed as parameter to say classes and are assigned to local vars in a class. Mutating those objects will mutate vars in a class. This manifests itself in strange behavior where multiple things change at the same time.
Recently I had this problem with a Python/Tkinter app where editing text in one text entry field changed the text in another as I typed :)
Here is an example on how you might use function id() to trace where those references are. By all means this is not a solution covering all possible cases, but you get the idea. Again IDs are used in the background and user does not see them:
class democlass:
classvar = 24
def __init__(self, var):
self.instancevar1 = var
self.instancevar2 = 42
def whoreferencesmylocalvars(self, fromwhere):
return {__l__: {__g__
for __g__ in fromwhere
if not callable(__g__) and id(eval(__g__)) == id(getattr(self,__l__))
}
for __l__ in dir(self)
if not callable(getattr(self, __l__)) and __l__[-1] != '_'
}
def whoreferencesthisclassinstance(self, fromwhere):
return {__g__
for __g__ in fromwhere
if not callable(__g__) and id(eval(__g__)) == id(self)
}
a = [1,2,3,4]
b = a
c = b
democlassinstance = democlass(a)
d = democlassinstance
e = d
f = democlassinstance.classvar
g = democlassinstance.instancevar2
print( 'My class instance is of', type(democlassinstance), 'type.')
print( 'My instance vars are referenced by:', democlassinstance.whoreferencesmylocalvars(globals()) )
print( 'My class instance is referenced by:', democlassinstance.whoreferencesthisclassinstance(globals()) )
OUTPUT:
My class instance is of <class '__main__.democlass'> type.
My instance vars are referenced by: {'instancevar2': {'g'}, 'classvar': {'f'}, 'instancevar1': {'a', 'c', 'b'}}
My class instance is referenced by: {'e', 'd', 'democlassinstance'}
Underscores in variable names are used to prevent name colisions. Functions use "fromwhere" argument so that you can let them know where to start searching for references. This argument is filled by a function that lists all names in a given namespace. Globals() is one such function.

id() does return the address of the object being referenced (in CPython), but your confusion comes from the fact that python lists are very different from C arrays. In a python list, every element is a reference. So what you are doing is much more similar to this C code:
int *arr[3];
arr[0] = malloc(sizeof(int));
*arr[0] = 1;
arr[1] = malloc(sizeof(int));
*arr[1] = 2;
arr[2] = malloc(sizeof(int));
*arr[2] = 3;
printf("%p %p %p", arr[0], arr[1], arr[2]);
In other words, you are printing the address from the reference and not an address relative to where your list is stored.
In my case, I have found the id() function handy for creating opaque handles to return to C code when calling python from C. Doing that, you can easily use a dictionary to look up the object from its handle and it's guaranteed to be unique.

I am starting out with python and I use id when I use the interactive shell to see whether my variables are assigned to the same thing or if they just look the same.
Every value is an id, which is a unique number related to where it is stored in the memory of the computer.

If you're using python 3.4.1 then you get a different answer to your question.
list = [1,2,3]
id(list[0])
id(list[1])
id(list[2])
returns:
1705950792
1705950808 # increased by 16
1705950824 # increased by 16
The integers -5 to 256 have a constant id, and on finding it multiple times its id does not change, unlike all other numbers before or after it that have different id's every time you find it.
The numbers from -5 to 256 have id's in increasing order and differ by 16.
The number returned by id() function is a unique id given to each item stored in memory and it is analogy wise the same as the memory location in C.

The is operator uses it to check whether two objects are identical (as opposed to equal). The actual value that is returned from id() is pretty much never used for anything because it doesn't really have a meaning, and it's platform-dependent.

The answer is pretty much never. IDs are mainly used internally to Python.
The average Python programmer will probably never need to use id() in their code.

It is the address of the object in memory, exactly as the doc says. However, it has metadata attached to it, properties of the object and location in the memory is needed to store the metadata. So, when you create your variable called list, you also create metadata for the list and its elements.
So, unless you an absolute guru in the language you can't determine the id of the next element of your list based on the previous element, because you don't know what the language allocates along with the elements.

I have an idea to use value of id() in logging.
It's cheap to get and it's quite short.
In my case I use tornado and id() would like to have an anchor to group messages scattered and mixed over file by web socket.

I'm a little bit late and i will talk about Python3. To understand what id() is and how it (and Python) works, consider next example:
>>> x=1000
>>> y=1000
>>> id(x)==id(y)
False
>>> id(x)
4312240944
>>> id(y)
4312240912
>>> id(1000)
4312241104
>>> x=1000
>>> id(x)
4312241104
>>> y=1000
>>> id(y)
4312241200
You need to think about everything on the right side as objects. Every time you make assignment - you create new object and that means new id. In the middle you can see a "wild" object which is created only for function - id(1000). So, it's lifetime is only for that line of code. If you check the next line - you see that when we create new variable x, it has the same id as that wild object. Pretty much it works like memory address.

As of in python 3 id is assigned to a value not a variable. This means that if you create two functions as below, all the three id's are the same.
>>> def xyz():
... q=123
... print(id(q))
...
>>> def iop():
... w=123
... print(id(w))
>>> xyz()
1650376736
>>> iop()
1650376736
>>> id(123)
1650376736

Be carefull (concerning the answer just below)...That's only true because 123 is between -5 and 256...
In [111]: q = 257
In [112]: id(q)
Out[112]: 140020248465168
In [113]: w = 257
In [114]: id(w)
Out[114]: 140020274622544
In [115]: id(257)
Out[115]: 140020274622768