Develop Reference

Is there any usage of self-referential lists or circular reference in list, eg. appending a list to itself

So if I have a list a and append a to it, I will get a list that contains it own reference.
>>> a = [1,2]
>>> a.append(a)
>>> a
[1, 2, [...]]
>>> a[-1][-1][-1]
[1, 2, [...]]
And this basically results in seemingly infinite recursions.
And not only in lists, dictionaries as well:
>>> b = {'a':1,'b':2}
>>> b['c'] = b
>>> b
{'a': 1, 'b': 2, 'c': {...}}
It could have been a good way to store the list in last element and modify other elements, but that wouldn't work as the change will be seen in every recursive reference.
I get why this happens, i.e. due to their mutability. However, I am interested in actual use-cases of this behavior. Can somebody enlighten me?

The use case is that Python is a dynamically typed language, where anything can reference anything, including itself.
List elements are references to other objects, just like variable names and attributes and the keys and values in dictionaries. The references are not typed, variables or lists are not restricted to only referencing, say, integers or floating point values. Every reference can reference any valid Python object. (Python is also strongly typed, in that the objects have a specific type that won't just change; strings remain strings, lists stay lists).
So, because Python is dynamically typed, the following:
foo = []
# ...
foo = False
is valid, because foo isn't restricted to a specific type of object, and the same goes for Python list objects.
The moment your language allows this, you have to account for recursive structures, because containers are allowed to reference themselves, directly or indirectly. The list representation takes this into account by not blowing up when you do this and ask for a string representation. It is instead showing you a [...] entry when there is a circular reference. This happens not just for direct references either, you can create an indirect reference too:
>>> foo = []
>>> bar = []
>>> foo.append(bar)
>>> bar.append(foo)
>>> foo
[[[...]]]
foo is the outermost [/]] pair and the [...] entry. bar is the [/] pair in the middle.
There are plenty of practical situations where you'd want a self-referencing (circular) structure. The built-in OrderedDict object uses a circular linked list to track item order, for example. This is not normally easily visible as there is a C-optimised version of the type, but we can force the Python interpreter to use the pure-Python version (you want to use a fresh interpreter, this is kind-of hackish):
>>> import sys
>>> class ImportFailedModule:
... def __getattr__(self, name):
... raise ImportError
...
>>> sys.modules["_collections"] = ImportFailedModule() # block the extension module from being loaded
>>> del sys.modules["collections"] # force a re-import
>>> from collections import OrderedDict
now we have a pure-python version we can introspect:
>>> od = OrderedDict()
>>> vars(od)
{'_OrderedDict__hardroot': <collections._Link object at 0x10a854e00>, '_OrderedDict__root': <weakproxy at 0x10a861130 to _Link at 0x10a854e00>, '_OrderedDict__map': {}}
Because this ordered dict is empty, the root references itself:
>>> od._OrderedDict__root.next is od._OrderedDict__root
True
just like a list can reference itself. Add a key or two and the linked list grows, but remains linked to itself, eventually:
>>> od["foo"] = "bar"
>>> od._OrderedDict__root.next is od._OrderedDict__root
False
>>> od._OrderedDict__root.next.next is od._OrderedDict__root
True
>>> od["spam"] = 42
>>> od._OrderedDict__root.next.next is od._OrderedDict__root
False
>>> od._OrderedDict__root.next.next.next is od._OrderedDict__root
True
The circular linked list makes it easy to alter the key ordering without having to rebuild the whole underlying hash table.

However, I am interested in actual use-cases of this behavior. Can somebody enlighten me?
I don't think there are many useful use-cases for this. The reason this is allowed is because there could be some actual use-cases for it and forbidding it would make the performance of these containers worse or increase their memory usage.
Python is dynamically typed and you can add any Python object to a list. That means one would need to make special precautions to forbid adding a list to itself. This is different from (most) typed-languages where this cannot happen because of the typing-system.
So in order to forbid such recursive data-structures one would either need to check on every addition/insertion/mutation if the newly added object already participates in a higher layer of the data-structure. That means in the worst case it has to check if the newly added element is anywhere where it could participate in a recursive data-structure. The problem here is that the same list can be referenced in multiple places and can be part of multiple data-structures already and data-structures such as list/dict can be (almost) arbitrarily deep. That detection would be either slow (e.g. linear search) or would take quite a bit of memory (lookup). So it's cheaper to simply allow it.
The reason why Python detects this when printing is that you don't want the interpreter entering an infinite loop, or get a RecursionError, or StackOverflow. That's why for some operations like printing (but also deepcopy) Python temporarily creates a lookup to detect these recursive data-structures and handles them appropriately.

Consider building a state machine that parse string of digits an check if you can divide by 25 you could model each node as list with 10 outgoing directions consider some connections going to them self
def canDiv25(s):
n0,n1,n1g,n2=[],[],[],[]
n0.extend((n1,n0,n2,n0,n0,n1,n0,n2,n0,n0))
n1.extend((n1g,n0,n2,n0,n0,n1,n0,n2,n0,n0))
n1g.extend(n1)
n2.extend((n1,n0,n2,n0,n0,n1g,n0,n2,n0,n0))
cn=n0
for c in s:
cn=cn[int(c)]
return cn is n1g
for i in range(144):
print("%d %d"%(i,canDiv25(str(i))),end='\t')
While this state machine by itself has little practical it show what could happen. Alternative you could have an simple Adventure game where each room is represented as a dictionary you can go for example NORTH but in that room there is of course a back link to SOUTH. Also sometimes game developers make it so that for example to simulate a tricky path in some dungeon the way in NORTH direction will point to the room itself.

A very simple application of this would be a circular linked list where the last node in a list references the first node. These are useful for creating infinite resources, state machines or graphs in general.
def to_circular_list(items):
head, *tail = items
first = { "elem": head }
current = first
for item in tail:
current['next'] = { "elem": item }
current = current['next']
current['next'] = first
return first
to_circular_list([1, 2, 3, 4])
If it's not obvious how that relates to having a self-referencing object, think about what would happen if you only called to_circular_list([1]), you would end up with a data structure that looks like
item = {
"elem": 1,
"next": item
}
If the language didn't support this kind of direct self referencing, it would be impossible to use circular linked lists and many other concepts that rely on self references as a tool in Python.

The reason this is possible is simply because the syntax of Python doesn't prohibit it, much in the way any C or C++ object can contain a reference to itself. An example might be: https://www.geeksforgeeks.org/self-referential-structures/
As #MSeifert said, you will generally get a RecursionError at some point if you're trying to access the list repeatedly from itself. Code that uses this pattern like this:
a = [1, 2]
a.append(a)
def loop(l):
for item in l:
if isinstance(item, list):
loop(l)
else: print(item)
will eventually crash without some sort of condition. I believe that even print(a) will also crash. However:
a = [1, 2]
while True:
for item in a:
print(item)
will run infinitely with the same expected output as the above. Very few recursive problems don't unravel into a simple while loop. For an example of recursive problems that do require a self-referential structure, look up Ackermann's function: http://mathworld.wolfram.com/AckermannFunction.html. This function could be modified to use a self-referential list.
There is certainly precedent for self-referential containers or tree structures, particularly in math, but on a computer they are all limited by the size of the call stack and CPU time, making it impractical to investigate them without some sort of constraint.

Python's Passing by References [duplicate]

This question already has answers here:
"is" operator behaves unexpectedly with integers
(11 answers)
Closed 5 years ago.
Hello I am trying to understand how Python's pass by reference works.
I have an example:
>>>a = 1
>>>b = 1
>>>id(a);id(b)
140522779858088
140522779858088
This makes perfect sense since a and b are both referencing the same value that they would have the identity. What I dont quite understand is how this example:
>>>a = 4.4
>>>b = 1.0+3.4
>>>id(a);id(b)
140522778796184
140522778796136
Is different from this example:
>>>a = 2
>>>b = 2 + 0
>>>id(a);id(b)
140522779858064
140522779858064
Is it because in the 3rd example the 0 int object is being viewed as "None" by the interpreter and is not being recognized as needing a different identity from the object which variable "a" is referencing(2)? Whereas in the 2nd example "b" is adding two different int objects and the interpreter is allocating memory for both of those objects to be added, which gives variable "a", a different identity from variable "b"?

In your first example, the names a and b are both "referencing" the same object because of interning. The assignment statement resulted in an integer with the same id only because it has reused a preexisting object that happened to be hanging around in memory already. That's not a reliable behavior of integers:
>>> a = 257
>>> b = 257
>>> id(a), id(b)
(30610608, 30610728)
As demonstrated above, if you pick a big enough integer then it will behave as the floats in your second example behaved. And interning small integers is optional in the Python language anyway, this happens to be a CPython implementation detail: it's a performance optimization intended to avoid the overhead of creating a new object. We can speed things up by caching commonly used integer instances, at the cost of a higher memory footprint of the Python interpreter.
Don't think about "reference" and "value" when dealing with Python, the model that works for C doesn't really work well here. Instead think of "names" and "objects".
The diagram above illustrates your third example. 2 is an object, a and b are names. We can have different names pointing at the same object. And objects can exist without any name.
Assigning a variable only attaches a nametag. And deleting a variable only removes a nametag. If you keep this idea in mind, then the Python object model will never surprise you again.

As stated here, CPython caches integers from -5 to 256. So all variables within this range with the same value share the same id (I wouldn't bet on that for future versions, though, but that's the current implementation)
There's no such thing for floats probably because there's an "infinity" of possible values (well not infinite but big because of floating point), so the chance of computing the same value by different means is really low compared to integers.
>>> a=4.0
>>> b=4.0
>>> a is b
False

Python variables are always references to objects. These objects can be divided into mutable and immutable objects.
A mutable type can be modified without its id being modified, thus every variable that points to this object will get updated. However, an immutable object can not be modified this way, so Python generates a new object with the changes and reassigns the variable to point to this new object, thus the id of the variable before the change will not match the id of the variable after the change.
Integers and floats are immutable, so after a change they will point to a different object and thus have different ids.
The problem is that CPython "caches" some common integer values so that there are not multiple objects with the same value in order to save memory, and 2 is one of those cache-ed integers, so every time a variable points to the integer 2 it will have the same id (its value will be different for different python executions).

python atomic data types

It was written here that Python has both atomic and reference object types. Atomic objects are: int, long, complex.
When assigning atomic object, it's value is copied, when assigning reference object it's reference is copied.
My question is:
why then, when i do the code bellow i get 'True'?
a = 1234
b = a
print id(a) == id(b)
It seems to me that i don't copy value, i just copy reference, no matter what type it is.

Assignment (binding) in Python NEVER copies data. It ALWAYS copies a reference to the value being bound.
The interpreter computes the value on the right-hand side, and the left-hand side is bound to the new value by referencing it. If expression on the right-hand side is an existing value (in other words, if no operators are required to compute its value) then the left-hand side will be a reference to the same object.
After
a = b
is executed,
a is b
will ALWAYS be true - that's how assignment works in Python. It's also true for containers, so x[i].some_attribute = y will make x[i].some_attribute is y true.
The assertion that Python has atomic types and reference types seems unhelpful to me, if not just plain untrue. I'd say it has atomic types and container types. Containers are things like lists, tuples, dicts, and instances with private attributes (to a first approximation).
As #BallPointPen helpfully pointed out in their comment, mutable values can be altered without needing to re-bind the reference. Since immutable values cannot be altered, references must be re-bound in order to refer to a different value.
Edit: Recently reading the English version of the quoted page (I'm afraid I don't understand Russian) I see "Python uses dynamic typing, and a combination of reference counting and a cycle-detecting garbage collector for memory management." It's possible the Russian page has mistranslated this to give a false impression of the language, or that it was misunderstood by the OP. But Python doesn't have "reference types" except in the most particular sense for weakrefs and similar constructs.

int types are immutable.
what you see is the reference for the number 1234 and that will never change.
for mutable object like list, dictionary you can use
import copy
a = copy.deepcopy(b)

Actually like #spectras said there are only references but there are immutable objects like floats, ints, tuples. For immutable objects (apart from memory consumption) it just does not matter if you pass around references or create copies.
The interpreter even does some optimizations making use of numbers with the same value being interchangeable making checking numbers for identity interesting because eg for
a=1
b=1
c=2/2
d=12345
e=12345*1
a is b is true and a is c is also true but d is e is false (== works normally as expected)
Immutable objects are atomic the way that changing them is threadsafe because you do not actually change the object itself but just put a new reference in a variable (which is threadsafe).

Why should I refer to "names" and "binding" in Python instead of "variables" and "assignment"?

Why should I refer to "names" and "binding" in Python instead of "variables" and "assignment"?
I know this question is a bit general but I really would like to know :)

In C and C++, a variable is a named memory location. The value of the variable is the value stored in that location. Assign to the variable and you modify that value. So the variable is the memory location, not the name for it.
In Python, a variable is a name used to refer to an object. The value of the variable is that object. So far sounds like the same thing. But assign to the variable and you don't modify the object itself, rather you alter which object the variable refers to. So the variable is the name, not the object.
For this reason, if you're considering the properties of Python in the abstract, or if you're talking about multiple languages at once, then it's useful to use different names for these two different things. To keep things straight you might avoid talking about variables in Python, and refer to what the assignment operator does as "binding" rather than "assignment".
Note that The Python grammar talks about "assignments" as a kind of statement, not "bindings". At least some of the Python documentation calls names variables. So in the context of Python alone, it's not incorrect to do the same. Different definitions for jargon words apply in different contexts.

In, for example, C, a variable is a location in memory identified by a specific name. For example, int i; means that there is a 4-byte (usually) variable identified by i. This memory location is allocated regardless of whether a value is assigned to it yet. When C runs i = 1000, it is changing the value stored in the memory location i to 1000.
In python, the memory location and size is irrelevant to the interpreter. The closest python comes to a "variable" in the C sense is a value (e.g. 1000) which exists as an object somewhere in memory, with or without a name attached. Binding it to a name happens by i = 1000. This tells python to create an integer object with a value of 1000, if it does not already exist, and bind to to the name 'i'. An object can be bound to multiple names quite easily, e.g:
>>> a = [] # Create a new list object and bind it to the name 'a'
>>> b = a # Get the object bound to the name 'a' and bind it to the name 'b'
>>> a is b # Are the names 'a' and 'b' bound to the same object?
True
This explains the difference between the terms, but as long as you understand the difference it doesn't really matter which you use. Unless you're pedantic.

I'm not sure the name/binding description is the easiest to understand, for example I've always been confused by it even if I've a somewhat accurate understanding of how Python (and cpython in particular) works.
The simplest way to describe how Python works if you're coming from a C background is to understand that all variables in Python are indeed pointers to objects and for example that a list object is indeed an array of pointers to values. After a = b both a and b are pointing to the same object.
There are a couple of tricky parts where this simple model of Python semantic seems to fail, for example with list augmented operator += but for that it's important to note that a += b in Python is not the same as a = a + b but it's a special increment operation (that can also be defined for user types with the __iadd__ method; a += b is indeed a = a.__iadd__(b)).
Another important thing to understand is that while in Python all variables are indeed pointers still there is no pointer concept. In other words you cannot pass a "pointer to a variable" to a function so that the function can change the variable: what in C++ is defined by
void increment(int &x) {
x += 1;
}
or in C by
void increment(int *x) {
*x += 1;
}
in Python cannot be defined because there's no way to pass "a variable", you can only pass "values". The only way to pass a generic writable place in Python is to use a callback closure.

who said you should? Unless you are discussing issues that are directly related to name binding operations; it is perfectly fine to talk about variables and assignments in Python as in any other language. Naturally the precise meaning is different in different programming languages.
If you are debugging an issue connected with "Naming and binding" then use this terminology because Python language reference uses it: to be as specific and precise as possible, to help resolve the problem by avoiding unnecessary ambiguity.
On the other hand, if you want to know what is the difference between variables in C and Python then these pictures might help.

I would say that the distinction is significant because of several of the differences between C and Python:
Duck typing: a C variable is always an instance of a given type - in Python it isn't the type that a name refers to can change.
Shallow copies - Try the following:
>>> a = [4, 5, 6]
>>> b = a
>>> b[1] = 0
>>> a
[4, 0, 6]
>>> b = 3
>>> a
[4, 0, 6]
This makes sense as a and b are both names that spend some of the time bound to a list instance rather than being separate variables.

Content in Python names

There is a lot of confusion with python names in the web and documentation doesn't seem to be that clear about names. Below are several things I read about python names.
names are references to objects (where are they? heap?) and what name holds is an address. (like Java).
names in python are like C++ references ( int& b) which means that it is another alias for a memory location; i.e. for int a , a is a memory location. if int& b = a means that b is another name the for same memory location
names are very similar to automatically dereferenced pointers variables in C.
Which of the above statements is/are correct?
Does Python names contain some kind of address in them or is it just a name to a memory location (like C++ & references)?
Where are python names stored, Stack or heap?
EDIT:
Check out the below lines from http://etutorials.org/Programming/Python.+Text+processing/Appendix+A.+A+Selective+and+Impressionistic+Short+Review+of+Python/A.2+Namespaces+and+Bindings/#
Whenever a (possibly qualified) name occurs on the right side of an assignment, or on a line by itself, the name is dereferenced to the object itself. If a name has not been bound inside some accessible scope, it cannot be dereferenced; attempting to do so raises a NameError exception. If the name is followed by left and right parentheses (possibly with comma-separated expressions between them), the object is invoked/called after it is dereferenced. Exactly what happens upon invocation can be controlled and overridden for Python objects; but in general, invoking a function or method runs some code, and invoking a class creates an instance. For example:
pkg.subpkg.func() # invoke a function from a namespace
x = y # deref 'y' and bind same object to 'x'
This makes sense.Just want to cross check how true it is.Comments and answers please

names are references to objects
Yes. You shouldn't care where the objects live if you just want to understand Python variables' semantics; they're somewhere in memory and Python implementations manage memory for you. How they do that depends on the implementation (CPython, Jython, PyPy...).
names in python are like C++ references
Not exactly. Reassigning a reference in C++ actually reassigns the memory location referenced, e.g. after
int i = 0;
int &r = i;
r = 1;
it is true that i == 1. You can't do this in Python except by using a mutable container object. The closest you can get to the C++ reference behavior is
i = [0] # single-element list
r = i # r is now another reference to the object referenced by i
r[0] = 1 # sets i[0]
are very similar to automatically dereferenced pointers variables in C
No, because then they'd be similar to C++ references in the above regard.
Does Python names contain some kind of address in them or is it just a name to a memory location?
The former is closer to the truth, assuming a straightforward implementation (again, PyPy might do things differently than CPython). In any case, Python variables are not storage locations, but rather labels/names for objects that may live anywhere in memory.
Every object in a Python process has an identity that can be obtained using the id function, which in CPython returns its memory address. You can check whether two variables (or expressions more generally) reference the same object by checking their id, or more directly by using is:
>>> i = [1, 2]
>>> j = i # a new reference
>>> i is j # same identity?
True
>>> j = [1, 2] # a new list
>>> i == j # same value?
True
>>> i is j # same identity?
False

Python names are, well, names. You have objects and names, that's it.
Creating an object, say [3, 4, 5] creates an object somewhere on the heap. You don't need to know how. Now you can put names to target this object, by assigning it to names:
x = [3, 4, 5]
That is, the assignment operator assigns names rather than values. x isn't [3, 4, 5], no, it's simply a name pointing to the [3, 4, 5] object. So doing this:
x = 1
Doesn't change the original [3, 4, 5] object, instead it assigns the object 1 to the name x. Also note that most expressions like [3, 4, 5], but also 8 + 3 create temporaries. Unless you assign a name to that temporary it will immediately die. There is no (except, for example in CPython for small numbers, but that aside) mechanism to keep objects alive that aren't referenced, and cache them. For example, this fails:
>>> x = [3, 4, 5]
>>> x is [3, 4, 5] # must be some object, right? no!
False
However, that's merely assignment (which is not overloadable in Python). In fact, objects and names in Python very well behave like automatically dereferencing pointers, except that they are automatically reference counted and die after they're not referenced anymore (in CPython, at least) and that they do not automatically dereference on assignment.
Thanks to this memory model, for example the C++ way of overloading index operations doesn't work. Instead, Python uses __setitem__ and __getitem__ because you can't return anything that's "assignable". Furthermore, the operators +=, *=, etc... work by creating temporaries and assigning that temporary back to the name.

Python objects are stored on a heap and are garbage collected via reference counting.
Variables are references to objects like in Java, and thus point 1 applies. I am not familiar with either C++ or automatically dereferenced pointer variables in C, to make a call on those.
Ultimately, it's the python interpreter that does the looking up of items in the interpreter structures, which usually are python lists and dictionaries and other such abstract containers; namespaces use dict (a hash table) for example, where the names and values are pointers to other python objects. These are managed explicitly by the mapping protocol.
To the python programmer, this is all hidden; you don't need to know where your objects live, just that they are still alive as long as you have something referencing them. You pass around these references when coding in python.