Develop Reference

Mutability, Locality, and Looping [duplicate]

This question already has answers here:
Local (?) variable referenced before assignment [duplicate]
(3 answers)
Closed 4 years ago.
I have some code from a beginner's coding exercise:
numbers = []
i = 0
def populate(maximum, step):
while i < maximum:
numbers.append(i)
i = i + step
populate(10, 2)
Which fails with the stack trace:
Traceback (most recent call last):
File "test_python.py", line 9, in <module>
populate(10, 2)
File "test_python.py", line 5, in populate
while i < maximum:
UnboundLocalError: local variable 'i' referenced before assignment
Here is my understanding of the problem so far...
i is an int and therefore it is immutable, and numbers is a list and is mutable
Since i is immutable, it can be read while it is out of scope. However, overwritting it while it is not in scope (ie. in the populate function), causes an UnboundLocalError
Since numbers is a mutable list, appending to it does not cause an overwrite and therefore does not cause an UnboundLocalError
If a simple change is made to the populate method, the program works successfully (due to i not being overwritten)
def populate(maximum, step):
new_i = i
while new_i < maximum:
numbers.append(i)
new_i = new_i + step
If I comment out the i = i + 1 line, the while loop (obviously) runs forever, but the program does not fail.
My question is then, why does Python fail when it hits the while loop on line 5, instead of the actual problem on line 7 (i = i + 1)? Is this some artifact of the interpreter taking the while loop as a block of code?
This piece of code fails in the correct spot:
def populate(maximum, step):
while i < maximum:
raise Exception("foo")
Stack trace:
Traceback (most recent call last):
File "test_python.py", line 12, in <module>
populate(10, 2)
File "test_python.py", line 6, in populate
raise Exception("foo")
Exception: foo
Another note: This only seems to be the case when the variable is used within the start of the control block (ie. while i < maximum). And the same behavior occurs for every type of control block: while, for, if, elif, etc.

Mutability is a red herring here. Mutable values are affected by scope the same way as immutable values. In fact, nothing in the Python language handles mutable values specially (this is a common myth, however).
The key insight is that the scope of a name is fixed in each scope. Within the scope of populate, every name must either be local or global: this decision is a part of the bytecode of the method.
A name that is only read can be looked up from an enclosing scope. But an assignment to a name anywhere within a scope forces that to be treated as a local variable everywhere in the scope. (Unless you use the global or nonlocal keywords.)
So if you're assigning to i anywhere in the method, then i must be a new variable local to the method, and not the global i. If you want i to mean the global i, simply add this line at the top of the method:
global i

"local variable referenced before assignment" [duplicate]

This question already has answers here:
Assigning to variable from parent function: "Local variable referenced before assignment" [duplicate]
(5 answers)
Short description of the scoping rules?
(9 answers)
Closed 8 years ago.
I come from a land of heavy Java and PHP experience so when it comes to Python much of the rules do not make sense to me.
I have a recursive Fibonacci function below that spits out the error:
Traceback (most recent call last):
File "C:\Users\Nic\workspace\lab8\src\Main.py", line 26, in <module>
print fibcount(27),"took",calls,"calls."
File "C:\Users\Nic\workspace\lab8\src\Main.py", line 19, in fibcount
calls += 1
UnboundLocalError: local variable 'calls' referenced before assignment
Here is my code:
calls = 0
def fibcount(n):
calls += 1
if n < 2:
return (1,1)
f1,c1 = fibcount(n-1)
f2,c2 = fibcount(n-2)
return (f1+f2,c1+c2+1)
print fibcount(27),"took",calls,"calls."
In Java this would obviously work because calls is a global variable in respect to the function fibcount() so it confuses me that calls is somehow not in scope.
What am I doing wrong?

In Python, you need to declare global variables as being global inside functions which reassign them to a different value. You would use global for this:
def fibcount(n):
global calls
...
Otherwise, Python will treat the calls variable as if it were local.
Note however that you only need to do this for global variables which you reassign. Simply reading their values will work fine.
Also, reassigning global variables like this is considered ugly in Python and should be avoided whenever possible. Most of the time, you should stick to passing values to functions and then reassigning variables to their return values. If you need to maintain state, use a class.
The only time global variables are warranted in Python is when you want to have module-level constants. And in that case, the convention is that they should have all-caps names:
MYCONST = 12345

Is it possible to delete an external object reference (variable) from within a function without using exec, eval, compile?

I believe I am on my way to thoroughly understanding how Python uses variables and references, etc, and how they are passed between functions.
As a result of the way Python does things, if I pass a reference variable to a function and attempt to delete the variable inside:
class C(): pass
c = C()
def euthanize(patient): del patient
euthanize(c)
The external object reference will not be deleted because the function has only removed a local reference variable (which was created when the function was called), not the reference variable from the external context:
assert c #No error
This is also the case for a class:
class C():
def die(self): c.__del__() #same result for 'del c' here
def __del__(self): pass
c = C()
c.die()
assert c #No error
This leads me to believe that there is no way, aside from using something like exec or eval, to delete a variable from outside of the context in which it was created. Is this correct?
Clarification
I'm sure I'm going to get a lot of "Why would you want to do this?" comments. Please refrain. I have come to understand that, for Python, if there isn't a somewhat straight forward way to accomplish a task, then it is probably something I shouldn't be doing in the first place and I should look into another approach. I am asking the question because I want to make sure I am correct that this is one of those situations, and therefore not something most people will find that they need to accomplish (and of course, if you REALLY DO need to do it, there is always exec!). I have noticed people are VERY quick to ask "WHY WOULD YOU WANT TO...?" on SO. This is understandable, but sometimes very annoying. Please just don't do it in this case, alright?

You are basically right, but I think the reason is somewhat broader than you seem to believe. As discussed in the link you mentioned, arguments are passed "by object reference". In other words, when you do euthanize(c), euthanize only gets the object referred to by c; it doesn't know how you referrred to it. In this case you used a bare name c, but euthanize doesn't know that. You might not have used a variable at all; your example would be equivalent if it was euthanize(C()), in which case there is never any name that refers to the C instance.
So, the reason you can't delete a "variable" from the enclosing namespace is that what you are trying to delete is a name in the enclosing namespace, while function arguments pass only values. (This isn't pass-by-value, though, because the value is an object reference, which is mentioned in your link so I'm assuming you get what I mean here.)
Your example with __del__ is a bit of a red herring. __del__ is not really deleting anything. It may be called if there are no further references to an object, but calling del c in your second example might or might not result in c.__del__ being called (right away or later). So even doing c = C() and then c.__del__() right in the same global namespace wouldn't delete the variable (that is, the name) called c.
As ch3ka's answer suggests, the only way to delete a name is to pass it around as a string and then use it to to do something like del globals(someName). However, even this won't always work; it works for globals, but you can't, for instance, use it in a nested function to delete a local variable of the enclosing function.

one way to do it, but probably not what you had in mind:
>>> class C(): pass
...
>>> c = C()
>>> def euthanize(patient):
... del globals()[patient]
...
>>> euthanize('c')
>>> c
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'c' is not defined
another way, not requiring the patient to be passed as string (but still does not clear closures and such):
>>> def euthanize(patient):
... for p in [x[0] for x in
globals().items() if
x[1] is patient]:
... del globals()[p]
...
>>> c
<__main__.C object at 0x7fe8c4a3d518>
>>> euthanize(c)
>>> c
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'c' is not defined

I came up with a way that you can sort-of kind-of do this. You have to use something like a container or a class attribute.
For example:
def euthanize(patient,index): del patient[index]
class C: pass
L = [C()]
euthanize(L,0)
assert len(L) == 0
This isn't exactly what I was asking in the question above, but it is similar because it does remove the external strong reference to the object L[0] which was created in the above code, and the object will eventually be GC'd. This can be tested using weakref.finalize:
import weakref
L = [C()]
weakref.finalize(L[0],lambda: print("L[0] finalized!"))
euthanize(L,0) #OUTPUT: L[0] finalized!
However, to be precise, the function is not deleting an object reference. L is the object reference. The function simply manipulates L. EDIT: And as BrenBarn noted below, if there are any other references to the member of L I am trying to euthanize, those are not removed.
Here's how it would look using a class attribute:
c = type('C',(object,),{})()
setattr(c,'X',1)
def euthanizeX(obj): del obj.X
assert c.X #AttributeError (expected)

Implementing Classes as dictionary in Python

I found below code at http://pydanny.com/python-dictionary-as-a-class.html. The code works fine and I kind of understand why it does but a similar code, below it gives error.
def newclass(**kwargs):
""" Use kwargs.update() method to handle inheritance """
def set(key, value):
""" Sets key/value to the kwargs.
Replicates self/this clumsily
"""
kwargs[key] = value
kwargs['set'] = set
return kwargs
My trial code:
def closing():
x=1
def closed():
print(x)
x=x+1
return(closed)
a=closing()
a()
Error Message:
Traceback (most recent call last):
File "<pyshell#606>", line 1, in <module>
a()
File "<pyshell#604>", line 4, in closed
print(x)
UnboundLocalError: local variable 'x' referenced before assignment
When I use 'nonlocal x' in the closed function it works but how come the initial code works without 'nonlocal'.
My understanding is that it's a closure and the inside function will keep reference of the outside (free) variable and whenever the inner function is called it will be able to act upon that closed variable but certainly I haven't understood some part of it properly.
Please help me clear the concept I am missing.
Thank you all the folks who answer. SO has been too helpful.

When I use 'nonlocal x' in the closed function it works but how come the initial code works without 'nonlocal'.
In the first snippet, you're mutating the existing kwargs value. You're not assigning a new value to it, or to any other name.
But in the second snippet, you're assigning a new value to x. That forces x to be local (unless you've said otherwise). Therefore, your x+1 is a reference to a local variable that hasn't been assigned yet. Hence the error.
For a more rigorous explanation, read Naming and binding in the documentation. The relevant bits are:
Each occurrence of a name in the program text refers to the binding of that name established in the innermost function block containing the use…
When a name is used in a code block, it is resolved using the nearest enclosing scope…
If a name is bound in a block, it is a local variable of that block, unless declared as nonlocal…
To learn how this works under the covers, you'll also need to read about functions and code objects in The standard type hierarchy. You probably don't need to know that part for understanding, but for debugging, you may want to inspect the __closure__ and other attributes.

UnboundLocalError in python confusing [duplicate]

This question already has answers here:
UnboundLocalError trying to use a variable (supposed to be global) that is (re)assigned (even after first use)
(14 answers)
Why does code like `str = str(...)` cause a TypeError, but only the second time?
(20 answers)
Closed 6 months ago.
Could anyone explain the exception the below code. It only works when I change the var sub in the display() to another name. There is no global variable sub as well. So what happened ?
def sub(a, b):
return a - b
def display():
sub = sub(2,1) // if change to sub1 or sth different to sub, it works
print sub

Any variable you assign to inside a scope is treated as a local variable (unless you declare it global, or, in python3, nonlocal), which means it is not looked up in the surrounding scopes.
A simplified example with the same error:
def a(): pass
def b(): a = a()
Now, consider the different scopes involved here:
The global namespace contains a and b.
The function a contains no local variables.
The function b contains an assignment to a - this means it is interpreted as a local variable and shadows the function a from the outer scope (in this case, the global scope). As a has not been defined inside of b before the call, it is an unbound local variable, hence the UnboundLocalError. This is exactly the same as if you had written this:
def b(): x = x()
The solution to this is simple: choose a different name for the result of the sub call.
It is important to note that the order of use and assignment makes no difference - the error would have still happened if you wrote the function like this:
def display():
value = sub(2,1) #UnboundLocalError here...
print value
sub = "someOtherValue" #because you assign a variable named `sub` here
This is because the list of local variables is generated when the python interpreter creates the function object.

This was originally a comment. The OP found this useful as an answer. Therefore, I am re-posting it as an answer
Initially, sub is a function. Then, it becomes the return value of a function. So when you say print sub, python doesn't know which sub you are referring to.
Edit:
First you define a function sub. Now, python knows what sub is.
When you create a variable and try to assign to it (say x = 2), python evaluates the stuff on the right hand side of the = and assigns the value of the evaluation as the value of the stuff on the left hand side of the =. Thus, everything on the right hand side should actually compute.
So if your statement was x = x+1, then x better have a value assigned to it before that line; and the previously defined x has to be of some type compatible with the addition of 1.
But suppose x is a function, and you make a variable called x in some other function, and try to assign to it, a value computed with function x, then this really starts to confuse python about which x you are referring to. This is really an oversimplification of this answer, which does a much better job of explaining variable scope and shadowing in python functions

For every variable used, Python determines whether it is a local or a nonlocal variable. Referencing a unknown variable marks it as nonlocal. Reusing the same name as a local variable later is considered a programmers mistake.
Consider this example:
def err():
print x # this line references x
x = 3 # this line creates a local variable x
err()
This gives you
Traceback (most recent call last):
File "asd.py", line 5, in <module>
err()
File "asd.py", line 2, in err
print x # this line references x
UnboundLocalError: local variable 'x' referenced before assignment
What happens is basically that Python keeps track of all references to names in code. When it reads the line print x Python knows that x is a variable from a outer scope (upvalue or global). However, in x = 3 the x is used as a local variable. As this is a inconsistency in the code, Python raises an UnboundLocalError to get the Programmers attention.

Python start executing your code and get the function first
def sub(a, b):
return a - b
So after executing this interpreter get the sub as a function. Now when come to next line it found
def display():
sub = sub(2,1) // if change to sub1 or sth different to sub, it works
print sub
so first line sub = sub (2, 1) will convert the sub function to sub variable. From this function you are returning the sub variable. So its create problem.