Loop Variables Overwrite Globals - python

In python, why do loop variables overwrite already defined global variables? It seems counterintuitive that a loop variable is put into the module's global namespace rather than a new local namespace just for the loop. Here's an example that shows what I'm talking about:
c = 3.14
print("before loop c = {}".format(c))
for c in range(3):
print("in loop c = {}".format(c))
print("after loop c = {}".format(c))
Running this code will print out:
before loop c = 3.14
in loop c = 0
in loop c = 1
in loop c = 2
after loop c = 2
Reusing names like this is almost definitely bad coding style, but it may happen accidentally in large modules and cause globals to be defined where you don't expect them to be. For example if I were to do this:
def f(x):
print("x is {}".format(c)) # finger slipped, wrote c instead of x
for c in range(3):
print("c is {}".format(c))
for a in "test":
f(a)
I would get:
c is 0
c is 1
c is 2
x is 2
x is 2
x is 2
x is 2
This question seems to indicate that for loops don't have their own namespace by design. What is the rationale behind this when it can cause bugs depending on when loops are run relative to the execution order of the program?

Related

I am having problems with booleans staying as I define them [duplicate]

Consider this example:
def A():
b = 1
def B():
# I can access 'b' from here.
print(b)
# But can i modify 'b' here?
B()
A()
For the code in the B function, the variable b is in a non-global, enclosing (outer) scope. How can I modify b from within B? I get an UnboundLocalError if I try it directly, and using global does not fix the problem since b is not global.
Python implements lexical, not dynamic scope - like almost all modern languages. The techniques here will not allow access to the caller's variables - unless the caller also happens to be an enclosing function - because the caller is not in scope. For more on this problem, see How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?.
On Python 3, use the nonlocal keyword:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
def foo():
a = 1
def bar():
nonlocal a
a = 2
bar()
print(a) # Output: 2
On Python 2, use a mutable object (like a list, or dict) and mutate the value instead of reassigning a variable:
def foo():
a = []
def bar():
a.append(1)
bar()
bar()
print a
foo()
Outputs:
[1, 1]
You can use an empty class to hold a temporary scope. It's like the mutable but a bit prettier.
def outer_fn():
class FnScope:
b = 5
c = 6
def inner_fn():
FnScope.b += 1
FnScope.c += FnScope.b
inner_fn()
inner_fn()
inner_fn()
This yields the following interactive output:
>>> outer_fn()
8 27
>>> fs = FnScope()
NameError: name 'FnScope' is not defined
I'm a little new to Python, but I've read a bit about this. I believe the best you're going to get is similar to the Java work-around, which is to wrap your outer variable in a list.
def A():
b = [1]
def B():
b[0] = 2
B()
print(b[0])
# The output is '2'
Edit: I guess this was probably true before Python 3. Looks like nonlocal is your answer.
No you cannot, at least in this way.
Because the "set operation" will create a new name in the current scope, which covers the outer one.
I don't know if there is an attribute of a function that gives the __dict__ of the outer space of the function when this outer space isn't the global space == the module, which is the case when the function is a nested function, in Python 3.
But in Python 2, as far as I know, there isn't such an attribute.
So the only possibilities to do what you want is:
1) using a mutable object, as said by others
2)
def A() :
b = 1
print 'b before B() ==', b
def B() :
b = 10
print 'b ==', b
return b
b = B()
print 'b after B() ==', b
A()
result
b before B() == 1
b == 10
b after B() == 10
.
Nota
The solution of Cédric Julien has a drawback:
def A() :
global b # N1
b = 1
print ' b in function B before executing C() :', b
def B() :
global b # N2
print ' b in function B before assigning b = 2 :', b
b = 2
print ' b in function B after assigning b = 2 :', b
B()
print ' b in function A , after execution of B()', b
b = 450
print 'global b , before execution of A() :', b
A()
print 'global b , after execution of A() :', b
result
global b , before execution of A() : 450
b in function B before executing B() : 1
b in function B before assigning b = 2 : 1
b in function B after assigning b = 2 : 2
b in function A , after execution of B() 2
global b , after execution of A() : 2
The global b after execution of A() has been modified and it may be not whished so
That's the case only if there is an object with identifier b in the global namespace
The short answer that will just work automagically
I created a python library for solving this specific problem. It is released under the unlisence so use it however you wish. You can install it with pip install seapie or check out the home page here https://github.com/hirsimaki-markus/SEAPIE
user#pc:home$ pip install seapie
from seapie import Seapie as seapie
def A():
b = 1
def B():
seapie(1, "b=2")
print(b)
B()
A()
outputs
2
the arguments have following meaning:
The first argument is execution scope. 0 would mean local B(), 1 means parent A() and 2 would mean grandparent <module> aka global
The second argument is a string or code object you want to execute in the given scope
You can also call it without arguments for interactive shell inside your program
The long answer
This is more complicated. Seapie works by editing the frames in call stack using CPython api. CPython is the de facto standard so most people don't have to worry about it.
The magic words you are probably most likely interesed in if you are reading this are the following:
frame = sys._getframe(1) # 1 stands for previous frame
parent_locals = frame.f_locals # true dictionary of parent locals
parent_globals = frame.f_globals # true dictionary of parent globals
exec(codeblock, parent_globals, parent_locals)
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),ctypes.c_int(1))
# the magic value 1 stands for ability to introduce new variables. 0 for update-only
The latter will force updates to pass into local scope. local scopes are however optimized differently than global scope so intoducing new objects has some problems when you try to call them directly if they are not initialized in any way. I will copy few ways to circumvent these problems from the github page
Assingn, import and define your objects beforehand
Assingn placeholder to your objects beforehand
Reassign object to itself in main program to update symbol table: x = locals()["x"]
Use exec() in main program instead of directly calling to avoid optimization. Instead of calling x do: exec("x")
If you are feeling that using exec() is not something you want to go with you can
emulate the behaviour by updating the the true local dictionary (not the one returned by locals()). I will copy an example from https://faster-cpython.readthedocs.io/mutable.html
import sys
import ctypes
def hack():
# Get the frame object of the caller
frame = sys._getframe(1)
frame.f_locals['x'] = "hack!"
# Force an update of locals array from locals dict
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),
ctypes.c_int(0))
def func():
x = 1
hack()
print(x)
func()
Output:
hack!
I don't think you should want to do this. Functions that can alter things in their enclosing context are dangerous, as that context may be written without the knowledge of the function.
You could make it explicit, either by making B a public method and C a private method in a class (the best way probably); or by using a mutable type such as a list and passing it explicitly to C:
def A():
x = [0]
def B(var):
var[0] = 1
B(x)
print x
A()
For anyone looking at this much later on a safer but heavier workaround is. Without a need to pass variables as parameters.
def outer():
a = [1]
def inner(a=a):
a[0] += 1
inner()
return a[0]
You can, but you'll have to use the global statment (not a really good solution as always when using global variables, but it works):
def A():
global b
b = 1
def B():
global b
print( b )
b = 2
B()
A()

python scoping rules for nested functions [duplicate]

Consider this example:
def A():
b = 1
def B():
# I can access 'b' from here.
print(b)
# But can i modify 'b' here?
B()
A()
For the code in the B function, the variable b is in a non-global, enclosing (outer) scope. How can I modify b from within B? I get an UnboundLocalError if I try it directly, and using global does not fix the problem since b is not global.
Python implements lexical, not dynamic scope - like almost all modern languages. The techniques here will not allow access to the caller's variables - unless the caller also happens to be an enclosing function - because the caller is not in scope. For more on this problem, see How can I access variables from the caller, even if it isn't an enclosing scope (i.e., implement dynamic scoping)?.
On Python 3, use the nonlocal keyword:
The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals. This is important because the default behavior for binding is to search the local namespace first. The statement allows encapsulated code to rebind variables outside of the local scope besides the global (module) scope.
def foo():
a = 1
def bar():
nonlocal a
a = 2
bar()
print(a) # Output: 2
On Python 2, use a mutable object (like a list, or dict) and mutate the value instead of reassigning a variable:
def foo():
a = []
def bar():
a.append(1)
bar()
bar()
print a
foo()
Outputs:
[1, 1]
You can use an empty class to hold a temporary scope. It's like the mutable but a bit prettier.
def outer_fn():
class FnScope:
b = 5
c = 6
def inner_fn():
FnScope.b += 1
FnScope.c += FnScope.b
inner_fn()
inner_fn()
inner_fn()
This yields the following interactive output:
>>> outer_fn()
8 27
>>> fs = FnScope()
NameError: name 'FnScope' is not defined
I'm a little new to Python, but I've read a bit about this. I believe the best you're going to get is similar to the Java work-around, which is to wrap your outer variable in a list.
def A():
b = [1]
def B():
b[0] = 2
B()
print(b[0])
# The output is '2'
Edit: I guess this was probably true before Python 3. Looks like nonlocal is your answer.
No you cannot, at least in this way.
Because the "set operation" will create a new name in the current scope, which covers the outer one.
I don't know if there is an attribute of a function that gives the __dict__ of the outer space of the function when this outer space isn't the global space == the module, which is the case when the function is a nested function, in Python 3.
But in Python 2, as far as I know, there isn't such an attribute.
So the only possibilities to do what you want is:
1) using a mutable object, as said by others
2)
def A() :
b = 1
print 'b before B() ==', b
def B() :
b = 10
print 'b ==', b
return b
b = B()
print 'b after B() ==', b
A()
result
b before B() == 1
b == 10
b after B() == 10
.
Nota
The solution of Cédric Julien has a drawback:
def A() :
global b # N1
b = 1
print ' b in function B before executing C() :', b
def B() :
global b # N2
print ' b in function B before assigning b = 2 :', b
b = 2
print ' b in function B after assigning b = 2 :', b
B()
print ' b in function A , after execution of B()', b
b = 450
print 'global b , before execution of A() :', b
A()
print 'global b , after execution of A() :', b
result
global b , before execution of A() : 450
b in function B before executing B() : 1
b in function B before assigning b = 2 : 1
b in function B after assigning b = 2 : 2
b in function A , after execution of B() 2
global b , after execution of A() : 2
The global b after execution of A() has been modified and it may be not whished so
That's the case only if there is an object with identifier b in the global namespace
The short answer that will just work automagically
I created a python library for solving this specific problem. It is released under the unlisence so use it however you wish. You can install it with pip install seapie or check out the home page here https://github.com/hirsimaki-markus/SEAPIE
user#pc:home$ pip install seapie
from seapie import Seapie as seapie
def A():
b = 1
def B():
seapie(1, "b=2")
print(b)
B()
A()
outputs
2
the arguments have following meaning:
The first argument is execution scope. 0 would mean local B(), 1 means parent A() and 2 would mean grandparent <module> aka global
The second argument is a string or code object you want to execute in the given scope
You can also call it without arguments for interactive shell inside your program
The long answer
This is more complicated. Seapie works by editing the frames in call stack using CPython api. CPython is the de facto standard so most people don't have to worry about it.
The magic words you are probably most likely interesed in if you are reading this are the following:
frame = sys._getframe(1) # 1 stands for previous frame
parent_locals = frame.f_locals # true dictionary of parent locals
parent_globals = frame.f_globals # true dictionary of parent globals
exec(codeblock, parent_globals, parent_locals)
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),ctypes.c_int(1))
# the magic value 1 stands for ability to introduce new variables. 0 for update-only
The latter will force updates to pass into local scope. local scopes are however optimized differently than global scope so intoducing new objects has some problems when you try to call them directly if they are not initialized in any way. I will copy few ways to circumvent these problems from the github page
Assingn, import and define your objects beforehand
Assingn placeholder to your objects beforehand
Reassign object to itself in main program to update symbol table: x = locals()["x"]
Use exec() in main program instead of directly calling to avoid optimization. Instead of calling x do: exec("x")
If you are feeling that using exec() is not something you want to go with you can
emulate the behaviour by updating the the true local dictionary (not the one returned by locals()). I will copy an example from https://faster-cpython.readthedocs.io/mutable.html
import sys
import ctypes
def hack():
# Get the frame object of the caller
frame = sys._getframe(1)
frame.f_locals['x'] = "hack!"
# Force an update of locals array from locals dict
ctypes.pythonapi.PyFrame_LocalsToFast(ctypes.py_object(frame),
ctypes.c_int(0))
def func():
x = 1
hack()
print(x)
func()
Output:
hack!
I don't think you should want to do this. Functions that can alter things in their enclosing context are dangerous, as that context may be written without the knowledge of the function.
You could make it explicit, either by making B a public method and C a private method in a class (the best way probably); or by using a mutable type such as a list and passing it explicitly to C:
def A():
x = [0]
def B(var):
var[0] = 1
B(x)
print x
A()
For anyone looking at this much later on a safer but heavier workaround is. Without a need to pass variables as parameters.
def outer():
a = [1]
def inner(a=a):
a[0] += 1
inner()
return a[0]
You can, but you'll have to use the global statment (not a really good solution as always when using global variables, but it works):
def A():
global b
b = 1
def B():
global b
print( b )
b = 2
B()
A()

Remember Array value after Function call

If I write this:
c = []
def cf(n):
c = range (5)
print c
if any((i>3) for i in c) is True:
print 'hello'
cf(1)
print c
Then I get:
[1, 2, 3, 4]
hello
[]
I'm really new to programming, so please explain it really simply, but how do I stop Python from forgetting what c is after the function has ended? I thought I could fix it by defining c before the function, but obviously that c is different to the one created just for the function loop.
In my example, I could obviously just write:
c = range (5)
def cf(n)
But the program I'm trying to write is more like this:
b = [blah]
c = []
def cf(n):
c = [transformation of b]
if (blah) is True:
'loop' cf
else:
cf(1)
g = [transformation of c that produces errors if c is empty or if c = b]
So I can't define c outside the function.
In python you can read global variables in functions, but you cant assigned to them by default. the reason is that whenever python finds c = it will create a local variable. Thus to assign to global one, you need explicitly specify that you are assigning to global variable.
So this will work, e.g.:
c = [1,2,3]
def cf():
print(c) # it prints [1,2,3], it reads global c
However, this does not as you would expect:
c = [1,2,3]
def cf():
c = 1 # c is local here.
print(c) # it prints 1
cf()
print(c) # it prints [1,2,3], as its value not changed inside cf()
So to make c be same, you need:
c = [1,2,3]
def cf():
global c
c = 1 # c is global here. it overwrites [1,2,3]
print(c) # prints 1
cf()
print(c) # prints 1. c value was changed inside cf()
To summarise a few of these answers, you have 3 basic options:
Declare the variable as global at the top of your function
Return the local instance of the variable at the end of your function
Pass the variable as an argument to your function
You can also pass the array c into the function after declaring it. As the array is a function argument the c passed in will be modified as long as we don't use an = statement. This can be achieved like this:
def cf(n, c):
c.extend(range(5))
print c
if any((i>3) for i in c) is True:
print 'hello'
if __name__ == '__main__':
c = []
cf(1, c)
print c
For an explanation of this see this
This is preferable to introducing global variables into your code (which is generally considered bad practice). ref
Try this
c = []
def cf(n):
global c
c = range (5)
print c
if any((i>3) for i in c) is True:
print 'hello'
cf(1)
print c
If you want your function to modify c then make it explicit, i.e. your function should return the new value of c. This way you avoid unwanted side effects:
def cf(n, b):
"""Given b loops n times ...
Returns
------
c: The modified value
"""
c = [transformation of b]
...
return c # <<<<------- This
c = cf(1)

Python: variables not cumulative passing through functions

I am writing a function for a text adventure I'm making that acts as a progress bar when the player receives experience (which then levels the player up upon reaching 100). For some reason altering the values of my variables inside the function does not change their values outside the function even though I've returned all three. This becomes apparent when I try calling the function twice, each time adding 85 to the variable a.
Within the function, the objective is to pass the value of a to b, check if b is greater than or equal to 100, if so add 1 to c and remove 100 from b, then reset a to 0, print the result, and return the new values.
a = new experience b = current experience c = player level
a = 0
b = 0
c = 1
def func_1(a, b, c):
b = b + a
loop1 = 0
while loop1 < 1:
if b >= 100:
print(" ")
print("Add 1!")
print(" ")
c = c + 1
b = b - 100
else:
loop1 = loop1 + 1
a = a - a
print("c is " + str(c))
print("b is " + str(b))
print("a is " + str(a))
return a
return b
return c
a = a + 85
func_1(a, b, c)
a = a + 85
func_1(a, b, c)
print a, b, c
I'm really new to programming so I apologize for any inefficiencies. Thank you for any help and let me know if something doesn't make sense/is unclear.
Couple of things I see here:
First, out of the three return statements at the end of your code, only the first, return a, is executed. A return statement immediately stops execution of the function; return b and return c are never touched.
Second, you're having some confusion with the scope of the variables. a, b, and c defined outside of the function are global variables, while the a, b, and c passed into the function are local to the scope of the function. Thus, any modifications to the variables in your function won't affect the global variables.
You can do two things. First, have a global declaration at the beginning of the function:
def func_1():
global a
global b
global c
# Function body
This indicates to the function that a, b, and c are global variables that exist outside the function. Thus, passing in arguments is no longer needed; the function will search for variables outside the function. However, this is bad programming practice. The better option is to have func_1 return the modified values and reassign the global values to these new values, like so:
def func_1(a, b, c):
# Function body
return (a, b, c)
Then, to call the function and modify the variables outside the function:
a, b, c = func_1(a, b, c)
A couple suggestions:
You have a lot of incrementing going on, and python has specialized syntax for it: b = b + a can be shortened to b += a, c = c + 1 to c += 1, b = b - 100 to b -= 100. Also, simply reset a to 0 with a = 0 instead of subtracting a - a. Also, you don't need print(" "); print() will suffice.
Next, your while loop is unnecessary. Your function only needs to check once whether b >= 100, so there's no need to set up a loop. Increment b and then use a simple if statement to level up if necessary:
def func_1(a, b, c):
b += a
if b >= 100:
print("\nAdd 1!\n")
c += 1
b -= 100
a = 0
# Rest of the function
Inside func_1() the names a,b,c are local. When you change them nothing happens to the external a,b,c. You return the values correctly in the function, but then when calling the function you need to assign the values to the variables like this: a,b,c=func_1(a,b,c).
Returning a value doesn't set it unless you explicitly assign it in the calling function:
a, b, c = func_1(a, b, c)
Assigning inside the function doesn't affect the outer ones because they are considered "local scope". To counter that declare them global to affect the outer variables:
def func_1():
global a
global b
global c
Only one of these should be implemented
It is generally preferred not to declare globals. Ideally you should make a class for all of this, but these two options would require the least refactoring of your existing code
Note that a global variable (the variables outside of the function) are completely separate from the local variables (the variables inside the function).
This doesn't work because you can only return once. When you returned a, the function immediately stopped.
Also, since you didn't set any variable to the returned value, the global variables outside of the loop were unaffected. What you can do is return the a, b, and c values as a tuple, and then set the a, b and c global variables to that tuple.
return (a, b, c)
a, b, c = func_1(a, b, c)

Can anyone help me understand Python variable scoping?

I wrote a test program that looked like this:
#!/usr/bin/python
def incrementc():
c = c + 1
def main():
c = 5
incrementc()
main()
print c
I'd think that since I called incrementc within the body of main, all variables from main would pass to incrementc. But when I run this program I get
Traceback (most recent call last):
File "test.py", line 10, in <module>
main()
File "test.py", line 8, in main
incrementc()
File "test.py", line 4, in incrementc
c = c + 1
UnboundLocalError: local variable 'c' referenced before assignment
Why isn't c passing through? And if I want a variable to be referenced by multiple functions, do I have to declare it globally? I read somewhere that global variables are bad.
Thanks!
You're thinking of dynamic scoping. The problem with dynamic scoping is that the behavior of incrementc would depend on previous function calls, which makes it very difficult to reason about the code. Instead most programming languages (also Python) use static scoping: c is visible only within main.
To accomplish what you want, you'd either use a global variable, or, better, pass c as a parameter. Now, because the primitives in Python are immutable, passing an integer can't be changed (it's effectively passed by value), so you'd have to pack it into a container, like a list. Like this:
def increment(l):
l[0] = l[0] + 1
def main():
c = [5]
increment(c)
print c[0]
main()
Or, even simpler:
def increment(l):
return l + 1
def main():
c = 5
print increment(c)
main()
Generally, global variables are bad because they make it very easy to write code that's hard to understand. If you only have these two functions, you can go ahead and make c global because it's still obvious what the code does. If you have more code, it's better to pass the variables as a parameter instead; this way you can more easily see who depends on the global variable.
When a variable is assigned to in a scope, Python assumes it's local for the whole scope unless you tell it otherwise.
So, to get this to work as you think it will, you need to use two global statements:
#!/usr/bin/python
def incrementc():
global c
c = c + 1
def main():
global c
c = 5
incrementc()
main()
print c
Otherwise, you're talking about a local variable named c in both situations.
The normal way to solve this, however, does not involve globals.
#!/usr/bin/python
def incrementc(c):
c = c + 1
return c
def main():
c = 5
c = incrementc(c)
return c
c = main()
print c
Here, in each function and in the global scope, c refers to a different variable, which you are passing around as an argument and with return values. If you wanted only one c, use a class:
class Foo:
def __init__(self, c):
self.c = c
self.incrementc()
def incrementc(self):
self.c = self.c + 1
foo = Foo(5)
print foo.c
The variable c isn't passing through because you do not hand any reference to c to the function incrementc.
What you're looking at here are 3 scopes, the global scope and those within the functions main and incrementc. In main you've properly defined a variable c, but increment c has no knowledge of this - so attempting to increment it is going to fail. Even if those two functions succeeded, trying to print c would fail in the global scope because it has no knowledge of the c you've defined in main.
You have a few options. One way to do this:
def incrementc(c):
c = c + 1
return c
def main():
c = 5
c = incrementc(c)
return c
c = main()
print c
Notice how c is being handed around. Of course, the name doesn't have to be preserved, you very well could write it like this:
def increment(z):
z = z + 1
return z
def main():
bar = 5
bar = increment(bar)
return bar
foo = main()
print foo
Another option that many would probably dislike (for good reason) is to use globals. In that case:
def incrementc():
global c # indicate intention to write to this global, not just read it
c = c + 1
def main():
global c # declares c in global space
c = 5
incrementc()
main()
print c
Any function in which you hope to MODIFY the GLOBAL instance of c, you need to inform the function. So you state, 'global c'. You can READ from the global without doing so. This would ensure (to some extent) that you don't make a mistake and overwrite a value in the global space unintentionally with a similar name, should you decide to use one in the local space of a function.
Hopefully that's clear enough, but feel free to ask for clarification on any point (I'm also open to being corrected if I've mistakenly described any part of this).
Global variables are bad.
Just like friends and enemys. Keep your friends close but keep your enemys even closer.
The function main last a local variable c, assignment the value 5
You then call the function inc..C. The c from main is now out of scope so you are trying to use a value of c that is not in scope - hence the error.

Categories

Resources