Hole-in-scope, dead code or why such output? - python

Code
def change1(list1):
list1[1] = list1[1] + 5
def change2(number):
number = number + 2
def main():
numbers = [4, 8, 12]
change1(numbers)
variable = 15
change2(variable)
i = 0
while i < 3:
print numbers[i]
i += 1
print variable
main()
When I read it, I thought it will output 4 8 12 15 but it outputs 4 13 12 15. I can see here that Python deals with integer and lists differently, I assumed that the last thing is impossible without global. I cannot understand the output, in such case, why would it not output 4 13 12 17?
You can see here almost identical code with different types and different reference:
$ python test2.py
4
13
12
15
$ python test3.py
4
13
12
17
$ cat test2.py test3.py
Pass-by-reference examples
test2.py: pass-by-reference and mutable data type -example. Table/list is not enough to affect the local variable in main, you need the Reference!
def change1(list1):
list1[1] = list1[1] + 5
def change2(number):
number = [x+2 for x in number]
def main():
numbers = [4, 8, 12]
change1(numbers)
variable = [15]
change2(variable)
i = 0
while i < 3:
print numbers[i]
i += 1
print variable[0]
main()
test3.py: pass-by-reference example, changing a mutable data type list/table outside the main function
def change1(list1):
list1[1] = list1[1] + 5
def change2(number):
number[0] += 2
def main():
numbers = [4, 8, 12]
change1(numbers)
variable = [15]
change2(variable)
i = 0
while i < 3:
print numbers[i]
i += 1
print variable[0]
main()
pass-by-value examples
test4.py: trying to find an example with pass-by-value, why it does not work?
$ cat test4.py
# Not yet a pass-by-value example!
global variable
variable = [15]
def change1(list1):
list1[1] = list1[1] + 5
def change2(number):
number = [x+2 for x in number]
def main():
numbers = [4, 8, 12]
change1(numbers)
#variable = 15
change2(variable)
i = 0
while i < 3:
print numbers[i]
i += 1
print variable[0]
main()
$ python test4.py
4
13
12
15 # I expected 17! Why no 17?

def change1(list1):
# `list1[1] =` means you are changing the object passed in
list1[1] = list1[1] + 5
def change2(number):
# `number = ` means you create a **new local variable**, number,
# based on the `number`you passed in
number = [x+2 for x in number]
So if you want to change existing objects, you have to referene them in some way, for example in
def change3(number):
# `number[:]` is the whole existing list and you overwrite it
number[:] = [x+2 for x in number]
Note the [ .. ] when changing a list.

Python parameters are passed by reference. You mutating only one object in change1.
However, numerical values and Strings are all immutable. You cannot change the value of a passed in immutable and see that value change in the caller. Dictionaries and Lists on the other hand are mutable, and changes made to them by a called function will be preserved when the function returns.
More: http://www.penzilla.net/tutorials/python/functions/

The definitive answer is that Python is actually "call by sharing", also known as "call by object" or "call by object reference".
This has been extensively discussed before. From that article:
From time to time, people who’ve read a little CS but not a lot CS (or too much of just one kind of CS) pop up on comp.lang.python and waste a lot of energy trying to tell everyone that Python’s using some calling model that it doesn’t really use. It always turns out that they don’t really understand Python’s model, and quite often, they don’t understand their favourite model either.
But nevermind, the only thing you need to know is that Python’s model is neither “call by value” nor “call by reference” (because any attempt to use those terms for Python requires you to use non-standard definitions of the words “-value” and “-reference”). The most accurate description is CLU’s “call by object” or “call by sharing“. Or, if you prefer, “call by object reference“.
You should also read this, if you haven’t done so already.
Python's semantics are most similar to the semantics of the language CLU. The CLU Reference Manual by Liskov et al describes the semantics like this:
"We call the argument passing technique call by sharing,
because the argument objects are shared between the
caller and the called routine. This technique does not
correspond to most traditional argument passing techniques
(it is similar to argument passing in LISP). In particular it
is not call by value because mutations of arguments per-
formed by the called routine will be visible to the caller.
And it is not call by reference because access is not given
to the variables of the caller, but merely to certain objects."

In change1 you exchange the value in the list with value + 5.
In change2 you add 5 to number. The result is a new object and is not just applied to the passed variable.
If you come from C++: No there is no int& var in Python.
You get the expected result when doing this:
def change2(number):
return number + 5
variable = 15
variable = change2(variable)
If you still don't want to return a value, you could create a MutableInt class.
class MutableInt(object):
def __init__(self, value = 0):
self._value = int(value)
def __add__(self, other):
self._value += int(other)
return self
def __sub__(self, other):
self._value -= int(other)
return self
...

All the examples show call-by-value. Python only has call-by-value. There is no call-by-reference. All values in python are references (it is not possible to have an "object" as the value). Hence it is references that are copied when passed to the function. Lists are mutable, so it is possible to mutate its contents through a shared reference. In change2 you are reassigning a local variable to point to another object, which, like all assignments to local variables, has no effect on any calling scope, since it is call-by-value.

Related

Is there a way to change a variable within a function if the variable is being passed as an argument?

If I had the code:
num = 3
def add(variable, number):
variable += number
add(num, 2)
print(num)
It would output 3 because inside the function, the temporary variable variable inside the function is being added to instead of the num variable. Is there a way to make it so when I pass num as the first argument, 3 would be added to num instead of variable and num would be changed to 5?
When you call a function with arguments, you pass the arguments value to the function. That function has no connection to the original variable and it cannot be reassigned to a new value.
The correct way to accomplish your goal here is to return a new value:
num = 3
def add(variable, number):
return variable + number
num = add(num, 2)
print(num)
You could write it like this:
num = 3
def add(n1, n2):
return n1 + n2
print(add(num, 2))
The original assignment is not going to update when printing num.
I think you need to explicitly assign the new value to the num variable. For example, you could do:
num = 3
def add(variable, number)
return variable + number
num = add(num, 2)
print(num) #Prints out 5
Your question touches on one of the more esotheric and easy to misunderstand features of Python: mutables and immutables.
Here is the official documentation on mutable and immutable sequence types;
https://docs.python.org/3/library/stdtypes.html#mutable-sequence-types
https://docs.python.org/3/library/stdtypes.html#immutable-sequence-types
As you can tell, only sequences and complex objects are mutable, and thus may be altered by the function.
Base types like integers and floats cannot be changes within the function as you describe, though strings can, as they count as sequences.
In your example, you're using an integer - so this will not work. However, with a string, array, dictionary or complex object (a class of your own design), this _will work.
If you're coming at it from another programming language like C++, think of mutables like references, and immutables like values. If you pass a value, any changes to the object referenced within the scope of the function do not live past the life of the function scope. Any changes to objects with references on them, however, do persist.
This post does an excellent job of explaining this in much more depth, and I would highly recommend skimming it;
https://medium.com/#meghamohan/mutable-and-immutable-side-of-python-c2145cf72747
The purpose of the add function is to add two numbers together which will result in a number. This is something completely different than doing operations on already existing variables. If you want to use variables both inside and outside function, you can use the keyword 'global'. This lets the function know that this is a globally defined variable and you will be able to do operations on it. So you can for example do:
num = 3
def add(number):
global num
num += number
add(2)
print(num)
Which will print 5
If you just want a function to add two numbers and assign this result to the already globally defined variable, you can use:
num = 3
def add(variable, number):
return variable + number
num = add(num, 2)
print(num)
This will also print 5

How to assign value to function parameter in python

I'd like to initialise a variable inside a function (so that the final print statement in my example outputs 10):
def init_param(param):
param = 10
n = 1
init_param(n)
print n # prints 1
Is this doable in python?
Arguments are assigned inside the function as it's local variables. So all principles apply here.
Immutable objects cannot be changed.
Mutable objects can be modified in place.
you're indenting to modify an immutable object, which is not possible. So your only options are :-
def init_param(param):
param = 10
return param
n = 1
n = init_param(n)
print n
which is pretty much useless OR
def init_param(param):
param[0] = 10
n = [1]
init_param(n)
print n
First of all python function passes the value by object and the reference name here param is just a reference to a value hold by n.
Now coming to the solution, yes it could be possible provided you pass the variable name
def init_param(var_name):
globals()[var_name] = 10
n = 1
init_param('n')
print n
Hope it will answer!
Short answer: no, you can't.
Longer answer: in
def init_param(param):
param = 10
the name param is local to the init_param function. Rebinding this name will change the value bound to the name param in the function's local scope, but will have absolutely no effect on the name n in the caller's scope - those names live in totally distinct namespaces. You can read Ned Batcheler's reference article on Python's names and binding for more in-depth explanations.
What would work would be to use a mutable container - a dict for example - and mutate this container, ie:
def init_param(params, name, value):
params[name] = value
params = {
"n": 1,
"answer": 42,
"parrot": "dead"
}
init_params(params, "n", 10)
print(params)
(if you don't understand why this one works, re-read Ned Batcheler's article linked above)
def init_param(param):
param = 10
n = 1
init_param(n)
print n
here n is a integer (immutable data type) so it will be passed by value and so value of n will be unchanged.
lets take a mutable data type (ex. list) then it will be passed by referenced and so values in list will be changed.
def init_param(a):
a[0] = 10
arr = [1]
init_param(arr)
print(arr[0]) # print 10
so you have to check first whether the data is mutable or immutable.
otherwise you can use global keyword to access global variables.
def f():
global n
n = 10
n = 1
f()
print(n) # print 10

Odd threading behavior in python

I have a problem where I need to pass the index of an array to a function which I define inline. The function then gets passed as a parameter to another function which will eventually call it as a callback.
The thing is, when the code gets called, the value of the index is all wrong. I eventually solved this by creating an ugly workaround but I am interested in understanding what is happening here. I created a minimal example to demonstrate the problem:
from __future__ import print_function
import threading
def works_as_expected():
for i in range(10):
run_in_thread(lambda: print('the number is: {}'.format(i)))
def not_as_expected():
for i in range(10):
run_later_in_thread(lambda: print('the number is: {}'.format(i)))
def run_in_thread(f):
threading.Thread(target=f).start()
threads_to_run_later = []
def run_later_in_thread(f):
threads_to_run_later.append(threading.Thread(target=f))
print('this works as expected:\n')
works_as_expected()
print('\nthis does not work as expected:\n')
not_as_expected()
for t in threads_to_run_later: t.start()
Here is the output:
this works as expected:
the number is: 0
the number is: 1
the number is: 2
the number is: 3
the number is: 4
the number is: 6
the number is: 7
the number is: 7
the number is: 8
the number is: 9
this does not work as expected:
the number is: 9
the number is: 9
the number is: 9
the number is: 9
the number is: 9
the number is: 9
the number is: 9
the number is: 9
the number is: 9
the number is: 9
Can someone explain what is happening here? I assume it has to do with enclosing scope or something, but an answer with a reference that explains this dark (to me) corner of python scoping would be valuable to me.
I'm running this on python 2.7.11
This is a result of how closures and scopes work in python.
What is happening is that i is bound within the scope of the not_as_expected function. So even though you're feeding a lambda function to the thread, the variable it's using is being shared between each lambda and each thread.
Consider this example:
def make_function():
i = 1
def inside_function():
print i
i = 2
return inside_function
f = make_function()
f()
What number do you think it will print? The i = 1 before the function was defined or the i = 2 after?
It's going to print the current value of i (i.e. 2). It doesn't matter what the value of i was when the function was made, it's always going to use the current value. The same thing is happening with your lambda functions.
Even in your expected results you can see it didn't always work right, it skipped 5 and displayed 7 twice. What is happening in that case is that each lambda is usually running before the loop gets to the next iteration. But in some cases (like the 5) the loop manages to get through two iterations before control is passed to one of the other threads, and i increments twice and a number is skipped. In other cases (like the 7) two threads manage to run while the loop is still in the same iteration and since i doesn't change between the two threads, the same value gets printed.
If you instead did this:
def function_maker(i):
return lambda: print('the number is: {}'.format(i))
def not_as_expected():
for i in range(10):
run_later_in_thread(function_maker(i))
The i variable gets bound inside function_maker along with the lambda function. Each lambda function will be referencing a different variable, and it will work as expected.
A closure in Python captures the free variables, not their current values at the time of the creation of the closure. For example:
def capture_test():
i = 1
def foo():
return i
def bar():
return i
print(foo(), bar()) # 1 1
i = 2
print(foo(), bar()) # 2 2
In Python you can also capture variables and write to them:
def incdec():
counter = 0
def inc(x):
nonlocal counter
counter += x
return counter
def dec(x):
nonlocal counter
counter -= x
return counter
return inc, dec
i1, d1 = incdec()
i2, d2 = incdec()
print(i1(10), i1(20), d1(3)) # 10 30 27
print(i2(100), d2(5), d2(20)) # 100 95 75
print(i1(7), d2(9)) # 34 66
As you see incdec returns a pair of two closures that captured the same variable and that are incrementing/decrementing it. The variable shared by i1/d1 is however different from the variable shared by i2/d2.
One common mistake is for example to expect that
L = []
for i in range(10):
L.append(lambda : i)
for x in L:
print(x())
will display the numbers from 0 to 9... all of the unnamed closures here captured the same variable i used to loop and all of them will return the same value when called.
The common Python idiom to solve this problem is
L.append(lambda i=i: i)
i.e. using the fact that default values for parameters are evaluated at the time the function is created. With this approach each closure will return a different value because they're returning their private local variable (a parameter that has a default).

Basic Python problems. ((self) parameter, for loop and bubblesort)

#!/usr/bin/python3
class BubbleSort:
def __init__(self):
self.x=[]
self.limit=0
def getElements(self):
self.limit=int(input("Enter the limit:"))
print("Enter {} number".format(self.limit))
for i in range(1,self.limit+1):
self.x.append(int(input()))
def sort(self):
for i in range(0,self.limit):
for j in range(0,(self.limit-1)-i):
if self.x[j+1] < self.x[j]:
temp1=self.x[j+1]
temp2=self.x[j]
del self.x[j]
del self.x[j+1]
self.x.insert(j+1,temp2)
self.x.insert(j,temp1)
print(self.x)
print("Sorted list is")
for i in self.x:
print(i)
def main():
b=BubbleSort()
b.getElements()
b.sort()
if __name__=="__main__":main()
This is a simple bubble sort program.
Problem 1: If I run the program, two same numbers come , for example, when i enter say -> 3 6 5 1 2
output -> 1 2 2 5 6
The 3 gets replaced by 2.
2: why do I see a lot of 'self' as parameter to a function in python? then what does self.x=[] and self.limit=0 do?
I am super new to methods as a concept and I tried reading, they do not help.
3.In the function getElements, what is self.limit ? why is it even required? we can just use a normal variable like 'x'
Ex: x= int(input("enter the limit:"))
print("enter {} number".format(x))
3.Explain the self.x.append(int(input())))
Doesnt append(int(something)) will add the value of something in the end of a list?
Check the for loop, say the self.limit is 5(entered value). then the loop requires 6 numbers right? equivalence in c++
for(i=1;i<=6;i++)
Assuming the left most number is not considered in python in the range function. right?
Mainly explain 'self' parameter in every function.
When you delete x[j], x[j+1] becomes x[j]. Replace:
del self.x[j]
del self.x[j+1]
with:
del self.x[j+1]
del self.x[j]
However, like #IanAuld said, deleting elements of an array while iterating over it is poor programming practice. Instead, try making a new array and copying the elements to the new array, leaving the original array unchanged in the process. Afterward, if you want, you can replace the old array with the new array.
And to answer your second question, self refers to the class so you can access other variables from the same class your function is in. For example, if you have:
#!/usr/bin/python3
class Foo:
bar = 3
def getValueOfBar(self):
return self.bar
x = Foo()
print( x.getValueOfBar() )
self refers to the parent class, so you can retrieve variables from the class that aren't in the scope of the function.

Confusion regarding mutable and immutable data types in Python 2.7

I have read that while writing functions it is good practice to copy the arguments into other variables because it is not always clear whether the variable is immutable or not. [I don't remember where so don't ask]. I have been writing functions according to this.
As I understand creating a new variable takes some overhead. It may be small but it is there. So what should be done? Should I be creating new variables or not to hold the arguments?
I have read this and this. I have confusion regarding as to why float's and int's are immutable if they can be changed this easily?
EDIT:
I am writing simple functions. I'll post example. I wrote the first one when after I read that in Python arguments should be copied and the second one after I realized by hit-and-trial that it wasn't needed.
#When I copied arguments into another variable
def zeros_in_fact(num):
'''Returns the number of zeros at the end of factorial of num'''
temp = num
if temp < 0:
return 0
fives = 0
while temp:
temp /= 5
fives += temp
return fives
#When I did not copy arguments into another variable
def zeros_in_fact(num):
'''Returns the number of zeros at the end of factorial of num'''
if num < 0:
return 0
fives = 0
while num:
num /= 5
fives += num
return fives
I think it's best to keep it simple in questions like these.
The second link in your question is a really good explanation; in summary:
Methods take parameters which, as pointed out in that explanation, are passed "by value". The parameters in functions take the value of variables passed in.
For primitive types like strings, ints, and floats, the value of the variable is a pointer (the arrows in the following diagram) to a space in memory that represents the number or string.
code | memory
|
an_int = 1 | an_int ----> 1
| ^
another_int = 1 | another_int /
When you reassign within the method, you change where the arrow points.
an_int = 2 | an_int -------> 2
| another_int --> 1
The numbers themselves don't change, and since those variables have scope only inside the functions, outside the function, the variables passed in remain the same as they were before: 1 and 1. But when you pass in a list or object, for example, you can change the values they point to outside of the function.
a_list = [1, 2, 3] | 1 2 3
| a_list ->| ^ | ^ | ^ |
| 0 2 3
a_list[0] = 0 | a_list ->| ^ | ^ | ^ |
Now, you can change where the arrows in the list, or object, point to, but the list's pointer still points to the same list as before. (There should probably actually only be one 2 and 3 in the diagram above for both sets of arrows, but the arrows would have gotten difficult to draw.)
So what does the actual code look like?
a = 5
def not_change(a):
a = 6
not_change(a)
print(a) # a is still 5 outside the function
b = [1, 2, 3]
def change(b):
b[0] = 0
print(b) # b is now [0, 2, 3] outside the function
Whether you make a copy of the lists and objects you're given (ints and strings don't matter) and thus return new variables or change the ones passed in depends on what functionality you need to provide.
What you are doing in your code examples involves no noticeable overhead, but it also doesn't accomplish anything because it won't protect you from mutable/immutable problems.
The way to think about this is that there are two kinds of things in Python: names and objects. When you do x = y you are operating on a name, attaching that name to the object y. When you do x += y or other augmented assignment operators, you also are binding a name (in addition to doing the operation you use, + in this case). Anything else that you do is operating on objects. If the objects are mutable, that may involve changing their state.
Ints and floats cannot be changed. What you can do is change what int or float a name refers to. If you do
x = 3
x = x + 4
You are not changing the int. You are changing the name x so that it now is attached to the number 7 instead of the number 3. On the other hand when you do this:
x = []
x.append(2)
You are changing the list, not just pointing the name at a new object.
The difference can be seen when you have multiple names for the same object.
>>> x = 2
>>> y = x
>>> x = x + 3 # changing the name
>>> print x
5
>>> print y # y is not affected
2
>>> x = []
>>> y = x
>>> x.append(2) # changing the object
>>> print x
[2]
>>> print y # y is affected
[2]
Mutating an object means that you alter the object itself, so that all names that point to it see the changes. If you just change a name, other names are not affected.
The second question you linked to provides more information about how this works in the context of function arguments. The augmented assignment operators (+=, *=, etc.) are a bit trickier since they operate on names but may also mutate objects at the same time. You can find other questions on StackOverflow about how this works.
If you are rebinding the name then mutability of the object it contains is irrelevant. Only if you perform mutating operations must you create a copy. (And if you read between the lines, that indirectly says "don't mutate objects passed to you".)

Categories

Resources