I understand that:
a=[lambda :k for k in [1,2,3]]
[e() for e in a]
returns [3, 3, 3] because it takes the value of k at runtime, and the last value of k is 3. However, I don't understand why, if I do:
k=50
[e() for e in a]
I still get [3, 3, 3]. Why this? I updated the value of k with 50, why do(es) the e() function(s) still read the old k value?
Although you just said it is clear, that
a=[lambda :k for k in [1,2,3]]
[e() for e in a]
returns [3,3,3], this needs a few words. And for sure this is not clear for everybody. The first list comprehensions creates three functions. They all take no arguments and they all return the variable k. Inside the list comprehension k is not a number. K is a variable. And your three functions return that variable. So you have three identical functions. Normally the k variable would be lost. It scopes only withing the list comprehension. You cannot access it any more outside the list comprehension. But in your special case k still lives. It is still captured in the three functions, because these return it. This is what is called a closure (of k). And as k was for iterating through the list [1,2,3] at the end it still is 3. So all three functions return 3. But that is not so obvious. Python could also set it back to zero again or a random number. There is no such statement that k remains at the last value. So even if your code returns [3, 3, 3], that is not obvious and that is not guaranteed.
As said k is not accessible any more outside the list comprehension. It is in the scope of the list comprehension only. Only with the three a-functions you have ever access to that k. And on the other hand, you cannot tell the a-funcions to use another k. The k from the list comprehension is in their closure. The k=50 has a completely different scope.
If you want the other k in the a-functions, you must define them this way:
a=[lambda k : k_compr for k_kompr in [1,2,3]]
[e() for e in a]
and pass the k to them. But I think you question is just informatory. Nobody would ever use such a list comrehension.
Related
While trying to implement an algorithm, I couldn't get python lists to mutate via a function. After reading up on the issue I was suggested by this StackOverflow answer to use [:] in order to mutate the array passed in the function argumemt.
However, as seen in the following code snippet, the issue still persists when trying to mutate the list l. I am expecting the output to be Before: [1,2,3,4]
After: [69, 69, 69, 69], but instead I get back the original value of l as shown below.
def mutate_list(a, b):
c = [69] * 4
a[:] = c[:2] # changed the elements, but array's still unchanged outside function
b[:] = c[2:]
if __name__ == '__main__':
l = [1, 2, 3, 4]
print("Before: {}" .format(l))
mutate_list(l[:2], l[2:])
print("After: {}" .format(l))
Output:
Before: [1, 2, 3, 4]
After : [1, 2, 3, 4]
Any insights into why this is happening?
The error is that you not pass actually the l but two slices of it. You should change it, for example:
def mutate_list(a):
c = [69] * 4
a[:2] = c[:2]
a[2:] = c[2:]
if __name__ == '__main__':
l = [1, 2, 3, 4]
print("Before: {}" .format(l))
mutate_list(l)
print("After: {}" .format(l))
its all about the scope, mutable concept is applicable on list but not to reference variable.
The variables a,b are local variables, hence the scope of the variable will be always function scope.
The operations which you have performed :
a[:]=c[:2]
b[:]=c[2:]
Note: a and b both are list now so you will get following output in the function:
[69,69],[69,69]
but if you use + operator which is use for adding operations then the out out will be like:
[69,69,69,69]
Now whatever I told you that will be a local scope, if you want that the list should be mutable across the program then you have to specify the scope of the list as global inside function and on that variable you can do changes. in this case you also dont need to pass any arguments:
def mutate_list():
global l # telling python to use this global variable in a local function
c = [69] * 4
l=c # assigning new values to actual list i.e l
Now before output will be [1,2,3,4]
and after will be [69,69,69,69]
As pointed out by others, the issue arose from the fact that the function parameters were slices of the original array and as such, the parameters were being passed by value (instead of being passed by reference).
According to #Selcuk 's suggestion, the correct way of doing such an operation would be to pass the original array along with its indices to the function and then perform any slicing inside the function.
NOTE: This concept comes in handy for (recursive) divide-and-conquer algorithms where subarrays must be mutated and combined to form the solution.
I'm doing my programming coursework, and I've come across an issue
gamecentre1 = [winnerscore, winner]
organise = []
organise.extend([gamecentre1])
from operator import attrgetter, itemgetter
sorted(organise, key= itemgetter(0))
print(organise)
f = open("gameresults.txt","w")
f.write("Here are the winners of the games: \n")
f.write(str(organise))
f.write("\n")
f.close()
I'm trying to add two variables to a list, and add that list to another list. Then, I want to organise that larger list based off of the integer variable of the sublist (the integer is the winnerscore). But, the problem is that I haven't been able to organise them properly, and I worry that since I have to append the same list with the same variables to the larger list without overwriting the existing list in it, the larger list will just have the same values over and over again.
This is because I want to store the variables every time the program runs, without getting rid of the values of the variable from the previous game.
How do I do this?
I'm trying to add two variables to a list, and add that list to another list. Then, I want to organise that larger list based off of the integer variable of the sublist
It sounds like what you want is a nested list - a big list of small lists, such that each small list is independent of the others. The key problem in your code right now that's blocking you from accomplishing this is extend() - this is essentially list concatenation, which isn't what you want. For example,
x = [1, 2]
x.extend([3, 4]) # x == [1, 2, 3, 4]
Instead, try using append(), which adds its argument as the next value in the list. For example:
x = []
x.append([3, 4]) # [[3, 4]]
x.append([1, 2]) # [[3, 4], [1, 2]]
Now in the above example, we have a list x that's two elements long, and each of those elements is itself a list of two elements. Now we can plug that into sort:
y = sorted(x, key=lambda i: i[0])
print(y)
# [[1, 2], [3, 4]]
(the lambda i: i[0] is really just a more elegant way of what you're doing with itemgettr(0) - it's a lambda, a small inline function, that takes one argument i and returns the 0th element of i. In this case, the i that gets passed in is one of the smaller lists).
Now you have a sorted list of smaller lists, and you can do whatever you need with them.
for practicing purposes, I tried to implement a function that receives two lists as parameters and returns the difference of them. So basically the elements which are the lists have not in common.
I coded the following functions:
list1 = [4,2,5,3,9,11]
list2 = [7,9,2,3,5,1]
def difference(list1,list2):
return (list(set(list1) - set(list2)))
difference(list1,list2)
AND
def difference_extra_credit(list1,list2):
return [value for value in list1 if value not in list2]
difference(list1,list2)
--> Basically both codes seem to work but I'm currently facing the problem that the lists need to have the same length in order for the functions to work. If the length is not the same, adding for instance an integer of 100 to list 1, it would not be shown as a difference between the lists if you print the functions.
I didn't manage to find a way to modify the code so that the length of the lists doesn't matter.. Does someone has an idea?
Thanks!
If you want symmetric difference, use the ^ operator instead of -
def difference(list1, list2):
return list(set(list1) ^ set(list2))
Here are the four set operators that combine two sets into one set.
| union : elements in one or both of the sets
& intersection : only elements common to both sets
- difference : elements in the left hand set that are not in the right hand set
^ symmetric difference : elements in either set but not in both.
I think this is a more readable way of writing the function
def symmetric_difference(a, b):
return {*a} ^ {*b}
(* unpacking in set literals requires python 3.5 or later)
Returning a set instead of a list makes it a bit more clear what the function does. The input arguments can be any iterable types, and since set is an unordered data type, returning a set makes it obvious that any ordering in the input data was not preserved.
>>> symmetric_difference(range(3, 8), [1,2,3,4])
{1, 2, 5, 6, 7}
>>> symmetric_difference('hello', 'world')
{'d', 'e', 'h', 'r', 'w'}
your both versions aren't symmetrical: if you exchange list1 and list2, the result won't be the same.
If you add a number in list2 (not in list1 as your question states), it's not seen as a difference, whereas it is one.
You want to perform a symmetric difference, so no matter the data in both lists (swapped or not) the result remains the same
def difference(list1,list2):
return list(set(list1).symmetric_difference(list2))
with your data:
[1, 4, 7, 11]
Trying out your code, it seemed to work fine with me regardless of the length of the lists - when I added 100 to list1, it showed up for both difference functions.
However, there appear to be a few issues with your code that could be causing the problems. Firstly, you accept arguments list1 and list2 for both functions, but these variables are the same name as your list variables. This seems not to cause an issue, but it means that the global variables are no longer accessible, and it is generally a better practice to avoid confusion by using different names for global variables and variables within functions.
Additionally, your function does not take the symmetric difference - it only loops over the variables in the first list, so unique variables in the second list will not be counted. To fix this easily, you could add a line combining your lists into a sum list, then looping over that entire list and checking if each value is in only one of the lists - this would use ^ to do an xor comparison of whether or not the variable is in the two lists, so that it returns true if it is in only one of the lists. This could be done like so:
def difference_extra_credit(l1,l2):
list = l1 + l2
return [value for value in list if (value in l1) ^ (value in l2)]
Testing this function on my own has resulted in the list [4, 11, 7, 1], and [4, 11, 100, 7, 1] if 100 is added to list1 or list2.
I'm still new to Python, and I've been making a small function that reverses a list of lists, both the original list and the lists inside. This is my code:
def deep_reverse(L):
L.reverse()
L = [i.reverse() for i in L]
Now this code works perfectly, but if I do a small change and rearrange the lines like this:
def deep_reverse(L):
L = [i.reverse() for i in L]
L.reverse()
suddenly it stops working! It only reverses the internal lists but not the original one. Putting some debugging print() statements inside, I can see the first code reverses the original list after the first line and it's printed, but the second code actually prints a list containing 'None' as elements after reversing the list. Can anyone please explain why this behavior and what is the difference between the two codes?
The reverse() function reverses a list in-place and returns None, that explains the weird behavior. A correct implementation would be:
def deep_reverse(L):
ans = [i[::-1] for i in L]
ans.reverse()
return ans
Also, it's a bad idea to reassign and/or mutate a parameter to a function, it can lead to unexpected results. Sometimes functions in the standard library do it for efficiency reasons (for example, sort() and reverse()), that's ok - but it can lead to confusions, like the one you just experienced. Your code doesn't have to be written in that fashion unless strictly necessary.
Your first deep_reverse function reassigned L, but it is not a global parameter and is not returned in your function. Hence this variable is lost. HOWEVER, you are mutating the list in place, hence the modifications remain which is why it still works! Your original function is equivalent to the following (note there is no final assignment):
def deep_reverse(L):
L.reverse()
[i.reverse() for i in L]
This should probably be written using a for-loop:
def deep_reverse_2(L):
L.reverse()
for i in L:
i.reverse()
L = [[1, 2, 3], [2, 3, 4]]
deep_reverse_2(L)
>>> L
[[4, 3, 2], [3, 2, 1]]
The second function does not work because you reassign L inside the function (it is now local to the function and not the same L variable you passed in to the function). They would have different memory locations if you checked using id. Given that nothing is returned, this new L list is lost, as are the modifications made to it.
This question already has answers here:
Creating functions (or lambdas) in a loop (or comprehension)
(6 answers)
Closed 8 months ago.
In python 2.6:
[x() for x in [lambda: m for m in [1,2,3]]]
results in:
[3, 3, 3]
I would expect the output to be [1, 2, 3]. I get the exact same problem even with a non list comprehension approach. And even after I copy m into a different variable.
What am I missing?
To make the lambdas remember the value of m, you could use an argument with a default value:
[x() for x in [lambda m=m: m for m in [1,2,3]]]
# [1, 2, 3]
This works because default values are set once, at definition time. Each lambda now uses its own default value of m instead of looking for m's value in an outer scope at lambda execution time.
The effect you’re encountering is called closures, when you define a function that references non-local variables, the function retains a reference to the variable, rather than getting its own copy. To illustrate, I’ll expand your code into an equivalent version without comprehensions or lambdas.
inner_list = []
for m in [1, 2, 3]:
def Lambda():
return m
inner_list.append(Lambda)
So, at this point, inner_list has three functions in it, and each function, when called, will return the value of m. But the salient point is that they all see the very same m, even though m is changing, they never look at it until called much later.
outer_list = []
for x in inner_list:
outer_list.append(x())
In particular, since the inner list is constructed completely before the outer list starts getting built, m has already reached its last value of 3, and all three functions see that same value.
Long story short, you don't want to do this. More specifically, what you're encountering is an order of operations problem. You're creating three separate lambda's that all return m, but none of them are called immediately. Then, when you get to the outer list comprehension and they're all called the residual value of m is 3, the last value of the inner list comprehension.
-- For comments --
>>> [lambda: m for m in range(3)]
[<function <lambda> at 0x021EA230>, <function <lambda> at 0x021EA1F0>, <function <lambda> at 0x021EA270>]
Those are three separate lambdas.
And, as further evidence:
>>> [id(m) for m in [lambda: m for m in range(3)]]
[35563248, 35563184, 35563312]
Again, three separate IDs.
Look at the __closure__ of the functions. All 3 point to the same cell object, which keeps a reference to m from the outer scope:
>>> print(*[x.__closure__[0] for x in [lambda: m for m in [1,2,3]]], sep='\n')
<cell at 0x00D17610: int object at 0x1E2139A8>
<cell at 0x00D17610: int object at 0x1E2139A8>
<cell at 0x00D17610: int object at 0x1E2139A8>
If you don't want your functions to take m as a keyword argument, as per unubtu's answer, you could instead use an additional lambda to evaluate m at each iteration:
>>> [x() for x in [(lambda x: lambda: x)(m) for m in [1,2,3]]]
[1, 2, 3]
Personally, I find this a more elegant solution. Lambda returns a function, so if we want to use the function, then we should use it. It's confusing to use the same symbol for the 'anonymous' variable in the lambda and for the generator, so in my example I use a different symbol to make it hopefully more clear.
>>> [ (lambda a:a)(i) for i in range(3)]
[0, 1, 2]
>>>
it's faster too.
>>> timeit.timeit('[(lambda a:a)(i) for i in range(10000)]',number=10000)
9.231263160705566
>>> timeit.timeit('[lambda a=i:a for i in range(10000)]',number=10000)
11.117988109588623
>>>
but not as fast as map:
>>> timeit.timeit('map(lambda a:a, range(10000))',number=10000)
5.746963977813721
(I ran these tests more than once, result was the same, this was done in python 2.7, results are different in python 3: the two list comprehensions are much closer in performance and both a lot slower, map remains much faster. )
#unubtu's answer is correct. I recreated the scenario in Groovy with closures. Perhaps is illustrates what is going on.
This is analogous to [x() for x in [lambda: m for m in [1,2,3]]]
arr = []
x = 0
while (x < 3) {
x++
arr.add({ -> x })
}
arr.collect { f -> f() } == [3, 3, 3]
This is analogous to [x() for x in [lambda m=m: m for m in [1,2,3]]]
arr = []
x = 0
while (x < 3) {
x++
arr.add({_x -> { -> _x }}(x))
}
arr.collect { f -> f() } == [1, 2, 3]
Note that this would not happen if i used [1,2,3].each {x -> ... } instead of a while loop. Groovy while loops and Python list comprehensions both share its closure between iterations.
I noticed that too. I concluded that lambda are created only once. So in fact your inner list comprehension will give 3 indentical functions all related to the last value of m.
Try it and check the id() of the elements.
[Note: this answer is not correct; see the comments]