This question already has answers here:
Understanding nested list comprehension [duplicate]
(2 answers)
Closed 3 years ago.
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
squared = [[x**2 for x in row] for row in matrix]
print(squared)
In the preceding data structure and list comprehension, what is the order of execution?
Visually it appears to process from right to left. Considering the nested list first has to be access before each of its individual items can be squared.
Order of elements
The PEP 202 is not very... comprehensive, but you can found some information in the Python Language Reference:
The comprehension consists of a single expression followed by at least one for clause and zero or more for or if clauses. In this case, the elements of the new container are those that would be produced by considering each of the for or if clauses a block, nesting from left to right, and evaluating the expression to produce an element each time the innermost block is reached.
[...]
The iterable expression in the leftmost for clause is evaluated directly in the enclosing scope and then passed as an argument to the implictly nested scope.
Therefore list comprehension proceeds for blocks from left to right.
But in your case, you have two list comprehensions having each one block:
outer list comprehension: do something for each row in matrix;
inner list comprehension(do something with row): do something else for each x in row.
That's why your list comprehension is better read from right to left.
It is easier to see with a function:
>>> def square_elements(row): return [x**2 for x in row] # inner
...
>>> matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> [square_elements(row) for row in matrix] # outer
[[1, 4, 9], [16, 25, 36], [49, 64, 81]]
Consider now a two blocks list comprehension:
>>> [x**2 for row in matrix for x in row]
[1, 4, 9, 16, 25, 36, 49, 64, 81]
You see the left to right order. In the first case, you have: do something for each row in matrix, and that something is: do something else for each x in row. In the second case it is: do something for each x of each row in matrix.
Execution order
That's the question you asked, even if I'm not sure that's the question you wanted to ask! In which order the operation are performed? The spec doesn't seem to say anything on this question but the answer is: it doesn't matter, as long as you avoid side effects in your list comprehensions. If you get a list out of a list comprehension (no exception raised), this list is guaranteed to complete, no matter how it was built.
But if you don't follow the "no side-effect" rule, the order in which side effects are performed may change the final result.
You can use a trick to test it: the print function returns None, hence if not print(...) is always True:
>>> matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> squared = [[x**2 for x in row if not print("value", x)] for row in matrix if not print("row", row)]
row [1, 2, 3]
value 1
value 2
value 3
row [4, 5, 6]
value 4
value 5
value 6
row [7, 8, 9]
value 7
value 8
value 9
>>> squared
[[1, 4, 9], [16, 25, 36], [49, 64, 81]]
The order seems "natural", but I don't think you should rely on it (more important: you should not have side effects in your list comprehensions).
The order is from left to right. That is all row elements of the first row will be squared first.
in list comprehension in Python
execute:
for item in lista:
item * 2
equal:
[item * 2 for item in lista]
Related
This question already has answers here:
What does "list comprehension" and similar mean? How does it work and how can I use it?
(5 answers)
Closed 2 years ago.
I'm sure this question was asked time ago, but I can't find it because I don't know how to search for it.
Lately I'm seeing this kind of for everywhere. So I was wondering how does it work, what's it using for, and these stuff. I would be grateful if you could put multiple examples, equivalences in relation to other codes, etc. Thanks a lot.
This is the kind of for I'm asking for.
x = [x for x in range(len(ys))]
or
n = [ [ls[i] for i in t] for t in indexes(len(ls),k) ]
Thanks!
This is a list comprehension.
List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition.
The basic structure you'll see often is [EXP for VAR in LIST] which iterates LIST, and for each element in the list, EXP is evaluated with VAR set to the respective list element.
Let's start with a list x that contains the integers 0 to 9:
>>> x = range(10)
>>> x
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
You can use a list comprehension in a similar manner to map() to alter each element:
>>> [i * 2 for i in x]
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
You can use it to filter out elements:
>>> [i for i in x if i % 2 == 0]
[0, 2, 4, 6, 8]
You can do both at the same time:
>>> [i * 2 for i in x if i % 2 == 0]
[0, 4, 8, 12, 16]
You don't even have to reference the iteration variable:
>>> ["foo" for i in x if i % 2 == 0]
['foo', 'foo', 'foo', 'foo', 'foo']
The second example in your question is an example of how to project a list for each element; in this particular case, the outer list being traversed contains iterables. This expression maps the inner iterables to lists. This can be used to map nested lists.
vals= [1]
for j in xrange(i):
vals.append([k for k in f(vals[j])])
This loop appends values to itself over a loop. If I compress this into a list comprehension, it doesn't work because it doesn't "dynamically" extend vals using itself on each iteration -- it processes vals as it is originally framed.
Is there a way to do a one line list comprehension that dynamically appends like this? Based on my research, it looks like maybe I am looking for a reduce function? (the equivalent of a fold)
You can indeed use reduce for this, using the initial list as the third parameter.
>>> def f(lst):
... return [x+1 for x in lst] + [len(lst)]
>>> reduce(lambda lst, i: lst + [f(lst[i])], range(5), [[1]])
[[1], [2, 1], [3, 2, 2], [4, 3, 3, 3], [5, 4, 4, 4, 4], [6, 5, 5, 5, 5, 5]]
(Note that the initial list should probably be [[1]], not [1], otherwise you are passing a number to f in the first iteration, but a list in all following iterations.)
Also note that concerning performance your original loop is probably a bit faster, as the reduce basically has to create two new lists in each iteration, while you just have to append to a list. Personally, I would go with a variation of the loop, removing the (probably useless) inner list comprehension and using [-1] to make clear that you are always using the previous result.
vals = [[1]]
for _ in xrange(n):
vals.append(f(vals[-1]))
a=[[1,2,3],[4,5,6],[7,8,9]]
.pop() has the capacity to not only remove an element of a list but also return that element.
I am looking for a similar function that can remove and return a whole list that could exist in the middle of another list.
E.g is there a function that will remove [4,5,6] from the above list a, and return it.
The reason for the question is that I'm sorting a list through itemgetter and there's a collision between the headings row (string) and the rest of the data (datetime). As such, I'm looking to effectively pop the list which represents the headings, do a sort, then insert it back in.
The nested lists are just values in the outer list. Just use .pop() on that outer list:
inner_list = a.pop(1)
Demo:
>>> a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> a.pop(1)
[4, 5, 6]
>>> a
[[1, 2, 3], [7, 8, 9]]
You could just use a slice to remove the first row from consideration if a header row is in the way:
result = rows[:1] + sorted(rows[1:], key=itemgetter(1))
This question already has answers here:
Transpose list of lists
(14 answers)
Closed 9 years ago.
I have a 2d list like this:
1 2 3
4 5 6
and I want to make this:
1 4
2 5
3 6
I've tried to do a for loop and switch each value but I keep getting an index out of bound error. Here's what I have:
for i in results:
for j in range(numCenturies):
rotated[i][j] = results [j][i]
From python documentation on zip function:
This function returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The returned list is truncated in length to the length of the shortest argument sequence. When there are multiple arguments which are all of the same length, zip() is similar to map() with an initial argument of None. With a single sequence argument, it returns a list of 1-tuples. With no arguments, it returns an empty list.
Example:
zip([1, 2, 3], [4, 5, 6]) # returns [(1, 4), (2, 5), (3, 6)]
If you need the result to be the list of lists, not the list of tuples, you can use list comprehension:
[list(x) for x in zip([1, 2, 3], [4, 5, 6], [7, 8, 9])] # returns [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
If all your variables are stored in one 2d list, and you want it pass it into zip function, you can use the following (I'll call it the star notation, because I can't remember the proper English term for it):
results = [[1, 2, 3], [4, 5, 6]]
zip(*results) # returns [(1, 4), (2, 5), (3, 6)]
http://docs.scipy.org/doc/numpy/reference/generated/numpy.transpose.html
>>> from numpy import transpose
>>> transpose([[1,2,3],[4,5,6]])
array([[1, 4],
[2, 5],
[3, 6]])
zip is the right way to do this, as shown by aga.
But if you want to know why your original code wasn't working:
for i in results:
for j in range(numCenturies):
rotated[i][j] = results [j][i]
There are two clear problems here, and likely two others. (Since you didn't show us enough of the code or data to be sure, I can't guarantee the two likely ones.)
Presumably results looks something like this:
results = [[1, 2, 3], [4, 5, 6]]
When you do for i in results, that means i will be each element in results—that is, it will be [1, 2, 3], and then [4, 5, 6]. You can't use a list as an index into a list, so this is guaranteed to give you a TypeError: list indices must be integers, not list.
To fix this, you need:
for i in range(len(results)):
… or …
for i, row in enumerate(results):
Next, results[j][i] is guaranteed to raise IndexError: list index out of range, because i is each row number, but you're trying to use it as a column number. If you're iterating over the rows and columns of results, you want this:
rotated[j][i] = results[i][j]
Next, unless you pre-filled rotated with 3 lists, each of which was pre-filled with 2 objects of some kind, you're going to get an IndexError: list assignment index out of range.
To fix this, you need to pre-fill rotated, something like this:
rotated = [[None for j in range(2)] for i in range(3)]
… or …
rotated = [[None, None], [None, None], [None, None]]
Finally, I'll bet numCenturies is 3, in which case you'll get another IndexError: list index out of range as soon as j reaches 2. The simplest thing to do here is to just use the length of the row; there's no chance of an off-by-one error that way.
Putting it all together:
rotated = [[None for j in range(2)] for i in range(3)]
for i, row in enumerate(results):
for j, value in enumerate(row):
rotated[j][i] = value
But in general, Python gives you easier ways to do things than pre-creating arrays and looping over indices to fill in the values. You can use append—or, better, a list comprehension. Or, even better, find a higher-level way to write your use, like a single call to zip.
I have the following python code:
x = range(0,10)
print x
for number in x:
print(number)
if number%2<> 0:
x.remove(number)
print x
Oddly, the out put is this:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
0
1
3
5
7
9
[0, 2, 4, 6, 8]
The first and last lines are right, but why are 2,4,6, and 8 not printed out? The print statement is not inside the if statement!
I'm using python(x,y) on windows 7. Also, I'm new to Python...I'm used to C++
You're removing items from the list (x.remove) while iterating over it (for number in x).
for-in maintains an index separately, and that is why modifying the list gives unexpected behavior.
The list is iterated using its index, but when you remove elements you skip some indices.
E.g:
[0,1,2,...] # (iterator is at second - print 1)
remove
[0,2,3,...] # (iterator is still at second)
iterator advances
[0,2,3,...] # (iterator is at third - print 3)
Add some print statements for clarity:
x = range(10)
for index, number in enumerate(x):
print "x is ", x
print "element is", number
print "index is ", index
print
if number % 2 == 0:
x.remove(number)
And the output:
x is [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
element is 0
index is 0
x is [1, 2, 3, 4, 5, 6, 7, 8, 9]
element is 2
index is 1
x is [1, 3, 4, 5, 6, 7, 8, 9]
element is 4
index is 2
x is [1, 3, 5, 6, 7, 8, 9]
element is 6
index is 3
x is [1, 3, 5, 7, 8, 9]
element is 8
index is 4
As you can see, index keeps going up by 1, even though you remove elements from the list. This is what causes the loop to skip elements.
As others have pointed out, looping over a list and removing elements from it isn't a good idea. Loop over a copy instead:
for number in x[:]:
Or:
for number in list(x):
Better yet, make a new list with a list comprehension:
[number for number in x if number % 2 == 0]
Basically you can have weird behavior when you iterate something while removing at the same time. What's happening is that you're skipping some values due to them being shifted to indexes that you already iterated over.
A better way of doing what you want (filter out some items), would be to use a list comprehension, for instance:
[x for x in range(10) if x%2==0]
You could simply use the range step to only create even numbers, but the above solution let's you filter out on any condition.
The reason why some numbers aren't printed is that the values are changing positions while you loop and remove them. When you remove the 1, you can imagine all the values being shifted by one position, the iterator is pointing to where the 2 used to be, but now the value there is 3, so the 2 is never printed. And this goes on for the rest of the values.
As Mark Rushakoff mentions, you shouldn't modify something while you're iterating over it. Change your for number in x to for number in x[:] and it will work as you expect, though. In this case you're iterating over a copy.
Don't modify a list you're iterating over. Others suggest copying the list or collecting a new list of things to remove. Instead, collect the ones you want to remain. This is faster than copying the list and removing from the copy not being iterated over, and faster than collecting the ones to remove and then removing them.
evens = []
for number in x:
if number%2 == 0:
evens += [number]
print(evens)