This question already has answers here:
Transpose list of lists
(14 answers)
Closed 9 years ago.
I have a 2d list like this:
1 2 3
4 5 6
and I want to make this:
1 4
2 5
3 6
I've tried to do a for loop and switch each value but I keep getting an index out of bound error. Here's what I have:
for i in results:
for j in range(numCenturies):
rotated[i][j] = results [j][i]
From python documentation on zip function:
This function returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. The returned list is truncated in length to the length of the shortest argument sequence. When there are multiple arguments which are all of the same length, zip() is similar to map() with an initial argument of None. With a single sequence argument, it returns a list of 1-tuples. With no arguments, it returns an empty list.
Example:
zip([1, 2, 3], [4, 5, 6]) # returns [(1, 4), (2, 5), (3, 6)]
If you need the result to be the list of lists, not the list of tuples, you can use list comprehension:
[list(x) for x in zip([1, 2, 3], [4, 5, 6], [7, 8, 9])] # returns [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
If all your variables are stored in one 2d list, and you want it pass it into zip function, you can use the following (I'll call it the star notation, because I can't remember the proper English term for it):
results = [[1, 2, 3], [4, 5, 6]]
zip(*results) # returns [(1, 4), (2, 5), (3, 6)]
http://docs.scipy.org/doc/numpy/reference/generated/numpy.transpose.html
>>> from numpy import transpose
>>> transpose([[1,2,3],[4,5,6]])
array([[1, 4],
[2, 5],
[3, 6]])
zip is the right way to do this, as shown by aga.
But if you want to know why your original code wasn't working:
for i in results:
for j in range(numCenturies):
rotated[i][j] = results [j][i]
There are two clear problems here, and likely two others. (Since you didn't show us enough of the code or data to be sure, I can't guarantee the two likely ones.)
Presumably results looks something like this:
results = [[1, 2, 3], [4, 5, 6]]
When you do for i in results, that means i will be each element in results—that is, it will be [1, 2, 3], and then [4, 5, 6]. You can't use a list as an index into a list, so this is guaranteed to give you a TypeError: list indices must be integers, not list.
To fix this, you need:
for i in range(len(results)):
… or …
for i, row in enumerate(results):
Next, results[j][i] is guaranteed to raise IndexError: list index out of range, because i is each row number, but you're trying to use it as a column number. If you're iterating over the rows and columns of results, you want this:
rotated[j][i] = results[i][j]
Next, unless you pre-filled rotated with 3 lists, each of which was pre-filled with 2 objects of some kind, you're going to get an IndexError: list assignment index out of range.
To fix this, you need to pre-fill rotated, something like this:
rotated = [[None for j in range(2)] for i in range(3)]
… or …
rotated = [[None, None], [None, None], [None, None]]
Finally, I'll bet numCenturies is 3, in which case you'll get another IndexError: list index out of range as soon as j reaches 2. The simplest thing to do here is to just use the length of the row; there's no chance of an off-by-one error that way.
Putting it all together:
rotated = [[None for j in range(2)] for i in range(3)]
for i, row in enumerate(results):
for j, value in enumerate(row):
rotated[j][i] = value
But in general, Python gives you easier ways to do things than pre-creating arrays and looping over indices to fill in the values. You can use append—or, better, a list comprehension. Or, even better, find a higher-level way to write your use, like a single call to zip.
Related
I have a numerical list:
myList = [1, 2, 3, 100, 5]
Now if I sort this list to obtain [1, 2, 3, 5, 100].
What I want is the indices of the elements from the
original list in the sorted order i.e. [0, 1, 2, 4, 3]
--- ala MATLAB's sort function that returns both
values and indices.
If you are using numpy, you have the argsort() function available:
>>> import numpy
>>> numpy.argsort(myList)
array([0, 1, 2, 4, 3])
http://docs.scipy.org/doc/numpy/reference/generated/numpy.argsort.html
This returns the arguments that would sort the array or list.
Something like next:
>>> myList = [1, 2, 3, 100, 5]
>>> [i[0] for i in sorted(enumerate(myList), key=lambda x:x[1])]
[0, 1, 2, 4, 3]
enumerate(myList) gives you a list containing tuples of (index, value):
[(0, 1), (1, 2), (2, 3), (3, 100), (4, 5)]
You sort the list by passing it to sorted and specifying a function to extract the sort key (the second element of each tuple; that's what the lambda is for. Finally, the original index of each sorted element is extracted using the [i[0] for i in ...] list comprehension.
myList = [1, 2, 3, 100, 5]
sorted(range(len(myList)),key=myList.__getitem__)
[0, 1, 2, 4, 3]
I did a quick performance check on these with perfplot (a project of mine) and found that it's hard to recommend anything else but
np.argsort(x)
(note the log scale):
Code to reproduce the plot:
import perfplot
import numpy as np
def sorted_enumerate(seq):
return [i for (v, i) in sorted((v, i) for (i, v) in enumerate(seq))]
def sorted_enumerate_key(seq):
return [x for x, y in sorted(enumerate(seq), key=lambda x: x[1])]
def sorted_range(seq):
return sorted(range(len(seq)), key=seq.__getitem__)
b = perfplot.bench(
setup=np.random.rand,
kernels=[sorted_enumerate, sorted_enumerate_key, sorted_range, np.argsort],
n_range=[2 ** k for k in range(15)],
xlabel="len(x)",
)
b.save("out.png")
The answers with enumerate are nice, but I personally don't like the lambda used to sort by the value. The following just reverses the index and the value, and sorts that. So it'll first sort by value, then by index.
sorted((e,i) for i,e in enumerate(myList))
Updated answer with enumerate and itemgetter:
sorted(enumerate(a), key=lambda x: x[1])
# [(0, 1), (1, 2), (2, 3), (4, 5), (3, 100)]
Zip the lists together: The first element in the tuple will the index, the second is the value (then sort it using the second value of the tuple x[1], x is the tuple)
Or using itemgetter from the operatormodule`:
from operator import itemgetter
sorted(enumerate(a), key=itemgetter(1))
Essentially you need to do an argsort, what implementation you need depends if you want to use external libraries (e.g. NumPy) or if you want to stay pure-Python without dependencies.
The question you need to ask yourself is: Do you want the
indices that would sort the array/list
indices that the elements would have in the sorted array/list
Unfortunately the example in the question doesn't make it clear what is desired because both will give the same result:
>>> arr = np.array([1, 2, 3, 100, 5])
>>> np.argsort(np.argsort(arr))
array([0, 1, 2, 4, 3], dtype=int64)
>>> np.argsort(arr)
array([0, 1, 2, 4, 3], dtype=int64)
Choosing the argsort implementation
If you have NumPy at your disposal you can simply use the function numpy.argsort or method numpy.ndarray.argsort.
An implementation without NumPy was mentioned in some other answers already, so I'll just recap the fastest solution according to the benchmark answer here
def argsort(l):
return sorted(range(len(l)), key=l.__getitem__)
Getting the indices that would sort the array/list
To get the indices that would sort the array/list you can simply call argsort on the array or list. I'm using the NumPy versions here but the Python implementation should give the same results
>>> arr = np.array([3, 1, 2, 4])
>>> np.argsort(arr)
array([1, 2, 0, 3], dtype=int64)
The result contains the indices that are needed to get the sorted array.
Since the sorted array would be [1, 2, 3, 4] the argsorted array contains the indices of these elements in the original.
The smallest value is 1 and it is at index 1 in the original so the first element of the result is 1.
The 2 is at index 2 in the original so the second element of the result is 2.
The 3 is at index 0 in the original so the third element of the result is 0.
The largest value 4 and it is at index 3 in the original so the last element of the result is 3.
Getting the indices that the elements would have in the sorted array/list
In this case you would need to apply argsort twice:
>>> arr = np.array([3, 1, 2, 4])
>>> np.argsort(np.argsort(arr))
array([2, 0, 1, 3], dtype=int64)
In this case :
the first element of the original is 3, which is the third largest value so it would have index 2 in the sorted array/list so the first element is 2.
the second element of the original is 1, which is the smallest value so it would have index 0 in the sorted array/list so the second element is 0.
the third element of the original is 2, which is the second-smallest value so it would have index 1 in the sorted array/list so the third element is 1.
the fourth element of the original is 4 which is the largest value so it would have index 3 in the sorted array/list so the last element is 3.
If you do not want to use numpy,
sorted(range(len(seq)), key=seq.__getitem__)
is fastest, as demonstrated here.
The other answers are WRONG.
Running argsort once is not the solution.
For example, the following code:
import numpy as np
x = [3,1,2]
np.argsort(x)
yields array([1, 2, 0], dtype=int64) which is not what we want.
The answer should be to run argsort twice:
import numpy as np
x = [3,1,2]
np.argsort(np.argsort(x))
gives array([2, 0, 1], dtype=int64) as expected.
Most easiest way you can use Numpy Packages for that purpose:
import numpy
s = numpy.array([2, 3, 1, 4, 5])
sort_index = numpy.argsort(s)
print(sort_index)
But If you want that you code should use baisc python code:
s = [2, 3, 1, 4, 5]
li=[]
for i in range(len(s)):
li.append([s[i],i])
li.sort()
sort_index = []
for x in li:
sort_index.append(x[1])
print(sort_index)
We will create another array of indexes from 0 to n-1
Then zip this to the original array and then sort it on the basis of the original values
ar = [1,2,3,4,5]
new_ar = list(zip(ar,[i for i in range(len(ar))]))
new_ar.sort()
`
s = [2, 3, 1, 4, 5]
print([sorted(s, reverse=False).index(val) for val in s])
For a list with duplicate elements, it will return the rank without ties, e.g.
s = [2, 2, 1, 4, 5]
print([sorted(s, reverse=False).index(val) for val in s])
returns
[1, 1, 0, 3, 4]
Import numpy as np
FOR INDEX
S=[11,2,44,55,66,0,10,3,33]
r=np.argsort(S)
[output]=array([5, 1, 7, 6, 0, 8, 2, 3, 4])
argsort Returns the indices of S in sorted order
FOR VALUE
np.sort(S)
[output]=array([ 0, 2, 3, 10, 11, 33, 44, 55, 66])
Code:
s = [2, 3, 1, 4, 5]
li = []
for i in range(len(s)):
li.append([s[i], i])
li.sort()
sort_index = []
for x in li:
sort_index.append(x[1])
print(sort_index)
Try this, It worked for me cheers!
firstly convert your list to this:
myList = [1, 2, 3, 100, 5]
add a index to your list's item
myList = [[0, 1], [1, 2], [2, 3], [3, 100], [4, 5]]
next :
sorted(myList, key=lambda k:k[1])
result:
[[0, 1], [1, 2], [2, 3], [4, 5], [3, 100]]
A variant on RustyRob's answer (which is already the most performant pure Python solution) that may be superior when the collection you're sorting either:
Isn't a sequence (e.g. it's a set, and there's a legitimate reason to want the indices corresponding to how far an iterator must be advanced to reach the item), or
Is a sequence without O(1) indexing (among Python's included batteries, collections.deque is a notable example of this)
Case #1 is unlikely to be useful, but case #2 is more likely to be meaningful. In either case, you have two choices:
Convert to a list/tuple and use the converted version, or
Use a trick to assign keys based on iteration order
This answer provides the solution to #2. Note that it's not guaranteed to work by the language standard; the language says each key will be computed once, but not the order they will be computed in. On every version of CPython, the reference interpreter, to date, it's precomputed in order from beginning to end, so this works, but be aware it's not guaranteed. In any event, the code is:
sizediterable = ...
sorted_indices = sorted(range(len(sizediterable)), key=lambda _, it=iter(sizediterable): next(it))
All that does is provide a key function that ignores the value it's given (an index) and instead provides the next item from an iterator preconstructed from the original container (cached as a defaulted argument to allow it to function as a one-liner). As a result, for something like a large collections.deque, where using its .__getitem__ involves O(n) work (and therefore computing all the keys would involve O(n²) work), sequential iteration remains O(1), so generating the keys remains just O(n).
If you need something guaranteed to work by the language standard, using built-in types, Roman's solution will have the same algorithmic efficiency as this solution (as neither of them rely on the algorithmic efficiency of indexing the original container).
To be clear, for the suggested use case with collections.deque, the deque would have to be quite large for this to matter; deques have a fairly large constant divisor for indexing, so only truly huge ones would have an issue. Of course, by the same token, the cost of sorting is pretty minimal if the inputs are small/cheap to compare, so if your inputs are large enough that efficient sorting matters, they're large enough for efficient indexing to matter too.
This question already has answers here:
Understanding nested list comprehension [duplicate]
(2 answers)
Closed 3 years ago.
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
squared = [[x**2 for x in row] for row in matrix]
print(squared)
In the preceding data structure and list comprehension, what is the order of execution?
Visually it appears to process from right to left. Considering the nested list first has to be access before each of its individual items can be squared.
Order of elements
The PEP 202 is not very... comprehensive, but you can found some information in the Python Language Reference:
The comprehension consists of a single expression followed by at least one for clause and zero or more for or if clauses. In this case, the elements of the new container are those that would be produced by considering each of the for or if clauses a block, nesting from left to right, and evaluating the expression to produce an element each time the innermost block is reached.
[...]
The iterable expression in the leftmost for clause is evaluated directly in the enclosing scope and then passed as an argument to the implictly nested scope.
Therefore list comprehension proceeds for blocks from left to right.
But in your case, you have two list comprehensions having each one block:
outer list comprehension: do something for each row in matrix;
inner list comprehension(do something with row): do something else for each x in row.
That's why your list comprehension is better read from right to left.
It is easier to see with a function:
>>> def square_elements(row): return [x**2 for x in row] # inner
...
>>> matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> [square_elements(row) for row in matrix] # outer
[[1, 4, 9], [16, 25, 36], [49, 64, 81]]
Consider now a two blocks list comprehension:
>>> [x**2 for row in matrix for x in row]
[1, 4, 9, 16, 25, 36, 49, 64, 81]
You see the left to right order. In the first case, you have: do something for each row in matrix, and that something is: do something else for each x in row. In the second case it is: do something for each x of each row in matrix.
Execution order
That's the question you asked, even if I'm not sure that's the question you wanted to ask! In which order the operation are performed? The spec doesn't seem to say anything on this question but the answer is: it doesn't matter, as long as you avoid side effects in your list comprehensions. If you get a list out of a list comprehension (no exception raised), this list is guaranteed to complete, no matter how it was built.
But if you don't follow the "no side-effect" rule, the order in which side effects are performed may change the final result.
You can use a trick to test it: the print function returns None, hence if not print(...) is always True:
>>> matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> squared = [[x**2 for x in row if not print("value", x)] for row in matrix if not print("row", row)]
row [1, 2, 3]
value 1
value 2
value 3
row [4, 5, 6]
value 4
value 5
value 6
row [7, 8, 9]
value 7
value 8
value 9
>>> squared
[[1, 4, 9], [16, 25, 36], [49, 64, 81]]
The order seems "natural", but I don't think you should rely on it (more important: you should not have side effects in your list comprehensions).
The order is from left to right. That is all row elements of the first row will be squared first.
in list comprehension in Python
execute:
for item in lista:
item * 2
equal:
[item * 2 for item in lista]
This question already has answers here:
Understanding slicing
(38 answers)
Closed 5 years ago.
I worked in Python 3.6, and I am a beginner. So can anyone give me a true way of how can slice a list into variable size sub-list. I tried this solution from this site, but it gave me fixed size of sliced list. To clarify:
if I have this list:
inputList= [0,1,2,3,4,5,6,7]
I want the output to be like e.g.:
outputList=[[0,1,2], [3,4], [5], [6,7]]
each time it depends on (for example) user input or some variable size.
Just use itertools.islice(). This has the added advantage that if you request a slice that you would normally take you out of bounds, you won't get an error. You'll just get as many items are left as possible.
>>> import itertools as it
>>> input_list = range(8)
>>> slices = (3, 2, 1, 2)
>>> iterable = iter(input_list)
>>> output_list = [list(it.islice(iterable, sl)) for sl in slices]
>>> output_list
[[0, 1, 2], [3, 4], [5], [6, 7]]
For example, if you had slices = (3, 2, 1, 3, 2), your result would be [[0, 1, 2], [3, 4], [5], [6, 7], []].
Basically, iter(input_list) creates an iterable of your list so you can fetch the next k values with islice().
You can do this in a loop making use of python's [:#] slice notation:
Let's say the user's input for slicing chunk sizes is stored in a list. They want chunks of 3, 2, 1 and 2, so the user input defining the chunks gets stored into a list that has has [3,2,1,2].
Then, loop through that list and use it to get your slices:
input_list = [0,1,2,3,4,5,6,7]
chunk_list = [3,2,1,2]
output_list = []
for i in chunk_list:
output_list.append(input_list[:i])
input_list = input_list[i:]
print(output_list)
Prints:
[[0, 1, 2], [3, 4], [5], [6, 7]]
Probably you just want to learn about slicing. See Understanding Python's slice notation
To get your example output, you could do
outputList = [inputList[:3], inputList[3:5], inputList[5:6], inputList[6:]]
vals= [1]
for j in xrange(i):
vals.append([k for k in f(vals[j])])
This loop appends values to itself over a loop. If I compress this into a list comprehension, it doesn't work because it doesn't "dynamically" extend vals using itself on each iteration -- it processes vals as it is originally framed.
Is there a way to do a one line list comprehension that dynamically appends like this? Based on my research, it looks like maybe I am looking for a reduce function? (the equivalent of a fold)
You can indeed use reduce for this, using the initial list as the third parameter.
>>> def f(lst):
... return [x+1 for x in lst] + [len(lst)]
>>> reduce(lambda lst, i: lst + [f(lst[i])], range(5), [[1]])
[[1], [2, 1], [3, 2, 2], [4, 3, 3, 3], [5, 4, 4, 4, 4], [6, 5, 5, 5, 5, 5]]
(Note that the initial list should probably be [[1]], not [1], otherwise you are passing a number to f in the first iteration, but a list in all following iterations.)
Also note that concerning performance your original loop is probably a bit faster, as the reduce basically has to create two new lists in each iteration, while you just have to append to a list. Personally, I would go with a variation of the loop, removing the (probably useless) inner list comprehension and using [-1] to make clear that you are always using the previous result.
vals = [[1]]
for _ in xrange(n):
vals.append(f(vals[-1]))
I am having a list of lists :
mat = [[1,2,3],[4,5,6],[1,2,3],[7,8,9],[4,5,6]]
and I want to convert into a set i.e. remove the repeating lists and creating a new list out of it which will only contain the unique lists.
In above case the required answer will be
[[1,2,3],[4,5,6],[7,8,9]]
But when I do set(mat), it gives me error
TypeError: unhashable type: 'list'
Can you please solve my problem. Thanks in advance!
Since the lists are mutable, they cannot be hashed. The best bet is to convert them to a tuple and form a set, like this
>>> mat = [[1,2,3],[4,5,6],[1,2,3],[7,8,9],[4,5,6]]
>>> set(tuple(row) for row in mat)
set([(4, 5, 6), (7, 8, 9), (1, 2, 3)])
We iterate through the mat, one list at a time, convert that to a tuple (which is immutable, so sets are cool with them) and the generator is sent to the set function.
If you want the result as list of lists, you can extend the same, by converting the result of set function call, to lists, like this
>>> [list(item) for item in set(tuple(row) for row in mat)]
[[4, 5, 6], [7, 8, 9], [1, 2, 3]]
Lists are mutable, therefore unhashable. Use tuples instead
In [114]: mat = [[1,2,3],[4,5,6],[1,2,3],[7,8,9],[4,5,6]]
In [115]: mat = [tuple(t) for t in mat]
In [116]: matset = set(mat)
In [117]: matset
Out[117]: {(1, 2, 3), (4, 5, 6), (7, 8, 9)}
In [118]: [list(t) for t in matset]
Out[118]: [[4, 5, 6], [7, 8, 9], [1, 2, 3]]
#thefourtheye's answer clearly depicts the problem you were facing with non-hashable data types and the way to by pass it so that you can create a set and remove duplicates. This should suffice for most of thef problems but, re-reading your question
In above case the required answer will be [[1,2,3],[4,5,6],[7,8,9]].
If the order is important, you need to use OrderedDict
>>> from collections import OrderedDict
>>> map(list, OrderedDict.fromkeys(map(tuple, mat)).keys())
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
List as key for dictionary or set is not valid. Because key should remain constant when we want to access values in dictionary for example. Values can change but keys always remain constant.
So in your case:
mat = [[1,2,3],[4,5,6],[1,2,3],[7,8,9],[4,5,6]]
We need to convert inner list to tuple. It can be done by
map(tuple, mat)
Now tuple can be used as key for set/dictionary because tuple's key cannot be changed.
dict.fromkeys(map(tuple, mat))
Since we need final answer as list of list we need to convert all tuple keys of dictionary as list. We don't care about values so we only read keys from dictionary and convert it to list.
mat = map(list, dict.fromkeys(map(tuple, mat)).keys())
Now mat would look something like this.
mat = [[1,2,3],[4,5,6],[7,8,9]]
In Python 3.8+ dictionary will preserve order for older version, OrderedDict can be used.