I have a list X which contains a list of lists of lists... e.g.
X = [ [[], [], [], []], [[], [] ,[], []] ]
When i try to convert this list above (X) into a numpy array using
np.array(), I get the following:
array([list([[],[],[],[]]),
list([list([[],[],[],[]]),....)
and the list goes on pun intended*.
What am I doing wrong here? I just need it in a normal array form such that I can index it in ways lists don't allow (i.e. array[:,1])
If your lists are NOT of the same length (in each nested dimension) you CANT do a traditional conversion to a NumPy array because it's necessary for a NumPy array of 2D or above to have the same number of elements in its first dimension.
So you cant convert [[1,2],[3,4,5]] to a numpy array directly. Applying np.array will give you a 2 element numpy array where each element is a list object as - array([list([1, 2]), list([3, 4, 5])], dtype=object). I believe this is the issue you are facing.
You cant create a 2D matrix for example that looks like -
[[1,2,3,?],
[4,5,6,7]]
What you may need to do is pad the elements of each list of lists of lists to a fixed length (equal lengths for each dimension) before converting to a NumPy array.
I would recommend iterating over each of the lists of lists of lists as done in the code I have written below to flatten your data, then transforming it the way you want.
If your lists are of the same length, then should not be a problem with numpy version 1.18.5 or above.
a = [[[1,2],[3,4]],[[5,6],[7,8]]]
np.array(a)
array([[[1, 2],
[3, 4]],
[[5, 6],
[7, 8]]])
However, if you are unable to still work with the list of list of lists, then you may need to iterate over each element first to flatten the list and then change it into a numpy array with the required shape as below -
a = [[[1,2],[3,4]],[[5,6],[7,8]]]
flat_a = [item for sublist in a for subsublist in sublist for item in subsublist]
np.array(flat_a).reshape(2,2,2)
array([[[1, 2],
[3, 4]],
[[5, 6],
[7, 8]]])
Try this:
>>> import numpy as np
>>> a = np.array([[[1,2],[3,4],[5,6]],[[7,8],[9,10],[11,12]]])
>>> a
array([[[ 1, 2],
[ 3, 4],
[ 5, 6]],
[[ 7, 8],
[ 9, 10],
[11, 12]]])
>>> a.reshape(4,-1)
array([[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, 12]])
>>> a.reshape(1,-1)
array([[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]])
Related
Given a list L = [0,1,2,3,4,5,6,7,8,9]. What's the best way to access/extract elements where their indices are given by a numpy array? Let nArr=np.array([0,-1,2,6]).
The resulting output should be another list P = [0,9,2,6].
It is clear that when the elements are uniform in shape, we can simply cast it into another numpy array, but what if it isn't? For example, M = [np.random.rand(5,5), np.random.rand(1)].
Stock Python has a convenience class, itemgetter:
In [27]: from operator import itemgetter
In [28]: L = [0,1,2,3,4,5,6,7,8,9]
In [29]: nArr=np.array([0,-1,2,6])
In [30]: itemgetter(*nArr)
Out[30]: operator.itemgetter(0, -1, 2, 6)
In [31]: itemgetter(*nArr)(L)
Out[31]: (0, 9, 2, 6)
Internally it does something equivalent to the list comprehension:
In [33]: [L[x] for x in nArr]
Out[33]: [0, 9, 2, 6]
So it isn't a fast operation like the array indexing (look at its code). It may be most useful as a way of doing sort or other operations where you'd like to define a key function that fetches multiple values.
https://stackoverflow.com/a/47585659/901925
Make a random nested list:
In [54]: arr = np.random.randint(0,10,(4,4))
In [55]: L = arr.tolist()
In [56]: L
Out[56]: [[9, 5, 8, 4], [1, 5, 5, 8], [8, 0, 5, 8], [1, 4, 5, 1]]
lexical sort by 'column':
In [57]: sorted(L)
Out[57]: [[1, 4, 5, 1], [1, 5, 5, 8], [8, 0, 5, 8], [9, 5, 8, 4]]
lexical sort by 'columns' 2 and 1 (in that order):
In [59]: sorted(L, key=itemgetter(2,1))
Out[59]: [[8, 0, 5, 8], [1, 4, 5, 1], [1, 5, 5, 8], [9, 5, 8, 4]]
To summarize the comments: lists do not support indexing by an array, like L[nArr] where nArr is an array of indexes. One normally uses list comprehension, [L[i] for i in nArr]. But if you want to, you can cast the list to a NumPy array of objects, which can then be indexed and sliced as any other NumPy array:
np.array(L, dtype=object)[nArr]
If you want a list returned, you can do:
np.array(L, dtype=object)[nArr].tolist()
But that's not nearly as pythonic as list comprehension, requires more memory, and very likely doesn't improve the speed.
I have a 3 dimensional numpy array. The dimension can go up to 128 x 64 x 8192. What I want to do is to change the order in the first dimension by interchanging pairwise.
The only idea I had so far is to create a list of the indices in the correct order.
order = [1,0,3,2...127,126]
data_new = data[order]
I fear, that this is not very efficient but I have no better idea so far
You could reshape to split the first axis into two axes, such that latter of those axes is of length 2 and then flip the array along that axis with [::-1] and finally reshape back to original shape.
Thus, we would have an implementation like so -
a.reshape(-1,2,*a.shape[1:])[:,::-1].reshape(a.shape)
Sample run -
In [170]: a = np.random.randint(0,9,(6,3))
In [171]: order = [1,0,3,2,5,4]
In [172]: a[order]
Out[172]:
array([[0, 8, 5],
[4, 5, 6],
[0, 0, 2],
[7, 3, 8],
[1, 6, 3],
[2, 4, 4]])
In [173]: a.reshape(-1,2,*a.shape[1:])[:,::-1].reshape(a.shape)
Out[173]:
array([[0, 8, 5],
[4, 5, 6],
[0, 0, 2],
[7, 3, 8],
[1, 6, 3],
[2, 4, 4]])
Alternatively, if you are looking to efficiently create those constantly flipping indices order, we could do something like this -
order = np.arange(data.shape[0]).reshape(-1,2)[:,::-1].ravel()
This question already has answers here:
How do you rotate a two dimensional array?
(64 answers)
Closed 7 years ago.
How can rotate multi-dimensional List without using the zip function, I have this list (but it can be longer):
testlist = [[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
[10,11,12]]
I want to rotate it 90 degrees clockwise and also 90 degrees counterclockwise.
ps: The reason i don't want to use zip is because this is a homework and i must write it in vanilla python .
Of course, the proper way to rotate the multi-dimensional list would be to use zip on the reversed list. I assume you've already found this in other questions here:
>>> testlist = [[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
[10,11,12]]
>>> list(zip(*testlist[::-1]))
[[10, 7, 4, 1], [11, 8, 5, 2], [12, 9, 6, 3]]
If you do not want to do that (and also don't want other builtin functions), you'll basically have to replicate the behaviour of zip. In a very simple form (e.g. just assuming that all the lists have the same length) you can do this in a nested list comprehension. Remember to rotate the list, too:
>>> [[x[i] for x in testlist[::-1]] for i in range(len(testlist[0]))]
[[10, 7, 4, 1], [11, 8, 5, 2], [12, 9, 6, 3]]
Obviously, using zip is much clearer and less error-prone. Of course, you could also spread this out to a few lines using two nested loops. Doing the same for 90 degrees counter-clockwise is left as an excercise to the reader.
I am assuming you mean transpose.
How about this?
import numpy as np
a=np.array(testlist)
print a.T #this is a view
You code would be like:
>>> import numpy as np
>>> s = np.array([[1,2,3], [4,5,6], [7,8,9]], int)
>>> s
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
>>> np.rot90(s)
array([[3, 6, 9],
[2, 5, 8],
[1, 4, 7]])
>>> np.rot90(s, 3) # Rotate counterclockwise 3 times
array([[9, 8, 7],
[6, 5, 4],
[3, 2, 1]])
Since you've stated it's for homework, I'll give you the steps, but not the code.
Rotate 90
Each row becomes a column (you can use 2 for loops and enumerate)
Reverse order of the rows (you can use slices with a negative step)
Rotate -90
Each row from Rotate 90 becomes a column (2 for loops and enumerate)
Reverse order of the rows (slice with a negative step)
I have two arrays of same size. In general dtype of these arrays is object (dtype = 'O'). What is the best way to access elements with same indicies from both arrays.
Possibility 1:
remove_indices = [i for i in range(len(array1)) if value in array1]
array1 = np.delete(array1, remove_indices, 0)
array2 = np.delete(array2, remove_indices, 0)
Possibility 2:
array3 = np.array([[array1[i], array2[i]] for i in range(len(array1))
if value not in array1[i]])
array1 = array3[:,0]
array2 = array3[:,1]
Note that Possibility 2 is faster. Is there any other solution with similar execution time (or faster)? How could I make Possiblity 2 more readable?
Not sure to understand well your examples, but sticking to What is the best way to access elements with same indicies from both arrays. make me think about zip. But using numpy why not using transpose ?
Like:
>>> array1 = numpy.array([0, 1, 2, 3, 4])
>>> array2 = numpy.array([5, 6, 7, 8, 9])
>>> numpy.array([array1, array2])
array([[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]])
>>> numpy.array([array1, array2]).T
array([[0, 5],
[1, 6],
[2, 7],
[3, 8],
[4, 9]])
For example, I have a ndarray that is:
a = np.array([1, 3, 5, 7, 2, 4, 6, 8])
Now I want to split a into two parts, one is all numbers <5 and the other is all >=5:
[array([1,3,2,4]), array([5,7,6,8])]
Certainly I can traverse a and create two new array. But I want to know does numpy provide some better ways?
Similarly, for multidimensional array, e.g.
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
[2, 4, 7]])
I want to split it according to the first column <3 and >=3, which result is:
[array([[1, 2, 3],
[2, 4, 7]]),
array([[4, 5, 6],
[7, 8, 9]])]
Are there any better ways instead of traverse it? Thanks.
import numpy as np
def split(arr, cond):
return [arr[cond], arr[~cond]]
a = np.array([1,3,5,7,2,4,6,8])
print split(a, a<5)
a = np.array([[1,2,3],[4,5,6],[7,8,9],[2,4,7]])
print split(a, a[:,0]<3)
This produces the following output:
[array([1, 3, 2, 4]), array([5, 7, 6, 8])]
[array([[1, 2, 3],
[2, 4, 7]]), array([[4, 5, 6],
[7, 8, 9]])]
It might be a quick solution
a = np.array([1,3,5,7])
b = a >= 3 # variable with condition
a[b] # to slice the array
len(a[b]) # count the elements in sliced array
1d array
a = numpy.array([2,3,4,...])
a_new = a[(a < 4)] # to get elements less than 5
2d array based on column(consider value of column i should be less than 5,
a = numpy.array([[1,2],[5,6],...]
a = a[(a[:,i] < 5)]
if your condition is multicolumn based, then you can make a new array applying the conditions on the columns. Then you can just compare the new array with value 5(according to my assumption) to get indexes and follow above codes.
Note that, whatever i have written in (), returns the index array.