Related
I'm trying to split a list into groups based on index pairs from another list, given:
>>> l = list(range(10))
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> idx = [0, 5]
I need to break up the list resulting in:
>>> l[0:5]
[0, 1, 2, 3, 4]
>>> l[5:]
[5, 6, 7, 8, 9]
The list idx will at a minimum always be [0], but may be of size n; values inside idx will always be sorted ascending.
Currently I have:
>>> l = list(range(10))
>>> idx = [0, 5]
>>> idx.append(None)
>>> [l[idx[i]:idx[i + 1]] for i in range(len(idx) - 1)]
[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]
Is there a way to accomplish this without explicitly appending Non and iterating over a range?
Edit: for another example...
Given:
>>> l = list(range(14))
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
>>> idx = [0, 5, 10]
Desired result:
[[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13]]
You could try about itertools.zip_longest:
from itertools import zip_longest
l = list(range(14))
idx = [0, 5, 10]
print([l[pre: next] for pre, next in zip_longest(idx,idx[1:])])
Result:
[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13]]
With numpy you can use numpy.split()
import numpy as np
res =[list(x) for x in np.split(l, idx) if x.size != 0]
print(res)
Output:
[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13]]
result = [l[curr_idx:curr_idx+idx[1]] for curr_idx in idx]
result
[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13]]
Let say we have a 2-D array like this:
>>> a
array([[1, 1, 2],
[0, 2, 2],
[2, 2, 0],
[0, 2, 0]])
For each line I want to replace each element by the maximum of the 2 others in the same line.
I've found how to do it for each column separately, using numpy.amax and an identity array, like this:
>>> np.amax(a*(1-np.eye(3)[0]), axis=1)
array([ 2., 2., 2., 2.])
>>> np.amax(a*(1-np.eye(3)[1]), axis=1)
array([ 2., 2., 2., 0.])
>>> np.amax(a*(1-np.eye(3)[2]), axis=1)
array([ 1., 2., 2., 2.])
But I would like to know if there is a way to avoid a for loop and get directly the result which in this case should look like this:
>>> numpy_magic(a)
array([[2, 2, 1],
[2, 2, 2],
[2, 2, 2],
[2, 0, 2]])
Edit: after a few hours playing in the console, I've finally come up with the solution I was looking for. Be ready for some mind blowing one line code:
np.amax(a[[range(a.shape[0])]*a.shape[1],:][(np.eye(a.shape[1]) == 0)[:,[range(a.shape[1])*a.shape[0]]].reshape(a.shape[1],a.shape[0],a.shape[1])].reshape((a.shape[1],a.shape[0],a.shape[1]-1)),axis=2).transpose()
array([[2, 2, 1],
[2, 2, 2],
[2, 2, 2],
[2, 0, 2]])
Edit2: Paul has suggested a much more readable and faster alternative which is:
np.max(a[:, np.where(~np.identity(a.shape[1], dtype=bool))[1].reshape(a.shape[1], -1)], axis=-1)
After timing these 3 alternatives, both Paul's solutions are 4 times faster in every contexts (I've benchmarked for 2, 3 and 4 columns with 200 rows). Congratulations for these amazing pieces of code!
Last Edit (sorry): after replacing np.identity with np.eye which is faster, we now have the fastest and most concise solution:
np.max(a[:, np.where(~np.eye(a.shape[1], dtype=bool))[1].reshape(a.shape[1], -1)], axis=-1)
Here are two solutions, one that is specifically designed for max and a more general one that works for other operations as well.
Using the fact that all except possibly one maximums in each row are the maximum of the entire row, we can use argpartition to cheaply find the indices of the largest two elements. Then in the position of the largest we put the value of the second largest and everywhere else the largest value. Works also for more than 3 columns.
>>> a
array([[6, 0, 8, 8, 0, 4, 4, 5],
[3, 1, 5, 0, 9, 0, 3, 6],
[1, 6, 8, 3, 4, 7, 3, 7],
[2, 1, 6, 2, 9, 1, 8, 9],
[7, 3, 9, 5, 3, 7, 4, 3],
[3, 4, 3, 5, 8, 2, 2, 4],
[4, 1, 7, 9, 2, 5, 9, 6],
[5, 6, 8, 5, 5, 3, 3, 3]])
>>>
>>> M, N = a.shape
>>> result = np.empty_like(a)
>>> largest_two = np.argpartition(a, N-2, axis=-1)
>>> rng = np.arange(M)
>>> result[...] = a[rng, largest_two[:, -1], None]
>>> result[rng, largest_two[:, -1]] = a[rng, largest_two[:, -2]]>>>
>>> result
array([[8, 8, 8, 8, 8, 8, 8, 8],
[9, 9, 9, 9, 6, 9, 9, 9],
[8, 8, 7, 8, 8, 8, 8, 8],
[9, 9, 9, 9, 9, 9, 9, 9],
[9, 9, 7, 9, 9, 9, 9, 9],
[8, 8, 8, 8, 5, 8, 8, 8],
[9, 9, 9, 9, 9, 9, 9, 9],
[8, 8, 6, 8, 8, 8, 8, 8]])
This solution depends on specific properties of max.
A more general solution that for example also works for sum instead of max would be. Glue two copies of a together (side-by-side, not on top of each other). So the rows are something like a0 a1 a2 a3 a0 a1 a2 a3. For an index x we can get all but ax by slicing [x+1:x+4]. To do this vectorized we use stride_tricks:
>>> a
array([[2, 6, 0],
[5, 0, 0],
[5, 0, 9],
[6, 4, 4],
[5, 0, 8],
[1, 7, 5],
[9, 7, 7],
[4, 4, 3]])
>>> M, N = a.shape
>>> aa = np.c_[a, a]
>>> ast = np.lib.stride_tricks.as_strided(aa, (M, N+1, N-1), aa.strides + aa.strides[1:])
>>> result = np.max(ast[:, 1:, :], axis=-1)
>>> result
array([[6, 2, 6],
[0, 5, 5],
[9, 9, 5],
[4, 6, 6],
[8, 8, 5],
[7, 5, 7],
[7, 9, 9],
[4, 4, 4]])
# use sum instead of max
>>> result = np.sum(ast[:, 1:, :], axis=-1)
>>> result
array([[ 6, 2, 8],
[ 0, 5, 5],
[ 9, 14, 5],
[ 8, 10, 10],
[ 8, 13, 5],
[12, 6, 8],
[14, 16, 16],
[ 7, 7, 8]])
List comprehension solution.
np.array([np.amax(a * (1 - np.eye(3)[j]), axis=1) for j in range(a.shape[1])]).T
Similar to #Ethan's answer but with np.delete(), np.max(), and np.dstack():
np.dstack([np.max(np.delete(a, i, 1), axis=1) for i in range(a.shape[1])])
array([[2, 2, 1],
[2, 2, 2],
[2, 2, 2],
[2, 0, 2]])
delete() "filters" out each column successively;
max() finds the row-wise maximum of the remaining two columns
dstack() stacks the resulting 1d arrays
If you have more than 3 columns, note that this will find the maximum of "all other" columns rather than the "2-greatest" columns per row. For example:
a2 = np.arange(25).reshape(5,5)
np.dstack([np.max(np.delete(a2, i, 1), axis=1) for i in range(a2.shape[1])])
array([[[ 4, 4, 4, 4, 3],
[ 9, 9, 9, 9, 8],
[14, 14, 14, 14, 13],
[19, 19, 19, 19, 18],
[24, 24, 24, 24, 23]]])
I have an m X 3 matrix and an array of length m.
I want to do the following
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], [13, 14, 15]])
b = np.array([1, 2, 1, 3, 3])
me = np.mean(a[np.where(b==1)][:, 0])
a[np.where(b==1)][:, 0] = me
The problem is that
a[np.where(b==1)][:, 0]
returns [1, 7] instead of [4, 4].
You are combining index arrays with slices:
[np.where(b==1)] is a index array, [:, 0] is a slice. The way you do it a copy is returned and therefore you set the new values on the copy. You should instead do:
a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12], [13, 14, 15]])
b = np.array([1, 2, 1, 3, 3])
me = np.mean(a[np.where(b==1)][:, 0])
a[np.where(b==1), 0] = me
Also see https://docs.scipy.org/doc/numpy/user/basics.indexing.html for combining index arrays with slices.
How can I create a NumPy array B which is a sub-array of a NumPy array A, by specifying which rows and columns (indicated by x and y respectively) are to be included?
For example:
A = numpy.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15]])
x = [0, 2]
y = [1, 3, 4]
B = # Do something .....
Should give the output:
>>> B
array([[2, 4, 5], [12, 14, 15]])
The best way to do this is to use the ix_ function: see the answer by MSeifert for details.
Alternatively, you could use chain the indexing operations using x and y:
>>> A[x][:,y]
array([[ 2, 4, 5],
[12, 14, 15]])
First x is used to select the rows of A. Next, [:,y] picks out the columns of the subarray specified by the elements of y.
The chaining is symmetric in this case: you can also choose the columns first with A[:,y][x] if you wish.
You can use np.ix_ which allows to broadcast the integer index arrays:
>>> A = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15]])
>>> x = [0, 2]
>>> y = [1, 3, 4]
>>> A[np.ix_(x, y)]
array([[ 2, 4, 5],
[12, 14, 15]])
From the documentation the ix_ function was designed so
[...] one can quickly construct index arrays that will index the cross product. a[np.ix_([1,3],[2,5])] returns the array [[a[1,2] a[1,5]], [a[3,2] a[3,5]]].
Here's a super verbose way to get what you want:
import numpy as np
a = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15]])
x = [0, 2]
y = [1,3,4]
a2 = a.tolist()
a3 = [[l for k,l in enumerate(j) if k in y] for i,j in enumerate(a2) if i in x]
b = np.array(a3)
But please follow #ajcr answer:
import numpy as np
a = np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15]])
x = [0, 2]
y = [1,3,4]
a[x][:,y]
let's say I have this:
(numpy array)
a=
[0 1 2 3],
[4 5 6 7],
[8 9 10 11]
to get [1,1] which is 5 its diagonal is zero; according to numpy, a.diagonal(0)= [0,5,10]. How do I get the reverse or the right to left diagonal [2,5,8] for [1,1]? Is this possible?
My original problem is an 8 by 8 (0:7).. I hope that helps
Get a new array each row reversed.
>>> import numpy as np
>>> a = np.array([
... [0, 1, 2, 3],
... [4, 5, 6, 7],
... [8, 9, 10, 11]
... ])
>>> a[:, ::-1]
array([[ 3, 2, 1, 0],
[ 7, 6, 5, 4],
[11, 10, 9, 8]])
>>> a[:, ::-1].diagonal(1)
array([2, 5, 8])
or using numpy.fliplr:
>>> np.fliplr(a).diagonal(1)
array([2, 5, 8])
Flip the array upside-down and use the same:
np.flipud(a).diagonal(0)[::-1]
Another way to achieve this is to use np.rot90
import numpy as np
a = np.array([[0, 1, 2, 3],
[4, 5, 6, 7],
[8, 9, 10, 11]])
my_diag = np.rot90(a).diagonal(-1)
Result:
>>> my_diag
array([2, 5, 8])
A number of answers so far. #Akavall is closest as you need to rotate or filip and transpose (equivilant operations). I haven't seen a response from the OP regarding expected behavior on the "long" part of the rectangle.
Generalized solution for a square matrix:
a = array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
>>> [(i, np.rot90(a).diagonal(2*i-a.shape[0]+1)) for i in range(a.shape[0])]
[(0, array([0])),
(1, array([ 2, 6, 10])),
(2, array([ 4, 8, 12, 16, 20])),
(3, array([14, 18, 22])),
(4, array([24]))]
As a function:
def reverse_diag(arr, n):
idx = 2*n - arr.shape[0]+1
return np.rot90(arr).diagonal(idx)
original matrix can be made square with a[:np.min(a.shape),:np.min(a.shape)]
EDIT: OP indicated the array is square.... Final Answer is the above