I'd like to calculate the "cumulative minimum" array--basically, the minimum value of an array up to each index such as:
import numpy as np
nums = np.array([5.,3.,4.,2.,1.,1.,2.,0.])
cumulative_min = np.zeros(nums.size, dtype=float)
for i,num in enumerate(nums):
cumulative_min[i] = np.min(nums[0:i+1])
This works (it returns the correct array([ 5., 3., 3., 2., 1., 1., 1., 0.])
), but I'd like to avoid the for loop if I can. I thought it might be faster to construct a 2-d array and use the np.amin() function, but I needed a loop for that as well.
For any 2-argument NumPy universal function, its accumulate method is the cumulative version of that function. Thus, numpy.minimum.accumulate is what you're looking for:
>>> numpy.minimum.accumulate([5,4,6,10,3])
array([5, 4, 4, 4, 3])
Create a matrix which lower triangle (np.tril) is filled with values of your array nums and your upper triangle (np.triu, with second parameter 1, so the diagonal stays free) is filled with the maximum of the array. (EDIT: instead of the maximum, the first element of the array is the better way. -> comments)
nums = np.array([5.,3.,4.,2.,1.,1.,2.,0.])
oneSquare = np.ones((nums.size, nums.size))
A = nums * np.tril(oneSquare)
B = np.triu(oneSquare, 1) * nums[0]
A, B
Out:
(array([[ 5., 0., 0., 0., 0., 0., 0., 0.],
[ 5., 3., 0., 0., 0., 0., 0., 0.],
[ 5., 3., 4., 0., 0., 0., 0., 0.],
[ 5., 3., 4., 2., 0., 0., 0., 0.],
[ 5., 3., 4., 2., 1., 0., 0., 0.],
[ 5., 3., 4., 2., 1., 1., 0., 0.],
[ 5., 3., 4., 2., 1., 1., 2., 0.],
[ 5., 3., 4., 2., 1., 1., 2., 0.]]),
array([[ 0., 5., 5., 5., 5., 5., 5., 5.],
[ 0., 0., 5., 5., 5., 5., 5., 5.],
[ 0., 0., 0., 5., 5., 5., 5., 5.],
[ 0., 0., 0., 0., 5., 5., 5., 5.],
[ 0., 0., 0., 0., 0., 5., 5., 5.],
[ 0., 0., 0., 0., 0., 0., 5., 5.],
[ 0., 0., 0., 0., 0., 0., 0., 5.],
[ 0., 0., 0., 0., 0., 0., 0., 0.]]))
Now take the minimum of each row:
(A+B).min(axis=1)
Out:
array([ 5., 3., 3., 2., 1., 1., 1., 0.])
Related
I have a multi-level indexed square matrix, that needs to be reordered.
Say I have a two-level indexing system x and y and the square matrix M has the shape (len(x)*len(y), len(x)*len(y)).
M is sorted by the x index and I want to transform it to be sorted by the y index. Here is an example to contruct an arbitary square matrix M:
import numpy as np
nx = 4 # equal to len(x), arbitary
ny = 3 # equal to len(y), arbitary
A=np.ones(ny*ny).reshape(ny,ny) #arbitary
B=np.ones(ny*ny).reshape(ny,ny)*2 #arbitary
C=np.ones(ny*ny).reshape(ny,ny)*3 #arbitary
D=np.ones(ny*ny).reshape(ny,ny)*4 #arbitary
E=np.arange(ny*ny).reshape(ny,ny) #arbitary
M = np.block([[A, np.zeros((ny,ny)), E, np.zeros((ny,ny))],
[np.zeros((ny,ny)), B, np.zeros((ny,ny)),np.zeros((ny,ny))],
[np.zeros((ny,ny)),np.zeros((ny,ny)),C, np.zeros((ny,ny))],
[np.zeros((ny,ny)), np.zeros((ny,ny)), np.zeros((ny,ny)), D]])
and the resulting matrix M may look like this
array([[1., 1., 1., 0., 0., 0., 0., 1., 2., 0., 0., 0.],
[1., 1., 1., 0., 0., 0., 3., 4., 5., 0., 0., 0.],
[1., 1., 1., 0., 0., 0., 6., 7., 8., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.]])
Now I want to transform the M into M_transformed that looks like this
array([[1., 0., 0., 0., 1., 0., 1., 0., 1., 0., 2., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.],
[1., 0., 3., 0., 1., 0., 4., 0., 1., 0., 5., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.],
[1., 0., 6., 0., 1., 0., 7., 0., 1., 0., 8., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.]])
I use a very elementary, 4 layers of for loops to solve this problem and I believe there must be a more straight forward way (like a library) to solve this issue, as the matrix M can grow very large depending on the length of x and y (nx and ny)
M_transformed = np.zeros(M.shape)
for i in range(nx):
for j in range(nx):
for k in range(ny):
for l in range(ny):
M_transformed[k * nx + i,l * nx + j] = M[i * ny + k, j * ny + l]
I did it with no calculations, just borrowing some ideas from how to do maxpooling and experimenting a lot with swaps of axes.
I came to solution with this plan:
And this is my solution:
w = (3, 3)
initial_shape = M.shape
M = M.reshape((M.shape[0]//w[0], w[0], M.shape[1]//w[1], w[1]))
M = M.swapaxes(0, 1)
M = M.swapaxes(2, 3)
M = M.reshape(initial_shape)
This question already has an answer here:
python: Multiply slice i of a matrix stack by column i of a matrix efficiently
(1 answer)
Closed 5 years ago.
There are really similar questions here, here, here, but I don't really understand how to apply them to my case precisely.
I have an array of matrices and an array of vectors and I need element-wise dot product. Illustration:
In [1]: matrix1 = np.eye(5)
In [2]: matrix2 = np.eye(5) * 5
In [3]: matrices = np.array((matrix1,matrix2))
In [4]: matrices
Out[4]:
array([[[ 1., 0., 0., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 1., 0., 0.],
[ 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 1.]],
[[ 5., 0., 0., 0., 0.],
[ 0., 5., 0., 0., 0.],
[ 0., 0., 5., 0., 0.],
[ 0., 0., 0., 5., 0.],
[ 0., 0., 0., 0., 5.]]])
In [5]: vectors = np.ones((5,2))
In [6]: vectors
Out[6]:
array([[ 1., 1.],
[ 1., 1.],
[ 1., 1.],
[ 1., 1.],
[ 1., 1.]])
In [9]: np.array([m # v for m,v in zip(matrices, vectors.T)]).T
Out[9]:
array([[ 1., 5.],
[ 1., 5.],
[ 1., 5.],
[ 1., 5.],
[ 1., 5.]])
This last line is my desired output. Unfortunately it is very inefficient, for instance doing matrices # vectors that computes unwanted dot products due to broadcasting (if I understand well, it returns the first matrix dot the 2 vectors and the second matrix dot the 2 vectors) is actually faster.
I guess np.einsum or np.tensordot might be helpful here but all my attempts have failed:
In [30]: np.einsum("i,j", matrices, vectors)
ValueError: operand has more dimensions than subscripts given in einstein sum, but no '...' ellipsis provided to broadcast the extra dimensions.
In [34]: np.tensordot(matrices, vectors, axes=(0,1))
Out[34]:
array([[[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.]]])
NB: my real-case scenario use more complicated matrices than matrix1 and matrix2
With np.einsum, you might use:
np.einsum("ijk,ki->ji", matrices, vectors)
#array([[ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.]])
You can use # as follows
matrices # vectors.T[..., None]
# array([[[ 1.],
# [ 1.],
# [ 1.],
# [ 1.],
# [ 1.]],
# [[ 5.],
# [ 5.],
# [ 5.],
# [ 5.],
# [ 5.]]])
As we can see it computes the right thing but arranges them wrong.
Therefore
(matrices # vectors.T[..., None]).squeeze().T
# array([[ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.]])
I have an application where I have to process 1000's of 2D arrays. The result of the processed array is based on half of a Kings Move neighborhood in the original array. I'm trying to avoid loops if I can due to speed considerations. So, here is an example Numpy Array:
array([[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 1., 1., 1., 0., 0., 0., 0.],
[ 0., 0., 0., 1., 5., 5., 5., 5., 0., 0., 0.],
[ 0., 1., 5., 5., 1., 1., 1., 1., 1., 0., 0.],
[ 5., 5., 5., 5., 1., 5., 1., 1., 1., 1., 0.],
[ 1., 1., 1., 5., 1., 1., 5., 5., 1., 1., 0.],
[ 5., 1., 5., 1., 1., 5., 5., 5., 1., 5., 0.],
[ 0., 5., 1., 5., 1., 1., 1., 1., 1., 0., 0.],
[ 0., 0., 1., 1., 1., 1., 1., 1., 0., 0., 0.],
[ 0., 0., 0., 1., 5., 5., 5., 0., 0., 0., 0.]])
At each element, I want the sum of the cell right above it, the upper right diagonal element, the cell to the immediate right and the lower diagonal. So, using the element at [6][0] I would want to sum 1 + 1 + 1 + 5.
Of course, I also have to handle the edge cases where one of the 4 cells is not there. I have started with the padded zeros on top and far right to manage some of that but I'm stuck right now. Any advice would be much appreciated!
What you're doing can be viewed as performing a convolution with a particular convolution kernel. Here's a solution using the scipy convolve2d function:
import numpy as np
import scipy as sp
import scipy.signal
x = np.random.randint(5,size=(10,10))
kernel = np.array([[0,1,1],[0,0,1],[0,0,1]])
kernel = np.fliplr(np.flipud(kernel))
check = sp.signal.convolve2d(x,kernel,mode='same')
print x
print check
Input array for reference,
u = array([[ 0., 0., 0., 0., 0.],
[ 0., 1., 1., 1., 0.],
[ 0., 1., 1., 1., 0.],
[ 0., 1., 1., 1., 0.],
[ 0., 0., 0., 0., 0.]])
python function using for loop
import numpy as np
u = np.zeros((5,5))
u[1:-1,1:-1]=1
def cds(n):
for i in range(1,4):
for j in range(1,4):
u[i,j] = u[i,j+1] + u[i,j-1] + u[i+1,j] + u[i-1,j]
return u
above function cds(5) provide the following result by using for loop,
u=array([[ 0., 0., 0., 0., 0.],
[ 0., 2., 4., 5., 0.],
[ 0., 4., 10., 16., 0.],
[ 0., 5., 16., 32., 0.],
[ 0., 0., 0., 0., 0.]])
same function using numpy
def cds(n):
u[1:-1,1:-1] = u[1:-1,2:] + u[1:-1,:-2] + u[2:,1:-1] + u[:-2,1:-1]
return u
But for the same input array(u), function cds(5) using NUMPY provide different result.,
u=array([[ 0., 0., 0., 0., 0.],
[ 0., 2., 3., 2., 0.],
[ 0., 3., 4., 3., 0.],
[ 0., 2., 3., 2., 0.],
[ 0., 0., 0., 0., 0.]])
The reason for this problem is, python "for loop" updates every u[i,j] value to the exsisting u array while looping but "numpy" didn't.....
I want same result from numpy as like as from the for loop.
Is there any way to solve this issue in NUMPY? please help me, Thanks in advance...
I am trying to remove the rows which have their mean equal to 0. I have tried this:
import numpy as np
a = np.zeros((4,4))
for i in range(len(a)):
if (i%2)==0 : a[i]= np.arange(4*i,4*(i+1))
array([[ 0., 1., 2., 3.],
[ 0., 0., 0., 0.],
[ 8., 9., 10., 11.],
[ 0., 0., 0., 0.]])
np.ma.MaskedArray(a, mask=(np.mean(a,0)==0))
np.ma.compress_rows(a)
I would like to get this:
a =([[ 0., 1., 2., 3.],
[ 8., 9., 10., 11.]])
Using index and conditions
In [187]: a[~(a.mean(axis=1)==0)]
Out[187]:
array([[ 0., 1., 2., 3.],
[ 8., 9., 10., 11.]])