How to find first minimum after first peak in array? - python

I am trying to find the first minimum after the first peak in a data array. Here is my code:
x = array
mins = argrelextrema(x, np.less)[0]
mins_above_zero = np.where(x[mins] > 0)[0]
ag = x[mins[mins_above_zero]].argmin()
true_minimum_index = mins[ag]
pyplot.scatter(mins, x[mins])
pyplot.plot(x)
pyplot.ylim(0, 2000)
It currently picks out too many minimums.
If I have a numpy array like this:
array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 1., 0., 1., 0.,
0., 0., 1., 1., 0., 0., 2., 0., 2., 1., 3.,
1., 2., 5., 8., 6., 55., 396., 608., 157., 40., 45.,
43., 51., 74., 89., 107., 121., 98., 111., 122., 170., 187.,
190., 229., 284., 372., 450., 457., 327., 328., 318., 288., 290.,
262., 235., 223., 177., 232., 217., 234., 261., 206., 192., 221.,
189., 181., 185., 162., 140., 144., 171., 176., 168., 213., 222.,
314., 397., 413., 429., 442., 352., 416., 439., 424., 480., 479.,
515., 522., 569., 543., 626., 666., 637., 680., 678., 747., 720.,
695., 674., 605., 490., 475., 332., 284., 252., 169., 140., 117.,
86., 71., 58., 55., 37., 45., 35., 25., 21., 16., 14.,
17., 12., 9., 7., 6., 0., 6., 6., 6., 3., 1.,
1., 4., 2., 1., 4., 0., 2., 2., 0., 1., 2.,
0., 0., 4., 0., 1., 1., 0., 0., 0., 0., 0.,
0., 1., 1., 0.])
That creates a plot like this except just with the minimum after the first peak:


Try playing with the order of argrelextrema
With your array data:
x = array
# Order 2 looks at more than just the immediate numbers around a variable
mins = argrelextrema(x, np.less, order=2)[0]
print(mins)
mins_above_zero = np.where(x[mins] > 0)[0]
ag = x[mins[mins_above_zero]].argmin()
true_minimum_index = mins[ag]
#Grabs the first relative minimum
mins = mins[0]
pyplot.scatter(mins, x[mins])
pyplot.plot(x)
pyplot.ylim(0, 2000)
Which creates:

Related

Reordering block matrix

I have a multi-level indexed square matrix, that needs to be reordered.
Say I have a two-level indexing system x and y and the square matrix M has the shape (len(x)*len(y), len(x)*len(y)).
M is sorted by the x index and I want to transform it to be sorted by the y index. Here is an example to contruct an arbitary square matrix M:
import numpy as np
nx = 4 # equal to len(x), arbitary
ny = 3 # equal to len(y), arbitary
A=np.ones(ny*ny).reshape(ny,ny) #arbitary
B=np.ones(ny*ny).reshape(ny,ny)*2 #arbitary
C=np.ones(ny*ny).reshape(ny,ny)*3 #arbitary
D=np.ones(ny*ny).reshape(ny,ny)*4 #arbitary
E=np.arange(ny*ny).reshape(ny,ny) #arbitary
M = np.block([[A, np.zeros((ny,ny)), E, np.zeros((ny,ny))],
[np.zeros((ny,ny)), B, np.zeros((ny,ny)),np.zeros((ny,ny))],
[np.zeros((ny,ny)),np.zeros((ny,ny)),C, np.zeros((ny,ny))],
[np.zeros((ny,ny)), np.zeros((ny,ny)), np.zeros((ny,ny)), D]])
and the resulting matrix M may look like this
array([[1., 1., 1., 0., 0., 0., 0., 1., 2., 0., 0., 0.],
[1., 1., 1., 0., 0., 0., 3., 4., 5., 0., 0., 0.],
[1., 1., 1., 0., 0., 0., 6., 7., 8., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 2., 2., 2., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 3., 3., 3., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 4., 4., 4.]])
Now I want to transform the M into M_transformed that looks like this
array([[1., 0., 0., 0., 1., 0., 1., 0., 1., 0., 2., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.],
[1., 0., 3., 0., 1., 0., 4., 0., 1., 0., 5., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.],
[1., 0., 6., 0., 1., 0., 7., 0., 1., 0., 8., 0.],
[0., 2., 0., 0., 0., 2., 0., 0., 0., 2., 0., 0.],
[0., 0., 3., 0., 0., 0., 3., 0., 0., 0., 3., 0.],
[0., 0., 0., 4., 0., 0., 0., 4., 0., 0., 0., 4.]])
I use a very elementary, 4 layers of for loops to solve this problem and I believe there must be a more straight forward way (like a library) to solve this issue, as the matrix M can grow very large depending on the length of x and y (nx and ny)
M_transformed = np.zeros(M.shape)
for i in range(nx):
for j in range(nx):
for k in range(ny):
for l in range(ny):
M_transformed[k * nx + i,l * nx + j] = M[i * ny + k, j * ny + l]

I did it with no calculations, just borrowing some ideas from how to do maxpooling and experimenting a lot with swaps of axes.
I came to solution with this plan:
And this is my solution:
w = (3, 3)
initial_shape = M.shape
M = M.reshape((M.shape[0]//w[0], w[0], M.shape[1]//w[1], w[1]))
M = M.swapaxes(0, 1)
M = M.swapaxes(2, 3)
M = M.reshape(initial_shape)

Plotting 3D Bar Chart with Plotly

I want to plot a 3d bar chart like in this example but I can not or dont know how to use the code for my data. This is how my data looks like:
z = np.array([[ -10., -40., 0., 20., 10., 0., 0., 0.],
[ 0., -50., -60., -20., 20., 0., 0., 0.],
[ -60., -140., -90., -20., 20., 0., 0., 0.],
[ 190., -70., -240., 20., 20., 0., 0., 0.],
[ 430., 70., -380., -20., 20., 0., 0., 0.],
[ 170., 0., -280., -110., 20., 0., 0., 0.],
[ 20., 10., -90., -80., 20., 0., 0., 0.],
[ 20., 20., 20., 20., 20., 0., 0., 0.]])
This is how it shoud look like but with my data and a grid of 8x8:
My z-values represent the height/depth of the bins. So in total I would have 8x8 bins on this 3D plot.
I want to use Plotly because the plot will be embedded in Plotly Dash and should be interactive.
So how do I need to change my data fromat to use the code from this example or is there another possibility to get such a plot in plotly?

How to do batch filling in pytorch

I need to fill images with other image patches during training. Since I’m training with mini-batch, is there any efficient way to do this?
For example, I have a mini-batch of images of size [B, 3, 128, 128]. I also have patches of size [B, 4, 3, 32, 32], where 4 is the number of patches. Besides, I have the bounding box indicating the location of patches of size [B, 4, 4]. How can I fill in the patches in a batch-wise way? You can ignore the overlap between patches.

import torch
B = 1 # num of batches
C = 3 # num of channels
S = 16 # size of img
img = torch.zeros(B,C,S,S)
s = 4 # size of patch
n_patch = 4 # num of patches
patches = torch.arange(B*C*n_patch*s*s).view(B,n_patch,C,s,s)
# generate 4*B boxes
# boxes: x_min,y_min,x_max,y_max. Here, we take four corner patches. Also, we simply set all batch share the same patch coordinates.
base_boxes = torch.tensor([[0,0,s,s],[S-s,0,S,s],[0,S-s,s,S],[S-s,S-s,S,S]])
bboxes = base_boxes.view(1,-1,4).repeat(B,1,1)
for batch_id in range(B):
for patch_id in range(n_patch):
bbox = bboxes[batch_id,patch_id,:]
img[batch_id,:,bbox[0]:bbox[2],bbox[1]:bbox[3]] = batches[batch_id,patch_id,...]
img
# tensor([[[[ 0., 1., 2., 3., 0., 0., 0., 0., 0., 0., 0., 0., 96., 97., 98., 99.],
# [ 4., 5., 6., 7., 0., 0., 0., 0., 0., 0., 0., 0., 100., 101., 102., 103.],
# [ 8., 9., 10., 11., 0., 0., 0., 0., 0., 0., 0., 0., 104., 105., 106., 107.],
# [ 12., 13., 14., 15., 0., 0., 0., 0., 0., 0., 0., 0., 108., 109., 110., 111.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 48., 49., 50., 51., 0., 0., 0., 0., 0., 0., 0., 0., 144., 145., 146., 147.],
# [ 52., 53., 54., 55., 0., 0., 0., 0., 0., 0., 0., 0., 148., 149., 150., 151.],
# [ 56., 57., 58., 59., 0., 0., 0., 0., 0., 0., 0., 0., 152., 153., 154., 155.],
# [ 60., 61., 62., 63., 0., 0., 0., 0., 0., 0., 0., 0., 156., 157., 158., 159.]],
# [[ 16., 17., 18., 19., 0., 0., 0., 0., 0., 0., 0., 0., 112., 113., 114., 115.],
# [ 20., 21., 22., 23., 0., 0., 0., 0., 0., 0., 0., 0., 116., 117., 118., 119.],
# [ 24., 25., 26., 27., 0., 0., 0., 0., 0., 0., 0., 0., 120., 121., 122., 123.],
# [ 28., 29., 30., 31., 0., 0., 0., 0., 0., 0., 0., 0., 124., 125., 126., 127.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 64., 65., 66., 67., 0., 0., 0., 0., 0., 0., 0., 0., 160., 161., 162., 163.],
# [ 68., 69., 70., 71., 0., 0., 0., 0., 0., 0., 0., 0., 164., 165., 166., 167.],
# [ 72., 73., 74., 75., 0., 0., 0., 0., 0., 0., 0., 0., 168., 169., 170., 171.],
# [ 76., 77., 78., 79., 0., 0., 0., 0., 0., 0., 0., 0., 172., 173., 174., 175.]],
# [[ 32., 33., 34., 35., 0., 0., 0., 0., 0., 0., 0., 0., 128., 129., 130., 131.],
# [ 36., 37., 38., 39., 0., 0., 0., 0., 0., 0., 0., 0., 132., 133., 134., 135.],
# [ 40., 41., 42., 43., 0., 0., 0., 0., 0., 0., 0., 0., 136., 137., 138., 139.],
# [ 44., 45., 46., 47., 0., 0., 0., 0., 0., 0., 0., 0., 140., 141., 142., 143.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
# [ 80., 81., 82., 83., 0., 0., 0., 0., 0., 0., 0., 0., 176., 177., 178., 179.],
# [ 84., 85., 86., 87., 0., 0., 0., 0., 0., 0., 0., 0., 180., 181., 182., 183.],
# [ 88., 89., 90., 91., 0., 0., 0., 0., 0., 0., 0., 0., 184., 185., 186., 187.],
# [ 92., 93., 94., 95., 0., 0., 0., 0., 0., 0., 0., 0., 188., 189., 190., 191.]]]])

Element wise dot product of matrices and vectors [duplicate]

This question already has an answer here:
python: Multiply slice i of a matrix stack by column i of a matrix efficiently
(1 answer)
Closed 5 years ago.
There are really similar questions here, here, here, but I don't really understand how to apply them to my case precisely.
I have an array of matrices and an array of vectors and I need element-wise dot product. Illustration:
In [1]: matrix1 = np.eye(5)
In [2]: matrix2 = np.eye(5) * 5
In [3]: matrices = np.array((matrix1,matrix2))
In [4]: matrices
Out[4]:
array([[[ 1., 0., 0., 0., 0.],
[ 0., 1., 0., 0., 0.],
[ 0., 0., 1., 0., 0.],
[ 0., 0., 0., 1., 0.],
[ 0., 0., 0., 0., 1.]],
[[ 5., 0., 0., 0., 0.],
[ 0., 5., 0., 0., 0.],
[ 0., 0., 5., 0., 0.],
[ 0., 0., 0., 5., 0.],
[ 0., 0., 0., 0., 5.]]])
In [5]: vectors = np.ones((5,2))
In [6]: vectors
Out[6]:
array([[ 1., 1.],
[ 1., 1.],
[ 1., 1.],
[ 1., 1.],
[ 1., 1.]])
In [9]: np.array([m # v for m,v in zip(matrices, vectors.T)]).T
Out[9]:
array([[ 1., 5.],
[ 1., 5.],
[ 1., 5.],
[ 1., 5.],
[ 1., 5.]])
This last line is my desired output. Unfortunately it is very inefficient, for instance doing matrices # vectors that computes unwanted dot products due to broadcasting (if I understand well, it returns the first matrix dot the 2 vectors and the second matrix dot the 2 vectors) is actually faster.
I guess np.einsum or np.tensordot might be helpful here but all my attempts have failed:
In [30]: np.einsum("i,j", matrices, vectors)
ValueError: operand has more dimensions than subscripts given in einstein sum, but no '...' ellipsis provided to broadcast the extra dimensions.
In [34]: np.tensordot(matrices, vectors, axes=(0,1))
Out[34]:
array([[[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.],
[ 0., 0., 0., 0., 0.]],
[[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0.],
[ 6., 6., 6., 6., 6.]]])
NB: my real-case scenario use more complicated matrices than matrix1 and matrix2

With np.einsum, you might use:
np.einsum("ijk,ki->ji", matrices, vectors)
#array([[ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.]])

You can use # as follows
matrices # vectors.T[..., None]
# array([[[ 1.],
# [ 1.],
# [ 1.],
# [ 1.],
# [ 1.]],
# [[ 5.],
# [ 5.],
# [ 5.],
# [ 5.],
# [ 5.]]])
As we can see it computes the right thing but arranges them wrong.
Therefore
(matrices # vectors.T[..., None]).squeeze().T
# array([[ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.],
# [ 1., 5.]])

Calculating cumulative minimum with numpy arrays

I'd like to calculate the "cumulative minimum" array--basically, the minimum value of an array up to each index such as:
import numpy as np
nums = np.array([5.,3.,4.,2.,1.,1.,2.,0.])
cumulative_min = np.zeros(nums.size, dtype=float)
for i,num in enumerate(nums):
cumulative_min[i] = np.min(nums[0:i+1])
This works (it returns the correct array([ 5., 3., 3., 2., 1., 1., 1., 0.])
), but I'd like to avoid the for loop if I can. I thought it might be faster to construct a 2-d array and use the np.amin() function, but I needed a loop for that as well.

For any 2-argument NumPy universal function, its accumulate method is the cumulative version of that function. Thus, numpy.minimum.accumulate is what you're looking for:
>>> numpy.minimum.accumulate([5,4,6,10,3])
array([5, 4, 4, 4, 3])

Create a matrix which lower triangle (np.tril) is filled with values of your array nums and your upper triangle (np.triu, with second parameter 1, so the diagonal stays free) is filled with the maximum of the array. (EDIT: instead of the maximum, the first element of the array is the better way. -> comments)
nums = np.array([5.,3.,4.,2.,1.,1.,2.,0.])
oneSquare = np.ones((nums.size, nums.size))
A = nums * np.tril(oneSquare)
B = np.triu(oneSquare, 1) * nums[0]
A, B
Out:
(array([[ 5., 0., 0., 0., 0., 0., 0., 0.],
[ 5., 3., 0., 0., 0., 0., 0., 0.],
[ 5., 3., 4., 0., 0., 0., 0., 0.],
[ 5., 3., 4., 2., 0., 0., 0., 0.],
[ 5., 3., 4., 2., 1., 0., 0., 0.],
[ 5., 3., 4., 2., 1., 1., 0., 0.],
[ 5., 3., 4., 2., 1., 1., 2., 0.],
[ 5., 3., 4., 2., 1., 1., 2., 0.]]),
array([[ 0., 5., 5., 5., 5., 5., 5., 5.],
[ 0., 0., 5., 5., 5., 5., 5., 5.],
[ 0., 0., 0., 5., 5., 5., 5., 5.],
[ 0., 0., 0., 0., 5., 5., 5., 5.],
[ 0., 0., 0., 0., 0., 5., 5., 5.],
[ 0., 0., 0., 0., 0., 0., 5., 5.],
[ 0., 0., 0., 0., 0., 0., 0., 5.],
[ 0., 0., 0., 0., 0., 0., 0., 0.]]))
Now take the minimum of each row:
(A+B).min(axis=1)
Out:
array([ 5., 3., 3., 2., 1., 1., 1., 0.])

Categories

Resources