I have 3d array and I need to set to zero its right part. For each 2d slice (n, :, :) of the array the index of the column should be taken from vector b. This index defines separating point - the left and right parts, as shown in the figure below.
a_before = [[[ 1 2 3 4]
[ 5 6 7 8]
[ 9 10 11 12]
[13 14 15 16]]
[[17 18 19 20]
[21 22 23 24]
[25 26 27 28]
[29 30 31 32]]
[[33 34 35 36]
[37 38 39 40]
[41 42 43 44]
[45 46 47 48]]]
a_before.shape = (3, 4, 4)
b = (2, 3, 1)
a_after_1 = [[[ 1 2 0 0]
[ 5 6 0 0]
[ 9 10 0 0]
[13 14 0 0]]
[[17 18 19 0]
[21 22 23 0]
[25 26 27 0]
[29 30 31 0]]
[[33 0 0 0]
[37 0 0 0]
[41 0 0 0]
[45 0 0 0]]]
After this, for each 2d slice (n, :, :) I have to take index of the column from c vector and multiply by the corresponding value taken from the vector d.
c = (1, 2, 0)
d = (50, 100, 150)
a_after_2 = [[[ 1 100 0 0]
[ 5 300 0 0]
[ 9 500 0 0]
[13 700 0 0]]
[[17 18 1900 0]
[21 22 2300 0]
[25 26 2700 0]
[29 30 3100 0]]
[[4950 0 0 0]
[5550 0 0 0]
[6150 0 0 0]
[6750 0 0 0]]]
I did it but my version looks ugly. Maybe someone can help me.
P.S. I would like to avoid for loops and use only numpy methods.
Thank You.
Here's a version without loops.
In [232]: A = np.arange(1,49).reshape(3,4,4)
In [233]: b = np.array([2,3,1])
In [234]: d = np.array([50,100,150])
In [235]: I,J = np.nonzero(b[:,None]<=np.arange(4))
In [236]: A[I,:,J]=0
In [237]: A[np.arange(3),:,b-1] *= d[:,None]
In [238]: A
Out[238]:
array([[[ 1, 100, 0, 0],
[ 5, 300, 0, 0],
[ 9, 500, 0, 0],
[ 13, 700, 0, 0]],
[[ 17, 18, 1900, 0],
[ 21, 22, 2300, 0],
[ 25, 26, 2700, 0],
[ 29, 30, 3100, 0]],
[[4950, 0, 0, 0],
[5550, 0, 0, 0],
[6150, 0, 0, 0],
[6750, 0, 0, 0]]])
Before I developed this, I wrote an iterative version. It helped me visualize the problem.
In [240]: Ac = np.arange(1,49).reshape(3,4,4)
In [241]:
In [241]: for i,v in enumerate(b):
...: Ac[i,:,v:]=0
...:
In [242]: for i,(bi,di) in enumerate(zip(b,d)):
...: Ac[i,:,bi-1]*=di
It may be easier to understand, and in that sense, less ugly!
The fact that your A has middle dimension that is "just-going-along" for the ride, complicates "vectorizing" the problem.
With a (3,4) 2d array, the solution is just:
In [251]: Ab = Ac[:,0,:]
In [252]: Ab[b[:,None]<=np.arange(4)]=0
In [253]: Ab[np.arange(3),b-1]*=d
Here it is:
import numpy as np
a = np.arange(1,49).reshape(3,4,4)
b = np.array([2,3,1])
c = np.array([1,2,0])
d = np.array([50,100,150])
for i in range(len(b)):
a[i,:,b[i]:] = 0
for i,j in enumerate(c):
a[i,:,j] = a[i,:,j]* d[i]
print(a)
#
[[[ 1 100 0 0]
[ 5 300 0 0]
[ 9 500 0 0]
[ 13 700 0 0]]
[[ 17 18 1900 0]
[ 21 22 2300 0]
[ 25 26 2700 0]
[ 29 30 3100 0]]
[[4950 0 0 0]
[5550 0 0 0]
[6150 0 0 0]
[6750 0 0 0]]]
Related
I want to develop an algorithm in python that returns an 18x9 matrix randomly populated with numbers from 1 to 90 using certain rules/conditions.
Rules
#1 - Maintain an array of 18x9 between 1 and 90.
#2 - First column contains 1-10, second column contains 11-20, etc.
#3 - Each row must have 5 numbers. Other columns should be set to 0.
#4 - Numbers must be arranged in ascending order from top to bottom in a column
What I have done so far?
import numpy as np
columns = 9
rows = 18
n_per_row = 5
matrix = np.zeros((rows, columns), dtype=int)
# Keep track of available places at each row.
available_places = {k: n_per_row for k in range(rows)}
# Shuffle order in which we fill the columns.
col_order = np.arange(columns)
np.random.shuffle(col_order)
for column in col_order:
# Indices of available rows.
indices = [c for c in available_places if available_places[c]]
# Sample which row to use of the available.
indices = np.random.choice(indices, size=min(len(indices), 10), replace=False)
# print(indices)
# Values for this column.
col_values = np.random.choice(list(np.arange(1, 10+1)), size=min(len(indices), 10), replace=False) + column*10
# Fill in ascending order.
matrix[sorted(indices), column] = sorted(col_values)
for idx in indices:
available_places[idx] -= 1
print(matrix)
Result
[[ 0 0 0 31 0 51 0 71 81]
[ 1 11 0 0 0 52 61 72 0]
[ 0 0 21 32 41 0 62 73 0]
[ 0 0 22 33 0 0 0 74 82]
[ 0 12 23 0 42 0 63 0 83]
[ 2 13 24 34 0 53 0 0 0]
[ 3 0 0 0 43 54 64 0 84]
[ 4 0 0 0 44 55 65 0 85]
[ 5 14 0 0 45 0 66 0 86]
[ 6 0 25 35 46 0 0 75 0]
[ 7 15 26 36 0 0 0 0 87]
[ 8 16 0 0 47 0 0 76 88]
[ 0 17 27 0 48 0 0 77 89]
[ 0 18 0 0 49 56 67 78 0]
[ 9 0 28 39 0 57 0 79 0]
[ 0 0 29 0 50 58 68 80 0]
[ 0 19 30 40 0 59 69 0 0]
[10 20 0 0 0 60 70 0 90]]
Expected Result: https://images.meesho.com/images/products/56485141/snksv_512.jpg
Final result according to the 4 rules
5 values per row
10 values per column starting with 1,11,21, etc in ascending order
( Notice these rules are not ok for a bingo as seen in the image )
============ final matrix ===============
--------------------------------
[1, 11, 21, 31, 41, 0, 0, 0, 0]
[2, 12, 0, 32, 42, 0, 61, 0, 0]
[0, 13, 0, 33, 0, 0, 62, 71, 81]
[3, 0, 0, 34, 0, 0, 63, 72, 82]
[0, 0, 22, 0, 0, 51, 64, 73, 83]
[4, 14, 23, 35, 0, 52, 0, 0, 0]
[5, 0, 24, 0, 43, 53, 0, 0, 84]
[6, 15, 0, 36, 44, 54, 0, 0, 0]
[7, 0, 0, 37, 0, 0, 65, 74, 85]
[0, 0, 0, 0, 45, 55, 66, 75, 86]
[8, 16, 25, 0, 0, 0, 67, 76, 0]
[0, 0, 26, 0, 46, 56, 0, 77, 87]
[9, 17, 0, 0, 0, 0, 68, 78, 88]
[10, 18, 0, 0, 0, 57, 0, 79, 89]
[0, 19, 27, 38, 47, 0, 0, 80, 0]
[0, 20, 28, 39, 48, 58, 0, 0, 0]
[0, 0, 29, 0, 49, 59, 69, 0, 90]
[0, 0, 30, 40, 50, 60, 70, 0, 0]
--------------------------------
Principles :
Establish first a matrix with 0 and 1 set as placeholders for future values.
Randomize 0 or 1 per cell in the matrix, but survey # of 1 in a row and # of 1 in a col to respect constraints.
As it could happen that random gives not enough 1 early, the both constraints cannot be satisfied at first try. Prog retry automatically and traces each try for observation. (max seen in my tests : 10 loops, mean : <=3 loops)
Once a satisfactory matrix of 0 & 1 is obtained, replace each 1 by the corresponding value for each col.
A solution :
import random
# #1 - Maintain an array of 18x9 (between 1 and 90)
maxRow = 18
maxCol = 9
# #2 - First column contains 1-10, second column contains 11-20, etc.
# ie first column 1 start from 1 and have 10 entries, column 2 start from 11 and have 10 entries, etc.
origins = [i*10 +1 for i in range(maxCol)] #[1, 11, 21, 31, 41, 51, 61, 71, 81]
maxInCol = [10 for i in range(maxCol)] #[10, 10, 10, 10, 10, 10, 10, 10, 10]
# comfort : display matrix
def showMatrix():
print('--------------------------------')
for row in range(len(matrix)):
print(matrix[row])
print('--------------------------------')
# comfort : count #values in a col
def countInCol(col):
count = 0
for row in range(maxRow):
count+=matrix[row][col]
return count
# verify the rules of 5 per row and 10 per cols
def verify():
ok = True
showMatrix()
# count elements in a col
for col in range(maxCol):
count = 0
for row in range(maxRow):
count+= matrix[row][col]
if(count!= maxInCol[col]):
print ('*** wrong # of elements in col {0} : {1} instead of {2}'.format(col, count,maxInCol[col]))
ok = False
# count elements in a row
for row in range(maxRow):
count = 0
for col in range(maxCol):
count+= matrix[row][col]
if(count!=5):
print('***** wrong # of elements in row {0} : {1}'.format(row, count))
ok = False
if (not ok): print( '********************************************')
return ok
# -- main ----
# need to iterate in case of no more value to complete a col
tour = 1
maxTour = 100 #security limit
while True:
# prepare a matrix of rows of cols of 0
matrix = [[0 for i in range(maxCol)] for i in range(18)]
# begin to fill some places with 1 instead of 0
for row in range(maxRow):
count = 0
for col in range(maxCol):
if (count==5): break # line is already full with 5 elt
# random a 0 or 1
placeHolder = random.choice([0,1])
# if the remaining cols of this row needs to be 1 to complete at 5/row
if (5-count) == (maxCol-col):
placeHolder = 1 # must complete the row
else:
inCol = countInCol(col)
# 10 places max in col
if (inCol)==maxInCol[col]: placeHolder = 0 # this col is full
# constraint : if the remaining rows of this col need to be 1 to complete the expected 10 values
if(maxRow-row) == (maxInCol[col]-inCol): placeHolder = 1
matrix[row][col] = placeHolder
count+= placeHolder
#-------- some case are not correct . prog loops
if verify():
print(' ok after {0} loop(s)'.format(tour))
break
# security infinite loop
if (tour>=maxTour): break
tour +=1
# now replace the placeholders by successive values per col
print('\n============ final matrix ===============')
for row in range(maxRow):
for col in range(maxCol):
if matrix[row][col]==1:
matrix[row][col] = origins[col]
origins[col]+=1
showMatrix()
HTH
I'm using scikit.morphology to do an erosion on a two-dimensional array. I need to also ascertain the distance of each cell to the minimum value identified in the erosion.
Example:
np.reshape(np.arange(1,126,step=5),[5,5])
array([[ 1, 6, 11, 16, 21],
[ 26, 31, 36, 41, 46],
[ 51, 56, 61, 66, 71],
[ 76, 81, 86, 91, 96],
[101, 106, 111, 116, 121]])
erosion(np.reshape(np.arange(1,126,step=5),[5,5]),selem=disk(3))
array([[ 1, 1, 1, 1, 6],
[ 1, 1, 1, 6, 11],
[ 1, 1, 1, 6, 11],
[ 1, 6, 11, 16, 21],
[26, 31, 36, 41, 46]])
Now what I want to do is also return an array that gives me the distance to the minimum like this:
array([[ 0, 1, 2, 3, 3],
[ 1, 1, 2, 3, 3],
[ 2, 2, 3, 3, 3],
[ 3, 3, 3, 3, 3],
[ 3, 3, 3, 3, 3]])
Is there a scikit tool that can do this? If not, any tips on how to efficiently achieve this result?
You can find the distances from the centre of your footprint using scipy.ndimage.distance_transform_cdt, then use SciPy's ndimage.generic_filter to return those values:
import numpy as np
from skimage.morphology import erosion, disk
from scipy import ndimage as ndi
input_arr = np.reshape(np.arange(1,126,step=5),[5,5])
footprint = disk(3)
def distance_from_min(values, distance_values):
d = np.inf
min_val = np.inf
for i in range(len(values)):
if values[i] <= min_val:
min_val = values[i]
d = distance_values[i]
return d
full_footprint = np.ones_like(footprint, dtype=float)
full_footprint[tuple(i//2 for i in footprint.shape)] = 0
# use `ndi.distance_transform_edt` instead for the euclidean distance
distance_footprint = ndi.distance_transform_cdt(
full_footprint, metric='taxicab'
)
# set values outside footprint to 0 for pretty-printing
distance_footprint[~footprint.astype(bool)] = 0
# then, extract it into values matching the values in generic_filter
distance_values = distance_footprint[footprint.astype(bool)]
output = ndi.generic_filter(
input_arr.astype(float),
distance_from_min,
footprint=footprint,
mode='constant',
cval=np.inf,
extra_arguments=(distance_values,),
)
print('input:\n', input_arr)
print('footprint:\n', footprint)
print('distance_footprint:\n', distance_footprint)
print('output:\n', output)
Which gives:
input:
[[ 1 6 11 16 21]
[ 26 31 36 41 46]
[ 51 56 61 66 71]
[ 76 81 86 91 96]
[101 106 111 116 121]]
footprint:
[[0 0 0 1 0 0 0]
[0 1 1 1 1 1 0]
[0 1 1 1 1 1 0]
[1 1 1 1 1 1 1]
[0 1 1 1 1 1 0]
[0 1 1 1 1 1 0]
[0 0 0 1 0 0 0]]
distance_footprint:
[[0 0 0 3 0 0 0]
[0 4 3 2 3 4 0]
[0 3 2 1 2 3 0]
[3 2 1 0 1 2 3]
[0 3 2 1 2 3 0]
[0 4 3 2 3 4 0]
[0 0 0 3 0 0 0]]
output:
[[0. 1. 2. 3. 3.]
[1. 2. 3. 3. 3.]
[2. 3. 4. 4. 4.]
[3. 3. 3. 3. 3.]
[3. 3. 3. 3. 3.]]
This function will be very slow, however. If you want to make it faster, you will need (a) a solution like Numba or Cython for the filter function, in conjunction with SciPy LowLevelCallables and (b) to hardcode the distance array into the distance function, because for LowLevelCallables it is more difficult to pass in extra arguments. Here is a full example with llc-tools, which you can install with pip install numba llc-tools.
import numpy as np
from scipy import ndimage as ndi
from skimage.morphology import erosion, disk
import llc
def filter_func_from_footprint(footprint):
# first, create a footprint where the values are the distance from the
# center
full_footprint = np.ones_like(footprint, dtype=float)
full_footprint[tuple(i//2 for i in footprint.shape)] = 0
# use `ndi.distance_transform_edt` instead for the euclidean distance
distance_footprint = ndi.distance_transform_cdt(
full_footprint, metric='taxicab'
)
# then, extract it into values matching the values in generic_filter
distance_footprint[~footprint.astype(bool)] = 0
distance_values = distance_footprint[footprint.astype(bool)]
# finally, create a filter function with the values hardcoded
#llc.jit_filter_function
def distance_from_min(values):
d = np.inf
min_val = np.inf
for i in range(len(values)):
if values[i] <= min_val:
min_val = values[i]
d = distance_values[i]
return d
return distance_from_min
if __name__ == '__main__':
input_arr = np.reshape(np.arange(1,126,step=5),[5,5])
footprint = disk(3)
eroded = erosion(input_arr, selem=footprint)
filter_func = filter_func_from_footprint(footprint)
result = ndi.generic_filter(
# use input_arr.astype(float) when using euclidean dist
input_arr,
filter_func,
footprint=disk(3),
mode='constant',
cval=np.inf,
)
print('input:\n', input_arr)
print('output:\n', result)
Which gives:
input:
[[ 1 6 11 16 21]
[ 26 31 36 41 46]
[ 51 56 61 66 71]
[ 76 81 86 91 96]
[101 106 111 116 121]]
output:
[[0 1 2 3 3]
[1 2 3 3 3]
[2 3 4 4 4]
[3 3 3 3 3]
[3 3 3 3 3]]
For more reading on low-level callables and llc-tools, in addition to the LowLevelCallable documentation on the SciPy site (linked above, plus links therein), you can read these two blog posts I wrote a few years ago:
SciPy's new LowLevelCallable is a game-changer
Prettier LowLevelCallables with Numba JIT and decorators
While trying to convert MATLAB code to Python, I am running into a multidimensional (4D) array multiplication issue.
How do I get the same result as MATLAB's using Python/NumPy?
Python 3 NumPy code & result:
A = np.arange(1,25).reshape((2, 3, 2, 2))
B = np.array([1,10,100])
A * B[np.newaxis,:, np.newaxis, np.newaxis]
array([[[[ 1, 2],
[ 3, 4]],
[[ 50, 60],
[ 70, 80]],
[[ 900, 1000],
[1100, 1200]]],
[[[ 13, 14],
[ 15, 16]],
[[ 170, 180],
[ 190, 200]],
[[2100, 2200],
[2300, 2400]]]])
MATLAB Code & result:
A = reshape(1:24, 2,3,2,2)
B = [1 10 100]
A .* B
ans(:,:,1,1) =
1 30 500
2 40 600
ans(:,:,2,1) =
7 90 1100
8 100 1200
ans(:,:,1,2) =
13 150 1700
14 160 1800
ans(:,:,2,2) =
19 210 2300
20 220 2400
Numpy has row-major indexing as Divakar commented. Furthermore it indexes from 0.
So you can do as follows:
import numpy as np
A = np.arange(1,25).reshape((2, 2, 3, 2))
B = np.array([1,10,100])
ans = A * B[np.newaxis, np.newaxis, :, np.newaxis]
ans = np.transpose(ans)
print(ans[:,:,0,0])
print(ans[:,:,1,0])
print(ans[:,:,0,1])
print(ans[:,:,1,1])
Out:
[[ 1 30 500]
[ 2 40 600]]
[[ 7 90 1100]
[ 8 100 1200]]
[[ 13 150 1700]
[ 14 160 1800]]
[[ 19 210 2300]
[ 20 220 2400]]
Couldn't transpose np.array
import numpy as np
arr = np.arange(16).reshape((2, 2, 4))
print(arr)
arr.transpose(1, 0, 2)
print('------------')
print(arr)
output:
[[[ 0 1 2 3]
[ 4 5 6 7]]
[[ 8 9 10 11]
[12 13 14 15]]]
------------
[[[ 0 1 2 3]
[ 4 5 6 7]]
[[ 8 9 10 11]
[12 13 14 15]]]
I think that's weird. Here the same example but it works. numpy==1.17.2 What could be wrong?
Try typing 'arr = arr.transpose(1, 0, 2)' in place of 'arr.transpose(1, 0, 2)'. You can also try typing 'print(arr.transpose(1, 0, 2)' in place of 'print(arr)'.
Say I generate a sequence of values, tile them by the range provided and then increment each value in each row by that current row ID, then mask some values outside of a desired range like below:
>>> range = 5
>>> matrix = np.arange(-5, 10, 1)
>>> matrix = np.tile(matrix, (range, 1))
>>> matrix = np.add(matrix, np.arange(0, range)[:, None])
>>> matrix = ma.masked_outside(matrix, 0, 10)
[[-- -- -- -- -- 0 1 2 3 4 5 6 7 8 9]
[-- -- -- -- 0 1 2 3 4 5 6 7 8 9 10]
[-- -- -- 0 1 2 3 4 5 6 7 8 9 10 --]
[-- -- 0 1 2 3 4 5 6 7 8 9 10 -- --]
[-- 0 1 2 3 4 5 6 7 8 9 10 -- -- --]]
How would you best convert the above output to a matrix of the format [non-masked value, row-id], i.e.:
[0,0], [1, 0], [2,0] ... [10, 4]
Also, is the original code too wasteful to achieve the final desired step?
Playing around with your matrix I produced this:
In [50]: np.stack((matrix.compressed(), np.where(~matrix.mask)[0]),1)
Out[50]:
array([[ 0, 0],
[ 1, 0],
[ 2, 0],
[ 3, 0],
[ 4, 0],
[ 5, 0],
[ 6, 0],
[ 7, 0],
[ 8, 0],
[ 9, 0],
....
We could probably skip the masked array step, creating the mask directly. The compressed for example is produced by matrix.data[~matrix.mask].
In [52]: mask = ~matrix.mask
In [53]: data = matrix.data
In [54]: np.stack((data[mask], np.where(mask)[0]), 1)