Related
I have an array for an example:
import numpy as np
data=np.array([[4,4,4,0,1,1,1,0,0,0,0,1,0,0,1],
[3,0,0,1,1,1,1,1,1,1,1,0,0,1,0],
[6,0,0,1,1,1,1,1,0,0,0,0,1,0,0]])
Requirement :
In the data array, if element 1's are consecutive as the square size
of ((3,3)) and more than square size no changes. Otherwise, replace
element value 1 with zero except the square size.
Expected output :
[[4 4 4 0 1 1 1 0 0 0 0 0 0 0 0]
[3 0 0 0 1 1 1 0 0 0 0 0 0 0 0]
[6 0 0 0 1 1 1 0 0 0 0 0 0 0 0]]
I will provide here as solutions two different approaches. One which doesn't and one which is using Python loops. Let's start with the common header:
import numpy as np
from skimage.util import view_as_windows as winview
data=np.array([[4,4,4,0,1,1,1,0,0,0,0,1,0,0,1],
[3,0,0,1,1,1,1,1,1,1,1,0,0,1,0],
[6,0,0,1,1,1,1,1,0,0,0,0,1,0,0]])
Below an approach without using Python loops resulting in shortest code, but requiring import of an additional module skimage:
clmn = np.where(np.all(winview(data,(3,3))[0],axis=(1,2)))[0][0]
data[data == 1] = 0 # set all ONEs to zero
data[0:3,clmn+3:] = 0 # set after match to zero
data[0:3,clmn:clmn+3] = 1 # restore ONEs
Another one is using Python loops and only two lines longer:
for clmn in range(0,data.shape[1]):
if np.all(data[0:3,clmn:clmn+3]):
data[data==1] = 0
data[0:3,clmn+3:] = 0
data[0:3,clmn:clmn+3] = 1
break
Instead of explaining how the above code using loops works I have put the 'explanations' into the names of the used variables so the code becomes hopefully self-explaining. With this explanations and some redundant code you can use the code below for another shaped haystack to search for in another array of same kind. For an array with more rows as the shape of the sub-array there will be necessary to loop also over the rows and optimize the code skipping some unnecessary checks.
import numpy as np
data=np.array([[4,4,4,0,1,1,1,0,0,0,0,1,0,0,1],
[3,0,0,1,1,1,1,1,1,1,1,0,0,1,0],
[6,0,0,1,1,1,1,1,0,0,0,0,1,0,0]])
indx_of_clmns_in_shape = 1
indx_of_rows_in_shape = 0
subarr_shape = (3, 3)
first_row = 0
first_clmn = 0
for clmn in range(first_clmn,data.shape[indx_of_clmns_in_shape],1):
sub_data = data[
first_row:first_row+subarr_shape[indx_of_rows_in_shape],
clmn:clmn+subarr_shape[indx_of_clmns_in_shape]]
if np.all(sub_data):
data[data == 1] = 0
data[first_row : subarr_shape[indx_of_rows_in_shape],
clmn+subarr_shape[indx_of_clmns_in_shape] : ] = 0
data[first_row : subarr_shape[indx_of_rows_in_shape],
clmn : clmn+subarr_shape[indx_of_clmns_in_shape]] = 1
break
# print(sub_data)
print(data)
all three versions of the code give the same result:
[[4 4 4 0 1 1 1 0 0 0 0 0 0 0 0]
[3 0 0 0 1 1 1 0 0 0 0 0 0 0 0]
[6 0 0 0 1 1 1 0 0 0 0 0 0 0 0]]
Should be easy to do with a double for loop and a second array
rows = len(source_array)
columns = len(source_array[0])
# Create a result array of same size
result_array = [[0 for _ in range(rows)] for _ in range(columns)]
for i in range(rows):
for j in range(columns):
# Copy non 1s
if source_array[i][j] != 1:
result_array[i][j] = source_array[i][j]
# if enough rows left to check then check
if i < rows - 3:
if j < columns - 3:
# Create set on the selected partition
elements = set(source_array[i][j:j+3] + source_array[i+1][j:j+3] + source_array[i+2][j:j+3])
# Copy 1s to new array
if len(elements) == 1 and 1 in elements:
for sq_i in range(i,i+3):
for sq_j in range(j,j+3):
result_array[sq_i][sq_j] = 1
I wanted to construct a 6 x 9 matrix with entries zeros and ones in a specific way as follows. In the zeroth row column, 0 to 2 should be 1 and in the first-row column,3 to 5 should be one and in the second-row column, 6 to 8 should be one, with all the other entries to be zeros. In the third row, element 0,3,6 should be one and the other should be zeros. In the fourth row, element 1,4,7 should be one and the other elements should be zeros. In the fifth row,2,5,8 should be one and the remaining should be zeros. Half of the rows follow one way enter the value 1 and the other half of the row follows different procedures to enter the value one. How do extend this some 20 x 100 case where the first 10 rows follow one procedure as mentioned above and the second half follows different procedures
The 6x9 by matrix looks as follows
[[1,1,1,0,0,0,0,0,0],
[0,0,0,1,1,1,0,0,0],
[0,0,0,0,0,0,1,1,1],
[1,0,0,1,0,0,1,0,0],
[0,1,0,0,1,0,0,1,0],
[0,0,1,0,0,1,0,0,1]]
EDIT: Code I used to create this matrix:
import numpy as np
m=int(input("Enter the value of m, no. of points = "))
pimatrix=np.zeros((2*m +1)*(m**2)).reshape((2*m+1),(m**2))
for i in range(2*m + 1):
for j in range(m**2):
if((i<m) and ((j<((i+1)*m) and j>=(i*m)))):
pimatrix[i][j]=1
if (i>(m-1)):
for k in range(-1,m-1,1):
if(j == i+(k*m)):
pimatrix[i][j]=1
if i==2*m:
pimatrix[i][j]=1
print(pimatrix)
Try to use numpy.put function numpy.put
The best approach depends on the rules you plan to follow, but an easy approach would be to initialise the array as an array of zeroes:
import numpy as np
a = np.zeros([3, 4], dtype = int)
You can then write the logic to loop over the appropriate rows and set 1's as needed. You can simply access any element of the array by its coordinates:
a[2,1] = 1
print(a)
Result:
[[0 0 0 0]
[0 0 0 0]
[0 1 0 0]]
Without a general rule, it's hard to say what your intended logic is exactly, but assuming these rules: the top half of the array has runs of three ones on each consecutive row, starting in the upper left and moving down a row at the end of every run, until it reaches the bottom of the top half, where it wraps around to the top; the bottom half has runs of single ones, following the same pattern.
Implementing that, with your given example:
import numpy as np
a = np.zeros([6, 9], dtype=int)
def set_ones(a, run_length, start, end):
for n in range(a.shape[1]):
a[start + ((n // run_length) % (end - start)), n] = 1
set_ones(a, 3, 0, a.shape[0] // 2)
set_ones(a, 1, a.shape[0] // 2, a.shape[0])
print(a)
Result:
[[1 1 1 0 0 0 0 0 0]
[0 0 0 1 1 1 0 0 0]
[0 0 0 0 0 0 1 1 1]
[1 0 0 1 0 0 1 0 0]
[0 1 0 0 1 0 0 1 0]
[0 0 1 0 0 1 0 0 1]]
I have created a 2d matrix using Scipy's coo_matrix, and have a matrix M as such:
df = pd.DataFrame(columns=["hub", "auth", "weight"])
M = coo_matrix((df.iloc[:,2], (df.iloc[:,0],df.iloc[:,1])), shape=(len(hubs) + len(auths), len(hubs) + len(auths)))
M = M.todense()
[[0 0 0 1 1 1 0]
[0 0 0 1 1 0 0]
[0 0 0 0 0 0 1]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]
[0 0 0 0 0 0 0]]
I can successfully slice the array to get its columns and the elements in each column:
col = M[:,3]
val = col[0]
where val is equal to 1. I try to do something similar to extract a row:
row = M[0]
val = row[2]
which should also return 1, but instead val returns
[[0 0 0 1 1 1 0]]
What am I doing wrong here?
Since it is a numpy array (as DYZ pointed it that .todense() is called on the original coo_matrix):
Notice that your original matrix, or 2d array is 7 x 7 (7 rows by 7 columns). When you call col = M[:,3], you are saying you want the 3rd column and all rows, which is a resulting 7 x 1 matrix (7 rows by 1 column). When you call col[2], you are actually calling col[2,:] or getting the 2nd row (which is now just a 1 x 1 matrix).
Now, if you call row = M[0], you are actually calling row = M[0,:] or getting the 0th row and all columns, which is a 1 x 7 matrix (1 column by 7 rows). Thus calling val = row[2] gives an indexerror as you only have 1 row in your new matrix. You could instead call val = row[:,2] to get the 2nd column.
I have a 3D image with size: Deep x Weight x Height (for example: 10x20x30, means 10 images, and each image has size 20x30.
Given a patch size is pd x pw x ph (such as pd <Deep, pw<Weight, ph<Height), for example patch size: 4x4x4. The center point location of the path will be: pd/2 x pw/2 x ph/2. Let's call the distance between time t and time t+1 of the center point be stride, for example stride=2.
I want to extract the original 3D image into patches with size and stride given above. How can I do it in python? Thank you
.
Use np.lib.stride_tricks.as_strided. This solution does not require the strides to divide the corresponding dimensions of the input stack. It even allows for overlapping patches (Just do not write to the result in this case, or make a copy.). It therefore is more flexible than other approaches:
import numpy as np
from numpy.lib import stride_tricks
def cutup(data, blck, strd):
sh = np.array(data.shape)
blck = np.asanyarray(blck)
strd = np.asanyarray(strd)
nbl = (sh - blck) // strd + 1
strides = np.r_[data.strides * strd, data.strides]
dims = np.r_[nbl, blck]
data6 = stride_tricks.as_strided(data, strides=strides, shape=dims)
return data6#.reshape(-1, *blck)
#demo
x = np.zeros((5, 6, 12), int)
y = cutup(x, (2, 2, 3), (3, 3, 5))
y[...] = 1
print(x[..., 0], '\n')
print(x[:, 0, :], '\n')
print(x[0, ...], '\n')
Output:
[[1 1 0 1 1 0]
[1 1 0 1 1 0]
[0 0 0 0 0 0]
[1 1 0 1 1 0]
[1 1 0 1 1 0]]
[[1 1 1 0 0 1 1 1 0 0 0 0]
[1 1 1 0 0 1 1 1 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0]
[1 1 1 0 0 1 1 1 0 0 0 0]
[1 1 1 0 0 1 1 1 0 0 0 0]]
[[1 1 1 0 0 1 1 1 0 0 0 0]
[1 1 1 0 0 1 1 1 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0]
[1 1 1 0 0 1 1 1 0 0 0 0]
[1 1 1 0 0 1 1 1 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0]]
Explanation. Numpy arrays are organised in terms of strides, one for each dimension, data point [x,y,z] is located in memory at address base + stridex * x + stridey * y + stridez * z.
The stride_tricks.as_strided factory allows to directly manipulate the strides and shape of a new array sharing its memory with a given array. Try this only if you know what you're doing because no checks are performed, meaning you are allowed to shoot your foot by addressing out-of-bounds memory.
The code uses this function to split up each of the three existing dimensions into two new ones, one for the corresponding within block coordinate (this will have the same stride as the original dimension, because adjacent points in a block corrspond to adjacent points in the whole stack) and one dimension for the block index along this axis; this will have stride = original stride x block stride.
All the code does is computing the correct strides and dimensions (= block dimensions and block counts along the three axes).
Since the data are shared with the original array, when we set all points of the 6d array to 1, they are also set in the original array exposing the block structure in the demo. Note that the commented out reshape in the last line of the function breaks this link, because it forces a copy.
the skimage module offer you an integrated solution with view_as_blocks.
The source is on line.
Take care to choose Deep,Weight,Height multiple of pd, pw, ph, because as_strided do not check bounds.
I have a 2-dimensional array of integers, we'll call it "A".
I want to create a 3-dimensional array "B" of all 1s and 0s such that:
for any fixed (i,j) sum(B[i,j,:])==A[i.j], that is, B[i,j,:] contains A[i,j] 1s in it
the 1s are randomly placed in the 3rd dimension.
I know how I would do this using standard python indexing but this turns out to be very slow.
I am looking for a way to do this that takes advantage of the features that can make Numpy fast.
Here is how I would do it using standard indexing:
B=np.zeros((X,Y,Z))
indexoptions=range(Z)
for i in xrange(Y):
for j in xrange(X):
replacedindices=np.random.choice(indexoptions,size=A[i,j],replace=False)
B[i,j,[replacedindices]]=1
Can someone please explain how I can do this in a faster way?
Edit: Here is an example "A":
A=np.array([[0,1,2,3,4],[0,1,2,3,4],[0,1,2,3,4],[0,1,2,3,4],[0,1,2,3,4]])
in this case X=Y=5 and Z>=5
Essentially the same idea as #JohnZwinck and #DSM, but with a shuffle function for shuffling a given axis:
import numpy as np
def shuffle(a, axis=-1):
"""
Shuffle `a` in-place along the given axis.
Apply numpy.random.shuffle to the given axis of `a`.
Each one-dimensional slice is shuffled independently.
"""
b = a.swapaxes(axis,-1)
# Shuffle `b` in-place along the last axis. `b` is a view of `a`,
# so `a` is shuffled in place, too.
shp = b.shape[:-1]
for ndx in np.ndindex(shp):
np.random.shuffle(b[ndx])
return
def random_bits(a, n):
b = (a[..., np.newaxis] > np.arange(n)).astype(int)
shuffle(b)
return b
if __name__ == "__main__":
np.random.seed(12345)
A = np.random.randint(0, 5, size=(3,4))
Z = 6
B = random_bits(A, Z)
print "A:"
print A
print "B:"
print B
Output:
A:
[[2 1 4 1]
[2 1 1 3]
[1 3 0 2]]
B:
[[[1 0 0 0 0 1]
[0 1 0 0 0 0]
[0 1 1 1 1 0]
[0 0 0 1 0 0]]
[[0 1 0 1 0 0]
[0 0 0 1 0 0]
[0 0 1 0 0 0]
[1 0 1 0 1 0]]
[[0 0 0 0 0 1]
[0 0 1 1 1 0]
[0 0 0 0 0 0]
[0 0 1 0 1 0]]]