I'm trying to analyse map terrain given by the StarCraft 2 bot API.
A beginner's task for this analysis was finding cliffs for reapers, which are special units in SC2 that can jump up and down cliffs.
To solve this, I analyse points where the point itself is not pathable (=cliff) and the northern and southern two points are pathable. Pathable points are marked as 1 and not pathable as 0 in the array.
The terrain map exists as a 2D numpy array. The following is a small excerpt from a larger 200x200 array:
import numpy as np
example = np.array([[0, 0, 0, 0],
[0, 1, 1, 0],
[0, 0, 0, 0],
[0, 1, 1, 0],
[0, 0, 0, 0]])
Here, the points [2, 1] and [2, 2] would match the criteria where the points themselves are not pathable (=0) and the points above and below them are pathable (=1).
This can be achieved by the following code:
above = np.roll(example, 1, axis=0) # Shift rows downwards
below = np.roll(example, -1, axis=0) # Shift rows upwards
result = np.zeros_like(example) # Create array with zeros
result[(example == 0) & (above == 1) & (below == 1)] = 1 # Set cells to 1 that match condition
print(repr(result))
# array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 1, 1, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]])
Now my question is if the same can be achieved with less code?
The np.roll function creates a new np.array object each time, so analysing hundreds of nearby points could probably result in 100 lines of uncessary code and high memory usage.
I'm trying to find something similar to
result = np.zeros_like(example)
result[(example == 0) & (example[-1, 0] == 1) & (example[1, 0 == 1)] = 1
# or
result[(example == 0) & ((example[-1:2, 0].sum() == 2)] = 1
Here the numbers in the brackets display the relative position to the currently analysed point, but I don't know if there is a way to get this to work with numpy.
Also the result for the zeroth row wouldn't be clear when checking the point "above" it: It could access either the last row, result in an error or return a default value (0 or 1).
Edit:
I found this post and it pointed me towards the scipy convolve2d function which can be applied here, which might be what I am looking for:
import numpy as np
from scipy import signal
example = np.array([[0, 0, 0, 0],
[0, 1, 1, 0],
[0, 0, 0, 0],
[0, 1, 1, 0],
[0, 0, 0, 0]])
kernel = np.zeros((3, 3), dtype=int)
kernel[::2, 1] = 1
print(repr(kernel))
# array([[0, 1, 0],
# [0, 0, 0],
# [0, 1, 0]])
result2 = signal.convolve2d(example, kernel, mode="same")
print(repr(result2))
# array([[0, 1, 1, 0],
# [0, 0, 0, 0],
# [0, 2, 2, 0],
# [0, 0, 0, 0],
# [0, 1, 1, 0]])
result2[result2 < 2] = 0
result2[result2 == 2] = 1
print(repr(result2))
# array([[0, 0, 0, 0],
# [0, 0, 0, 0],
# [0, 1, 1, 0],
# [0, 0, 0, 0],
# [0, 0, 0, 0]])
Edit2:
Another solution may be scipy.ndimage.minimum_filter which seems to work similarly:
import numpy as np
from scipy import ndimage
example = np.array([[0, 0, 0, 0],
[0, 1, 1, 0],
[0, 0, 0, 0],
[0, 1, 1, 0],
[0, 0, 0, 0]])
kernel = np.zeros((3, 3), dtype=int)
kernel[::2, 1] = 1
print(repr(kernel))
# array([[0, 1, 0],
# [0, 0, 0],
# [0, 1, 0]])
result3 = ndimage.minimum_filter(example, footprint=kernel_vertical, mode="constant")
print(repr(result3))
# array([[0, 0, 0, 0],
# [0, 0, 0, 0],
# [0, 1, 1, 0],
# [0, 0, 0, 0],
# [0, 0, 0, 0]])
Related
I have a 10x10 array with zeros and ones.
I would like to:
find the position of each cell with a value of 1.
replace all the neighbors with 1. neighbors= any cell to a n=1 distance (also diagonal).
Example:
array([[0, 1, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 1, 1]])
output:
array([[1, 1, 1, 1, 0],
[1, 1, 1, 1, 0],
[1, 1, 1, 1, 0],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]])
I am trying finding indexes but It does not work:
a=np.where(a==1)+1
From other post I also try getting the neighbors with this function:
def n_closest(x,n,d=1):
return x[n[0]-d:n[0]+d+1,n[1]-d:n[1]+d+1]
But this does not work for the edges
Thanks
If you don't mind using scipy, a 2D convolution will solve the problem quickly:
import numpy as np
from scipy import signal
# Input array
X = np.array([[0, 1, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 1, 1]])
# We apply a 2D convolution with a 3x3 kernel and we check which value are bigger than 0.
R = (signal.convolve2d(X,np.ones((3,3)),mode='same')>0).astype(int)
# R = array([[1, 1, 1, 0, 0],
# [1, 1, 1, 1, 0],
# [1, 1, 1, 1, 0],
# [1, 1, 1, 1, 1],
# [1, 1, 1, 1, 1]])
# Finally we extract the index
x,y = np.where(R)
So here is what I can get with torch.eye(3,4) now
The matrix I get:
[[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0]]
Is there any (easy)way to transform it, or make such a mask in this format:
The matrix I want:
[[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]]
You can do it by using torch.diagonal and specifying the diagonal you want:
>>> torch.diag(torch.tensor([1,1,1]), diagonal=1)[:-1]
tensor([[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]])
If :attr:diagonal = 0, it is the main diagonal.
If :attr:diagonal > 0, it is above the main diagonal.
If :attr:diagonal < 0, it is below the main diagonal.
Here is another solution using torch.diagflat(), and using a positive offset for shifting/moving the diagonal above the main diagonal.
# diagonal values to fill
In [253]: diagonal_vals = torch.ones(3, dtype=torch.long)
# desired tensor but ...
In [254]: torch.diagflat(diagonal_vals, offset=1)
Out[254]:
tensor([[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1],
[0, 0, 0, 0]])
The above operation gives us a square matrix; however, we need a non-square matrix of shape (3,4). So, we'll just ignore the last row with simple indexing:
# shape (3, 4) with 1's above the main diagonal
In [255]: torch.diagflat(diagonal_vals, offset=1)[:-1]
Out[255]:
tensor([[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]])
I have a numpy array of shape (X,Y,Z). I want to check each of the Z dimension and delete the non-zero dimension really fast.
Detailed explanation:
I would like to check array[:,:,0] if any entry is non-zero.
If yes, ignore and check array[:,:,1].
Else if No, delete dimension array[:,:,0]
Also not 100% sure what your after but I think you want
np.squeeze(array, axis=2)
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.squeeze.html
I'm not certain what you want but this hopefully points in the right direction.
Edit 1st Jan:
Inspired by #J.Warren's use of np.squeeze I think np.compress may be more appropriate.
This does the compression in one line
np.compress((a!=0).sum(axis=(0,1)), a, axis=2) #
To explain the first parameter in np.compress
(a!=0).sum(axis=(0, 1)) # sum across both the 0th and 1st axes.
Out[37]: array([1, 1, 0, 0, 2]) # Keep the slices where the array !=0
My first answer which may no longer be relevant
import numpy as np
a=np.random.randint(2, size=(3,4,5))*np.random.randint(2, size=(3,4,5))*np.random.randint(2, size=(3,4,5))
# Make a an array of mainly zeroes.
a
Out[31]:
array([[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 1, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 0, 0, 0]],
[[0, 0, 0, 0, 0],
[0, 0, 0, 0, 1],
[0, 0, 0, 0, 0],
[1, 0, 0, 0, 0]]])
res=np.zeros(a.shape[2], dtype=np.bool)
for ix in range(a.shape[2]):
res[ix] = (a[...,ix]!=0).any()
res
Out[34]: array([ True, True, False, False, True], dtype=bool)
# res is a boolean array of which slices of 'a' contain nonzero data
a[...,res]
# use this array to index a
# The output contains the nonzero slices
Out[35]:
array([[[0, 0, 0],
[0, 0, 1],
[0, 0, 0],
[0, 0, 0]],
[[0, 1, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]],
[[0, 0, 0],
[0, 0, 1],
[0, 0, 0],
[1, 0, 0]]])
I have some data that I want to "one-hot encode" and it is represented as a 1-dimensional vector of positions.
Is there any function in NumPy that can expand my x into my x_ohe?
I'm trying to avoid using for-loops in Python at all costs for operations like this after watching Jake Vanderplas's talk
x = np.asarray([0,0,1,0,2])
x_ohe = np.zeros((len(x), 3), dtype=int)
for i, pos in enumerate(x):
x_ohe[i,pos] = 1
x_ohe
# array([[1, 0, 0],
# [1, 0, 0],
# [0, 1, 0],
# [1, 0, 0],
# [0, 0, 1]])
If x only contains non negative integers, you can compare x with a sequence use numpy broadcasting and convert the result to ints:
(x[:,None] == np.arange(x.max()+1)).astype(int)
#array([[1, 0, 0],
# [1, 0, 0],
# [0, 1, 0],
# [1, 0, 0],
# [0, 0, 1]])
Or initialize first, then assign ones use advanced indexing:
x_ohe = np.zeros((len(x), 3), dtype=int)
x_ohe[np.arange(len(x)), x] = 1
x_ohe
#array([[1, 0, 0],
# [1, 0, 0],
# [0, 1, 0],
# [1, 0, 0],
# [0, 0, 1]])
A one liner :
np.equal.outer(x,range(3)).astype(int)
array([[1, 0, 0],
[1, 0, 0],
[0, 1, 0],
[1, 0, 0],
[0, 0, 1]])
np.equal.outer(x,np.unique(x)).astype(int) works also here.
I have a ndarray, and I want to set all the non-maximum elements in the last dimension to be zero.
a = np.array([[[1,8,3,4],[6,7,10,6],[11,12,15,4]],
[[4,2,3,4],[4,7,9,8],[41,14,15,3]],
[[4,22,3,4],[16,7,9,8],[41,12,15,43]]
])
print(a.shape)
(3,3,4)
I can get the indexes of maximum elements by np.argmax():
b = np.argmax(a, axis=2)
b
array([[1, 2, 2],
[0, 2, 0],
[1, 0, 3]])
Obviously, b has 1 dimension less than a. Now, I want to get a new 3-d array that has all zeros except for where the maximum values are.
I want to get this array:
np.array([[[0,1,0,0],[0,0,1,0],[0,0,1,0]],
[[1,0,0,1],[0,0,1,0],[1,0,0,0]],
[[0,1,0,0],[1,0,0,0],[0,0,0,1]]
])
One way to achieve this, I tried creating these temporary arrays
b = np.repeat(b[:,:,np.newaxis], 4, axis=2)
t = np.repeat(np.arange(4).reshape(4,1), 9, axis=1).T.reshape(b.shape)
z = np.zeros(shape=a.shape, dtype=int)
z[t == b] = 1
z
array([[[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 1, 0]],
[[1, 0, 0, 0],
[0, 0, 1, 0],
[1, 0, 0, 0]],
[[0, 1, 0, 0],
[1, 0, 0, 0],
[0, 0, 0, 1]]])
Any idea how to get this in a more efficient way?
Here's one way that uses broadcasting:
In [108]: (a == a.max(axis=2, keepdims=True)).astype(int)
Out[108]:
array([[[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 1, 0]],
[[1, 0, 0, 1],
[0, 0, 1, 0],
[1, 0, 0, 0]],
[[0, 1, 0, 0],
[1, 0, 0, 0],
[0, 0, 0, 1]]])