Develop Reference

Python: how to count the adjacent values with values of 1 in a geotiff array?

Let's that we have geotiff of 0 and 1.
import rasterio
src = rasterio.open('myData.tif')
data = src.read(1)
data
array([[0, 1, 1, 0],
[1, 0, 0, 1],
[0, 0, 1, 0],
[1, 0, 1, 1]])
I would like to have for each pixel 1 the sum of all adjacent pixels forming a cluster of ones and to have something like the following:
array([[0, 2, 2, 0],
[1, 0, 0, 1],
[0, 0, 3, 0],
[1, 0, 3, 3]])

You can use scipy.ndimage.label:
from scipy.ndimage import label
out = np.zeros_like(data)
labels, N = label(data)
for i in range(N):
mask = labels==i+1
out[mask] = mask.sum()
output:
array([[0, 2, 2, 0],
[1, 0, 0, 1],
[0, 0, 3, 0],
[1, 0, 3, 3]])

Python, Numpy. Find values in 2d array and replace neighbors with 1

I have a 10x10 array with zeros and ones.
I would like to:
find the position of each cell with a value of 1.
replace all the neighbors with 1. neighbors= any cell to a n=1 distance (also diagonal).
Example:
array([[0, 1, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 1, 1]])
output:
array([[1, 1, 1, 1, 0],
[1, 1, 1, 1, 0],
[1, 1, 1, 1, 0],
[1, 1, 1, 1, 1],
[1, 1, 1, 1, 1]])
I am trying finding indexes but It does not work:
a=np.where(a==1)+1
From other post I also try getting the neighbors with this function:
def n_closest(x,n,d=1):
return x[n[0]-d:n[0]+d+1,n[1]-d:n[1]+d+1]
But this does not work for the edges
Thanks

If you don't mind using scipy, a 2D convolution will solve the problem quickly:
import numpy as np
from scipy import signal
# Input array
X = np.array([[0, 1, 0, 0, 0],
[0, 0, 0, 0, 0],
[0, 0, 1, 0, 0],
[1, 0, 0, 0, 0],
[0, 0, 0, 1, 1]])
# We apply a 2D convolution with a 3x3 kernel and we check which value are bigger than 0.
R = (signal.convolve2d(X,np.ones((3,3)),mode='same')>0).astype(int)
# R = array([[1, 1, 1, 0, 0],
# [1, 1, 1, 1, 0],
# [1, 1, 1, 1, 0],
# [1, 1, 1, 1, 1],
# [1, 1, 1, 1, 1]])
# Finally we extract the index
x,y = np.where(R)

How to create an array of binary digits of given unsigned integer numbers with Numpy?

I have an array of numbers between 0 and 3 and I want to create a 2D array of their binary digits.
in the future may be I need to have array of numbers between 0 and 7 or 0 to 15.
Currently my array is defined like this:
a = np.array([[0], [1], [2], [3]], dtype=np.uint8)
I used numpy unpackbits function:
b = np.unpackbits(a, axis=1)
and the result is this :
array([[0, 0, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0, 1],
[0, 0, 0, 0, 0, 0, 1, 0],
[0, 0, 0, 0, 0, 0, 1, 1]], dtype=uint8)
As you can see it created a 2d array with 8 items in column while I'm looking for 2 columns 2d array.
here is my desired array:
array([[0, 0],
[0, 1],
[1, 0],
[1, 1]])
Is this related to data type uint8 ?
what is your idea?

One way of approaching the problem is to just adapt your b to match your desired output via a simple slicing, similarly to what suggested in #GrzegorzSkibinski answer:
import numpy as np
def gen_bits_by_val(values):
n = int(max(values)).bit_length()
return np.unpackbits(values, axis=1)[:, -n:].copy()
print(gen_bits_by_val(a))
# [[0 0]
# [0 1]
# [1 0]
# [1 1]]
Alternatively, you could create a look-up table, similarly to what suggested in #WarrenWeckesser answer, using the following:
import numpy as np
def gen_bits_by_num(n):
values = np.arange(2 ** n, dtype=np.uint8).reshape(-1, 1)
return np.unpackbits(values, axis=1)[:, -n:].copy()
bits2 = gen_bits_by_num(2)
print(bits2)
# [[0 0]
# [0 1]
# [1 0]
# [1 1]]
which allows for all kind of uses thereby indicated, e.g.:
bits4 = gen_bits_by_num(4)
print(bits4[[1, 3, 12]])
# [[0 0 0 1]
# [0 0 1 1]
# [1 1 0 0]]
EDIT
Considering #PaulPanzer answer the line:
return np.unpackbits(values, axis=1)[:, -n:]
has been replaced with:
return np.unpackbits(values, axis=1)[:, -n:].copy()
which is more memory efficient.
It could have been replaced with:
return np.unpackbits(values << (8 - n), axis=1, count=n)
with similar effects.

You can use the count keyword. It cuts from the right so you also have to shift bits before applying unpackbits.
b = np.unpackbits(a<<6, axis=1, count=2)
b
# array([[0, 0],
# [0, 1],
# [1, 0],
# [1, 1]], dtype=uint8)
This produces a "clean" array:
b.flags
# C_CONTIGUOUS : True
# F_CONTIGUOUS : False
# OWNDATA : True
# WRITEABLE : True
# ALIGNED : True
# WRITEBACKIFCOPY : False
# UPDATEIFCOPY : False
In contrast, slicing the full 8-column output of unpackbits is in a sense a memory leak because the discarded columns will stay in memory as long as the slice lives.

You can truncate b to keep just the columns since the first column with 1:
b=b[:, int(np.argwhere(b.max(axis=0)==1)[0]):]

For such a small number of bits, you can use a lookup table.
For example, here bits2 is an array with shape (4, 2) that holds the bits of the integers 0, 1, 2, and 3. Index bits2 with the values from a to get the bits:
In [43]: bits2 = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
In [44]: a = np.array([[0], [1], [2], [3]], dtype=np.uint8)
In [45]: bits2[a[:, 0]]
Out[45]:
array([[0, 0],
[0, 1],
[1, 0],
[1, 1]])
This works fine for 3 or 4 bits, too:
In [46]: bits4 = np.array([[0, 0, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0], [0, 0, 1, 1], [0, 1, 0, 0], [
...: 0, 1, 0, 1], [0, 1, 1, 0], [0, 1, 1, 1], [1, 0, 0, 0], [1, 0, 0, 1], [1, 0, 1, 0], [1, 0,
...: 1, 1], [1, 1, 0, 0], [1, 1, 0, 1], [1, 1, 1, 0], [1, 1, 1, 1]])
In [47]: bits4
Out[47]:
array([[0, 0, 0, 0],
[0, 0, 0, 1],
[0, 0, 1, 0],
[0, 0, 1, 1],
[0, 1, 0, 0],
[0, 1, 0, 1],
[0, 1, 1, 0],
[0, 1, 1, 1],
[1, 0, 0, 0],
[1, 0, 0, 1],
[1, 0, 1, 0],
[1, 0, 1, 1],
[1, 1, 0, 0],
[1, 1, 0, 1],
[1, 1, 1, 0],
[1, 1, 1, 1]])
In [48]: x = np.array([0, 1, 5, 14, 9, 8, 15])
In [49]: bits4[x]
Out[49]:
array([[0, 0, 0, 0],
[0, 0, 0, 1],
[0, 1, 0, 1],
[1, 1, 1, 0],
[1, 0, 0, 1],
[1, 0, 0, 0],
[1, 1, 1, 1]])

scipy.ndimage.label: include error margin

After reading an interesting topic on scipy.ndimage.label (Variable area threshold for identifying objects - python), I'd like to include an 'error margin' in the labelling.
In the above linked discussion:
How can the blue dot on top be included, too (let's say it is wrongly disconnected from the orange, biggest, object)?
I found the structure attribute, which should be able to include that dot by changing the array (from np.ones(3,3,3) to anything more than that (I'd like it to be 3D). However, adjusting the 'structure' attribute to a larger array does not seem to work, unfortunately. It either gives an error of dimensions (RuntimeError: structure and input must have equal rank
) or it does not change anything..
Thanks!
this is the code:
labels, nshapes = ndimage.label(a, structure=np.ones((3,3,3)))
in which a is a 3D array.

Here's a possible approach that uses scipy.ndimage.binary_dilation. It is easier to see what is going on in a 2D example, but I'll show how to generalize to 3D at the end.
In [103]: a
Out[103]:
array([[0, 0, 0, 1, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0],
[1, 1, 0, 0, 1, 0, 0],
[1, 1, 0, 0, 0, 1, 1],
[0, 0, 0, 0, 0, 1, 1],
[1, 1, 1, 0, 0, 0, 0]])
In [104]: from scipy.ndimage import label, binary_dilation
Extend each "shape" by one pixel down and to the right:
In [105]: b = binary_dilation(a, structure=np.array([[0, 0, 0], [0, 1, 1], [0, 1, 1]])).astype(int)
In [106]: b
Out[106]:
array([[0, 0, 0, 1, 1, 0, 0],
[0, 0, 0, 1, 1, 0, 0],
[1, 1, 1, 0, 1, 1, 0],
[1, 1, 1, 0, 1, 1, 1],
[1, 1, 1, 0, 0, 1, 1],
[1, 1, 1, 1, 0, 1, 1]])
Apply label to the padded array:
In [107]: labels, numlabels = label(b)
In [108]: numlabels
Out[108]: 2
In [109]: labels
Out[109]:
array([[0, 0, 0, 1, 1, 0, 0],
[0, 0, 0, 1, 1, 0, 0],
[2, 2, 2, 0, 1, 1, 0],
[2, 2, 2, 0, 1, 1, 1],
[2, 2, 2, 0, 0, 1, 1],
[2, 2, 2, 2, 0, 1, 1]], dtype=int32)
By multiplying a by labels, we get the desired array of labels of a:
In [110]: alab = labels*a
In [111]: alab
Out[111]:
array([[0, 0, 0, 1, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 0],
[2, 2, 0, 0, 1, 0, 0],
[2, 2, 0, 0, 0, 1, 1],
[0, 0, 0, 0, 0, 1, 1],
[2, 2, 2, 0, 0, 0, 0]])
(This assumes that the values in a are 0 or 1. If they are not, you can use alab = labels * (a > 0).)
For a 3D input, you have to change the structure argument to binary_dilation:
struct = np.zeros((3, 3, 3), dtype=int)
struct[1:, 1:, 1:] = 1
b = binary_dilation(a, structure=struct).astype(int)

Numpy: filling the non-maximum elements of ndarray with zeros

I have a ndarray, and I want to set all the non-maximum elements in the last dimension to be zero.
a = np.array([[[1,8,3,4],[6,7,10,6],[11,12,15,4]],
[[4,2,3,4],[4,7,9,8],[41,14,15,3]],
[[4,22,3,4],[16,7,9,8],[41,12,15,43]]
])
print(a.shape)
(3,3,4)
I can get the indexes of maximum elements by np.argmax():
b = np.argmax(a, axis=2)
b
array([[1, 2, 2],
[0, 2, 0],
[1, 0, 3]])
Obviously, b has 1 dimension less than a. Now, I want to get a new 3-d array that has all zeros except for where the maximum values are.
I want to get this array:
np.array([[[0,1,0,0],[0,0,1,0],[0,0,1,0]],
[[1,0,0,1],[0,0,1,0],[1,0,0,0]],
[[0,1,0,0],[1,0,0,0],[0,0,0,1]]
])
One way to achieve this, I tried creating these temporary arrays
b = np.repeat(b[:,:,np.newaxis], 4, axis=2)
t = np.repeat(np.arange(4).reshape(4,1), 9, axis=1).T.reshape(b.shape)
z = np.zeros(shape=a.shape, dtype=int)
z[t == b] = 1
z
array([[[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 1, 0]],
[[1, 0, 0, 0],
[0, 0, 1, 0],
[1, 0, 0, 0]],
[[0, 1, 0, 0],
[1, 0, 0, 0],
[0, 0, 0, 1]]])
Any idea how to get this in a more efficient way?

Here's one way that uses broadcasting:
In [108]: (a == a.max(axis=2, keepdims=True)).astype(int)
Out[108]:
array([[[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 1, 0]],
[[1, 0, 0, 1],
[0, 0, 1, 0],
[1, 0, 0, 0]],
[[0, 1, 0, 0],
[1, 0, 0, 0],
[0, 0, 0, 1]]])