Develop Reference

Creating Mask that Applies to Vectors in 3D Array

I could not find a previous post that specifically addressed how to create masks that work against vectors in a 3D array. I have only found previous questions and answers that either address only how masks can be applied to individual elements in a 3D array or vectors in a 2D array. So as the title states, that is exactly what I wish to do here. I want to remove all zero vectors from a 3D (x,y,z) array and the only method I can think of is to create two for loops that run over both x and (y,:) as shown in the code below. However, this does not work either because of the error message I receive when I try to run this.
'list' object cannot be safely interpreted as an integer
Moreover, even if I do get this method to work somehow, I know that using a double for loop will make this masking process very time consuming because eventually I want to apply this to array sizes in the millions. So this develops into my main question; What would be the fastest method to accomplish this?
Code:
import numpy as np
data = np.array([[[0,0,0],[1,2,3],[4,5,6],[0,0,0]],[[7,8,9],[0,0,0],[0,0,0],[10,11,12]]],dtype=float)
datanonzero = np.empty([[],[]],dtype=float)
for maskclear1 in range(0,2):
for maskclear2 in range(0,4):
datanonzero[maskclear1,maskclear2,:] = data[~np.all(data[maskclear1,maskclear2,0:3] == 0, axis=0)

import numpy as np
data = np.array([[[0,0,0],[1,2,3],[4,5,6],[0,0,0]],[[7,8,9],[0,0,0],[0,0,0],[10,11,12]]],dtype=float)
flatten_data = data.reshape(-1, 3)
datanonzero = [ data[~np.all(vec == 0, axis=0)] for vec in flatten_data ]
datanonzero = np.reshape(datanonzero, (2,-1))

Alternative to loop for for boolean / nonzero indexing of numpy array

I need to select only the non-zero 3d portions of a 3d binary array (or alternatively the true values of a boolean array). Currently I am able to do so with a series of 'for' loops that use np.any, but this does work but seems awkward and slow, so currently investigating a more direct way to accomplish the task.
I am rather new to numpy, so the approaches that I have tried include a) using
np.nonzero, which returns indices that I am at a loss to understand what to do with for my purposes, b) boolean array indexing, and c) boolean masks. I can generally understand each of those approaches for simple 2d arrays, but am struggling to understand the differences between the approaches, and cannot get them to return the right values for a 3d array.
Here is my current function that returns a 3D array with nonzero values:
def real_size(arr3):
true_0 = []
true_1 = []
true_2 = []
print(f'The input array shape is: {arr3.shape}')
for zero_ in range (0, arr3.shape[0]):
if arr3[zero_].any()==True:
true_0.append(zero_)
for one_ in range (0, arr3.shape[1]):
if arr3[:,one_,:].any()==True:
true_1.append(one_)
for two_ in range (0, arr3.shape[2]):
if arr3[:,:,two_].any()==True:
true_2.append(two_)
arr4 = arr3[min(true_0):max(true_0) + 1, min(true_1):max(true_1) + 1, min(true_2):max(true_2) + 1]
print(f'The nonzero area is: {arr4.shape}')
return arr4
# Then use it on a small test array:
test_array = np.zeros([2, 3, 4], dtype = int)
test_array[0:2, 0:2, 0:2] = 1
#The function call works and prints out as expected:
non_zero = real_size(test_array)
>> The input array shape is: (2, 3, 4)
>> The nonzero area is: (2, 2, 2)
# So, the array is correct, but likely not the best way to get there:
non_zero
>> array([[[1, 1],
[1, 1]],
[[1, 1],
[1, 1]]])
The code works appropriately, but I am using this on much larger and more complex arrays, and don't think this is an appropriate approach. Any thoughts on a more direct method to make this work would be greatly appreciated. I am also concerned about errors and the results if the input array has for example two separate non-zero 3d areas within the original array.
To clarify the problem, I need to return one or more 3D portions as one or more 3d arrays beginning with an original larger array. The returned arrays should not include extraneous zeros (or false values) in any given exterior plane in three dimensional space. Just getting the indices of the nonzero values (or vice versa) doesn't by itself solve the problem.

Assuming you want to eliminate all rows, columns, etc. that contain only zeros, you could do the following:
nz = (test_array != 0)
non_zero = test_array[nz.any(axis=(1, 2))][:, nz.any(axis=(0, 2))][:, :, nz.any(axis=(0, 1))]
An alternative solution using np.nonzero:
i = [np.unique(_) for _ in np.nonzero(test_array)]
non_zero = test_array[i[0]][:, i[1]][:, :, i[2]]
This can also be generalized to arbitrary dimensions, but requires a bit more work (only showing the first approach here):
def real_size(arr):
nz = (arr != 0)
result = arr
axes = np.arange(arr.ndim)
for axis in range(arr.ndim):
zeros = nz.any(axis=tuple(np.delete(axes, axis)))
result = result[(slice(None),)*axis + (zeros,)]
return result
non_zero = real_size(test_array)

Improving performance of a Python function that outputs the pixels that are different between two images

I'm working on a computer vision project and am looking to build a fast function that compares two images and outputs only the pixels where the differences between the pixels of the two images are sufficiently different. Other pixels get set to (0,0,0). In practice, I want the camera to detect objects and ignore the background.
My issue is the function doesn't run fast enough. What are some ways to speed things up?
def get_diff_image(fixed_image):
#get new image
new_image = current_image()
#get diff
diff = abs(fixed_image-new_image)
#creating a filter array
filter_array = np.empty(shape = (fixed_image.shape[0], fixed_image.shape[1]))
for idx, row in enumerate(diff):
for idx2, pixel in enumerate(row):
mag = np.linalg.norm(pixel)
if mag > 40:
filter_array[idx][idx2] = True
else:
filter_array[idx][idx2] = False
#applying filter
filter_image = np.copy(new_image)
filter_image[filter_array == False] = [0, 0, 0]
return filter_image

As others have mentioned, your biggest slow down in this code is iterating over every pixel in Python. Since Python is an interpreted language, these iterations take much longer than their equivalents in C/C++, which numpy uses under the hood.
Conveniently, you can specify an axis for numpy.linalg.norm, so you can get all the magnitudes in one numpy command. In this case, your pixels are on axis 2, so we'll take the norm on that axis, like this:
mags = np.linalg.norm(diff, axis=2)
Here, mags will have the same shape as filter_array, and each location will hold the magnitude of the corresponding pixel.
Using a boolean operator on a numpy array returns an array of bools, so:
filter_array = mags > 40
With the loops removed, the whole thing looks like this:
def get_diff_image(fixed_image):
#get new image
new_image = current_image()
#get diff
diff = abs(fixed_image-new_image)
#creating a filter array
mags = np.linalg.norm(diff, axis=2)
filter_array = mags > 40
#applying filter
filter_image = np.copy(new_image)
filter_image[filter_array == False] = [0, 0, 0]
return filter_image
But there is still more efficiency to be gained.
As noted by pete2fiddy, the magnitude of a vector doesn't depend on its direction. The absolute value operator only changes direction, not magnitude, so we just don't need it here. Sweet!
The biggest remaining performance gain is to avoid copying the image. If you need to preserve the original image, start by allocating zeros for the output array since zeroing memory is often hardware accelerated. Then, copy only the required pixels. If you don't need the original and only plan to use the filtered one, then modifying the image in-place will give much better performance.
Here's an updated function with those changes in place:
def get_diff_image(fixed_image):
#get new image
new_image = current_image()
# Compute difference magnitudes
mags = np.linalg.norm(fixed_image - new_image, axis=2)
# Preserve original image
filter_image = np.zeros_like(new_image)
filter_image[mag > 40] = new_image[mag > 40]
return filter_image
# Avoid copy entirely (overwrites original!)
# new_image[mag < 40] = [0, 0, 0]
# return new_image

The slow part is your loop.
for idx, row in enumerate(diff):
for idx2, pixel in enumerate(row):
mag = np.linalg.norm(pixel)
if mag > 40:
filter_array[idx][idx2] = True
else:
filter_array[idx][idx2] = False
Doing this in Python code is much slower than doing this in numpy's implicit loop over an array. I believe that this would apply to the norm call in which case you would have mag = np.linalg.norm(diff) which would create the array mag. You could the use numpy's greater function applied to the whole array to get filter_array.
The loop over the array will then be done in numpy's C code which is an order of magnitude faster (in general).

Nested loops in python run very slowly. However, this can often be mitigated through letting numpy which is largely written in C. Try giving this a go:
def get_image_difference_magnitudes(image1, image2):
'''order of subtraction does not matter, because the magnitude
of any vector is always a positive scalar'''
image_subtractions = image1-image2
'''checks if the image has channels or not, i.e.
if the two images are grayscaled or not. Calculating the magnitude
of change differs for both cases'''
if len(image1.shape) == 2:
return np.abs(image_subtractions)
'''returns the magnitude of change for each pixel. The "axis = 2"
part of np.linalg.norm specifies that, rather than the output that
does not specify the axis, it will return an image that replaces each
pixel, which contains a vector of the difference between the two images,
with the magnitude of the vector at that pixel'''
return np.linalg.norm(image_subtractions, axis = 2)
def difference_magnitude_threshold_images(image1, image2, threshold):
image_subtraction_magnitudes = get_image_difference_magnitudes(image1, image2)
'''creates a numpy array of "False" that is the same shape as the
width and height of the image. Slicing the shape to 2 makes it so
that the mask is only width x height, and not width x height x 3,
in the case of RGB'''
threshold_mask = np.zeros(image1.shape[:2], dtype = np.bool)
'''checks each element of image_subtraction_magnitudes, and for
all instances where the magnitude of difference at that pixel is
greater than threshold, set the mask = to True'''
threshold_mask[image_subtraction_magnitudes > threshold] = True
return threshold_mask
The above also has the benefit of being only a few lines long (were you to fit it into one function). The way you apply the filter looks fine to me -- I'm not sure if there is a faster way to do it. Also, I apologize for the formatting. I wasn't sure how to indent within a code block, so I left the function names outside of the code block.

Removing completely isolated cells from Python array?

I'm trying to reduce noise in a binary python array by removing all completely isolated single cells, i.e. setting "1" value cells to 0 if they are completely surrounded by other "0"s. I have been able to get a working solution by removing blobs with sizes equal to 1 using a loop, but this seems like a very inefficient solution for large arrays:
import numpy as np
import scipy.ndimage as ndimage
import matplotlib.pyplot as plt
# Generate sample data
square = np.zeros((32, 32))
square[10:-10, 10:-10] = 1
np.random.seed(12)
x, y = (32*np.random.random((2, 20))).astype(np.int)
square[x, y] = 1
# Plot original data with many isolated single cells
plt.imshow(square, cmap=plt.cm.gray, interpolation='nearest')
# Assign unique labels
id_regions, number_of_ids = ndimage.label(square, structure=np.ones((3,3)))
# Set blobs of size 1 to 0
for i in xrange(number_of_ids + 1):
if id_regions[id_regions==i].size == 1:
square[id_regions==i] = 0
# Plot desired output, with all isolated single cells removed
plt.imshow(square, cmap=plt.cm.gray, interpolation='nearest')
In this case, eroding and dilating my array won't work as it will also remove features with a width of 1. I feel the solution lies somewhere within the scipy.ndimage package, but so far I haven't been able to crack it. Any help would be greatly appreciated!

A belated thanks to both Jaime and Kazemakase for their replies. The manual neighbour-checking method did remove all isolated patches, but also removed patches attached to others by one corner (i.e. to the upper-right of the square in the sample array). The summed area table works perfectly and is very fast on the small sample array, but slows down on larger arrays.
I ended up following a approach using ndimage which seems to work efficiently for very large and sparse arrays (0.91 sec for 5000 x 5000 array vs 1.17 sec for summed area table approach). I first generated a labelled array of unique IDs for each discrete region, calculated sizes for each ID, masked the size array to focus only on size == 1 blobs, then index the original array and set IDs with a size == 1 to 0:
def filter_isolated_cells(array, struct):
""" Return array with completely isolated single cells removed
:param array: Array with completely isolated single cells
:param struct: Structure array for generating unique regions
:return: Array with minimum region size > 1
"""
filtered_array = np.copy(array)
id_regions, num_ids = ndimage.label(filtered_array, structure=struct)
id_sizes = np.array(ndimage.sum(array, id_regions, range(num_ids + 1)))
area_mask = (id_sizes == 1)
filtered_array[area_mask[id_regions]] = 0
return filtered_array
# Run function on sample array
filtered_array = filter_isolated_cells(square, struct=np.ones((3,3)))
# Plot output, with all isolated single cells removed
plt.imshow(filtered_array, cmap=plt.cm.gray, interpolation='nearest')
Result:

You can manually check the neighbors and avoid the loop using vectorization.
has_neighbor = np.zeros(square.shape, bool)
has_neighbor[:, 1:] = np.logical_or(has_neighbor[:, 1:], square[:, :-1] > 0) # left
has_neighbor[:, :-1] = np.logical_or(has_neighbor[:, :-1], square[:, 1:] > 0) # right
has_neighbor[1:, :] = np.logical_or(has_neighbor[1:, :], square[:-1, :] > 0) # above
has_neighbor[:-1, :] = np.logical_or(has_neighbor[:-1, :], square[1:, :] > 0) # below
square[np.logical_not(has_neighbor)] = 0
That way looping over the square is performed internally by numpy, which is rather more efficient than looping in python. There are two drawbacks of this solution:
If your array is very sparse there may be more efficient ways to check the neighborhood of non-zero points.
If your array is very large the has_neighbor array might consume too much memory. In this case you could loop over sub-arrays of smaller size (trade-off between python loops and vectorization).
I have no experience with ndimage, so there may be a better solution built in somewhere.

The typical way of getting rid of isolated pixels in image processing is to do a morphological opening, for which you have a ready-made implementation in scipy.ndimage.morphology.binary_opening. This would affect the contours of your larger areas as well though.
As for a DIY solution, I would use a summed area table to count the number of items in every 3x3 subimage, subtract from that the value of the central pixel, then zero all center points where the result came out to zero. To properly handle the borders, first pad the array with zeros:
sat = np.pad(square, pad_width=1, mode='constant', constant_values=0)
sat = np.cumsum(np.cumsum(sat, axis=0), axis=1)
sat = np.pad(sat, ((1, 0), (1, 0)), mode='constant', constant_values=0)
# These are all the possible overlapping 3x3 windows sums
sum3x3 = sat[3:, 3:] + sat[:-3, :-3] - sat[3:, :-3] - sat[:-3, 3:]
# This takes away the central pixel value
sum3x3 -= square
# This zeros all the isolated pixels
square[sum3x3 == 0] = 0
The implementation above works, but is not especially careful about not creating intermediate arrays, so you can probably shave off some execution time by refactoring adequately.

Best practice for fancy indexing a numpy array along multiple axes

I'm trying to optimize an algorithm to reduce memory usage, and I've identified this particular operation as a pain point.
I have a symmetric matrix, an index array along the rows, and another index array along the columns (which is just all values that I wasn't selecting in the row index). I feel like I should just be able to pass in both indexes at the same time, but I find myself being forced to select along one axis and then the other, which is causing some memory issues because I don't actually need the copy of the array that's returned, just statistics I'm calculating from it. Here's what I am trying to do:
from scipy.spatial.distance import pdist, squareform
from sklearn import datasets
import numpy as np
iris = datasets.load_iris().data
dx = pdist(iris)
mat = squareform(dx)
outliers = [41,62,106,108,109,134,135]
inliers = np.setdiff1d( range(iris.shape[0]), outliers)
# What I want to be able to do:
scores = mat[inliers, outliers].min(axis=0)
Here's what I'm actually doing to make this work:
# What I'm being forced to do:
s1 = mat[:,outliers]
scores = s1[inliers,:].min(axis=0)
Because I'm fancy indexing, s1 is a new array instead of a view. I only need this array for one operation, so if I could eliminate returning a copy here or at least make the new array smaller (i.e. by respecting the second fancy index selection while I'm doing the first one instead of two separate fancy index operations) that would be preferable.

"Broadcasting" applies to indexing. You could convert inliers into column matrix (e.g. inliers.reshape(-1,1) or inliers[:, np.newaxis], so it has shape (m,1)) and index mat with that in the first column:
s1 = mat[inliers.reshape(-1,1), outliers]
scores = s1.min(axis=0)

There's a better way in terms of readability:
result = mat[np.ix_(inliers, outliers)].min(0)
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ix_.html#numpy.ix_

Try:
outliers = np.array(outliers) # just to be sure they are arrays
result = mat[inliers[:, np.newaxis], outliers[np.newaxis, :]].min(0)