Good evening,
I'm trying to learn NumPy and have written a simple Linear transformation that applies to an image using for loops:
import numpy as np
M = np.array([
[width, 0],
[0, height]
])
T = np.array([
[1, 3],
[0, 1]
])
def transform_image(M, T):
T_rel_M = abs(M # T)
new_img = np.zeros(T_rel_M.sum(axis=1).astype("int")).T
for i in range(0, 440):
for j in range(0, 440):
x = np.array([j, i])
coords = (T # x)
x = coords[0]
y = coords[1]
new_img[y, -x] = image[i, -j]
return new_img
plt.imshow(transform_image(M, T))
It does what I want and spits out a transformation that is correct, except that I think there is a way to do this without the loops.
I tried doing some stuff with meshgrid but I couldn't figure out how to get the pixels from the image in the same way I do it in the loop (using i and j). I think I figured out how to apply the transformation but then getting the pixels from the image in the correct spots wouldn't work.
Any ideas?
EDIT:
Great help with below solutions, lezaf's solution was very similar to what I tried before, the only step missing that I couldn't figure out was assigning the pixels from the old to the new image. I made some changes to the code to exclude transposing, and also added a astype("int") so it works with float values in the T matrix:
def transform_image(M, T):
T_rel_M = abs(M # T)
new_img = np.zeros(T_rel_M.sum(axis=1).astype("int")).T
x_combs = np.array(np.meshgrid(np.arange(width), np.arange(height))).reshape(2,-1)
coords = (T # x_combs).astype("int")
new_img[coords[1, :], -coords[0, :]] = image[x_combs[1, :], -x_combs[0, :]]
return new_img
A more efficient solution is the following:
def transform_image(M, T):
T_rel_M = abs(M # T)
new_img = np.zeros(T_rel_M.sum(axis=1).astype("int")).T
# This one replaces the double for-loop
x_combs = np.array(np.meshgrid(np.arange(440), np.arange(440))).T.reshape(-1,2)
# Calculate the new coordinates
coords = (T#x_combs.T)
# Apply changes to new_img
new_img[coords[1, :], -coords[0, :]] = image[x_combs[:, 1], -x_combs[:,0]]
I updated my solution removing the for-loop, so now is a lot more straightforward.
After this change, the time of the optimized code is 50 ms compared to the initial 3.06 s of the code in question.
There seems to have some confusions between width/height, x/y, ... so not 100% my code won't need adaptation. But I think, the main idea is the one you are looking for
def transform_image(M, T):
T_rel_M = abs(M # T)
j,i=np.meshgrid(range(width), range(height))
ji=np.array((j.flatten(), i.flatten()))
coords = (T#ji).astype(int)
new_img=np.zeros((coords[1].max()+1, coords[0].max()+1), dtype=np.uint8)
new_img[coords[1], coords[0]] = image.flatten()
The main idea here is to build a set of coordinates of the input image with meshgrid. I don't want a 2d-array of coordinates. Just a list of coordinates (a list of pairs i,j). Hence the flatten. So ji is a huge 2×N array, N being the number of pixels (so width×height).
coords is the transformation of all those coordinates.
Since your original code seemed to have some inconsistency with size (the rotated image did not fit in the new_img), I choose the easy way to compute the size of new_img, and just compute the max of those coordinates (a bit overkill: the max of the four corners would be enough)
And then, I use this set of coordinates as indexes for new_img, to which I affect the matching image, that is image flatten
So, no for loop at all.
(Note that I've dropped the -x thing also. Just because I struggled to understand. I could have putted it back now that I have a working solution. But I am not 100% sure if it wasn't there because you also tried/errored some strange adjustment. But anyway, I think what you were looking for is how to use meshgrid to create a set of coordinates and process them without loop. Even if you may need to adapt my solution, you have it: flatten the coordinates of meshgrid, transform them with a matrix multiplication, and use them as index for places of all pixels of the original image)
Edit : variant
def transform_image(M, T):
T_rel_M = abs(M # T)
ji=np.array(np.meshgrid(range(width), range(height)))
coords = np.einsum('ik,kjl', T, ji).astype(int)
new_img=np.zeros((max(coords[1,0,-1],coords[1,-1,0], coords[1,-1,-1])+1, max(coords[0,0,-1], coords[0,-1,0], coords[0,-1,-1])+1), dtype=np.uint8)
new_img[coords[1].flatten(), coords[0].flatten()] = image.flatten()
return new_img
The idea is the same. But instead of flattening directly ji original coordinates, I keep them as is. Then use einsum to perform a matrix multiplication on a 3D array (which returns also a 2d 2×width×height arrays, whose each [:,j,i] value is just the transformation of [j,i]. So, it is just the same as previous #, except that it works even if, instead of having a 2×N set of coordinates we have a 2×width×height one).
Which has 2 advantages
Apparently it is sensibly faster to create ji than way
It allows the usage of just corners to find the size of the new image, as I've mentioned before (that was more difficult when coords was flatten from its creation).
Timing
Solution
Timing
Yours
4.5 s
lezaf's
3.2 s
This one
49 ms
The variant
41 ms
Related
I wrote the following function, which takes as inputs three 1D array (namely int_array, x, and y) and a number lim. The output is a number as well.
def integrate_to_lim(int_array, x, y, lim):
if lim >= np.max(x):
res = 0.0
if lim <= np.min(x):
res = int_array[0]
else:
index = np.argmax(x > lim) # To find the first element of x larger than lim
partial = int_array[index]
slope = (y[index-1] - y[index]) / (x[index-1] - x[index])
rest = (x[index] - lim) * (y[index] + (lim - x[index]) * slope / 2.0)
res = partial + rest
return res
Basically, outside form the limit cases lim>=np.max(x) and lim<=np.min(x), the idea is that the function finds the index of the first value of the array x larger than lim and then uses it to make some simple calculations.
In my case, however lim can also be a fairly big 2D array (shape ~2000 times ~1000 elements)
I would like to rewrite it such that it makes the same calculations for the case that lim is a 2D array.
Obviously, the output should also be a 2D array of the same shape of lim.
I am having a real struggle figuring out how to vectorize it.
I would like to stick only to the numpy package.
PS I want to vectorize my function because efficiency is important and as I understand using for loops is not a good choice in this regard.
Edit: my attempt
I was not aware of the function np.take, which made the task way easier.
Here is my brutal attempt that seems to work (suggestions on how to clean up or to make the code faster are more than welcome).
def integrate_to_lim_vect(int_array, x, y, lim_mat):
lim_mat = np.asarray(lim_mat) # Make sure that it is an array
shape_3d = list(lim_mat.shape) + [1]
x_3d = np.ones(shape_3d) * x # 3 dimensional version of x
lim_3d = np.expand_dims(lim_mat, axis=2) * np.ones(x_3d.shape) # also 3d
# I use np.argmax on the 3d matrices (is there a simpler way?)
index_mat = np.argmax(x_3d > lim_3d, axis=2)
# Silly calculations
partial = np.take(int_array, index_mat)
y1_mat = np.take(y, index_mat)
y2_mat = np.take(y, index_mat - 1)
x1_mat = np.take(x, index_mat)
x2_mat = np.take(x, index_mat - 1)
slope = (y1_mat - y2_mat) / (x1_mat - x2_mat)
rest = (x1_mat - lim_mat) * (y1_mat + (lim_mat - x1_mat) * slope / 2.0)
res = partial + rest
# Make the cases with np.select
condlist = [lim_mat >= np.max(x), lim_mat <= np.min(x)]
choicelist = [0.0, int_array[0]] # Shoud these options be a 2d matrix?
output = np.select(condlist, choicelist, default=res)
return output
I am aware that if the limit is larger than the maximum value in the array np.argmax returns the index zero (leading to wrong results). This is why I used np.select to check and correct for these cases.
Is it necessary to define the three dimensional matrices x_3d and lim_3d, or there is a simpler way to find the 2D matrix of the indices index_mat?
Suggestions, especially to improve the way I expanded the dimension of the arrays, are welcome.
I think you can solve this using two tricks. First, a 2d array can be easily flattened to a 1d array, and then your answers can be converted back into a 2d array with reshape.
Next, your use of argmax suggests that your array is sorted. Then you can find your full set of indices using digitize. Thus instead of a single index, you will get a complete array of indices. All the calculations you are doing are intrinsically supported as array operations in numpy, so that should not cause any problems.
You will have to specifically look at the limiting cases. If those are rare enough, then it might be okay to let the answers be derived by the default formula (they will be garbage values), and then replace them with the actual values you desire.
I work with raster images and the module rasterio to imprt them as numpy arrays. I would like to cut a portion of size (1000, 1000) out of the middle of each (to avoid the out-of-bound masks of the image).
image = np.random.random_sample((2000, 2000))
s = image.shape
mid = [round(x / 2) for x in s] # middle point of both axes
margins = [[y + x for y in [-500, 500]] for x in mid] # 1000 range around every middle point
The result is a list of 2 lists, for the cut range on each axis. But this is where I stump: range() doesn't accept lists, and I'm attempting the following brute force method:
cut_image = image[range(margins[0][0], margins[0][1]), range(margins[1][0], margins[1][1])]
However:
cut_image.shape
## (1000,)
Slicing an array loses dimension information which is exactly what I don't want.
Consider me confused.
Looking for a more tasteful solution.
As the other answer points it out, you're not really slicing your array, but using indexing on it.
If you want to slice your array (and you're right, that's more elegant than using list of indices) , you'll be happier with slices. That objects represents the start:end:step syntax.
In your case,
import numpy as np
WND = 50
image = np.random.random_sample((200, 300))
s = image.shape
mid = [round(x / 2) for x in s] # middle point of both axes
margins = [[y + x for y in [-WND, WND]] for x in mid] # 1000 range around every middle point
# array[slice(start, end)] -> array[start:end]
x_slice = slice(margins[0][0], margins[0][1])
y_slice = slice(margins[1][0], margins[1][1])
print(x_slice, y_slice)
# slice(50, 150, None) slice(100, 200, None)
cut_image = image[x_slice, y_slice]
print(cut_image.shape)
# (100,100)
Indexing ?
You might wonder what was happening in your question that resulted in only 1000 elements instead of the expected 1000*1000.
Here is a simpler example of indexing with lists on different dimensions
# n and n2 have identical values
n = a[[i0, i1, i2],[j0, j1, j2]]
n2 = np.array([a[i0, j0], a[i1, j1], a[i2, j2]]
This being clarified, you'll understand that instead of taking a block matrix, your code only returns the diagonal coefficients of that block matrix :)
The issue here is that what you're doing is known as integer indexing, instead of slice indexing. The bahaviour changes and may seem counterintuitive when not acquainted with it. You can check the docs for more details.
Here's how you could do it with basic slicing:
# center coordinates of the image
x_0, y_0 = np.asarray(image.shape)//2
# slice taken from the center point
out = image[x_0-x_0//2:x_0+x_0//2, y_0-y_0//2:y_0+y_0//2]
print(out.shape)
# (1000, 1000)
Attempting to do forward warping of a homography matrix in OpenCV. You don't have to know what that means to understand the issue though.
Assume there are 2 images (an image is a 2D Numpy array of pixel values), A and B, and an array match that looks like
[[ 6.96122642e+01 -1.06556338e+03 1.02251944e+00]
[ 6.92265938e+01 -1.06334423e+03 1.02246589e+00]
[ 6.88409234e+01 -1.06112508e+03 1.02241234e+00]
... ]
The first column is X, second Y, and third is a scalar. These XY values are image A pixel indices and correspond to the imageB indexes
[[0,0],
[0,1],
[0,2]
... ]
I want to use this info to quickly set imageB values from imageA. I have this working but it is not as fast as I'd like
yAs = np.int32(np.round( match[:, 0] / match[:, 2] )
xAs = np.int32(np.round( match[:, 1] / match[:, 2] )
it = np.nditer(pixelsImageB[0], flags=['f_index'])
while not it.finished:
i = it.index
xA = xAs[i]
yA = yAs[i]
if in_bounds(xA, yA, imageA):
yB = pixB[0][i]
xB = pixB[1][i]
imageB[xB,yB] = imageA[xA,yA]
it.iternext()
But I'm not sure how to make this fast in Numpy, naively doing this loop is very slow. I'm a total scrub at advanced indexing, broadcasting, and the like. Any ideas?
The fastest way would be to not reinvent the wheel and use cv.WarpPerspective function.
Alternatively, you can use Pillow Image.transform method which according to docs has slight advantage over OpenCV in that it also supports bicubic interpolation, which should produce output of better quality.
I'm working on a computer vision project and am looking to build a fast function that compares two images and outputs only the pixels where the differences between the pixels of the two images are sufficiently different. Other pixels get set to (0,0,0). In practice, I want the camera to detect objects and ignore the background.
My issue is the function doesn't run fast enough. What are some ways to speed things up?
def get_diff_image(fixed_image):
#get new image
new_image = current_image()
#get diff
diff = abs(fixed_image-new_image)
#creating a filter array
filter_array = np.empty(shape = (fixed_image.shape[0], fixed_image.shape[1]))
for idx, row in enumerate(diff):
for idx2, pixel in enumerate(row):
mag = np.linalg.norm(pixel)
if mag > 40:
filter_array[idx][idx2] = True
else:
filter_array[idx][idx2] = False
#applying filter
filter_image = np.copy(new_image)
filter_image[filter_array == False] = [0, 0, 0]
return filter_image
As others have mentioned, your biggest slow down in this code is iterating over every pixel in Python. Since Python is an interpreted language, these iterations take much longer than their equivalents in C/C++, which numpy uses under the hood.
Conveniently, you can specify an axis for numpy.linalg.norm, so you can get all the magnitudes in one numpy command. In this case, your pixels are on axis 2, so we'll take the norm on that axis, like this:
mags = np.linalg.norm(diff, axis=2)
Here, mags will have the same shape as filter_array, and each location will hold the magnitude of the corresponding pixel.
Using a boolean operator on a numpy array returns an array of bools, so:
filter_array = mags > 40
With the loops removed, the whole thing looks like this:
def get_diff_image(fixed_image):
#get new image
new_image = current_image()
#get diff
diff = abs(fixed_image-new_image)
#creating a filter array
mags = np.linalg.norm(diff, axis=2)
filter_array = mags > 40
#applying filter
filter_image = np.copy(new_image)
filter_image[filter_array == False] = [0, 0, 0]
return filter_image
But there is still more efficiency to be gained.
As noted by pete2fiddy, the magnitude of a vector doesn't depend on its direction. The absolute value operator only changes direction, not magnitude, so we just don't need it here. Sweet!
The biggest remaining performance gain is to avoid copying the image. If you need to preserve the original image, start by allocating zeros for the output array since zeroing memory is often hardware accelerated. Then, copy only the required pixels. If you don't need the original and only plan to use the filtered one, then modifying the image in-place will give much better performance.
Here's an updated function with those changes in place:
def get_diff_image(fixed_image):
#get new image
new_image = current_image()
# Compute difference magnitudes
mags = np.linalg.norm(fixed_image - new_image, axis=2)
# Preserve original image
filter_image = np.zeros_like(new_image)
filter_image[mag > 40] = new_image[mag > 40]
return filter_image
# Avoid copy entirely (overwrites original!)
# new_image[mag < 40] = [0, 0, 0]
# return new_image
The slow part is your loop.
for idx, row in enumerate(diff):
for idx2, pixel in enumerate(row):
mag = np.linalg.norm(pixel)
if mag > 40:
filter_array[idx][idx2] = True
else:
filter_array[idx][idx2] = False
Doing this in Python code is much slower than doing this in numpy's implicit loop over an array. I believe that this would apply to the norm call in which case you would have mag = np.linalg.norm(diff) which would create the array mag. You could the use numpy's greater function applied to the whole array to get filter_array.
The loop over the array will then be done in numpy's C code which is an order of magnitude faster (in general).
Nested loops in python run very slowly. However, this can often be mitigated through letting numpy which is largely written in C. Try giving this a go:
def get_image_difference_magnitudes(image1, image2):
'''order of subtraction does not matter, because the magnitude
of any vector is always a positive scalar'''
image_subtractions = image1-image2
'''checks if the image has channels or not, i.e.
if the two images are grayscaled or not. Calculating the magnitude
of change differs for both cases'''
if len(image1.shape) == 2:
return np.abs(image_subtractions)
'''returns the magnitude of change for each pixel. The "axis = 2"
part of np.linalg.norm specifies that, rather than the output that
does not specify the axis, it will return an image that replaces each
pixel, which contains a vector of the difference between the two images,
with the magnitude of the vector at that pixel'''
return np.linalg.norm(image_subtractions, axis = 2)
def difference_magnitude_threshold_images(image1, image2, threshold):
image_subtraction_magnitudes = get_image_difference_magnitudes(image1, image2)
'''creates a numpy array of "False" that is the same shape as the
width and height of the image. Slicing the shape to 2 makes it so
that the mask is only width x height, and not width x height x 3,
in the case of RGB'''
threshold_mask = np.zeros(image1.shape[:2], dtype = np.bool)
'''checks each element of image_subtraction_magnitudes, and for
all instances where the magnitude of difference at that pixel is
greater than threshold, set the mask = to True'''
threshold_mask[image_subtraction_magnitudes > threshold] = True
return threshold_mask
The above also has the benefit of being only a few lines long (were you to fit it into one function). The way you apply the filter looks fine to me -- I'm not sure if there is a faster way to do it. Also, I apologize for the formatting. I wasn't sure how to indent within a code block, so I left the function names outside of the code block.
I'm trying to reduce noise in a binary python array by removing all completely isolated single cells, i.e. setting "1" value cells to 0 if they are completely surrounded by other "0"s. I have been able to get a working solution by removing blobs with sizes equal to 1 using a loop, but this seems like a very inefficient solution for large arrays:
import numpy as np
import scipy.ndimage as ndimage
import matplotlib.pyplot as plt
# Generate sample data
square = np.zeros((32, 32))
square[10:-10, 10:-10] = 1
np.random.seed(12)
x, y = (32*np.random.random((2, 20))).astype(np.int)
square[x, y] = 1
# Plot original data with many isolated single cells
plt.imshow(square, cmap=plt.cm.gray, interpolation='nearest')
# Assign unique labels
id_regions, number_of_ids = ndimage.label(square, structure=np.ones((3,3)))
# Set blobs of size 1 to 0
for i in xrange(number_of_ids + 1):
if id_regions[id_regions==i].size == 1:
square[id_regions==i] = 0
# Plot desired output, with all isolated single cells removed
plt.imshow(square, cmap=plt.cm.gray, interpolation='nearest')
In this case, eroding and dilating my array won't work as it will also remove features with a width of 1. I feel the solution lies somewhere within the scipy.ndimage package, but so far I haven't been able to crack it. Any help would be greatly appreciated!
A belated thanks to both Jaime and Kazemakase for their replies. The manual neighbour-checking method did remove all isolated patches, but also removed patches attached to others by one corner (i.e. to the upper-right of the square in the sample array). The summed area table works perfectly and is very fast on the small sample array, but slows down on larger arrays.
I ended up following a approach using ndimage which seems to work efficiently for very large and sparse arrays (0.91 sec for 5000 x 5000 array vs 1.17 sec for summed area table approach). I first generated a labelled array of unique IDs for each discrete region, calculated sizes for each ID, masked the size array to focus only on size == 1 blobs, then index the original array and set IDs with a size == 1 to 0:
def filter_isolated_cells(array, struct):
""" Return array with completely isolated single cells removed
:param array: Array with completely isolated single cells
:param struct: Structure array for generating unique regions
:return: Array with minimum region size > 1
"""
filtered_array = np.copy(array)
id_regions, num_ids = ndimage.label(filtered_array, structure=struct)
id_sizes = np.array(ndimage.sum(array, id_regions, range(num_ids + 1)))
area_mask = (id_sizes == 1)
filtered_array[area_mask[id_regions]] = 0
return filtered_array
# Run function on sample array
filtered_array = filter_isolated_cells(square, struct=np.ones((3,3)))
# Plot output, with all isolated single cells removed
plt.imshow(filtered_array, cmap=plt.cm.gray, interpolation='nearest')
Result:
You can manually check the neighbors and avoid the loop using vectorization.
has_neighbor = np.zeros(square.shape, bool)
has_neighbor[:, 1:] = np.logical_or(has_neighbor[:, 1:], square[:, :-1] > 0) # left
has_neighbor[:, :-1] = np.logical_or(has_neighbor[:, :-1], square[:, 1:] > 0) # right
has_neighbor[1:, :] = np.logical_or(has_neighbor[1:, :], square[:-1, :] > 0) # above
has_neighbor[:-1, :] = np.logical_or(has_neighbor[:-1, :], square[1:, :] > 0) # below
square[np.logical_not(has_neighbor)] = 0
That way looping over the square is performed internally by numpy, which is rather more efficient than looping in python. There are two drawbacks of this solution:
If your array is very sparse there may be more efficient ways to check the neighborhood of non-zero points.
If your array is very large the has_neighbor array might consume too much memory. In this case you could loop over sub-arrays of smaller size (trade-off between python loops and vectorization).
I have no experience with ndimage, so there may be a better solution built in somewhere.
The typical way of getting rid of isolated pixels in image processing is to do a morphological opening, for which you have a ready-made implementation in scipy.ndimage.morphology.binary_opening. This would affect the contours of your larger areas as well though.
As for a DIY solution, I would use a summed area table to count the number of items in every 3x3 subimage, subtract from that the value of the central pixel, then zero all center points where the result came out to zero. To properly handle the borders, first pad the array with zeros:
sat = np.pad(square, pad_width=1, mode='constant', constant_values=0)
sat = np.cumsum(np.cumsum(sat, axis=0), axis=1)
sat = np.pad(sat, ((1, 0), (1, 0)), mode='constant', constant_values=0)
# These are all the possible overlapping 3x3 windows sums
sum3x3 = sat[3:, 3:] + sat[:-3, :-3] - sat[3:, :-3] - sat[:-3, 3:]
# This takes away the central pixel value
sum3x3 -= square
# This zeros all the isolated pixels
square[sum3x3 == 0] = 0
The implementation above works, but is not especially careful about not creating intermediate arrays, so you can probably shave off some execution time by refactoring adequately.