Develop Reference

Fast Way to Perform Array Computation in Python

I have an image that I want to perform some calculations on. The image pixels will be represented as f(x, y) where x is the column number and y is the row number of each pixel. I want to perform a calculation using the following formula:
Here is the code that does the calculation:
import matplotlib.pyplot as plt
import numpy as np
import os.path
from PIL import Image
global image_width, image_height
# A. Blur Measurement
def measure_blur(f):
D_sub_h = [[0 for y in range(image_height)] for x in range(image_width)]
for x in range(image_width):
for y in range(image_height):
if(y == 0):
f_x_yp1 = f[x][y+1]
f_x_ym1 = 0
elif(y == (image_height -1)):
f_x_yp1 = 0
f_x_ym1 = f[x][y -1]
else:
f_x_yp1 = f[x][y+1]
f_x_ym1 = f[x][y -1]
D_sub_h[x][y] = abs(f_x_yp1 - f_x_ym1)
return D_sub_h
if __name__ == '__main__':
image_counter = 1
while True:
if not os.path.isfile(str (image_counter) + '.jpg'):
break
image_path = str(image_counter) + '.jpg'
image = Image.open(image_path )
image_height, image_width = image.size
print("Image Width : " + str(image_width))
print("Image Height : " + str(image_height))
f = np.array(image)
D_sub_h = measure_blur(f)
image_counter = image_counter + 1
The problem with this code is when the image size becomes large, such as (5000, 5000), it takes a very long time to complete. Is there any way or function I can use to make the execution time faster by not doing one by one or manual computation?

Since you specifically convert the input f to a numpy array, I am assuming you want to use numpy. In that case, the allocation of D_sub_h needs to change from a list to an array:
D_sub_h = np.empty_like(f)
If we assume that everything outside your array is zeros, then the first row and last row can be computed as the second and negative second-to-last rows, respectively:
D_sub_h[0, :] = f[1, :]
D_sub_h[-1, :] = -f[-2, :]
The remainder of the data is just the difference between the next and previous index at each location, which is idiomatically computed by shifting views: f[2:, :] - f[:-2, :]. This formulation creates a temporary array. You can avoid doing that by using np.subtract explicitly:
np.subtract(f[2:, :], f[:-2, :], out=D_sub_h[1:-1, :])
The entire thing takes four lines in this formulation, and is fully vectorized, which means that loops run quickly under the hood, without most of Python's overhead:
def measure_blur(f):
D_sub_h = np.empty_like(f)
D_sub_h[0, :] = f[1, :]
D_sub_h[-1, :] = -f[-2, :]
np.subtract(f[2:, :], f[:-2, :], out=D_sub_h[1:-1, :])
return D_sub_h
Notice that I return the value instead of printing it. When you write functions, get in the habit of returning a value. Printing can be done later, and effectively discards the computation if it replaces a proper return.
The way shown above is fairly efficient with regards to time and space. If you want to write a one liner that uses a lot of temporary arrays, you can also do:
D_sub_h = np.concatenate((f[1, None], f[2:, :] - f[:-2, :], -f[-2, None]), axis=0)

Find average colour of each section of an image

I am looking for the best way to achieve the following using Python:
Import an image.
Add a grid of n sections (4 shown in this example below).
For each section find the dominant colour.
Desired output
Output an array, list, dict or similar capturing these dominant colour values.
Maybe even a Matplotlib graph showing the colours (like pixel art).
What have I tried?
The image could be sliced using image slicer:
import image_slicer
image_slicer.slice('image_so_grid.png', 4)
I could then potentially use something like this to get the average colour but Im sure there are better ways to do this.
What are the best ways to do this with Python?

This works for 4 sections, but you'll need to figure out how to make it work for 'n' sections:
import cv2
img = cv2.imread('image.png')
def fourSectionAvgColor(image):
rows, cols, ch = image.shape
colsMid = int(cols/2)
rowsMid = int(rows/2)
numSections = 4
section0 = image[0:rowsMid, 0:colsMid]
section1 = image[0:rowsMid, colsMid:cols]
section2 = image[rowsMid: rows, 0:colsMid]
section3 = image[rowsMid:rows, colsMid:cols]
sectionsList = [section0, section1, section2, section3]
sectionAvgColorList = []
for i in sectionsList:
pixelSum = 0
yRows, xCols, chs = i.shape
pixelCount = yRows*xCols
totRed = 0
totBlue = 0
totGreen = 0
for x in range(xCols):
for y in range(yRows):
bgr = i[y,x]
b = bgr[0]
g = bgr[1]
r = bgr[2]
totBlue = totBlue+b
totGreen = totGreen+g
totRed = totRed+r
avgBlue = int(totBlue/pixelCount)
avgGreen = int(totGreen/pixelCount)
avgRed = int(totRed/pixelCount)
avgPixel = (avgBlue, avgGreen, avgRed)
sectionAvgColorList.append(avgPixel)
return sectionAvgColorList
print(fourSectionAvgColor(img))
cv2.waitKey(0)
cv2.destroyAllWindows()

You can use scikit-image's view_as_blocks together with numpy.mean. You specify the block size instead of the number of blocks:
import numpy as np
from skimage import data, util
import matplotlib.pyplot as plt
astro = data.astronaut()
blocks = util.view_as_blocks(astro, (8, 8, 3))
print(astro.shape)
print(blocks.shape)
mean_color = np.mean(blocks, axis=(2, 3, 4))
fig, ax = plt.subplots()
ax.imshow(mean_color.astype(np.uint8))
Output:
(512, 512, 3)
(64, 64, 1, 8, 8, 3)
Don't forget the cast to uint8 because matplotlib and scikit-image expect floating point images to be in [0, 1], not [0, 255]. See the scikit-image documentation on data types for more info.

Iterate over image and select lightness/saturation values in a given hue range not using nested for loops

Story behind:
I am trying to build a simple hand detection that is adaptive to environmental changes (light mainly) and therefore re-calibrates every few minutes using a histogram of a rough estimate where the hand is (using YOLO-Darknet).
Idea is to get the finger position at the end.
Currently I am getting the hue value of the hand and the lightness and saturation are fixed.
I have written a short nested for loop for the lightness and saturation search where it goes through all the elements in the hue range and finds its max and min value to write it in the array.
Now my question:
I know that python has functionalities for writing such nested loops and list modification/manipulation short and elegant, how do I do this?
Following I have an MWE and an example image.
import numpy as np
import cv2
hue = 0
light = 1
satur = 2
img = cv2.imread('Untitled.png')
hls = cv2.cvtColor(img, cv2.COLOR_BGR2HLS)
# structure of histograms:
# 0, 2, 6 = histogram values for H, L and S
# 1, 3, 5 = bin values for H, L and S
histograms = [0] * 6
# Only 5 bins for a rough estimate of the skin color (so not too much is lost)
histograms[0], histograms[1] = np.histogram(hls[:, :, 0], 5)
histograms[2], histograms[3] = np.histogram(hls[:, :, 1], 5)
histograms[4], histograms[5] = np.histogram(hls[:, :, 2], 5)
# structure of HLS_bins:
# [Hue, Lightness, Saturation] [min, min, min]
# [Hue, Lightness, Saturation] [max, max, max]
HLS_bins = [[0, 200, 30], [0, 255, 255]]
# TODO alternative approach to the one below:
# todo...find the bin for the highest occuring color and select the Lightness
# todo...and Saturation according to the corresponding values
# write in loop (elegant/generalized way) ?
# select the highest occurence of the hue
max_value_hue = max(histograms[0])
max_index_hue = list(histograms[0]).index(max_value_hue)
HLS_bins[0][0] = histograms[1][max_index_hue]
HLS_bins[1][0] = histograms[1][max_index_hue + 1]
min_value_light = 255
max_value_light = 0
min_value_saturation = 255
max_value_saturation = 0
for row in range(np.shape(hls)[0]):
for col in range(np.shape(hls)[1]):
if hls[row][col][hue] > HLS_bins[0][0] and hls[row][col][hue] < HLS_bins[1][0]:
if hls[row][col][light] > max_value_light:
max_value_light = hls[row][col][light]
if hls[row][col][light] < min_value_light:
min_value_light = hls[row][col][light]
if hls[row][col][satur] > max_value_saturation:
max_value_saturation = hls[row][col][satur]
if hls[row][col][satur] < min_value_saturation:
min_value_saturation = hls[row][col][satur]
HLS_bins[0][1] = min_value_light
HLS_bins[1][1] = max_value_light
HLS_bins[0][2] = min_value_saturation
HLS_bins[1][2] = max_value_saturation
HLS_bins = np.array(HLS_bins, dtype="uint8")
print(HLS_bins)
Most should have guessed it already, it's about this part of code:
for row in range(np.shape(hls)[0]):
for col in range(np.shape(hls)[1]):
if hls[row][col][hue] > HLS_bins[0][0] and hls[row][col][hue] < HLS_bins[1][0]:
if hls[row][col][light] > max_value_light:
max_value_light = hls[row][col][light]
if hls[row][col][light] < min_value_light:
min_value_light = hls[row][col][light]
if hls[row][col][satur] > max_value_saturation:
max_value_saturation = hls[row][col][satur]
if hls[row][col][satur] < min_value_saturation:
min_value_saturation = hls[row][col][satur]
So, how to write this nice and elegant?

If what you want is to obtain the max and min values of your Lightness and Saturation (the last 2 channels), a way of doing it is by using the np.max() and np.min() method on your image array directly.
To obtain such values of the desired channels you can Slice them from the image to then query for the values:
import cv2
img = cv2.imread('Untitled.png')
hls = cv2.cvtColor(img, cv2.COLOR_BGR2HLS)
#reading Lightness channel: No. 1
#this slice basically means: "give me all rows and all cols from channel 1"
max_value_light = hls[:,:,1].max()
min_value_light = hls[:,:,1].min()
#reading Saturation channel: No. 2
# and this slice means: "give me all rows and all cols from channel 2"
max_value_saturation = hls[:,:,2].max()
min_value_saturation = hls[:,:,2].min()
Edit: Based on your clarification, if you want to query such max/min values, but only those that fall within certain interval [hue_min, hue_max] you could use np.where() along with max:
#obtain the min hue that falls within the interval
#hue is channel 0, so we slice for all x,y pixels and for that channel
theMin = hls[np.where((hls[:,:,0]>hue_min) & (hls[:,:,0]<hue_max))][:,0].min()
#same story with the rest, compare to your _min and _max and use it's index

Python element-wise vectorised boolean operations to classify image pixels based on their colour

I have an RGB image which I am loading into a 2D array using PIL
img = Image.open(path)
imgData = numpy.array(img)
I need to efficiently translate this into a 2D array of RGB tuples (in some sense a 3D array) the same size containing a rough 'classification' of each pixel - 'red', 'green', 'white' or 'other' - at each index based on which 'colour region' they lie within. This is for purposes of image recognition.
My current implementation uses a element-wise for loop but is very slow (an 8MP image takes 1+ minutes):
for i in range(cols): # for every col
for j in range(rows): # for every row
r,g,b = imgData[i,j]
if b > 220: # white
n = 3
elif r > 230: # red
n = 2
else: # green
n = 1
mapData[i,j] = n
(I realise that the order of the if statements here affects the precedence of the classifications - this is not a major issue for now although I would prefer to define the colour spaces exclusively)
I am running Python 3.6.4 and happy to use NumPy or not. Having done a bunch of research, it seems like there are a number of faster and more 'pythonic' and vectorised ways to do this but I have not been able to get any working.
Any help would be much appreciated
Thanks!

Using np.where makes this pretty fast.
mapData = np.where(imgData[:,:,2] > 220, 3, np.where(imgData[:,:,0]>230, 2, 1))
But when applying this to a picture the only results where ones. Did I miss anything or should the cases be made in a different way?

Your algorithm as of the moment can be captured like this:
r, g, b = imgData[...,0], imgData[...,1], imgData[...,2]
mapData = np.ones_like(r, dtype=int)
mapData[r > 230] = 2
mapData[b > 220] = 3
Note the order of operations in assigning these numbers.

Colour classification is usually done by treating RGB colours as vectors. Normalize each one to the magnitude, then find the distance to your target colour.
For example, the skin detector in smartcrop.js works like this (using pyvips):
def pythag(im):
return sum([x ** 2 for x in im]) ** 0.5
skin = [0.78, 0.57, 0.44]
score = 1 - pythag(img / pythag(img) - skin)
Now score is a float image with values in 0 - 1 which is 1 for pixels most likely to be skin-coloured. Note that it ignores brightness: you'll need another rule to chop off very dark areas.
In your case I guess you'd need an array set of target vectors, then compute all the colour probabilities, and finally label the output pixel with the index of the highest-scoring vector. Something like:
import sys
import pyvips
def pythag(im):
return sum([x ** 2 for x in im]) ** 0.5
def classify(img, target):
return 1 - pythag(img / pythag(img) - target)
# find [index, max] of an array of pyvips images
def argmax(ar):
if len(ar) == 1:
return [0, ar[0]]
else:
index, mx = argmax(ar[:-1])
return [(ar[-1] > mx).ifthenelse(len(ar) - 1, index),
(ar[-1] > mx).ifthenelse(ar[-1], mx)]
skin = [0.78, 0.57, 0.44]
red = [1, 0, 0]
green = [0, 1, 0]
targets = [red, green, skin]
# we're not doing any coordinate transformations, so we can stream the image
img = pyvips.Image.new_from_file(sys.argv[1], access="sequential")
scores = [classify(img, x) for x in targets]
index, mx = argmax(scores)
index.write_to_file(sys.argv[2])
(plug: pyvips is typically 2x or 3x faster than numpy and needs much less memory)

Interpolate between two images

I'm trying to interpolate between two images in Python.
Images are of shapes (188, 188)
I wish to interpolate the image 'in-between' these two images. Say Image_1 is at location z=0 and Image_2 is at location z=2. I want the interpolated image at location z=1.
I believe this answer (MATLAB) contains a similar problem and solution.
Creating intermediate slices in a 3D MRI volume with MATLAB
I've tried to convert this code to Python as follows:
from scipy.interpolate import interpn
from scipy.interpolate import griddata
# Construct 3D volume from images
# arr.shape = (2, 182, 182)
arr = np.r_['0,3', image_1, image_2]
slices,rows,cols = arr.shape
# Construct meshgrids
[X,Y,Z] = np.meshgrid(np.arange(cols), np.arange(rows), np.arange(slices));
[X2,Y2,Z2] = np.meshgrid(np.arange(cols), np.arange(rows), np.arange(slices*2));
# Run n-dim interpolation
Vi = interpn([X,Y,Z], arr, np.array([X1,Y1,Z1]).T)
However, this produces an error:
ValueError: The points in dimension 0 must be strictly ascending
I suspect I am not constructing my meshgrid(s) properly but am kind of lost on whether or not this approach is correct.
Any ideas?
---------- Edit -----------
Found some MATLAB code that appears to solve this problem:
Interpolating Between Two Planes in 3d space
I attempted to convert this to Python:
from scipy.ndimage.morphology import distance_transform_edt
from scipy.interpolate import interpn
def ndgrid(*args,**kwargs):
"""
Same as calling ``meshgrid`` with *indexing* = ``'ij'`` (see
``meshgrid`` for documentation).
"""
kwargs['indexing'] = 'ij'
return np.meshgrid(*args,**kwargs)
def bwperim(bw, n=4):
"""
perim = bwperim(bw, n=4)
Find the perimeter of objects in binary images.
A pixel is part of an object perimeter if its value is one and there
is at least one zero-valued pixel in its neighborhood.
By default the neighborhood of a pixel is 4 nearest pixels, but
if `n` is set to 8 the 8 nearest pixels will be considered.
Parameters
----------
bw : A black-and-white image
n : Connectivity. Must be 4 or 8 (default: 8)
Returns
-------
perim : A boolean image
From Mahotas: http://nullege.com/codes/search/mahotas.bwperim
"""
if n not in (4,8):
raise ValueError('mahotas.bwperim: n must be 4 or 8')
rows,cols = bw.shape
# Translate image by one pixel in all directions
north = np.zeros((rows,cols))
south = np.zeros((rows,cols))
west = np.zeros((rows,cols))
east = np.zeros((rows,cols))
north[:-1,:] = bw[1:,:]
south[1:,:] = bw[:-1,:]
west[:,:-1] = bw[:,1:]
east[:,1:] = bw[:,:-1]
idx = (north == bw) & \
(south == bw) & \
(west == bw) & \
(east == bw)
if n == 8:
north_east = np.zeros((rows, cols))
north_west = np.zeros((rows, cols))
south_east = np.zeros((rows, cols))
south_west = np.zeros((rows, cols))
north_east[:-1, 1:] = bw[1:, :-1]
north_west[:-1, :-1] = bw[1:, 1:]
south_east[1:, 1:] = bw[:-1, :-1]
south_west[1:, :-1] = bw[:-1, 1:]
idx &= (north_east == bw) & \
(south_east == bw) & \
(south_west == bw) & \
(north_west == bw)
return ~idx * bw
def signed_bwdist(im):
'''
Find perim and return masked image (signed/reversed)
'''
im = -bwdist(bwperim(im))*np.logical_not(im) + bwdist(bwperim(im))*im
return im
def bwdist(im):
'''
Find distance map of image
'''
dist_im = distance_transform_edt(1-im)
return dist_im
def interp_shape(top, bottom, num):
if num<0 and round(num) == num:
print("Error: number of slices to be interpolated must be integer>0")
top = signed_bwdist(top)
bottom = signed_bwdist(bottom)
r, c = top.shape
t = num+2
print("Rows - Cols - Slices")
print(r, c, t)
print("")
# rejoin top, bottom into a single array of shape (2, r, c)
# MATLAB: cat(3,bottom,top)
top_and_bottom = np.r_['0,3', top, bottom]
#top_and_bottom = np.rollaxis(top_and_bottom, 0, 3)
# create ndgrids
x,y,z = np.mgrid[0:r, 0:c, 0:t-1] # existing data
x1,y1,z1 = np.mgrid[0:r, 0:c, 0:t] # including new slice
print("Shape x y z:", x.shape, y.shape, z.shape)
print("Shape x1 y1 z1:", x1.shape, y1.shape, z1.shape)
print(top_and_bottom.shape, len(x), len(y), len(z))
# Do interpolation
out = interpn((x,y,z), top_and_bottom, (x1,y1,z1))
# MATLAB: out = out(:,:,2:end-1)>=0;
array_lim = out[-1]-1
out[out[:,:,2:out] >= 0] = 1
return out
I call this as follows:
new_image = interp_shape(image_1,image_2, 1)
Im pretty sure this is 80% of the way there but I still get this error when running:
ValueError: The points in dimension 0 must be strictly ascending
Again, I am probably not constructing my meshes correctly. I believe np.mgrid should produce the same result as MATLABs ndgrid though.
Is there a better way to construct the ndgrid equivalents?

I figured this out. Or at least a method that produces desirable results.
Based on: Interpolating Between Two Planes in 3d space
def signed_bwdist(im):
'''
Find perim and return masked image (signed/reversed)
'''
im = -bwdist(bwperim(im))*np.logical_not(im) + bwdist(bwperim(im))*im
return im
def bwdist(im):
'''
Find distance map of image
'''
dist_im = distance_transform_edt(1-im)
return dist_im
def interp_shape(top, bottom, precision):
'''
Interpolate between two contours
Input: top
[X,Y] - Image of top contour (mask)
bottom
[X,Y] - Image of bottom contour (mask)
precision
float - % between the images to interpolate
Ex: num=0.5 - Interpolate the middle image between top and bottom image
Output: out
[X,Y] - Interpolated image at num (%) between top and bottom
'''
if precision>2:
print("Error: Precision must be between 0 and 1 (float)")
top = signed_bwdist(top)
bottom = signed_bwdist(bottom)
# row,cols definition
r, c = top.shape
# Reverse % indexing
precision = 1+precision
# rejoin top, bottom into a single array of shape (2, r, c)
top_and_bottom = np.stack((top, bottom))
# create ndgrids
points = (np.r_[0, 2], np.arange(r), np.arange(c))
xi = np.rollaxis(np.mgrid[:r, :c], 0, 3).reshape((r**2, 2))
xi = np.c_[np.full((r**2),precision), xi]
# Interpolate for new plane
out = interpn(points, top_and_bottom, xi)
out = out.reshape((r, c))
# Threshold distmap to values above 0
out = out > 0
return out
# Run interpolation
out = interp_shape(image_1,image_2, 0.5)
Example output:

I came across a similar problem where I needed to interpolate the shift between frames where the change did not merely constitute a translation but also changes to the shape itself . I solved this problem by :
Using center_of_mass from scipy.ndimage.measurements to calculate the center of the object we want to move in each frame
Defining a continuous parameter t where t=0 first and t=1 last frame
Interpolate the motion between two nearest frames (with regard to a specific t value) by shifting the image back/forward via shift from scipy.ndimage.interpolation and overlaying them.
Here is the code:
def inter(images,t):
#input:
# images: list of arrays/frames ordered according to motion
# t: parameter ranging from 0 to 1 corresponding to first and last frame
#returns: interpolated image
#direction of movement, assumed to be approx. linear
a=np.array(center_of_mass(images[0]))
b=np.array(center_of_mass(images[-1]))
#find index of two nearest frames
arr=np.array([center_of_mass(images[i]) for i in range(len(images))])
v=a+t*(b-a) #convert t into vector
idx1 = (np.linalg.norm((arr - v),axis=1)).argmin()
arr[idx1]=np.array([0,0]) #this is sloppy, should be changed if relevant values are near [0,0]
idx2 = (np.linalg.norm((arr - v),axis=1)).argmin()
if idx1>idx2:
b=np.array(center_of_mass(images[idx1])) #center of mass of nearest contour
a=np.array(center_of_mass(images[idx2])) #center of mass of second nearest contour
tstar=np.linalg.norm(v-a)/np.linalg.norm(b-a) #define parameter ranging from 0 to 1 for interpolation between two nearest frames
im1_shift=shift(images[idx2],(b-a)*tstar) #shift frame 1
im2_shift=shift(images[idx1],-(b-a)*(1-tstar)) #shift frame 2
return im1_shift+im2_shift #return average
if idx1<idx2:
b=np.array(center_of_mass(images[idx2]))
a=np.array(center_of_mass(images[idx1]))
tstar=np.linalg.norm(v-a)/np.linalg.norm(b-a)
im1_shift=shift(images[idx2],-(b-a)*(1-tstar))
im2_shift=shift(images[idx1],(b-a)*(tstar))
return im1_shift+im2_shift
Result example

I don't know the solution to your problem, but I don't think it's possible to do this with interpn.
I corrected the code that you tried, and used the following input images:
But the result is:
Here's the corrected code:
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm
from scipy import interpolate
n = 8
img1 = np.zeros((n, n))
img2 = np.zeros((n, n))
img1[2:4, 2:4] = 1
img2[4:6, 4:6] = 1
plt.figure()
plt.imshow(img1, cmap=cm.Greys)
plt.figure()
plt.imshow(img2, cmap=cm.Greys)
points = (np.r_[0, 2], np.arange(n), np.arange(n))
values = np.stack((img1, img2))
xi = np.rollaxis(np.mgrid[:n, :n], 0, 3).reshape((n**2, 2))
xi = np.c_[np.ones(n**2), xi]
values_x = interpolate.interpn(points, values, xi, method='linear')
values_x = values_x.reshape((n, n))
print(values_x)
plt.figure()
plt.imshow(values_x, cmap=cm.Greys)
plt.clim((0, 1))
plt.show()
I think the main difference between your code and mine is in the specification of xi. interpn tends to be somewhat confusing to use, and I've explained it in greater detail in an older answer. If you're curious about the mechanics of how I've specified xi, see this answer of mine explaining what I've done.
This result is not entirely surprising, because interpn just linearly interpolated between the two images: so the parts which had 1 in one image and 0 in the other simply became 0.5.
Over here, since one image is the translation of the other, it's clear that we want an image that's translated "in-between". But how would interpn interpolate two general images? If you had one small circle and one big circle, is it in any way clear that there should be a circle of intermediate size "between" them? What about interpolating between a dog and a cat? Or a dog and a building?
I think you are essentially trying to "draw lines" connecting the edges of the two images and then trying to figure out the image in between. This is similar to sampling a moving video at a half-frame. You might want to check out something like optical flow, which connects adjacent frames using vectors. I'm not aware if and what python packages/implementations are available though.