Develop Reference

Scanning lists more efficiently in python

I have some code, which works as intended, however takes about 4 and a half hours to run, I understand that there are about 50 billion calculations my poor pc needs to do but I thought it would be worth asking!
This code gets an image, and wants to find every possible region of 331*331 pixels in the given image, and find how many black pixels there are in each, I will use this data to create a heatmap of black pixel density, and also a list of all of the values found:
image = Image.open(self.selectedFile)
pixels = list(image.getdata())
width, height = image.size
pixels = [pixels[i * width:(i+1) * width] for i in range(height)]
#print(pixels)
rightShifts = width - 331
downShifts = height - 331
self.totalRegionsLabel['text'] = f'Total Regions: {rightShifts * downShifts}'
self.blackList = [0 for i in range(0, rightShifts*downShifts)]
self.heatMap = [[] for i in range(0, downShifts)]
for x in range(len(self.heatMap)):
self.heatMap[x] = [0 for i in range(0, rightShifts)]
for x in range(rightShifts):
for y in range(downShifts):
blackCount = 0
for z in range(x + 331):
for w in range(y + 331):
if pixels[z][w] == 0:
blackCount += 1
self.blackList[x+1*y] = blackCount
self.heatMap[x][y] = blackCount
print(self.blackList)

You have several problems here, as I pointed out. Your z/w loops are always starting at the upper left, so by the time you get towards the end, you're summing the entire image, not just a 331x331 subset. You also have much confusion in your axes. In an image, [y] is first, [x] is second. An image is rows of columns. You need to remember that.
Here's an implementation as I suggested above. For each column, I do a full sum on the top 331x331 block. Then, for every row below, I just subtract the top row and add the next row below.
self.heatMap = [[0]*rightShifts for i in range(downShifts)]
for x in range(rightShifts):
# Sum up the block at the top.
blackCount = 0
for row in range(331):
for col in range(331):
if pixels[row][x+col] == 0:
blackCount += 1
self.heatMap[0][x] = blackCount
for y in range(1,downShifts):
# To do the next block down, we subtract the top row and
# add the bottom.
for col in range(331):
blackCount += pixels[y+330][x+col] - pixels[y-1][x+col]
self.heatMap[y][x] = blackCount
You could tweak this even more by alternating the columns. So, at the bottom of the first column, scoot to the right by subtracting the first column and adding the next new column. then scoot back up to the top. That's a lot more trouble.

The two innermost for-loops seem to be transformable to some numpy code if using this package is not an issue. It would give something like:
pixels = image.get_data() # it is probably already a numpy array
# Get an array filled with either True or False, with True whenever pixel is black:
pixel_is_black = (pixels[x:(x+331), y:(y+331)] == 0)
pixel_is_black *= 1 # Transform True and False to respectively 1 and 0. Maybe not needed
self.blackList[x+y] = pixel_is_black.sum() # self explanatory
This is the simplest optimization I can think of, you probably can do much better with clever numpy tricks.

I would recommend using some efficient vector computations through the numpy and opencv libraries.
First, binarize your image so that black pixels are set to zero, and any other color pixels (gray to white) are set to 1. Then, apply a 2D filter to the image of shape 331 x 331 where each value in the filter kernel is (1 / (331 x 331) - this will take the average of all the values in each 331x331 area and assign it to the center pixel.
This gives you a heatmap, where each pixel value is the proportion of non-black pixels in the surrounding 331 x 331 region. A darker pixel (value closer to zero) means more pixels in that region are black.
For some background, this approach uses image processing techniques called image binarization and box blur
Example code:
import cv2
import numpy as np
# setting up a fake image, with some white spaces, gray spaces, and black spaces
img_dim = 10000
fake_img = np.full(shape=(img_dim, img_dim), fill_value=255, dtype=np.uint8) # white
fake_img[: img_dim // 3, : img_dim // 3] = 0 # top left black
fake_img[2 * img_dim // 3 :, 2 * img_dim // 3 :] = 0 # bottom right black
fake_img[img_dim // 3 : 2 * img_dim // 3, img_dim // 3 : 2 * img_dim // 3] = 127 # center gray
# show the fake image
cv2.imshow("", fake_img)
cv2.waitKey()
cv2.destroyAllWindows()
# solution to your problem
binarized = np.where(fake_img == 0, 0, 1) # have 0 values where black, 1 values else
my_filter = np.full(shape=(331, 331), fill_value=(1 / (331 * 331))) # set up filter
heatmap = cv2.filter2D(fake_img, 1, my_filter) # apply filter, which takes average of values in 331x331 block
# show the heatmap
cv2.imshow("", heatmap)
cv2.waitKey()
cv2.destroyAllWindows()
I ran this on my laptop, with a huge (fake) image of 10000 x 10000 pixels, almost instantly.

Sorry I should have deleted this post before you all put the effort in, however, some of these workarounds are really smart and interesting, I ended up coming up with a solution independently that is the same as what Tim Robbers first suggested, I used the array I had and built a second one on which every item in a row is the number of black cells preceding it, and then for each row in a region instead of scanning every item, just scan the preceding value and the final value and you are good:
image = Image.open(self.selectedFile).convert('L') #convert to luminance mode as RGB information is irrelevant
pixels = list(image.getdata()) #get the value of every pixel in the image
width, height = image.size
pixels = [pixels[i * width:(i+1) * width] for i in range(height)] #split the pixels array into a two dimensional array with the dimensions to match the image
#This program scans every possible 331*331 square starting from the top left, so it will move right width - 331 pixels and down height - 331 pixels
rightShifts = width - 331
downShifts = height - 331
self.totalRegionsLabel['text'] = f'Total Regions: {rightShifts * downShifts}' #This wont update till the function has completed running
#The process of asigning new values to values in an array is faster than appending them so this is why I prefilled the arrays:
self.heatMap = [[] for i in range(0, downShifts)]
for x in range(len(self.heatMap)):
self.heatMap[x] = [0 for i in range(0, rightShifts)]
cumulativeMatrix = [] #The cumulative matrix replaces each value in each row with how many zeros precede it
for y in range(len(pixels)):
cumulativeMatrix.append([])
cumulativeMatrix[y].append(0)
count = 0
for x in range(len(pixels[y])):
if pixels[y][x] == 0:
count += 1
cumulativeMatrix[y].append(count)
regionCount = 0
maxValue = 0 #this is the lowest possible maximum value
minValue = 109561 #this is the largest possible minimum value
self.blackList = []
#loop through all possible regions
for y in range(downShifts):
for x in range(rightShifts):
blackPixels = 0
for regionY in range(y, y + 331):
lowerLimit = cumulativeMatrix[regionY][x]
upperLimit = cumulativeMatrix[regionY][x+332]
blackPixels += (upperLimit - lowerLimit)
if blackPixels > maxValue:
maxValue = blackPixels
if blackPixels < minValue:
minValue = blackPixels
self.blackList.append(blackPixels)
self.heatMap[y][x] = blackPixels
regionCount += 1
This brought run time to under a minute and thus solved my problem, however, thank you for your contributions I have learned a lot from reading them!

Try to look into the map() function. It uses C to streamline iterations.
You can speed up your for loops like this:
pixels = list(map(lambda i: x[i*width:(i+1)*width], range(height)))

Add glow effect to traffic light Imageprocessing skimage

I am learning image processing and came across this following code. What it does is glow the red light of the trafficlights image. I did understood the code by don't know how the new_R variable is calculated. Although it is easy to interpret that it corresponds to value in R and G scale where the values of pixel is less than 40 in R and 10 in G.
I just want to know why G layer values are added and what if i want to glow the green light or orange light what should be the upper or lower bound in calculation of new_G or new_O as new_R is done.
from sklearn import io as ip
import numpy as np
I = ip.imread('trafficlights.png')
R = np.logical_and(I[:, :, 0] > 40, I[:, :, 1] < 10)
new_R = gaussian_filt(255 * R, 30)
J = I.copy()
J[:, :, 0] = ip.imadd(new_R, J[:, :, 0])
import matplotlib.pyplot as plt
fig, ax = plt.subplots(1, 2)
plt.sca(ax[0])
ip.imshow(I)
plt.sca(ax[1])
ip.imshow(J)
ip.imsave('traffic_redglow.png', J)
if you want to reproduce then here are the other functions
def imadd(I, K):
'''
Add to the pixels of I.
- If K is a number, adds K to each pixel value of I.
- If K is an image (NumPy array) of the same size as I,
the pixel values of K are added to the pixel values of I.
Clips the result to avoid overflow.
Example:
# Add 10 to each pixel of I
J = imadd(I, 10)
# Subtract 20 from each pixel of I
J = imadd(I, -20)
# Add images I1 and I2 together
J = imadd(I1, I2)
'''
import numbers
if isinstance(K, numbers.Number):
J = I.astype('int32')
J += K
elif isinstance(K, np.ndarray):
assert K.shape == I.shape, f'Cannot add images with sizes {I.shape} and {K.shape}.'
J = I.astype('int32') + K.astype('int32')
else:
raise TypeError('K must be a number or an array.')
np.clip(J, 0, 255, out=J)
J = J.astype('uint8')
return J
def gaussian_filt(I, sigma, pad=0):
'''
Filters the image I with a Gaussian filter,
with standard deviation sigma.
Returns a new filtered image.
Optional argument pad specifies the boundary options,
and can take the following values:
- if pad is a number (0-255, default 0), pixels outside the bounds of the image are assigned this value.
- 'reflect': outer pixel values are mirror-reflections of inner pixel values
across the image border.
- 'nearest': outer pixel values are equal to the nearest inner pixel values.
- 'wrap': outer pixel values are computed by assuming that the image is periodic (i.e. tiled).
Example:
# Use Gaussian filter on I with a standard deviation of 0.5,
# pad boundary pixels with white
J = gaussian_filt(I, 0.5, pad=255)
# Reflect boundary pixel values across the image border
J = gaussian_filt(I, 0.5, pad='reflect')
'''
import numbers
assert isinstance(pad, numbers.Number) or pad in ['reflect', 'nearest', 'wrap'], \
'Choose a correct value for pad: a number (0-255), ''reflect'', ''nearest'', or ''wrap''.'
if isinstance(pad, numbers.Number):
md = 'constant'
c = pad
else:
md = pad
c = 0
return gaussian_filter(I, sigma, mode=md, cval=c)
Thanks a lot for going through

Python element-wise vectorised boolean operations to classify image pixels based on their colour

I have an RGB image which I am loading into a 2D array using PIL
img = Image.open(path)
imgData = numpy.array(img)
I need to efficiently translate this into a 2D array of RGB tuples (in some sense a 3D array) the same size containing a rough 'classification' of each pixel - 'red', 'green', 'white' or 'other' - at each index based on which 'colour region' they lie within. This is for purposes of image recognition.
My current implementation uses a element-wise for loop but is very slow (an 8MP image takes 1+ minutes):
for i in range(cols): # for every col
for j in range(rows): # for every row
r,g,b = imgData[i,j]
if b > 220: # white
n = 3
elif r > 230: # red
n = 2
else: # green
n = 1
mapData[i,j] = n
(I realise that the order of the if statements here affects the precedence of the classifications - this is not a major issue for now although I would prefer to define the colour spaces exclusively)
I am running Python 3.6.4 and happy to use NumPy or not. Having done a bunch of research, it seems like there are a number of faster and more 'pythonic' and vectorised ways to do this but I have not been able to get any working.
Any help would be much appreciated
Thanks!

Using np.where makes this pretty fast.
mapData = np.where(imgData[:,:,2] > 220, 3, np.where(imgData[:,:,0]>230, 2, 1))
But when applying this to a picture the only results where ones. Did I miss anything or should the cases be made in a different way?

Your algorithm as of the moment can be captured like this:
r, g, b = imgData[...,0], imgData[...,1], imgData[...,2]
mapData = np.ones_like(r, dtype=int)
mapData[r > 230] = 2
mapData[b > 220] = 3
Note the order of operations in assigning these numbers.

Colour classification is usually done by treating RGB colours as vectors. Normalize each one to the magnitude, then find the distance to your target colour.
For example, the skin detector in smartcrop.js works like this (using pyvips):
def pythag(im):
return sum([x ** 2 for x in im]) ** 0.5
skin = [0.78, 0.57, 0.44]
score = 1 - pythag(img / pythag(img) - skin)
Now score is a float image with values in 0 - 1 which is 1 for pixels most likely to be skin-coloured. Note that it ignores brightness: you'll need another rule to chop off very dark areas.
In your case I guess you'd need an array set of target vectors, then compute all the colour probabilities, and finally label the output pixel with the index of the highest-scoring vector. Something like:
import sys
import pyvips
def pythag(im):
return sum([x ** 2 for x in im]) ** 0.5
def classify(img, target):
return 1 - pythag(img / pythag(img) - target)
# find [index, max] of an array of pyvips images
def argmax(ar):
if len(ar) == 1:
return [0, ar[0]]
else:
index, mx = argmax(ar[:-1])
return [(ar[-1] > mx).ifthenelse(len(ar) - 1, index),
(ar[-1] > mx).ifthenelse(ar[-1], mx)]
skin = [0.78, 0.57, 0.44]
red = [1, 0, 0]
green = [0, 1, 0]
targets = [red, green, skin]
# we're not doing any coordinate transformations, so we can stream the image
img = pyvips.Image.new_from_file(sys.argv[1], access="sequential")
scores = [classify(img, x) for x in targets]
index, mx = argmax(scores)
index.write_to_file(sys.argv[2])
(plug: pyvips is typically 2x or 3x faster than numpy and needs much less memory)

Iterating a function over the RGB values of individual pixels

I've already opened my image and can access the individual pixels' RGB values, but what I'm trying to do now is apply a function to the RGB values of each of the pixels individually. That is, I don't want to apply it the same way across all the pixels in the entire image; I want to apply it differently depending on whether, for each individual pixel, the blue value is > red > green (rather than green > red > blue, etc, etc).
So my question is, how do I access the individual RGB elements within each pixel (as opposed to accessing all of the red, green, and blue values across the entire image at once)? Eventually my question will be "what's the fastest way to do this?" since it's obviously going to take a while to apply a function across each pixel individually but for now I'd be happy just to have any solution at all.
Thanks for any suggestions.
EDIT for clarity/more specificity:
I'm actually trying to apply a different set of instructions depending the ordering of 127.5 - abs(127.5 - red/green/blue)), not simply on the order of red>green>blue (as stated initially above, bc I was trying to simplify). Once that ordering is determined for a given pixel, then the appropriate set of instructions is applied. Again, this is pixel-by-pixel -- I'm not ordering things based on ALL red values across the image, just the rgbs of the individual pixels. So what I'm trying to do would look something like this (here I'm playing out just one of the six possible orders; I've omitted the five other possibilities for brevity):
def rgb_manip(red,green,blue):
r_max = int(127.5 - abs(127.5 - red))
g_max = int(127.5 - abs(127.5 - green))
b_max = int(127.5 - abs(127.5 - blue))
if r_max >= g_max >= b_max:
if r_max >= g_max + b_max:
new_red = red + g_max + b_max
new_green = green - g_max
new_blue = blue - b_max
else:
new_red = red + r_max
new_green = green - r_max + b_max
new_blue = blue - b_max
# elif... And so on, with a different set of instructions for each of the 6 possibilities depending on the order of the r_max, g_max, b_max values (e.g., r_max >= b_max >= g_max or g_max >= r_max >= b_max, etc, etc)

If you convert your image into an array, you can access the RGB values for one pixel, or one of the R, G, or B values for all pixels:
from __future__ import division
import numpy as np
from PIL import Image
im = Image.open(imfile)
arr = np.asarray(im)
arr[..., 0] # All Red values
arr[..., 1] # All Green values
arr[..., 2] # All Blue values
arr[0, 0] # RGB for first corner pixel
arr[m, n] # RGB for pixel at [m, n]
arr[m, n, 0] # R value for pixel [m, n]
arr[m, n, c] # value for color c at pixel [m, n]
You can get a ranking for each pixel using argsort, as in:
def transform(a, c=255/2):
return c - np.abs(c - a)
ranking = transform(arr).argsort(axis=-1)
which ranks the criterion values from smallest to largest value along the last (color) axis. So this gives a new array where each 'color' array instead of being the RGB values are the sorting of the transformed R, B, and G values (call them "R', B', G'"), so if the corner pixel had G' > B' > R', then ranking[0, 0] would be [0, 2, 1] because R' (0) is smallest, then next is B' (2), finally the largest is G' (1).
The advantage of doing the above is that you have an array that says which method to use on which pixel. It can have at most six orderings of the transformed channels. I suggest defining a separate function like so for each of orderings. Then, only one decision must be made within the function (the second nested if/else in your example), and it can be done with np.where which applies one thing to parts of an array where a condition is met, and another thing to the rest. This only works for two options, but if there are multiple options (if/elif/else), other techniques can work equally well.
def bgr(a):
""" for when B' < G' < R'
"""
t = transform(a)
red, green, blue = a.transpose([2,0,1])
# same as: red, green, blue = a[..., 0], a[..., 1], a[..., 2]
r_max, g_max, b_max = t.transpose([2,0,1])
assert np.all((b_max <= g_max) & (g_max <= r_max)), "doesn't match rank"
condition = r_max >= g_max + b_max
new_red = np.where(condition, red + g_max + b_max, red + r_max)
new_green = np.where(condition, green - g_max, green - r_max + b_max)
new_blue = blue - b_max
return np.dstack([new_red, new_green, new_blue])
this function only works for the first if in yours. I would make a new function for each of those six things, and fill them into a dict like so:
functions = {
(0, 1, 2) : rgb, # for R'<G'<B'
(0, 2, 1) : rbg, # for R'<B'<G'
#etc...
}
If your output has RGB values too:
out = np.empty_like(arr)
Then loop through all six rankings/functions:
for rank, func in functions.items():
mask = np.all(transform(arr).argsort(-1) == rank, -1)
out[mask] = func(arr[mask])

Generate coordinates inside Polygon

I want to bin the values of polygons to a fine regular grid.
For instance, I have the following coordinates:
data = 2.353
data_lats = np.array([57.81000137, 58.15999985, 58.13000107, 57.77999878])
data_lons = np.array([148.67999268, 148.69999695, 148.47999573, 148.92999268])
My regular grid looks like this:
delta = 0.25
grid_lons = np.arange(-180, 180, delta)
grid_lats = np.arange(90, -90, -delta)
llx, lly = np.meshgrid( grid_lons, grid_lats )
rows = lly.shape[0]
cols = llx.shape[1]
grid = np.zeros((rows,cols))
Now I can find the grid pixel that corresponds to the center of my polygon very easily:
centerx, centery = np.mean(data_lons), np.mean(data_lats)
row = int(np.floor( centery/delta ) + (grid.shape[0]/2))
col = int(np.floor( centerx/delta ) + (grid.shape[1]/2))
grid[row,col] = data
However, there are probably a couple of grid pixels that still intersect with the polygon. Hence, I would like to generate a bunch of coordinates inside my polygon (data_lons, data_lats) and find their corresponding grid pixel as before. Do you a suggestion to generate the coordinates randomly or systematically? I failed, but am still trying.
Note: One data set contains around ~80000 polygons so it has to be really fast (a couple of seconds). That is also why I chose this approach, because it does not account the area of overlap... (like my earlier question Data binning: irregular polygons to regular mesh which is VERY slow)

I worked on a quick and dirty solution by simply calculating the coordinates between corner pixels. Take a look:
dlats = np.zeros((data_lats.shape[0],4))+np.nan
dlons = np.zeros((data_lons.shape[0],4))+np.nan
idx = [0,1,3,2,0] #rearrange the corner pixels
for cc in range(4):
dlats[:,cc] = np.mean((data_lats[:,idx[cc]],data_lats[:,idx[cc+1]]), axis=0)
dlons[:,cc] = np.mean((data_lons[:,idx[cc]],data_lons[:,idx[cc+1]]), axis=0)
data_lats = np.column_stack(( data_lats, dlats ))
data_lons = np.column_stack(( data_lons, dlons ))
Thus, the red dots represent the original corners - the blue ones the intermediate pixels between them.
I can do this one more time and include the center pixel (geo[:,[4,9]])
dlats = np.zeros((data.shape[0],8))
dlons = np.zeros((data.shape[0],8))
for cc in range(8):
dlats[:,cc] = np.mean((data_lats[:,cc], geo[:,4]), axis=0)
dlons[:,cc] = np.mean((data_lons[:,cc], geo[:,9]), axis=0)
data_lats = np.column_stack(( data_lats, dlats, geo[:,4] ))
data_lons = np.column_stack(( data_lons, dlons, geo[:,9] ))
This works really nice, and I can assign each point directly to its corresponding grid pixel like this:
row = np.floor( data_lats/delta ) + (llx.shape[0]/2)
col = np.floor( data_lons/delta ) + (llx.shape[1]/2)
However the final binning now takes ~7sec!!! How can I speed this code up:
for ii in np.arange(len(data)):
for cc in np.arange(data_lats.shape[1]):
final_grid[row[ii,cc],col[ii,cc]] += data[ii]
final_grid_counts[row[ii,cc],col[ii,cc]] += 1

You'll need to test the following approach to see if it is fast enough. First, you should modify all your lats and lons into, to make them (possibly fractional) indices into your grid:
idx_lats = (data_lats - lat_grid_start) / lat_grid step
idx_lons = (data_lons - lon_grid_start) / lon_grid step
Next, we want to split your polygons into triangles. For any convex polygon, you could take the center of the polygon as one vertex of all triangles, and then the vertices of the polygon in consecutive pairs. But if your polygon are all quadrilaterals, it is going to be faster to divide them into only 2 triangles, using vertices 0, 1, 2 for the first, and 0, 2, 3 for the second.
To know if a certain point is inside a triangle, I am going to use the barycentric coordinates approach described here. This first function checks whether a bunch of points are inside a triangle:
def check_in_triangle(x, y, x_tri, y_tri) :
A = np.vstack((x_tri[0], y_tri[0]))
lhs = np.vstack((x_tri[1:], y_tri[1:])) - A
rhs = np.vstack((x, y)) - A
uv = np.linalg.solve(lhs, rhs)
# Equivalent to (uv[0] >= 0) & (uv[1] >= 0) & (uv[0] + uv[1] <= 1)
return np.logical_and(uv >= 0, axis=0) & (np.sum(uv, axis=0) <= 1)
Given a triangle by its vertices, you can get the lattice points inside it, by running the above function on the lattice points in the bounding box of the triangle:
def lattice_points_in_triangle(x_tri, y_tri) :
x_grid = np.arange(np.ceil(np.min(x_tri)), np.floor(np.max(x_tri)) + 1)
y_grid = np.arange(np.ceil(np.min(y_tri)), np.floor(np.max(y_tri)) + 1)
x, y = np.meshgrid(x_grid, y_grid)
x, y = x.reshape(-1), y.reshape(-1)
idx = check_in_triangle(x, y, x_tri, y_tri)
return x[idx], y[idx]
And for a quadrilateral, you simply call this last function twice :
def lattice_points_in_quadrilateral(x_quad, y_quad) :
return map(np.concatenate,
zip(lattice_points_in_triangle(x_quad[:3], y_quad[:3]),
lattice_points_in_triangle(x_quad[[0, 2, 3]],
y_quad[[0, 2, 3]])))
If you run this code on your example data, you will get two empty arrays returned: that's because the order of the quadrilateral points is a surprising one: indices 0 and 1 define one diagonal, 2 and 3 the other. My function above was expecting the vertices to be ordered around the polygon. If you really are doing things this other way, you need to change the second call to lattice_points_in_triangle inside lattice_points_in_quadrilateral so that the indices used are [0, 1, 3] instead of [0, 2, 3].
And now, with that change :
>>> idx_lats = (data_lats - (-180) ) / 0.25
>>> idx_lons = (data_lons - (-90) ) / 0.25
>>> lattice_points_in_quadrilateral(idx_lats, idx_lons)
[array([952]), array([955])]
If you change the resolution of your grid to 0.1:
>>> idx_lats = (data_lats - (-180) ) / 0.1
>>> idx_lons = (data_lons - (-90) ) / 0.1
>>> lattice_points_in_quadrilateral(idx_lats, idx_lons)
[array([2381, 2380, 2381, 2379, 2380, 2381, 2378, 2379, 2378]),
array([2385, 2386, 2386, 2387, 2387, 2387, 2388, 2388, 2389])]
Timing wise this approach is going to be, in my system, about 10x too slow for your needs:
In [8]: %timeit lattice_points_in_quadrilateral(idx_lats, idx_lons)
1000 loops, best of 3: 269 us per loop
So you are looking at over 20 sec. to process your 80,000 polygons.