Develop Reference

Speed up nested for loop with NumPy

I'm trying to write a package about image processing with some numpy operations. I've observe that the operations inside the nested loop are costly and want to speed it up.
Input is an 512 by 1024 image and be preprocessing into a
edge set, which is a list of (Ni,2) ndarrays for each array i.
And next, the nested for loop code will pass edge set and do some math stuffs.
###proprocessing: img ===> countour set
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
high_thresh, _ = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY +
cv2.THRESH_OTSU)
lowThresh = 0.5*high_thresh
b = cv2.Canny(img, lowThresh, high_thresh)
edgeset, _ =
cv2.findContours(b,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
imgH = img.shape[0] ## 512
imgW = img.shape[1] ## 1024
num_edges = len(edgeset) ## ~900
min_length_segment_vp = imgH/6 ## ~100
### nested for loop
for i in range(num_edges):
if(edgeset[i].shape[0] > min_length_segment_vp):
#points: (N, 1, 2) ==> uv: (N, 2)
uv = edgeset[i].reshape(edgeset[i].shape[0],
edgeset[i].shape[2])
uv = np.unique(uv, axis=0)
theta = -(uv[:, 1]-imgH/2)*np.pi/imgH
phi = (uv[:, 0]-imgW/2)*2*np.pi/imgW
xyz = np.zeros((uv.shape[0], 3))
xyz[:, 0] = np.sin(phi) * np.cos(theta)
xyz[:, 1] = np.cos(theta) * np.cos(phi)
xyz[:, 2] = np.sin(theta)
##xyz: (N, 3)
N=xyz.shape[0]
for _ in range(10):
if(xyz.shape[0] > N * 0.1):
bestInliers = np.array([])
bestOutliers = np.array([])
####
#### watch this out!
####
for _ in range(1000):
id0 = random.randint(0, xyz.shape[0]-1)
id1 = random.randint(0, xyz.shape[0]-1)
if(id0 == id1):
continue
n = np.cross(xyz[id0, :], xyz[id1, :])
n = n / np.linalg.norm(n)
cosTetha = n # xyz.T
inliers = np.abs(cosTetha) < threshold
outliers = np.where(np.invert(inliers))[0]
inliers = np.where(inliers)[0]
if inliers.shape[0] > bestInliers.shape[0]:
bestInliers = inliers
bestOutliers = outliers
What I have tried:
I changed np.cross and np.norm into my custom cross and norm
only work for shape (3,) ndarray. This gives me a from ~0.9s into
~0.3s in my i5-4460 cpu.
I profile my code and find that now the code inside the most inner loop still cost 2/3 of time.
What I think I can try next:
Compile code into cython and add some cdef notation.
Translate whole file into C++.
Use some faster library for calculation like numexpr.
Vectorization of the loop process (but I don't know how).
Can I do more faster? Please give me some suggestions! Thanks!

The question is quite broad so I'll only give a few non-obvious tips based on my own experience.
If you use Cython, you might want to change the for loops into while loops. I've managed to get quite big (x5) speed-ups just from this, although it may not help for all possible cases;
Sometimes code that would be considered inefficient in regular Python, such as a nested while (or for) loop to apply a function to an array one element at a time, can be optimized by Cython to be faster than the equivalent vectorized Numpy approach;
Find out which Numpy functions cost the most time, and write your own in a way that Cython can most easily optimise them (see above point).

Quick pixel manipulation with Pillow and/or NumPy

I'm trying to improve the speed of my image manipulation as it's been too slow for actual use.
What I need to do is apply a complex transformation on the colour of every pixel on an image. The manipulation is basically apply a vector transform like T(r, g, b, a) => (r * x, g * x, b * y, a) or in layman's terms, it's a multiplication of Red and Green values by a constant, a different multiplication for Blue and keep Alpha. But I also need to manipulate it differently if the RGB colour falls under some specific colours, in those cases they must follow a dictionary/transformation table where RGB => newRGB again keeping alpha.
The algorithm would be:
for each pixel in image:
if pixel[r, g, b] in special:
return special[pixel[r, g, b]] + pixel[a]
else:
return T(pixel)
It's simple but speed has been sub-optimal. I believe there's some way using numpy vectors, but I could not find how.
Important details about the implementation:
I don't care about the original buffer/image (manipulation can be in place)
I can use wxPython, Pillow and NumPy
Order or dimension of the array is not important as long as the buffer keeps the length
The buffer is obtained from a wxPython Bitmap and special and (RG|B)_pal are transformation tables, the end result will become a wxPython Bitmap too. They're obtained like these:
# buffer
bitmap = wx.Bitmap # it's valid wxBitmap here, this is just to let you know it exists
buff = bytearray(bitmap.GetWidth() * bitmap.GetHeight() * 4)
bitmap.CopyToBuffer(buff, wx.BitmapBufferFormat_RGBA)
self.RG_mult= 0.75
self.B_mult = 0.83
self.RG_pal = []
self.B_pal = []
for i in range(0, 256):
self.RG_pal.append(int(i * self.RG_mult))
self.B_pal.append(int(i * self.B_mult))
self.special = {
# RGB: new_RGB
# Implementation specific for the fastest access
# with buffer keys are 24bit numbers, with PIL keys are tuples
}
Implementations I tried include direct buffer manipulation:
for x in range(0, bitmap.GetWidth() * bitmap.GetHeight()):
index = x * 4
r = buf[index]
g = buf[index + 1]
b = buf[index + 2]
rgb = buf[index:index + 3]
if rgb in self.special:
special = self.special[rgb]
buf[index] = special[0]
buf[index + 1] = special[1]
buf[index + 2] = special[2]
else:
buf[index] = self.RG_pal[r]
buf[index + 1] = self.RG_pal[g]
buf[index + 2] = self.B_pal[b]
Use Pillow with getdata():
pil = Image.frombuffer("RGBA", (bitmap.GetWidth(), bitmap.GetHeight()), buf)
pil_buf = []
for colour in pil.getdata():
colour_idx = colour[0:3]
if (colour_idx in self.special):
special = self.special[colour_idx]
pil_buf.append((
special[0],
special[1],
special[2],
colour[3],
))
else:
pil_buf.append((
self.RG_pal[colour[0]],
self.RG_pal[colour[1]],
self.B_pal[colour[2]],
colour[3],
))
pil.putdata(pil_buf)
buf = pil.tobytes()
Pillow with point() and getdata() (fastest I achieved, more than twice times faster than others)
pil = Image.frombuffer("RGBA", (bitmap.GetWidth(), bitmap.GetHeight()), buf)
r, g, b, a = pil.split()
r = r.point(lambda r: r * self.RG_mult)
g = g.point(lambda g: g * self.RG_mult)
b = b.point(lambda b: b * self.B_mult)
pil = Image.merge("RGBA", (r, g, b, a))
i = 0
for colour in pil.getdata():
colour_idx = colour[0:3]
if (colour_idx in self.special):
special = self.special[colour_idx]
pil.putpixel(
(i % bitmap.GetWidth(), i // bitmap.GetWidth()),
(
special[0],
special[1],
special[2],
colour[3],
)
)
i += 1
buf = pil.tobytes()
I also tried working with numpy.where but then I could not get it to work. With numpy.apply_along_axis it worked but the performance was terrible. Other tries with numpy I could not access the RGB together, only as separated bands.

Pure Numpy Version
This first optimization relies on the fact, that one probably has way less special colors than pixels. I use numpy to do all the inner loops. This works well with images of up to 1MP. If You have multiple images I'd recommend the parallel approach.
Let's define a test case:
import requests
from io import BytesIO
from PIL import Image
import numpy as np
# Load some image, so we have the same
response = requests.get("https://upload.wikimedia.org/wikipedia/commons/4/41/Rick_Astley_Dallas.jpg")
# Make areas of known color
img = Image.open(BytesIO(response.content)).rotate(10, expand=True).rotate(-10,expand=True, fillcolor=(255,255,255)).convert('RGBA')
print("height: %d, width: %d (%.2f MP)"%(img.height, img.width, img.width*img.height/10e6))
height: 5034, width: 5792 (2.92 MP)
Define our special colors
specials = {
(4,1,6):(255,255,255),
(0, 0, 0):(255, 0, 255),
(255, 255, 255):(0, 255, 0)
}
Algorithm
def transform_map(img, specials, R_factor, G_factor, B_factor):
# Your transform
def transform(x, a):
a *= x
return a.clip(0, 255).astype(np.uint8)
# Convert to array
img_array = np.asarray(img)
# Extract channels
R = img_array.T[0]
G = img_array.T[1]
B = img_array.T[2]
A = img_array.T[3]
# Find Special colors
# First, calculate a uniqe hash
color_hashes = (R + 2**8 * G + 2**16 * B)
# Find inidices of special colors
special_idxs = []
for k, v in specials.items():
key_arr = np.array(list(k))
val_arr = np.array(list(v))
spec_hash = key_arr[0] + 2**8 * key_arr[1] + 2**16 * key_arr[2]
special_idxs.append(
{
'mask': np.where(np.isin(color_hashes, spec_hash)),
'value': val_arr
}
)
# Apply transform to whole image
R = transform(R, R_factor)
G = transform(G, G_factor)
B = transform(B, B_factor)
# Replace values where special colors were found
for idx in special_idxs:
R[idx['mask']] = idx['value'][0]
G[idx['mask']] = idx['value'][1]
B[idx['mask']] = idx['value'][2]
return Image.fromarray(np.array([R,G,B,A]).T, mode='RGBA')
And finally some bench marks on a Intel Core i5-6300U # 2.40GHz
import time
times = []
for i in range(10):
t0 = time.time()
# Test
transform_map(img, specials, 1.2, .9, 1.2)
#
t1 = time.time()
times.append(t1-t0)
np.round(times, 2)
print('average run time: %.2f +/-%.2f'%(np.mean(times), np.std(times)))
average run time: 9.72 +/-0.91
EDIT Parallelization
With the same setup as above, we can get a 2x speed increase on large images. (Small ones are faster without numba)
from numba import njit, prange
from numba.core import types
from numba.typed import Dict
# Map dict of special colors or transform over array of pixel values
#njit(parallel=True, locals={'px_hash': types.uint32})
def check_and_transform(img_array, d, T):
#Save Shape for later
shape = img_array.shape
# Flatten image for 1-d iteration
img_array_flat = img_array.reshape(-1,3).copy()
N = img_array_flat.shape[0]
# Replace or map
for i in prange(N):
px_hash = np.uint32(0)
px_hash += img_array_flat[i,0]
px_hash += types.uint32(2**8) * img_array_flat[i,1]
px_hash += types.uint32(2**16) * img_array_flat[i,2]
try:
img_array_flat[i] = d[px_hash]
except Exception:
img_array_flat[i] = (img_array_flat[i] * T).astype(np.uint8)
# return image
return img_array_flat.reshape(shape)
# Wrapper for function above
def map_or_transform_jit(image: Image, specials: dict, T: np.ndarray):
# assemble numba typed dict
d = Dict.empty(
key_type=types.uint32,
value_type=types.uint8[:],
)
for k, v in specials.items():
k = types.uint32(k[0] + 2**8 * k[1] + 2**16 * k[2])
v = np.array(v, dtype=np.uint8)
d[k] = v
# get rgb channels
img_arr = np.array(img)
rgb = img_arr[:,:,:3].copy()
img_shape = img_arr.shape
# apply map
rgb = check_and_transform(rgb, d, T)
# set color channels
img_arr[:,:,:3] = rgb
return Image.fromarray(img_arr, mode='RGBA')
# Benchmark
import time
times = []
for i in range(10):
t0 = time.time()
# Test
test_img = map_or_transform_jit(img, specials, np.array([1, .5, .5]))
#
t1 = time.time()
times.append(t1-t0)
np.round(times, 2)
print('average run time: %.2f +/- %.2f'%(np.mean(times), np.std(times)))
test_img
average run time: 3.76 +/- 0.08

Python gray-scale conversion of an image

So I made this script that takes an image and turns it into a gray scale of itself.
I know that a lot of modules can do this automatically like .convert('grey') but I want to do it manually by myself to learn more about python programming.
It works ok but its very slow, for a 200pX200p image it takes 10 seconds so, what can I modify for making it go faster?
it works like this, it takes a pixel, calculates the averange of R, G and B values, set the three to the averange value, adds 40 to each one for more brightness and writes the pixel.
Here is the code:
import imageio
import os
from PIL import Image, ImageDraw
from random import randrange
img = '/storage/emulated/0/DCIM/Camera/IMG_20190714_105429.jpg'
f = open('network.csv', 'a+')
pic = imageio.imread(img)
picture = Image.open(img)
draw = ImageDraw.Draw(picture)
f.write('\n')
def por():
cien = pic.shape[0] * pic.shape[1]
prog = pic.shape[1] * (h - 1) + w
porc = prog * 100 / cien
porc = round(porc)
porc = str(porc)
print(porc + '%')
rh = int(pic.shape[0])
wh = int(pic.shape[1])
for h in range(rh):
for w in range(wh):
prom = int(pic[h , w][0]) + int(pic[h, w][1]) + int(pic[h, w][2])
prom = prom / 3
prom = round(prom)
prom = int(prom)
prom = prom + 40
por()
draw.point( (w,h), (prom,prom,prom))
picture.save('/storage/emulated/0/DCIM/Camera/Modificada.jpg')

PIL does this for you.
from PIL import Image
img = Image.open('image.png').convert('grey')
img.save('modified.png')

The Method you are using for conversion of RGB to greyscale, is called Averaging.
from PIL import Image
image = Image.open(r"image_path").convert("RGB")
mapping = list(map(lambda x: int(x[0]*.33 + x[1]*.33 + x[2]*.33), list(image.getdata())))
Greyscale_img = Image.new("L", (image.size[0], image.size[1]), 255)
Greyscale_img.putdata(mapping)
Greyscale_img.show()
The above method (Averaging) isn't recommended for conversion of an colored image into greyscale. As it treats each color channel equally, assuming human perceives all colors equally (which is not the truth).
You should rather use something like ITU-R 601-2 luma transform (method used by PIL for converting RGB to L) for the conversion. As it uses perceptual luminance-preserving conversion to grayscale.
For that Just replace the line
mapping = list(map(lambda x: int(x[0]*.33 + x[1]*.33 + x[2]*.33), list(image.getdata())))
with
mapping = list(map(lambda x: int(x[0]*(299/1000) + x[1]*(587/1000) + x[2]*(114/1000)), list(image.getdata())))
P.S.:- I didn't add 40 to each pixel value, as it doesn't really makes any sense in the conversion of the image to greyscale.

Python is an interpreted language and not really fast enough for pixel loops. cython is a sister project that can compile Python into an executable and can be faster than plain Python for code like this.
You could also try using a Python math library like numpy or pyvips. These add array operations to Python: you can write lines like a += 12 * b where a and b are whole images and they'll operate on every pixel at the same time. You get the control of being able to specify every detail of the operation yourself combined with the speed of something like C.
For example, in pyvips you could write:
import sys
import pyvips
x = pyvips.Image.new_from_file(sys.argv[1], access="sequential")
x = 299 / 1000 * x[0] + 587 / 1000 * x[1] + 114 / 1000 * x[2]
x.write_to_file(sys.argv[2])
Copying the equation from Vasu Deo.S's excellent answer, then run with something like:
./grey2.py ~/pics/k2.jpg x.png
To read the JPG image k2.jpg and write a greyscale PNG called x.png.
You can approximate conversion in linear space with a pow before and after, assuming your source image is sRGB:
x = x ** 2.2
x = 299 / 1000 * x[0] + 587 / 1000 * x[1] + 114 / 1000 * x[2]
x = x ** (1 / 2.2)
Though that's not exactly correct since it's missing the linear part of the sRGB power function.
You could also simply use x = x.colourspace('b-w'), pyvips's built-in greyscale operation.

How to efficiently apply a function to each channel of every pixel in an image (for color conversion)?

I'm trying to implement Reinhard's method to use the color distribution of a target image to color normalize a passed in image for a research project. I've gotten the code to work and it outputs correctly but it's pretty slow. It takes about 20 minutes to iterate through 300 images. I'm pretty sure the bottleneck is how I'm handling applying the function to each image. I'm currently iterating through each pixel of the image and applying the functions below to each channel.
def reinhard(target, img):
#converts image and target from BGR colorspace to l alpha beta
lAB_img = cv2.cvtColor(img, cv2.COLOR_BGR2Lab)
lAB_tar = cv2.cvtColor(target, cv2.COLOR_BGR2Lab)
#finds mean and standard deviation for each color channel across the entire image
(mean, std) = cv2.meanStdDev(lAB_img)
(mean_tar, std_tar) = cv2.meanStdDev(lAB_tar)
#iterates over image implementing formula to map color normalized pixels to target image
for y in range(512):
for x in range(512):
lAB_tar[x, y, 0] = (lAB_img[x, y, 0] - mean[0]) / std[0] * std_tar[0] + mean_tar[0]
lAB_tar[x, y, 1] = (lAB_img[x, y, 1] - mean[1]) / std[1] * std_tar[1] + mean_tar[1]
lAB_tar[x, y, 2] = (lAB_img[x, y, 2] - mean[2]) / std[2] * std_tar[2] + mean_tar[2]
mapped = cv2.cvtColor(lAB_tar, cv2.COLOR_Lab2BGR)
return mapped
My supervisor told me that I could try using a matrix to apply the function all at once to improve the runtime but I'm not exactly sure how to go about doing that.

The original and the target：
Color transfer reuslts using Reinhard'method in 5 ms:
I prefer to implement the formulat in numpy vectorized operations other than python loops.
# implementing the formula
#(Io - mo)/so*st + mt = Io * (st/so) + mt - mo*(st/so)
ratio = (std_tar/std_ori).reshape(-1)
offset = (mean_tar - mean_ori*std_tar/std_ori).reshape(-1)
lab_tar = cv2.convertScaleAbs(lab_ori*ratio + offset)
Here is the code:
# 2019/02/19 by knight-金
# https://stackoverflow.com/a/54757659/3547485
import numpy as np
import cv2
def reinhard(target, original):
# cvtColor: COLOR_BGR2Lab
lab_tar = cv2.cvtColor(target, cv2.COLOR_BGR2Lab)
lab_ori = cv2.cvtColor(original, cv2.COLOR_BGR2Lab)
# meanStdDev: calculate mean and stadard deviation
mean_tar, std_tar = cv2.meanStdDev(lab_tar)
mean_ori, std_ori = cv2.meanStdDev(lab_ori)
# implementing the formula
#(Io - mo)/so*st + mt = Io * (st/so) + mt - mo*(st/so)
ratio = (std_tar/std_ori).reshape(-1)
offset = (mean_tar - mean_ori*std_tar/std_ori).reshape(-1)
lab_tar = cv2.convertScaleAbs(lab_ori*ratio + offset)
# convert back
mapped = cv2.cvtColor(lab_tar, cv2.COLOR_Lab2BGR)
return mapped
if __name__ == "__main__":
ori = cv2.imread("ori.png")
tar = cv2.imread("tar.png")
mapped = reinhard(tar, ori)
cv2.imwrite("mapped.png", mapped)
mapped_inv = reinhard(ori, tar)
cv2.imwrite("mapped_inv.png", mapped)

I managed to figure it out after looking at the numpy documentation. I just needed to replace my nested for loop with proper array accessing. It took less than a minute to iterate through all 300 images with this.
lAB_tar[:,:,0] = (lAB_img[:,:,0] - mean[0])/std[0] * std_tar[0] + mean_tar[0]
lAB_tar[:,:,1] = (lAB_img[:,:,1] - mean[1])/std[1] * std_tar[1] + mean_tar[1]
lAB_tar[:,:,2] = (lAB_img[:,:,2] - mean[2])/std[2] * std_tar[2] + mean_tar[2]

Creating an image from a dictionary using PIL

I have a dictionary that maps coordinate tuples (in the range (0,0) to (199, 199) to grayscale values (integers between 0 and 255.) Is there a good way to create an PIL Image that has the specified values at the specified coordinates? I'd prefer a solution that only uses PIL to one that uses scipy.

You can try image.putpixel() to change the color of a pixel at a particular position. Example code -
from PIL import Image
from random import randint
d = {(x,y):randint(0,255) for x in range(200) for y in range(200)}
im = Image.new('L',(200,200))
for i in d:
im.putpixel(i,d[i])
im.save('blah.png')
It gave me a result like -

You could do it with putpixel(), but that could potentially involve tens of thousands of calls. How much this matters depends on how many coordinate tuples are defined in the dictionary. I've included the method shown in each of the current answers for comparison (including my own before any benchmarking was added, but just now I made a small change to how it initializes the data buffer which measurably sped it up).
To make a level playing field, for testing purposes the input dictionary randomly selects only ½ of the possible pixels in the image to define and allows the rest to be set to a default background color. Anand S Kumar's answer currently doesn't do the latter, but the slightly modified version shown below does.
All produce the same image from the data.
from __future__ import print_function
import sys
from textwrap import dedent
import timeit
N = 100 # number of executions of each algorithm
R = 3 # number of repeations of executions
# common setup for all algorithms - is not included in algorithm timing
setup = dedent("""
from random import randint, sample, seed
from PIL import Image
seed(42)
background = 0 # default color of pixels not defined in dictionary
width, height = 200, 200
# create test dict of input data defining half of the pixel coords in image
coords = sample([(x,y) for x in xrange(width) for y in xrange(height)],
width * height // 2)
d = {coord: randint(0, 255) for coord in coords}
""")
algorithms = {
"Anand S Kumar": dedent("""
im = Image.new('L', (width, height), color=background) # set bgrd
for i in d:
im.putpixel(i, d[i])
"""),
"martineau": dedent("""
data = bytearray([background] * width * height)
for (x, y), v in d.iteritems():
data[x + y*width] = v
im = Image.frombytes('L', (width, height), str(data))
"""),
"PM 2Ring": dedent("""
data = [background] * width * height
for i in d:
x, y = i
data[x + y * width] = d[i]
im = Image.new('L', (width, height))
im.putdata(data)
"""),
}
# execute and time algorithms, collecting results
timings = [
(label,
min(timeit.repeat(algorithms[label], setup=setup, repeat=R, number=N)),
) for label in algorithms
]
print('fastest to slowest execution speeds (Python {}.{}.{})\n'.format(
*sys.version_info[:3]),
' ({:,d} executions, best of {:d} repetitions)\n'.format(N, R))
longest = max(len(timing[0]) for timing in timings) # length of longest label
ranked = sorted(timings, key=lambda t: t[1]) # ascending sort by execution time
fastest = ranked[0][1]
for timing in ranked:
print("{:>{width}} : {:9.6f} secs, rel speed {:4.2f}x, {:6.2f}% slower".
format(timing[0], timing[1], round(timing[1]/fastest, 2),
round((timing[1]/fastest - 1) * 100, 2), width=longest))
Output:
fastest to slowest execution speeds (Python 2.7.10)
(100 executions, best of 3 repetitions)
martineau : 0.255203 secs, rel speed 1.00x, 0.00% slower
PM 2Ring : 0.307024 secs, rel speed 1.20x, 20.31% slower
Anand S Kumar : 1.835997 secs, rel speed 7.19x, 619.43% slower

As martineau suggests putpixel() is ok when you're modifying a few random pixels, but it's not so efficient for building whole images. My approach is similar to his, except I use a list of ints and .putdata(). Here's some code to test these 3 different approaches.
from PIL import Image
from random import seed, randint
width, height = 200, 200
background = 0
seed(42)
d = dict(((x, y), randint(0, 255)) for x in range(width) for y in range(height))
algorithm = 2
print('Algorithm', algorithm)
if algorithm == 0:
im = Image.new('L', (width, height))
for i in d:
im.putpixel(i, d[i])
elif algorithm == 1:
buff = bytearray((background for _ in xrange(width * height)))
for (x,y), v in d.items():
buff[y*width + x] = v
im = Image.frombytes('L', (width,height), str(buff))
elif algorithm == 2:
data = [background] * width * height
for i in d:
x, y = i
data[x + y * width] = d[i]
im = Image.new('L', (width, height))
im.putdata(data)
#im.show()
fname = 'qrand%d.png' % algorithm
im.save(fname)
print(fname, 'saved')
Here are typical timings on my 2GHz machine running Python 2.6.6
$ time ./qtest.py
Algorithm 0
qrand0.png saved
real 0m0.926s
user 0m0.768s
sys 0m0.040s
$ time ./qtest.py
Algorithm 1
qrand1.png saved
real 0m0.733s
user 0m0.548s
sys 0m0.020s
$ time ./qtest.py
Algorithm 2
qrand2.png saved
real 0m0.638s
user 0m0.520s
sys 0m0.032s