Develop Reference

Confusion with red eye detection

First of all, I am not asking anyone to do my homework. I would like to get an explanation or clarification about my difficulties in understanding the following question.
I just finished my image processing test, but one question that I could not solve due to my confusion.
The question is:
Write the code to detect the red eye in a given image in RGB color space using the following formula for HSL color space:
LS_ratio = L / S
eye_pixel = (L >= 64) and (S >= 100) and (LS_ratio > 0.5) and (LS_ratio < 1.5) and ((H <= 7) or (H >= 162))
Please note that in above formula, H, S and L represent a single pixel value for the image in HSL color space and the value of ‘eye_pixel’ will be either True or False depending on the values of H, S and L (i.e. it will be either a red eye color pixel or not).
Your task is to write the code to check all pixels in the image. Store the result as a numpy array and display the resulted image.
My code is:
from __future__ import print_function
import numpy as np
import argparse
import cv2
#argument paser
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image", required = True, help = "Path to the image")
args = vars(ap.parse_args())
#load the image
image = cv2.imread(args["image"])
#Convert image to HLS
hls = cv2.cvtColor(image, cv2.COLOR_BGR2HLS)
#Split HLS Channels
H = hls[:, :, 0]
S = hls[:, :, 1]
L = hls[:, :, 2]
LS_ratio = L / S
#eye_pixel = (L >= 64) and (S >= 100) and (LS_ratio > 0.5) and (LS_ratio < 1.5) and ((H <= 7) or (H >= 162))
#if HSL pixel
#eye pixel either red or not
#show the image
#cv2.imshow("Image", np.hstack([image, red_eye]))
#debug
print("Lightness is: {}".format(L))
print("Saturation is: {}".format(S))
print("Hue is: {}".format(H))
#print("LS ratio: {}", LS_ratio)
cv2.waitKey(0)
Suppose that the image is:
I literally feel confused about what needs to be done. Highly appreciate if anyone helps explains to me what should be done.
Thank you.

All you need to do is implement the formula in term of the entire H, L, S images.
#Convert image to HLS
hls = cv2.cvtColor(image, cv2.COLOR_BGR2HLS)
#Split HLS Channels
H = hls[:, :, 0]
L = hls[:, :, 1]
S = hls[:, :, 2]
LS_ratio = L/(S + 1e-6)
redeye = ((L>=64) * (S>=100) * np.logical_or(H<=7, H>=162) * (LS_ratio>0.5) * (LS_ratio<1.5)).astype(bool)
Here redeye is a bool array the same size of your original image, where each pixel contains a True or False, representing whether if it's a redeye pixel or not. If I display the image:
redeye = cv2.cvtColor(redeye.astype(np.uint8)*255, cv2.COLOR_GRAY2BGR)
cv2.imshow('image-redeye', np.hstack([image, redeye]))

Region of interest - Object detection with SparseOpticalFlow

I am using the SparseOpticalFlow in python. With YOLOv3 I've extracted the boundary boxes of the object I need.
Now I would love to give those boundaries to the SOF algorithm to track only what I want to track.
I've found that there's "goodfeaturestotrack" function that finds the corners for the tracking. This function needs a "mask" param where it said: "mask – Optional region of interest. If the image is not empty (it needs to have the type CV_8UC1 and the same size as image ), it specifies the region in which the corners are detected."
prev = cv.goodFeaturesToTrack(prev_gray, mask = None, **feature_params)
This "mask" is what I am looking for.
My question is: what are the parameter to create the mask? I mean, it needs width, height, x, y or what else? Anyone can help?
The boundaries boxes param given from the YOLOv3 detector are: floating x, y, width, height. Where x and y start from the top left corner.
EDIT:
That's what I've tried to do.
# Initialize the mask with all black pixels
mask = np.zeros_like(first_frame)
# Get the coordinates and dimensions of the detect_box
x = 82
y = 69
w = 35
h = 55
# Set the selected rectangle within the mask to white
mask[y:y+h, x:x+w] = 255
prev = cv.goodFeaturesToTrack(prev_gray, mask = mask, **feature_params)
But i get this error...:
PS C:\Users\YodhResearch\Desktop\GIULIO FERRARI FOLDER\Giulio _ CSV\Py Script\sparse python> python .\sparseopt.py
Traceback (most recent call last):
File ".\sparseopt.py", line 32, in <module>
prev = cv.goodFeaturesToTrack(prev_gray, mask = mask, **feature_params)
cv2.error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\imgproc\src\featureselect.cpp:365: error: (-215:Assertion failed) _mask.empty() || (_mask.type() == CV_8UC1 && _mask.sameSize(_image)) in
function 'cv::goodFeaturesToTrack'
EDIT2:
Solution:
width = cap.get(cv.CAP_PROP_FRAME_WIDTH) # float
height = cap.get(cv.CAP_PROP_FRAME_HEIGHT) # float
print("w = %f" % width)
print("h = %f" % height)
# Initialize the mask with all black pixels
mask = [[0]*int(width)]*int(height)
mask = np.asarray(mask)
mask = mask.astype(np.uint8)
print( np.shape(mask) )
# Get the coordinates and dimensions of the detect_box
x = 82
y = 69
w = 35
h = 55
# Set the selected rectangle within the mask to white
mask[y:y+h, x:x+w] = 255
prev = cv.goodFeaturesToTrack(prev_gray, mask = mask, **feature_params)
# Creates an image filled with zero intensities with the same dimensions as the frame - for later drawing purposes
mask = np.zeros_like(first_frame)

How to blur the image according to segmentation map

Forgive me if I am unable to explain well because I am not native speaker.
I am working on blurring the part of image according to the white part of segmentation map. For example here is my segmentation image ( bmp image ).
.
Now what I want is to blur the part of original image where the pixels are white in the segmentation map. I just wrote the following code to so.
mask = mask >= 0.5
mask = np.reshape(mask, (512, 512))
mh, mw = 512, 512
mask_n = np.ones((mh, mw, 3))
mask_n[:,:,0] *= mask
mask_n[:,:,1] *= mask
mask_n[:,:,2] *= mask
# discard padded area
ih, iw, _ = image_n.shape
delta_h = mh - ih
delta_w = mw - iw
top = delta_h // 2
bottom = mh - (delta_h - top)
left = delta_w // 2
right = mw - (delta_w - left)
mask_n = mask_n[top:bottom, left:right, :]
# addWeighted
image_n = image_n *1 + cv2.blur(mask_n * 0.8, (800, 800))
Please help me, Thanks.

You can do it in the following steps:
Load original image and mask image.
Blur the whole original image and save it in a different variable.
Use np.where() method to select the pixels from the mask where you want blurred values and then replace it.
See the sample code below:
import cv2
import numpy as np
img = cv2.imread("./image.png")
blurred_img = cv2.GaussianBlur(img, (21, 21), 0)
mask = cv2.imread("./mask.png")
output = np.where(mask==np.array([255, 255, 255]), blurred_img, img)
cv2.imwrite("./output.png", output)

Here's an alternative to the solution proposed by #Chris Henri. It relies on scipy.ndimage.filters.gaussian_filter and NumPy's boolean indexing:
from skimage import io
import numpy as np
from scipy.ndimage.filters import gaussian_filter
import matplotlib.pyplot as plt
mask = io.imread('https://i.stack.imgur.com/qJiKf.png')
img = np.random.random(size=mask.shape[:2])
idx = mask.min(axis=-1) == 255
blurred = gaussian_filter(img, sigma=3)
blurred[~idx] = 0
fig, axs = plt.subplots(1, 3, figsize=(12, 4))
for ax, im in zip(axs, [img, mask, blurred]):
ax.imshow(im, cmap='gray')
ax.set_axis_off()
plt.show(fig)

Here is yet another alternative to do so, useful though when you have a 2D segmentation array indicating the segmented object class of pixel (mutually exclusive) for every index (i,j), and a 3D image on which you want to apply the blur.
def gaussian_blur(image: np.ndarray,
segmentation: np.ndarray,
classes_of_interest: list,
gaussian_variance: float = 10) -> np.ndarray:
'''
Function that applies a gaussian filter to the image,
specifically to the pixels contained in the possible segmented classes.
Returns an image (np.ndarray) where the gaussian blur intensity is
regulated by the parameter gaussian_variance.
'''
#Apply masking to select only the indices where the specific class is present
mask = np.isin(segmentation, classes_of_interest)
#Creating a 3D mask for all the channels and place it at channel axis
mask_3d = np.stack([mask,mask,mask], axis=2)
#Mask the image according to the 3D mask
img_masked = np.where(mask_3d, img, 0).astype(np.int8)
#Define gaussian blur noisy function
def noisy(image):
row,col,ch= image.shape
mean = 0
var = gaussian_variance
sigma = np.sqrt(var)
gauss = np.random.normal(mean,sigma,(row,col,ch))
gauss = gauss.reshape(row,col,ch)
#Sums up gaussian noise to img
noisy = image + gauss
return noisy.astype(np.uint8)
#Blurs the masked segmentation
img_masked_noisy = noisy(img_masked)
#Puts the blurred part back in the original image as substitution
img[mask_3d] = img_masked_noisy[mask_3d]
return img
And here is a toy example:
import numpy as np
possible_classes = [1,2,3]
#Setting up a toy example with a small image,
#shape (N, N, 3)
img = np.floor(np.random.random(size=(8,8,3)) * 256).astype(np.uint8)
#Setting up a fake segmentation with 3 mutually exclusive possible classes,
#shape (N, N)
segmentation = np.random.choice(possible_classes, size=(8,8))
new_img_blurred = gaussian_blur(img,
segmentation= segmentation,
classes_of_interest= possible_classes[:2])

Writing robust (size invariant) circle detection (Watershed)

Edit: Quick Summary so far: I use the watershed algorithm but I have probably a problem with threshold. It didn't detect the brighter circles.
New: Fast radial symmetry transform approach which didn't quite work eiter (Edit 6).
I want to detect circles with different sizes. The use case is to detect coins on an image and to extract them solely. -> Get the single coins as single image files.
For this I used the Hough Circle Transform of open-cv:
(https://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/hough_circle/hough_circle.html)
import sys
import cv2 as cv
import numpy as np
def main(argv):
## [load]
default_file = "data/newcommon_1euro.jpg"
filename = argv[0] if len(argv) > 0 else default_file
# Loads an image
src = cv.imread(filename, cv.IMREAD_COLOR)
# Check if image is loaded fine
if src is None:
print ('Error opening image!')
print ('Usage: hough_circle.py [image_name -- default ' + default_file + '] \n')
return -1
## [load]
## [convert_to_gray]
# Convert it to gray
gray = cv.cvtColor(src, cv.COLOR_BGR2GRAY)
## [convert_to_gray]
## [reduce_noise]
# Reduce the noise to avoid false circle detection
gray = cv.medianBlur(gray, 5)
## [reduce_noise]
## [houghcircles]
rows = gray.shape[0]
circles = cv.HoughCircles(gray, cv.HOUGH_GRADIENT, 1, rows / 8,
param1=100, param2=30,
minRadius=0, maxRadius=120)
## [houghcircles]
## [draw]
if circles is not None:
circles = np.uint16(np.around(circles))
for i in circles[0, :]:
center = (i[0], i[1])
# circle center
cv.circle(src, center, 1, (0, 100, 100), 3)
# circle outline
radius = i[2]
cv.circle(src, center, radius, (255, 0, 255), 3)
## [draw]
## [display]
cv.imshow("detected circles", src)
cv.waitKey(0)
## [display]
return 0
if __name__ == "__main__":
main(sys.argv[1:])
I tried all parameters (rows, param1, param2, minRadius, and maxRadius) to optimize the results. This worked very well for one specific image but other images with different sized coins didn't work.
Examples:
Parameters
circles = cv.HoughCircles(gray, cv.HOUGH_GRADIENT, 1, rows / 16,
param1=100, param2=30,
minRadius=0, maxRadius=120)
With the same parameters:
Changed to rows/8
I also tried two other approaches of this thread: writing robust (color and size invariant) circle detection with opencv (based on Hough transform or other features)
The approach of fireant leads to this result:
The approach of fraxel didn't work either.
For the first approach: This happens with all different sizes and also the min and max radius.
How could I change the code, so that the coin size is not important or that it finds the parameters itself?
Thank you in advance for any help!
Edit:
I tried the watershed algorithm of Open-cv, as suggested by Alexander Reynolds: https://docs.opencv.org/3.4/d3/db4/tutorial_py_watershed.html
import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt
img = cv.imread('data/P1190263.jpg')
gray = cv.cvtColor(img,cv.COLOR_BGR2GRAY)
ret, thresh = cv.threshold(gray,0,255,cv.THRESH_BINARY_INV+cv.THRESH_OTSU)
# noise removal
kernel = np.ones((3,3),np.uint8)
opening = cv.morphologyEx(thresh,cv.MORPH_OPEN,kernel, iterations = 2)
# sure background area
sure_bg = cv.dilate(opening,kernel,iterations=3)
# Finding sure foreground area
dist_transform = cv.distanceTransform(opening,cv.DIST_L2,5)
ret, sure_fg = cv.threshold(dist_transform,0.7*dist_transform.max(),255,0)
# Finding unknown region
sure_fg = np.uint8(sure_fg)
unknown = cv.subtract(sure_bg,sure_fg)
# Marker labelling
ret, markers = cv.connectedComponents(sure_fg)
# Add one to all labels so that sure background is not 0, but 1
markers = markers+1
# Now, mark the region of unknown with zero
markers[unknown==255] = 0
markers = cv.watershed(img,markers)
img[markers == -1] = [255,0,0]
#Display:
cv.imshow("detected circles", img)
cv.waitKey(0)
It works very well on the test image of the open-cv website:
But it performs very bad on my own images:
I can't really think of a good reason why it's not working on my images?
Edit 2:
As suggested I looked at the intermediate images. The thresh looks not good in my opinion. Next, there is no difference between opening and dist_transform. The corresponding sure_fg shows the detected images.
thresh:
opening:
dist_transform:
sure_bg:
sure_fg:
Edit 3:
I tried all distanceTypes and maskSizes I could find, but the results were quite the same (https://www.tutorialspoint.com/opencv/opencv_distance_transformation.htm)
Edit 4:
Furthermore, I tried to change the (first) threshold function. I used different threshold values instead of the OTSU Function. The best one was with 160, but it was far from good:
In the tutorial it looks like this:
It seems like the coins are somehow too bright to be detected by this algorithm, but I don't know how to improve it?
Edit 5:
Changing the overall contrast and brightness of the image (with cv.convertScaleAbs) didn't improve the results. Increasing the contrast however should increase the "difference" between foreground and background, at least on the normal image. But it even got worse. The corresponding threshold image didn't improved (didn't get more white pixel).
Edit 6: I tried another approach, the fast radial symmetry transform (from here https://github.com/ceilab/frst_python)
import cv2
import numpy as np
def gradx(img):
img = img.astype('int')
rows, cols = img.shape
# Use hstack to add back in the columns that were dropped as zeros
return np.hstack((np.zeros((rows, 1)), (img[:, 2:] - img[:, :-2]) / 2.0, np.zeros((rows, 1))))
def grady(img):
img = img.astype('int')
rows, cols = img.shape
# Use vstack to add back the rows that were dropped as zeros
return np.vstack((np.zeros((1, cols)), (img[2:, :] - img[:-2, :]) / 2.0, np.zeros((1, cols))))
# Performs fast radial symmetry transform
# img: input image, grayscale
# radii: integer value for radius size in pixels (n in the original paper); also used to size gaussian kernel
# alpha: Strictness of symmetry transform (higher=more strict; 2 is good place to start)
# beta: gradient threshold parameter, float in [0,1]
# stdFactor: Standard deviation factor for gaussian kernel
# mode: BRIGHT, DARK, or BOTH
def frst(img, radii, alpha, beta, stdFactor, mode='BOTH'):
mode = mode.upper()
assert mode in ['BRIGHT', 'DARK', 'BOTH']
dark = (mode == 'DARK' or mode == 'BOTH')
bright = (mode == 'BRIGHT' or mode == 'BOTH')
workingDims = tuple((e + 2 * radii) for e in img.shape)
# Set up output and M and O working matrices
output = np.zeros(img.shape, np.uint8)
O_n = np.zeros(workingDims, np.int16)
M_n = np.zeros(workingDims, np.int16)
# Calculate gradients
gx = gradx(img)
gy = grady(img)
# Find gradient vector magnitude
gnorms = np.sqrt(np.add(np.multiply(gx, gx), np.multiply(gy, gy)))
# Use beta to set threshold - speeds up transform significantly
gthresh = np.amax(gnorms) * beta
# Find x/y distance to affected pixels
gpx = np.multiply(np.divide(gx, gnorms, out=np.zeros(gx.shape), where=gnorms != 0),
radii).round().astype(int);
gpy = np.multiply(np.divide(gy, gnorms, out=np.zeros(gy.shape), where=gnorms != 0),
radii).round().astype(int);
# Iterate over all pixels (w/ gradient above threshold)
for coords, gnorm in np.ndenumerate(gnorms):
if gnorm > gthresh:
i, j = coords
# Positively affected pixel
if bright:
ppve = (i + gpx[i, j], j + gpy[i, j])
O_n[ppve] += 1
M_n[ppve] += gnorm
# Negatively affected pixel
if dark:
pnve = (i - gpx[i, j], j - gpy[i, j])
O_n[pnve] -= 1
M_n[pnve] -= gnorm
# Abs and normalize O matrix
O_n = np.abs(O_n)
O_n = O_n / float(np.amax(O_n))
# Normalize M matrix
M_max = float(np.amax(np.abs(M_n)))
M_n = M_n / M_max
# Elementwise multiplication
F_n = np.multiply(np.power(O_n, alpha), M_n)
# Gaussian blur
kSize = int(np.ceil(radii / 2))
kSize = kSize + 1 if kSize % 2 == 0 else kSize
S = cv2.GaussianBlur(F_n, (kSize, kSize), int(radii * stdFactor))
return S
img = cv2.imread('data/P1190263.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
result = frst(gray, 60, 2, 0, 1, mode='BOTH')
cv2.imshow("detected circles", result)
cv2.waitKey(0)
I only get this nearly black output (it has some very dark grey shadows). I don't know what to change and would be thankful for help!

Masking a grayscale Image with a 2D Gaussian Mask fails

This seems fairly simple, yet I am not getting the desired result. Can someone please explain to me why? I have the code below to generate a 2D Gaussian mask with the mean at the center and sigma as 1/3 of the image height for a 32x32 pixel image as follows:
def gauss2D(image):
x,y = image.shape[:2]
shape = (x,y)
sigma = 1/3 * min(x,y)
m,n = [(ss-1.)/2. for ss in shape]
y,x = np.ogrid[-m:m+1,-n:n+1]
h = np.exp( -(x*x + y*y) / (2.*sigma*sigma) )
h[ h < np.finfo(h.dtype).eps*h.max() ] = 0
h = h / h.max()
return h
The following is the masking image I want to use :
The code I use to mask the image is as follows:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
msk = gauss2D(gray)
masked_data = gray * msk
And the resulting image is this:
A blank white image, sometimes a few pieces in the corners show through for some different images.
I also tried a bitwise AND to apply the mask but I keep getting this error that I can't seem to fix:
res = cv2.bitwise_and(gray, gray, mask = msk)
cv2.error: OpenCV(3.4.1) /Users/travis/build/skvark/opencv-python/opencv/modules/core/src/arithm.cpp:241: error: (-215) (mtype == 0 || mtype == 1) && _mask.sameSize(*psrc1) in function binary_op

Try this, It works for me.
import numpy as np
import scipy.stats as st
def gkern(kernlen=21, nsig=3):
"""Returns a 2D Gaussian kernel array."""
interval = (2*nsig+1.)/(kernlen)
x = np.linspace(-nsig-interval/2., nsig+interval/2., kernlen+1)
kern1d = np.diff(st.norm.cdf(x))
kernel_raw = np.sqrt(np.outer(kern1d, kern1d))
kernel = kernel_raw/kernel_raw.sum()
return kernel
Input :
import matplotlib.pyplot as plt
plt.imshow(gkern(21), interpolation='none')