Develop Reference

Flood fill function not producing good results

I applied the floodfill function in opencv to extract the foreground from the background but some of the objects in the image were not recognized by the algorithm so I would like to know how I can improve my detections and what modifications are necessary.
image = cv2.imread(args["image"])
image = cv2.resize(image, (800, 800))
h,w,chn = image.shape
ratio = image.shape[0] / 800.0
orig = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (5, 5), 0)
edged = cv2.Canny(gray, 75, 200)
# show the original image and the edge detected image
print("STEP 1: Edge Detection")
cv2.imshow("Image", image)
cv2.imshow("Edged", edged)
warped1 = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
T = threshold_local(warped1, 11, offset = 10, method = "gaussian")
warped1 = (warped1 > T).astype("uint8") * 255
print("STEP 3: Apply perspective transform")
seed = (10, 10)
foreground, birdEye = floodFillCustom(image, seed)
cv2.circle(birdEye, seed, 50, (0, 255, 0), -1)
cv2.imshow("originalImg", birdEye)
cv2.circle(birdEye, seed, 100, (0, 255, 0), -1)
cv2.imshow("foreground", foreground)
cv2.imshow("birdEye", birdEye)
gray = cv2.cvtColor(foreground, cv2.COLOR_BGR2GRAY)
cv2.imshow("gray", gray)
cv2.imwrite("gray.jpg", gray)
threshImg = cv2.threshold(gray, 1, 255, cv2.THRESH_BINARY)[1]
h_threshold,w_threshold = threshImg.shape
area = h_threshold*w_threshold
cv2.imshow("threshImg", threshImg)[![enter image description here][1]][1]
The floodFillCustom function is as follows -
def floodFillCustom(originalImage, seed):
originalImage = np.maximum(originalImage, 10)
foreground = originalImage.copy()
cv2.floodFill(foreground, None, seed, (0, 0, 0),
loDiff=(10, 10, 10), upDiff=(10, 10, 10))
return [foreground, originalImage]

A little bit late, but here's an alternative solution for segmenting the tools. It involves converting the image to the CMYK color space and extracting the K (Key) component. This component can be thresholded to get a nice binary mask of the tools, the procedure is very straightforward:
Convert the image to the CMYK color space
Extract the K (Key) component
Threshold the image via Otsu's thresholding
Apply some morphology (a closing) to clean up the mask
(Optional) Get bounding rectangles of all the tools
Let's see the code:
# Imports
import cv2
import numpy as np
# Read image
imagePath = "C://opencvImages//"
inputImage = cv2.imread(imagePath+"DAxhk.jpg")
# Create deep copy for results:
inputImageCopy = inputImage.copy()
# Convert to float and divide by 255:
imgFloat = inputImage.astype(np.float) / 255.
# Calculate channel K:
kChannel = 1 - np.max(imgFloat, axis=2)
# Convert back to uint 8:
kChannel = (255*kChannel).astype(np.uint8)
The first step is to convert the BGR image to CMYK. There's no direct conversion in OpenCV for this, so I applied directly the conversion formula. We can get every color space component from that formula, but we are only interested on the K channel. The conversion is easy, but we need to be careful with the data types. We need to operate on float arrays. After getting the K channel, we convert back the image to an unsigned 8-bit array, this is the resulting image:
Let's threshold this image using Otsu's thresholding method:
# Threshold via Otsu:
_, binaryImage = cv2.threshold(kChannel, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
This yields the following binary image:
Looks very nice! Additionally, we can clean it up a little bit (joining the little gaps) using a morphological closing. Let's apply a rectangular structuring element of size 5 x 5 and use 2 iterations:
# Use a little bit of morphology to clean the mask:
# Set kernel (structuring element) size:
kernelSize = 5
# Set morph operation iterations:
opIterations = 2
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform closing:
binaryImage = cv2.morphologyEx(binaryImage, cv2.MORPH_CLOSE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
Which results in this:
Very cool. What follows is optional. We can get the bounding rectangles for every tool by looking for the outer (external) contours:
# Find the contours on the binary image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# Look for the outer bounding boxes (no children):
for _, c in enumerate(contours):
# Get the contours bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the dimensions of the bounding rectangle:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Set bounding rectangle:
color = (0, 0, 255)
cv2.rectangle( inputImageCopy, (int(rectX), int(rectY)),
(int(rectX + rectWidth), int(rectY + rectHeight)), color, 5 )
cv2.imshow("Bounding Rectangles", inputImageCopy)
cv2.waitKey(0)
Which produces the final image:

noise reduction for character recognition improvements?

I'm using a KNN to detect characters, however, it is sensitive to background noise. the image is basically what I'm using and I have developed a mini script to try get the best threshold image. would anyone have any suggestions/ changed to get better results? make the X more viewable. ( attached version of current output and what the input is)
import cv2
import numpy as np
image = cv2.imread("resize.png")
img = image
img = cv2.blur(img, (5, 5))
imgGray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # get grayscale image
imgBlurred = cv2.GaussianBlur(imgGray, (11, 11), 5) # blur
# cv2.imshow("test",imgBlurred)
imgThresh = cv2.adaptiveThreshold(imgBlurred, # input image
255, # make pixels that pass the threshold full white
cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
# use gaussian rather than mean, seems to give better results
cv2.THRESH_BINARY_INV,
# invert so foreground will be white, background will be black
13, # size of a pixel neighborhood used to calculate threshold value
2) # constant subtracted from the mean or weighted mean
imgThreshCopy = imgThresh.copy() # make a copy of the thresh image, this in necessary b/c findContours modifies the image
kernel = np.ones((3, 3), np.uint8)
kernel2 = np.ones((7, 7), np.uint8)
kernel3 = np.ones((1, 1), np.uint8)
imgThreshCopy = cv2.morphologyEx(imgThreshCopy, cv2.MORPH_OPEN, kernel)
imgThreshCopy = cv2.morphologyEx(imgThreshCopy, cv2.MORPH_CLOSE, kernel2)
imgThreshCopy = cv2.dilate(imgThreshCopy, kernel3, iterations=150)
res = imgThreshCopy
cv2.imwrite("test.jpg", res)
cv2.imshow("image", res)
cv2.waitKey(0)
output
input

How can I remove internal noises of gear completely in this image

gear
I want generalize method so that any type of noises inside the gear can be remove. I am using OpenCV with python
I have already try with lots filter and noise removing methods but I am not getting proper output. here is my code
import cv2
import numpy as np
import imutils
from imutils import perspective
from scipy.spatial import distance as dist
img1 = cv2.imread("5cam.png")
img = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
rows, cols = img.shape
dst = cv2.fastNlMeansDenoising(img, 15, 10, 7, 21)
gaussian_blurred_images = cv2.GaussianBlur(dst, (9, 9), 0)
_, thresh = cv2.threshold(gaussian_blurred_images, 200, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
kernel = np.ones((7, 7), np.uint8)
dilation = cv2.dilate(thresh, kernel)
canny = cv2.Canny(dilation, 200, 255)
contours = cv2.findContours(canny, mode=cv2.RETR_EXTERNAL, method=cv2.CHAIN_APPROX_NONE)[0]
areas1 = []
for ctr in contours:
areas = cv2.contourArea(ctr)
areas1.append(areas)
amax = max(areas1)
max_contour = [contours[areas1.index(amax)]]
cv2.drawContours(img1, max_contour, -1, (0, 255, 255), 2)
cv2.imshow("g", dst)
cv2.imshow("thresh", thresh)
cv2.imshow("c", canny)
cv2.imshow("img", img1)
cv2.waitKey(0)
cv2.destroyAllWindows()

first you'll need to improve the lighting and scene.
it needs to be more diffuse and not straight on, to prevent reflection on the gear. place lights to the side all around. don't use the camera's built in flash. use/build a "softbox" which is a white sheet of paper or fabric that diffuses the light before it hits the object (either translucent or used like a "mirror").
it needs to be more uniform. your picture shows a "vignette", darkening near the picture's outside. the previous step will probably fix that.
be careful about smudges, dirt on the background
move the camera further away and use zoom if possible. that will improve the overall sharpness of the picture (more depth of focus) and reduce lens distortion (if you care about that).
then you'll need a different approach. I would suggest trying segmentation based on hue and saturation (select the uniformly blue background).
use cv.cvtColor to transform the image into the HSV color space
then use numpy indexing/masking (or cv.inRange) to select a small range of hue (somewhere around green-blue, which is probably a hue of around 180 degrees, or 90 in cvtColor's CV_8U hue values) and saturations (medium to full). for example: mask = ((hsv_img >= (90, 170, 0)) & (hsv_img <= (100, 255, 255))).all(axis=2)
that approach, on the unimproved lighting, gets me this far. on better lighting it should be even better.

How do i ignore specific shapes on image processing, OpenCV, Python

I am working on cell segmentation and tracking. I've set of microscopical images. There are some circular noises caused by lamella. When I'm using my algorithm that may cause loss of cells some parts. I want to say to my program, "hey look those circular things are just noise, and just deny it, and work on real cell's membrane." The other one is, micro noises. There are some points with high or low contrast. I want to say to my program, "Hey, deny points, if its 10x10 pixels radius are the same with backgrounds contrast."
Work platform: Python 3.7.2, OpenCV 3.4.5
I hope, i clearly mentioned what my problem is. I am sharing one of those images.
4 circles on left are point noises.
2 circles on right are lamella noises.
enter image description here
import numpy
import cv2 as cv
from matplotlib import pyplot as plt
img = cv.imread('test001.tif')
gg = img.copy()
img_gray = cv.cvtColor(gg, cv.COLOR_BGR2GRAY)
clache = cv.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
img_gray = clache.apply(img_gray)
_, img_bin = cv.threshold(img_gray, 50, 255,
cv.THRESH_OTSU)
img_bin = cv.morphologyEx(img_bin, cv.MORPH_OPEN,
numpy.ones((10, 9), dtype=int))
img_bin = cv.morphologyEx(img_bin, cv.MORPH_DILATE,
numpy.ones((5, 5), dtype=int), iterations= 1)
def segment(im1, img):
#morphological transformations
border = cv.dilate(img, None, iterations=10)
border = border - cv.erode(border, None, iterations=1)
#invert the image so black becomes white, and vice versa
img = -img
#applies distance transform and shows visualization
dt = cv.distanceTransform(img, 2, 3)
dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(numpy.uint8)
#reapply contrast to strengthen boundaries
clache = cv.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
dt = clache.apply(dt)
#rethreshold the image
_, dt = cv.threshold(dt, 127, 255, cv.THRESH_BINARY)
ret, markers = cv.connectedComponents(dt)
markers = markers+1
# Complete the markers
markers[border == 255] = 255
markers = markers.astype(numpy.int32)
#apply watershed
cv.watershed(im1, markers)
markers[markers == -1] = 0
markers = markers.astype(numpy.uint8)
#return the image as one list, and the labels as another.
return dt, markers
dt, result = segment(img, img_bin)
cv.imshow('img',img)
cv.imshow('dt',dt)
cv.imshow('img_bin',img_bin)
cv.imshow('res',result)
Below one is serving as a guinea pig.
import numpy
import cv2 as cv
from matplotlib import pyplot as plt
img = cv.imread('test001.tif')
gg = img.copy()
img_gray = cv.cvtColor(gg, cv.COLOR_BGR2GRAY)
clache = cv.createCLAHE(clipLimit=2.0, tileGridSize=(20,20))
img_gray = clache.apply(img_gray)
cv.imshow('1img',img)
cv.imshow('2gray',img_gray)
#Threshold
_, img_bin = cv.threshold(img_gray, 127, 255, cv.THRESH_BINARY+cv.THRESH_OTSU)
cv.imshow('3threshold',img_bin)
#MorpClose
img_bin = cv.morphologyEx(img_bin, cv.MORPH_CLOSE,numpy.ones((5,5), dtype=int))
cv.imshow('4morp_close',img_bin)
#MorpErosion
img_bin = cv.erode(img_bin,numpy.ones((3,3),dtype=int),iterations = 1)
cv.imshow('5erosion',img_bin)
#MorpOpen
img_bin = cv.morphologyEx(img_bin, cv.MORPH_OPEN, numpy.ones((2, 2), dtype=int))
#cv.imshow('6morp_open',img_bin)
#MorpDilate
img_bin = cv.morphologyEx(img_bin, cv.MORPH_DILATE,numpy.ones((1, 1), dtype=int), iterations= 1)
#cv.imshow('7morp_dilate',img_bin)
#MorpBlackHat
img_bin = cv.morphologyEx(img_bin, cv.MORPH_BLACKHAT,numpy.ones((4,4),dtype=int))
#cv.imshow('8morpTophat',img_bin)

For those small dots you can try eroding and dilating:
You need to convert the image to grayscale and then process it, I'd create a mask with the eroded and dilated parts to remove those dots and then use that mask on the original image to delete the dots without compromising the resolution of you initial image.
For the big blury blobs, maybe add some noise to the image and compare with the original?
If most of those blobs are cv2.HoughCircles described here, it does something like this:
Of course you can tune those parameters to fit what you want and just ignore those parts of the image. Try that and also the noise, that might help reduce the false positives.
Best of luck!

Camera calibration, reverse projection of pixel to direction

I am using OpenCV to estimate a webcam's intrinsic matrix from a series of chessboard images - as detailed in this tutorial, and reverse project a pixel to a direction (in term of azimuth/elevation angles).
The final goal is to let the user select a point on the image, estimate the direction of this point in relation to the center of the webcam, and use this as DOA for a beam-forming algorithm.
So once I have estimated the intrinsic matrix, I reverse project the user-selected pixel (see code below) and display it as azimuth/elevation angles.
result = [0, 0, 0] # reverse projected point, in homogeneous coord.
while 1:
_, img = cap.read()
if flag: # If the user has clicked somewhere
result = np.dot(np.linalg.inv(mtx), [mouse_x, mouse_y, 1])
result = np.arctan(result) # convert to angle
flag = False
cv2.putText(img, '({},{})'.format(mouse_x, mouse_y), (20, 440), cv2.FONT_HERSHEY_SIMPLEX,
0.5, (0, 255, 0), 2, cv2.LINE_AA)
cv2.putText(img, '({:.2f},{:.2f})'.format(180/np.pi*result[0], 180/np.pi*result[1]), (20, 460),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2, cv2.LINE_AA)
cv2.imshow('image', img)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
My problem is that I'm not sure whether my results are coherent. The major incoherence is that, the point of the image corresponding to the {0,0} angle is noticeably off the image center, as seen below (camera image has been replaced by a black background for privacy reasons) :
I don't really see a simple yet efficient way of measuring the accuracy (the only method I could think of was to use a servo motor with a laser on it, just under the camera and point it to the computed direction).
Here is the intrinsic matrix after calibration with 15 images :
I get an error of around 0.44 RMS which seems satisfying.
My calibration code :
nCalFrames = 12 # number of frames for calibration
nFrames = 0
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001) # termination criteria
objp = np.zeros((9*7, 3), np.float32)
objp[:, :2] = np.mgrid[0:9, 0:7].T.reshape(-1, 2)
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
cap = cv2.VideoCapture(0)
previousTime = 0
gray = 0
while 1:
# Capture frame-by-frame
_, img = cap.read()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, corners = cv2.findChessboardCorners(gray, (9, 7), None)
# If found, add object points, image points (after refining them)
if ret:
corners2 = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
if time.time() - previousTime > 2:
previousTime = time.time()
imgpoints.append(corners2)
objpoints.append(objp)
img = cv2.bitwise_not(img)
nFrames = nFrames + 1
# Draw and display the corners
img = cv2.drawChessboardCorners(img, (9, 7), corners, ret)
cv2.putText(img, '{}/{}'.format(nFrames, nCalFrames), (20, 460), cv2.FONT_HERSHEY_SIMPLEX,
2, (0, 255, 0), 2, cv2.LINE_AA)
cv2.putText(img, 'press \'q\' to exit...', (255, 15), cv2.FONT_HERSHEY_SIMPLEX,
0.5, (0, 0, 255), 1, cv2.LINE_AA)
# Display the resulting frame
cv2.imshow('Webcam Calibration', img)
if nFrames == nCalFrames:
break
if cv2.waitKey(1) & 0xFF == ord('q'):
break
RMS_error, mtx, disto_coef, _, _ = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)
EDIT: another test method would be to use a whiteboard with known angles points and estimate the error by comparing with experimental results, but I don't know how to set up such a system

Regarding your first concern, it is normal to have the principal point off the image center. The estimated point, which is the point of zero elevation and azimuth, is the one that minimizes the radial distortion coefficients, and for a low value wide angle lens (e.g., that of a typical webcam) it can be easily off by noticeable amount.
Your calibration should be ok up to the call to calibrateCamera. However, in your code snippet it seems your ignoring the distortion coefficients. What is missing is initUndistortRectifyMap, which lets you also re-center the principal point if that matters.
h, w = img.shape[:2]
# compute new camera matrix with central principal point
new_mtx,roi = cv2.getOptimalNewCameraMatrix(mtx,disto_coef,(w,h),1,(w,h))
print(new_mtx)
# compute undistort maps
mapx,mapy = cv2.initUndistortRectifyMap(mtx,disto_coef,None,new_mtx,(w,h),5)
It essentially makes focal length equal in both dimensions and centers the principal point (see OpenCV python documentation for parameters).
Then, at each
_, img = cap.read()
you must undistort the image before rendering
# apply the remap
img = cv2.remap(img,mapx,mapy,cv2.INTER_LINEAR)
# crop the image
x,y,w,h = roi
img = img[y:y+h, x:x+w]
here, I put background to green to emphasize the barrel distortion. The output could be something like this (camera image replaced by checkerboard for privacy reasons):
If you do all these, your calibration target is accurate and your calibration samples fill the entire image area you should be quite confident of the computation. However, to validate the measured azimuth and elevation with respect to the undistorted image's pixel readings, I'd maybe suggest tape measure from the lenses first principal point and a calibration plate placed in normal angle right in front of the camera. There you can compute the expected angles and compare.
Hope this helps.