I want to detect the center of a cross. But since the two rectangles are connected, I don't know how to find it. I have these images for example:
Cross 1
Cross 2
I would like to find the "red dot".
The idea is that the point where a vertical and horizontal line touch is the intersection. A potential approach is:
Obtain binary image. Load image, convert to grayscale, Gaussian blur, then Otsu's threshold.
Obtain horizontal and vertical line masks. Create horizontal and vertical structuring elements with cv2.getStructuringElement then perform cv2.morphologyEx to isolate the lines.
Find joints. We cv2.bitwise_and the two masks together to get the joints.
Find centroid on joint mask. We find contours then calculate the centroid to get the intersection point.
Input image -> Horizontal mask -> Vertical mask -> Joints
Detected intersection in green
Results for the other image
Input image -> Horizontal mask -> Vertical mask -> Joints
Detected intersection in green
Code
import cv2
import numpy as np
# Load image, grayscale, Gaussian blur, Otsus threshold
image = cv2.imread('4.PNG')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Find horizonal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (150,5))
horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
# Find vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,150))
vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
# Find joints
joints = cv2.bitwise_and(horizontal, vertical)
# Find centroid of the joints
cnts = cv2.findContours(joints, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
# Find centroid and draw center point
M = cv2.moments(c)
cx = int(M['m10']/M['m00'])
cy = int(M['m01']/M['m00'])
cv2.circle(image, (cx, cy), 15, (36,255,12), -1)
cv2.imshow('horizontal', horizontal)
cv2.imshow('vertical', vertical)
cv2.imshow('joints', joints)
cv2.imshow('image', image)
cv2.waitKey()
Here's a possible solution. It is based on my answer here: How can i get the inner contour points without redundancy in OpenCV - Python. The main idea is to convolve the image with a special kernel that identifies intersections. After this operation, you create a mask with possible intersection points, apply some morphology and get the coordinates.
You did not provide your input image, I'm testing this algorithm with the "cross" image you posted. This is the code:
# Imports:
import cv2
import numpy as np
# Image path
path = "D://opencvImages//"
fileName = "cross.png" # Your "cross" image
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Prepare a deep copy of the input for results:
inputImageCopy = inputImage.copy()
# Grayscale conversion:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
Now, the convolution must receive an image where the shapes have been reduced to a 1 pixel width. This can be done computing the skeleton of the image. The skeleton is a version of the binary image where lines have been normalized to have a width of 1 pixel. We can then convolve the image with a 3 x 3 kernel and look for specific pixel patterns.
Before computing the skeleton, we will add a border around the image. This prevents some artifacts that the skeleton yields if a shape extends all the way to the borders of the image:
# Add borders to prevent skeleton artifacts:
borderThickness = 1
borderColor = (0, 0, 0)
grayscaleImage = cv2.copyMakeBorder(grayscaleImage, borderThickness, borderThickness, borderThickness, borderThickness,
cv2.BORDER_CONSTANT, None, borderColor)
# Compute the skeleton:
skeleton = cv2.ximgproc.thinning(grayscaleImage, None, 1)
This is the skeleton, free of artifacts:
Now, let's find the intersections. The approach is based on Mark Setchell's info on this post. The post mainly shows the method for finding end-points of a shape, but I extended it to also identify line intersections. The main idea is that the convolution yields a very specific value where patterns of black and white pixels are found in the input image. Refer to the post for the theory behind this idea, but here, we are looking for a value of 130:
# Threshold the image so that white pixels get a value of 10 and
# black pixels a value of 0:
_, binaryImage = cv2.threshold(skeleton, 128, 10, cv2.THRESH_BINARY)
# Set the intersections kernel:
h = np.array([[1, 1, 1],
[1, 10, 1],
[1, 1, 1]])
# Convolve the image with the kernel:
imgFiltered = cv2.filter2D(binaryImage, -1, h)
# Prepare the final mask of points:
(height, width) = binaryImage.shape
pointsMask = np.zeros((height, width, 1), np.uint8)
# Perform convolution and create points mask:
thresh = 130
# Locate the threshold in the filtered image:
pointsMask = np.where(imgFiltered == thresh, 255, 0)
# Convert and shape the image to a uint8 height x width x channels
# numpy array:
pointsMask = pointsMask.astype(np.uint8)
pointsMask = pointsMask.reshape(height, width, 1)
This is the pointsMask image:
I we apply some morphology we can join individual pixels into blobs. Here, a dilation will do:
# Set kernel (structuring element) size:
kernelSize = 7
# Set operation iterations:
opIterations = 3
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform Dilate:
pointsMask = cv2.morphologyEx(pointsMask, cv2.MORPH_DILATE, morphKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
This is the result of applying the dilation:
Now, we can find the coordinates of the white pixels and compute their mean values (or centroids):
# Get the coordinates of the end-points:
(Y, X) = np.where(pointsMask == 255)
# Get the centroid:
y = int(np.mean(Y))
x = int(np.mean(X))
Let's draw a circle using these coordinates on the original image:
# Draw the intersection point:
# Set circle color:
color = (0, 0, 255)
# Draw Circle
cv2.circle(inputImageCopy, (x, y), 3, color, -1)
# Show Image
cv2.imshow("Intersections", inputImageCopy)
cv2.waitKey(0)
This is the final result:
Related
I am trying to make a perspective transformation with OpenCV in python. I want to align the image and find the coordinates of top left top right and bottom left and bottom right contours. here is my code so far where I can identify all the contours.
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(3,3),0)
edges = cv2.Canny(blur,50,100)
contours, hierarchy = cv2.findContours(edges,cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) #find contours
cv2.drawContours(img,contours,-1,(0,255,0),2)
cv2.imshow('Contours',img)
cv2.waitKey(0)
Here is an image of what I get.
I'd be very thankful if you could try to help me solve this problem!
Original Image
Demo Inputs:
Demo Inputs
Demo Input 1:
Demo Input 1
Demo Input 2:
Demo Input 2
Demo Input 3:
Demo Input 3
Demo Input 4:
Demo Input 4
Desired Output:
Desired Output
Here is one way to do that in Python/OpenCV.
Read the input
Read the template (for its dimension)
Convert the input to gray and threshold
Pad the threshold (to preserve the corners when apply morphology)
Apply morphology close
Remove the padding
Get the largest external contour
Get its perimeter and approximate to 4 corners as the input corners for the warping
Get the output corners for the warping from the dimensions of the template
Get the perspective transformation matrix
Warp the input to match the template
Save the results
Input:
Template:
import cv2
import numpy as np
# read image
img = cv2.imread("omr_test.jpg")
hh, ww = img.shape[:2]
# read template
template = cv2.imread("omr_template.jpg")
ht, wd = template.shape[:2]
# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# do otsu threshold on gray image
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
# pad thresh with black to preserve corners when apply morphology
pad = cv2.copyMakeBorder(thresh, 20, 20, 20, 20, borderType=cv2.BORDER_CONSTANT, value=0)
# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15,15))
morph = cv2.morphologyEx(pad, cv2.MORPH_CLOSE, kernel)
# remove padding
morph = morph[20:hh+20, 20:ww+20]
# get largest external contour
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)
# get perimeter and approximate a polygon
peri = cv2.arcLength(big_contour, True)
corners = cv2.approxPolyDP(big_contour, 0.04 * peri, True)
# draw polygon on input image from detected corners
polygon = img.copy()
cv2.polylines(polygon, [corners], True, (0,255,0), 2, cv2.LINE_AA)
# Alternate: cv2.drawContours(page,[corners],0,(0,0,255),1)
# print the number of found corners and the corner coordinates
# They seem to be listed counter-clockwise from the top most corner
print(len(corners))
print(corners)
# reformat input corners to x,y list
icorners = []
for corner in corners:
pt = [ corner[0][0],corner[0][1] ]
icorners.append(pt)
icorners = np.float32(icorners)
# get corresponding output corners form width and height
ocorners = [ [0,0], [0,ht], [wd,ht], [wd,0] ]
ocorners = np.float32(ocorners)
# get perspective tranformation matrix
M = cv2.getPerspectiveTransform(icorners, ocorners)
# do perspective
warped = cv2.warpPerspective(img, M, (wd, ht))
# write results
cv2.imwrite("omr_test_thresh.jpg", thresh)
cv2.imwrite("omr_test_morph.jpg", morph)
cv2.imwrite("omr_test_polygon.jpg", polygon)
cv2.imwrite("omr_test_warped.jpg", warped)
# display it
cv2.imshow("thresh", thresh)
cv2.imshow("pad", pad)
cv2.imshow("morph", morph)
cv2.imshow("polygon", polygon)
cv2.imshow("warped", warped)
cv2.waitKey(0)
Threshold Image:
Morphology Image:
Polygon Image:
Warped Input:
Here is a correction that solves whether the input image is rotated cw or ccw in Python/OpenCV.
Read the input
Read the template (for its dimensions)
Convert the input to gray and threshold
Pad the threshold (to preserve the corners when applying morphology)
Apply morphology close
Remove the padding
Get the largest external contour
Get its perimeter and approximate to 4 corners as the input corners for the warping
Sort the corners by Y and test the first two sorted corners for X. If diff=X2-X1 is negative, then the output corners need to be list in a slightly differently order.
Get the output corners for the warping from the dimensions of the template and the diff of the sorted corners.
Get the perspective transformation matrix
Warp the input to match the template
Save the results
Input 1 (cw rotated):
:
Input 2 (ccw rotated):
import cv2
import numpy as np
# read image
#img = cv2.imread("omr_test.jpg")
img = cv2.imread("omr_test2.jpg")
hh, ww = img.shape[:2]
# read template
template = cv2.imread("omr_template.jpg")
ht, wd = template.shape[:2]
# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# do otsu threshold on gray image
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
# pad thresh with black to preserve corners when apply morphology
pad = cv2.copyMakeBorder(thresh, 20, 20, 20, 20, borderType=cv2.BORDER_CONSTANT, value=0)
# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15,15))
morph = cv2.morphologyEx(pad, cv2.MORPH_CLOSE, kernel)
# remove padding
morph = morph[20:hh+20, 20:ww+20]
# get largest external contour
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)
# get perimeter and approximate a polygon
peri = cv2.arcLength(big_contour, True)
corners = cv2.approxPolyDP(big_contour, 0.04 * peri, True)
# draw polygon on input image from detected corners
polygon = img.copy()
cv2.polylines(polygon, [corners], True, (0,255,0), 2, cv2.LINE_AA)
# print the number of found corners and the corner coordinates
# They seem to be listed counter-clockwise from the top most corner
print(len(corners))
print(corners)
# reformat input corners to x,y list
sortcorners = []
for corner in corners:
pt = [ corner[0][0],corner[0][1] ]
sortcorners.append(pt)
icorners = np.float32(sortcorners)
# sort corners on y
def takeSecond(elem):
return elem[1]
sortcorners.sort(key=takeSecond)
# check if second corner x is left or right of first corner x
x1 = sortcorners[0][0]
x2 = sortcorners[1][0]
diff = x2 - x1
print(x1, x2)
# get corresponding output corners from width and height
if diff >= 0:
ocorners = [ [0,0], [0,ht], [wd,ht], [wd,0] ]
else:
ocorners = [ [wd,0], [0,0], [0,ht], [wd,ht]]
ocorners = np.float32(ocorners)
# get perspective tranformation matrix
M = cv2.getPerspectiveTransform(icorners, ocorners)
# do perspective
warped = cv2.warpPerspective(img, M, (wd, ht))
# write results
cv2.imwrite("omr_test2_thresh.jpg", thresh)
cv2.imwrite("omr_test2_morph.jpg", morph)
cv2.imwrite("omr_test2_polygon.jpg", polygon)
cv2.imwrite("omr_test2_warped.jpg", warped)
# display it
cv2.imshow("thresh", thresh)
cv2.imshow("pad", pad)
cv2.imshow("morph", morph)
cv2.imshow("polygon", polygon)
cv2.imshow("warped", warped)
cv2.waitKey(0)
Result for first input (cw rotated):
Result for second input (ccw rotated):
I want to retrieve all contours of the image below, but ignore text.
Image:
When I try to find the contours of the current image I get the following:
I have no idea how to go about this as I am new to using OpenCV and image processing. I want to get ignore the text, how can I achieve this? If ignoring is not possible but making a single bounding box surrounding the text is, than that would be good too.
Edit:
Criteria that I need to match:
The contours may very in size and shape.
The colors from the image may differ.
The colors and size of the text inside the image may differ.
Here is one way to do that in Python/OpenCV.
Read the input
Convert to grayscale
Get Canny edges
Apply morphology close to ensure they are closed
Get all contour hierarchy
Filter contours to keep only those above threshold in perimeter
Draw contours on input
Draw each contour on a black background
Save results
Input:
import numpy as np
import cv2
# read input
img = cv2.imread('short_title.png')
# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# get canny edges
edges = cv2.Canny(gray, 1, 50)
# apply morphology close to ensure they are closed
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
edges = cv2.morphologyEx(edges, cv2.MORPH_CLOSE, kernel)
# get contours
contours = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
contours = contours[0] if len(contours) == 2 else contours[1]
# filter contours to keep only large ones
result = img.copy()
i = 1
for c in contours:
perimeter = cv2.arcLength(c, True)
if perimeter > 500:
cv2.drawContours(result, c, -1, (0,0,255), 1)
contour_img = np.zeros_like(img, dtype=np.uint8)
cv2.drawContours(contour_img, c, -1, (0,0,255), 1)
cv2.imwrite("short_title_contour_{0}.jpg".format(i),contour_img)
i = i + 1
# save results
cv2.imwrite("short_title_gray.jpg", gray)
cv2.imwrite("short_title_edges.jpg", edges)
cv2.imwrite("short_title_contours.jpg", result)
# show images
cv2.imshow("gray", gray)
cv2.imshow("edges", edges)
cv2.imshow("result", result)
cv2.waitKey(0)
Grayscale:
Edges:
All contours on input:
Contour 1:
Contour 2:
Contour 3:
Contour 4:
Here are two options for erasing the text:
Using pytesseract OCR.
Finding white (and small) connected components.
Both solution build a mask, dilate the mask and use cv2.inpaint for erasing the text.
Using pytesseract:
Find text boxes using pytesseract.image_to_boxes.
Fill the boxes in the mask with 255.
Code sample:
import cv2
import numpy as np
from pytesseract import pytesseract, Output
# Tesseract path
pytesseract.tesseract_cmd = "C:\\Program Files\\Tesseract-OCR\\tesseract.exe"
img = cv2.imread('ShortAndInteresting.png')
# https://stackoverflow.com/questions/20831612/getting-the-bounding-box-of-the-recognized-words-using-python-tesseract
boxes = pytesseract.image_to_boxes(img, lang='eng', config=' --psm 6') # Run tesseract, returning the bounding boxes
h, w, _ = img.shape # assumes color image
mask = np.zeros((h, w), np.uint8)
# Fill the bounding boxes on the image
for b in boxes.splitlines():
b = b.split(' ')
mask = cv2.rectangle(mask, (int(b[1]), h - int(b[2])), (int(b[3]), h - int(b[4])), 255, -1)
mask = cv2.dilate(mask, np.ones((5, 5), np.uint8)) # Dilate the boxes in the mask
clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS) # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).
# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()
Mask:
Finding white (and small) connected components:
Use mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255)) for finding the text (assume the text is white).
Finding connected components in the mask using cv2.connectedComponentsWithStats(mask, 4)
Remove large components from the mask - fill components with large area with zeros.
Code sample:
import cv2
import numpy as np
img = cv2.imread('ShortAndInteresting.png')
mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255))
nlabel, labels, stats, centroids = cv2.connectedComponentsWithStats(mask, 4) # Finding connected components with statistics
# Remove large components from the mask (fill components with large area with zeros).
for i in range(1, nlabel):
area = stats[i, cv2.CC_STAT_AREA] # Get area
if area > 1000:
mask[labels == i] = 0 # Remove large connected components from the mask (fill with zero)
mask = cv2.dilate(mask, np.ones((5, 5), np.uint8)) # Dilate the text in the maks
cv2.imwrite('mask2.png', mask)
clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS) # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).
# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()
Mask:
Clean image:
Note:
My assumption is that you know how to split the image into contours, and the only issue is the present of the text.
I would recommend using flood fill, find the seed point for each color region, flood fill it to ignore the text values within. Hope that helps!
Refer to example of using floodfill here: https://www.programcreek.com/python/example/89425/cv2.floodFill
Example below copied from link above
def fillhole(input_image):
'''
input gray binary image get the filled image by floodfill method
Note: only holes surrounded in the connected regions will be filled.
:param input_image:
:return:
'''
im_flood_fill = input_image.copy()
h, w = input_image.shape[:2]
mask = np.zeros((h + 2, w + 2), np.uint8)
im_flood_fill = im_flood_fill.astype("uint8")
cv.floodFill(im_flood_fill, mask, (0, 0), 255)
im_flood_fill_inv = cv.bitwise_not(im_flood_fill)
img_out = input_image | im_flood_fill_inv
return img_out
I have a irregular shape like below:
I need to get the centre of that white area, I just tried with contour in openCV like below
img=cv.imread('mypic',0)
ret,thresh = cv.threshold(img,127,255,cv.THRESH_BINARY_INV)
cnts = cv.findContours(thresh.copy(), cv.RETR_EXTERNAL,
cv.CHAIN_APPROX_SIMPLE)
cnt = contours[0]
x,y,w,h = cv.boundingRect(cnt)
res_img = cv.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
cv.imwrite('output.png',res_img)
But those cnts doeesn't give me very good results, as you can see the original image and its two small black point below the picture. Can someone point me to a good solution to get a centre of an irregular shape like above?
As I suggested, perform an erosion followed by dilation (an opening operation) on the binary image, then compute central moments and use this information to calculate the centroid. These are the steps:
Get a binary image from the input via Otsu's Thresholding
Compute central moments using cv2.moments
Compute the blob's centroid using the previous information
Let's see the code:
import cv2
import numpy as np
# Set image path
path = "C:/opencvImages/"
fileName = "pn43H.png"
# Read Input image
inputImage = cv2.imread(path+fileName)
# Convert BGR to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu + bias adjustment:
threshValue, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
This is the binary image you get. Notice the small noise:
The opening operation will get rid of the smalls blobs. A rectangular structuring element will suffice, let's use 3 iterations:
# Apply an erosion + dilation to get rid of small noise:
# Set kernel (structuring element) size:
kernelSize = 3
# Set operation iterations:
opIterations = 3
# Get the structuring element:
maxKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform closing:
openingImage = cv2.morphologyEx(binaryImage, cv2.MORPH_OPEN, maxKernel, None, None, opIterations, cv2.BORDER_REFLECT101)
This is the filtered image:
Now, compute the central moments and then the blob's centroid:
# Calculate the moments
imageMoments = cv2.moments(openingImage)
# Compute centroid
cx = int(imageMoments['m10']/imageMoments['m00'])
cy = int(imageMoments['m01']/imageMoments['m00'])
# Print the point:
print("Cx: "+str(cx))
print("Cy: "+str(cy))
Additionally, let's draw this point onto the binary image to check out the results:
# Draw centroid onto BGR image:
bgrImage = cv2.cvtColor(binaryImage, cv2.COLOR_GRAY2BGR)
bgrImage = cv2.line(bgrImage, (cx,cy), (cx,cy), (0,255,0), 10)
This is the result:
One can think of the centroid calculated using the image moments as the "mass" center of the object in relation to the pixel intensity. Depending on the actual shape of the object it may not even be inside the object.
An alternative would be calculating the center of the bounding circle:
thresh = cv2.morphologyEx(thresh, cv2.MORPH_DILATE, np.ones((3, 3)))
contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = [c for c in contours if cv2.contourArea(c) > 100]
(x, y), r = cv2.minEnclosingCircle(contours[0])
output = thresh.copy()
cv2.circle(output, (int(x), int(y)), 3, (0, 0, 0), -1)
cv2.putText(output, f"{(int(x), int(y))}", (int(x-50), int(y-10)), cv2.FONT_HERSHEY_PLAIN, 1, (0, 0, 0), 1)
cv2.circle(output, (int(x), int(y)), int(r), (255, 0, 0), 2)
The output of that code looks like this:
You may want to try ConnectedComponentsWithStats function. This returns centroids, areas and bounding box parameters. Also blur and morphology(dilate/erode) helps a lot with noice, as noted above. If you're generous enough with erode, you`re gonna get almost no stray pixels after processing.
I have two intersecting ellipses in a black and white image. I am trying to use OpenCV findContours to identify the separate shapes as separate contours using this code (and attached image below).
import numpy as np
import matplotlib.pyplot as plt
import cv2
import skimage.morphology
img_3d = cv2.imread("C:/temp/test_annotation_overlap.png")
img_grey = cv2.cvtColor(img_3d, cv2.COLOR_BGR2GRAY)
contours = cv2.findContours(img_grey, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)[-2]
fig, ax = plt.subplots(len(contours)+1,1, figsize=(5, 20))
thicker_img_grey = skimage.morphology.dilation(img_grey, skimage.morphology.disk(radius=3))
ax[0].set_title("ORIGINAL IMAGE")
ax[0].imshow(thicker_img_grey, cmap="Greys")
for i, contour in enumerate(contours):
new_img = np.zeros_like(img_grey)
cv2.drawContours(new_img, contour, -1, (255,255,255), 10)
ax[i+1].set_title(f"Contour {i}")
ax[i+1].imshow(new_img, cmap="Greys")
plt.show()
However four contours are found, none of which are the original contour:
How can I configure OpenCV.findContours to identify the two separate shapes? (Note I have already played around with Hough circles and found it unreliable for the images I am analysing)
Maybe I overkilled with this approach but it could be used as a working approach. You could find all the contours on the image - you will get the two contours that are like a "semicircle", the contour of the intersection and the contour that is the outer shape of the two addjointed circles. Smallest three contours should be the two semicircles and the intersection. If you draw combinations of two out of these three contours, you will get three mask out of which two will have the combination of one semicircle and the intersection. If you perform closing on the mask you will get your circle. Then you should simply make an algorithm to detect which two masks represent a full circle and you will get your result. Here is the sample solution:
import numpy as np
import cv2
# Function for returning solidity of contour - ratio of contour area to its
# convex hull area.
def checkSolidity(cnt):
area = cv2.contourArea(cnt)
hull = cv2.convexHull(cnt)
hull_area = cv2.contourArea(hull)
solidity = float(area)/hull_area
return solidity
img_orig = cv2.imread("circles.png")
# Had to dilate the image so the contour was completly connected.
img = cv2.dilate(img_orig, np.ones((3, 3), np.uint8))
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Grayscale transformation.
# Otsu threshold.
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)[1]
# Search for contours.
contours = cv2.findContours(thresh, cv2.CHAIN_APPROX_NONE, cv2.RETR_TREE)[0]
# Sorting contours from smallest to biggest.
contours.sort(key=lambda cnt: cv2.contourArea(cnt))
# Three contours - two semi circles and the intersection of the circles.
cnt1 = contours[0]
cnt2 = contours[1]
cnt3 = contours[2]
# Create three empty images
h, w = img.shape[:2]
mask1 = np.zeros((h, w), np.uint8)
mask2 = np.zeros((h, w), np.uint8)
mask3 = np.zeros((h, w), np.uint8)
# Draw all combinations of two out of three contours on the masks.
# The goal here is to draw one semicircle and the intersection together.
cv2.drawContours(mask1, [cnt1], 0, (255, 255, 255), -1)
cv2.drawContours(mask1, [cnt3], 0, (255, 255, 255), -1)
cv2.drawContours(mask2, [cnt2], 0, (255, 255, 255), -1)
cv2.drawContours(mask2, [cnt3], 0, (255, 255, 255), -1)
cv2.drawContours(mask3, [cnt1], 0, (255, 255, 255), -1)
cv2.drawContours(mask3, [cnt2], 0, (255, 255, 255), -1)
# Perform closing operation on the masks so that you get uniform contours.
kernel_size = 25
kernel = np.ones((kernel_size, kernel_size), np.uint8)
mask1 = cv2.morphologyEx(mask1, cv2.MORPH_CLOSE, kernel)
mask2 = cv2.morphologyEx(mask2, cv2.MORPH_CLOSE, kernel)
mask3 = cv2.morphologyEx(mask3, cv2.MORPH_CLOSE, kernel)
masks = [] # List for storing all the masks.
masks.append(mask1)
masks.append(mask2)
masks.append(mask3)
# List where you will append solidity of the found biggest contour of every mask.
solidity = []
for mask in masks:
cnts = cv2.findContours(mask, cv2.CHAIN_APPROX_NONE, cv2.RETR_TREE)[0]
cnt = max(cnts, key=lambda c: cv2.contourArea(c))
s = checkSolidity(cnt)
solidity.append(s)
# Index of the mask with smallest solidity.
min_solidity = solidity.index(min(solidity))
# The mask with the contour that has smallest solidity should be the one that
# has two semicirles drawn instead of one semicircle and the intersection.
#You could build a better function to check which mask is the one with
# two semicircles... like maybe the contour with the largest
# height and width of the bounding box etc.
# I chose solidity because it is enough for this example.
# Selection of colors.
colors = {
0: (0, 0, 255),
1: (0, 255, 0),
2: (255, 0, 0),
}
# Draw contours of the mask other two masks - those two that have the
# semicircle and the intersection.
for i, s in enumerate(solidity):
if s != solidity[min_solidity]:
cnts = cv2.findContours(
masks[i], cv2.CHAIN_APPROX_NONE, cv2.RETR_TREE)[0]
cnt = max(cnts, key=lambda c: cv2.contourArea(c))
cv2.drawContours(img_orig, [cnt], 0, colors[i], 1)
# Display result
cv2.imshow("img", img_orig)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
Philosophically, you want to find two circles, because you search for them, you expect centers and radii. Graphically the figures are connected, we can see them separated because we know what a "circle" is and extrapolate the coordinates, which match the parts which overlap.
So what about finding the minimum enclosing circle for each contour (or in some cases fitEllipse and use the their parameters): https://docs.opencv.org/master/dd/d49/tutorial_py_contour_features.html
Then say draw that circle in a clear image and take the pixel coordinates which are not zero - by a mask or compute them by drawing a circle step by step.
Then compare these coordinates with the coordinates in the other contours with some precision and append the matching coordinates to the current contour.
Finally: draw the extended contour on a clear canvas and apply HoughCircles for a single non-overlapping circle. (Or compute the center and radius, the coordinates of a circle and comparing to the contour with a precision.)
For reference here I will post the solution I came up with based on some ideas here and a few more. This solution was 99.9% effective and recovering ellipses from images often including many overlapping, contained, and with other image noise such as lines, text and so on.
The code is too length and distributed to post here but the pseudocode is as follows.
Segment the image
Run cv2 findContours with RETR_EXTERNAL to get separate regions in the image
For each image, fill in the interior, apply mask, and extract the region to be processed independently of other regions.
Remaining steps are executed for each region independently
Run cv2 findContours with RETR_LIST to get all internal and external contours
For each contour found, apply polygon smoothing to reduce effect of pixellation
For each smoothed contour, identify the continuous segments within that contour which have the same curvature sign i.e. segments which are entirely curving right, or curving left (just compute angles and sign changes)
Within each segment, fit an ellipse model with least squares (scikit-learn EllipseModel)
Perform the Lee algorithm on the original image to compute for each pixel its minimum distance to a white pixel
For each model, perform a greedy local neighbourhood search to improve the fit against the original - the fit being the fitted ellipse maximum distance to white pixel, from the output of the lee algorithm
Not simple or elegant but is highly accurate for the content I am dealing with confirmed from manual review of a large number of images.
How would I go around identifying and extracting rectangles from an image such as the one shown below.
Note that my rectangles might be incomplete and have some missing edges and some sides might be partial lines.
Thanks !
This can be solved using morphological operations such as eroding and dilating. This two operations will help creating closed rectangles.
After that you can use the tutorial from this page to detect simple shapes such as rectangles.
I implemented a quick demo which worked for the image you provided.
main.py:
import cv2
import numpy as np
from shapeDetector import ShapeDetector
import imutils
img = cv2.imread('t.png')
kernel = np.ones((5,5),np.uint8)
erosion = cv2.erode(img,kernel,iterations = 10)
dilate = cv2.dilate(erosion,kernel,iterations = 10)
The erosion thickens all the lines, so to get back to a normal width we need to dilate after eroding. I recommend to comment the dilate operation once to see how erode works and vice versa.
This operations will transform your image like this
The detection algorithm I used expects white lines on black background.
Thats why we need to invert the image.
cv2.bitwise_not ( dilate, dilate )
After that, we can use the code from the tutorial.
image = dilate
resized = imutils.resize(image, width=300)
ratio = image.shape[0] / float(resized.shape[0])
# convert the resized image to grayscale, blur it slightly,
# and threshold it
gray = cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
thresh = cv2.threshold(blurred, 60, 255, cv2.THRESH_BINARY)[1]
#thresh = dilate
# find contours in the thresholded image and initialize the
# shape detector
cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if imutils.is_cv2() else cnts[1]
sd = ShapeDetector()
# loop over the contours
for c in cnts:
# compute the center of the contour, then detect the name of the
# shape using only the contour
M = cv2.moments(c)
cX = int((M["m10"] / M["m00"]) * ratio)
cY = int((M["m01"] / M["m00"]) * ratio)
shape = sd.detect(c)
# multiply the contour (x, y)-coordinates by the resize ratio,
# then draw the contours and the name of the shape on the image
c = c.astype("float")
c *= ratio
c = c.astype("int")
cv2.drawContours(image, [c], -1, (0, 255, 0), 2)
cv2.putText(image, shape, (cX, cY), cv2.FONT_HERSHEY_SIMPLEX,
0.5, (255, 255, 255), 2)
# show the output image
cv2.imshow("Image", image)
cv2.waitKey(0)
shapeDetector.py:
# import the necessary packages
import cv2
class ShapeDetector:
def __init__(self):
pass
def detect(self, c):
# initialize the shape name and approximate the contour
shape = "unidentified"
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.04 * peri, True)
# if the shape is a triangle, it will have 3 vertices
if len(approx) == 3:
shape = "triangle"
# if the shape has 4 vertices, it is either a square or
# a rectangle
elif len(approx) == 4:
# compute the bounding box of the contour and use the
# bounding box to compute the aspect ratio
(x, y, w, h) = cv2.boundingRect(approx)
ar = w / float(h)
# a square will have an aspect ratio that is approximately
# equal to one, otherwise, the shape is a rectangle
shape = "square" if ar >= 0.95 and ar <= 1.05 else "rectangle"
# if the shape is a pentagon, it will have 5 vertices
elif len(approx) == 5:
shape = "pentagon"
# otherwise, we assume the shape is a circle
else:
shape = "circle"
# return the name of the shape
return shape
Result:
I would recommend taking a Hough transform to identify angles of interest, then identify pairs of angles with large mass in Hough space differing by exactly 90 degrees, then loop over each such pair, identify significantly represented spatial offsets for all such "vertical" and "horizontal" line candidates, and use some heuristics from there to identify likely rectangles from these orthongonal line candidates.
For example, after identifying a set of lines for a given angle pair, you likely have a relatively small number of lines to evaluate, so you can do something more like a brute force O(N^2) search through pairs of lines to look for relevant corners (defined as being "empty" on one side and "dense" on the other in both directions), and then match the corners up to make a list of rectangles.