I would like to detect circular erosions and dilations on a line. For dilations, I tried to recursively erode the image and on every recursion, I check width/height aspect ratio. If the ratio is smaller than 4, I assume that it the contour is circular and for each such contour I calculate circle center and radius from moments and area. This is the function that detects circular dilations:
def detect_circular_dilations(img, contours):
contours_current, hierarchy = cv2.findContours(img, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
if len(contours_current) == 0:
return get_circles_from_contours(contours)
for c in contours_current:
x, y, w, h = cv2.boundingRect(c)
if w > h:
aspect_ratio = float(w) / h
else:
aspect_ratio = float(h) / w
if aspect_ratio < 4 and w < 20 and h < 20 and w > 5 and h > 5:
contours.append(c)
return detect_circular_dilations(cv2.erode(img, None, iterations=1), contours)
An example of circular dilations that I want to detect are the following:
Another problem that I haven't solve is the detection of circular erosions. An example of circular erosion is the following:
Here I've marked the circular erosion I would like to detect with red rectangle. There might be some smaller circular patterns (on the left) that shouldn't be treated as actual circular erosion.
Does anyone know what is the best way to detect such circular shapes? For circular dilations, I would appreciate any comment/suggestion in order to potentially make detection more robust.
Thank you!
What I would try is to find two edges of the line with cv2.Canny() and search for contours. If you sort your contour by the width of their bounding box, the first two contours will be your lines edges. After that you can calculate the minimum distance of each point in one edge to the other edge. Then you can calculate the median of the distances and say that if a point has bigger or shorter distance than the median (+- tolerance) than that point is ether the dilation or erosion of the line and append it to a list. You can sort out noises if needed by itterating through the lists and remove the points if they are not consecutive (on x axis).
Here is a simple example:
import cv2
import numpy as np
from scipy import spatial
def detect_dilation(median, mindist, tolerance):
count = 0
for i in mindist:
if i > median + tolerance:
dilate.append((reshape_e1[count][0], reshape_e1[count][1]))
elif i < median - tolerance:
erode.append((reshape_e1[count][0], reshape_e1[count][1]))
else:
pass
count+=1
def other_axis(dilate, cnt):
temp = []
for i in dilate:
temp.append(i[0])
for i in cnt:
if i[0] in temp:
dilate.append((i[0],i[1]))
img = cv2.imread('1.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray,100,200)
_, contours, hierarchy = cv2.findContours(edges,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
contours.sort(key= lambda cnt :cv2.boundingRect(cnt)[3])
edge_1 = contours[0]
edge_2 = contours[1]
reshape_e1 = np.reshape(edge_1, (-1,2))
reshape_e2 =np.reshape(edge_2, (-1,2))
tree = spatial.cKDTree(reshape_e2)
mindist, minid = tree.query(reshape_e1)
median = np.median(mindist)
dilate = []
erode = []
detect_dilation(median,mindist,5)
other_axis(dilate, reshape_e2)
other_axis(erode, reshape_e2)
dilate = np.array(dilate).reshape((-1,1,2)).astype(np.int32)
erode = np.array(erode).reshape((-1,1,2)).astype(np.int32)
x,y,w,h = cv2.boundingRect(dilate)
cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
x,y,w,h = cv2.boundingRect(erode)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,0,255),2)
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
Edit:
If the picture has a line that is broken (that means more contours) you would have to treat each contour as a seperate line. You could achieve this by making a region of interest with the help of cv2.boundingRect(). But as I tried it with the new uploaded picture the process is not very robust since you have to change the tolerance to get the desired result. Since I don't know what other images look like, you may need a better way to get the average distance and the tolerance factor. Any way here is a sample of what I described (with 15 for tolerance):
import cv2
import numpy as np
from scipy import spatial
def detect_dilation(median, mindist, tolerance):
count = 0
for i in mindist:
if i > median + tolerance:
dilate.append((reshape_e1[count][0], reshape_e1[count][1]))
elif i < median - tolerance:
erode.append((reshape_e1[count][0], reshape_e1[count][1]))
else:
pass
count+=1
def other_axis(dilate, cnt):
temp = []
for i in dilate:
temp.append(i[0])
for i in cnt:
if i[0] in temp:
dilate.append((i[0],i[1]))
img = cv2.imread('2.jpg')
gray_original = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, thresh_original = cv2.threshold(gray_original, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Filling holes
_, contours, hierarchy = cv2.findContours(thresh_original,cv2.RETR_CCOMP,cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
cv2.drawContours(thresh_original,[cnt],0,255,-1)
_, contours, hierarchy = cv2.findContours(thresh_original,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE)
for cnt in contours:
x2,y,w2,h = cv2.boundingRect(cnt)
thresh = thresh_original[0:img.shape[:2][1], x2+20:x2+w2-20] # Region of interest for every "line"
edges = cv2.Canny(thresh,100,200)
_, contours, hierarchy = cv2.findContours(edges,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
contours.sort(key= lambda cnt: cv2.boundingRect(cnt)[3])
edge_1 = contours[0]
edge_2 = contours[1]
reshape_e1 = np.reshape(edge_1, (-1,2))
reshape_e2 =np.reshape(edge_2, (-1,2))
tree = spatial.cKDTree(reshape_e2)
mindist, minid = tree.query(reshape_e1)
median = np.median(mindist)
dilate = []
erode = []
detect_dilation(median,mindist,15)
other_axis(dilate, reshape_e2)
other_axis(erode, reshape_e2)
dilate = np.array(dilate).reshape((-1,1,2)).astype(np.int32)
erode = np.array(erode).reshape((-1,1,2)).astype(np.int32)
x,y,w,h = cv2.boundingRect(dilate)
if len(dilate) > 0:
cv2.rectangle(img[0:img.shape[:2][1], x2+20:x2+w2-20],(x,y),(x+w,y+h),(255,0,0),2)
x,y,w,h = cv2.boundingRect(erode)
if len(erode) > 0:
cv2.rectangle(img[0:img.shape[:2][1], x2+20:x2+w2-20],(x,y),(x+w,y+h),(0,0,255),2)
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
Problems like this are often solved using the distance transform and the medial axis transform. These are related in a way, as the medial axis runs along the ridge of the distance transform. The general idea is:
Compute the distance transform of the image (for each foreground pixel, returns the distance to the nearest background pixel; some libraries implement this the other way, in which case you need to compute the distance transform of the inverted image).
Compute the medial axis (or skeleton).
The values of the distance transform along the medial axis are the relevant values, we ignore all other pixels. Here we see the local radius of the line.
Local maxima are centroids of the dilations. Use a threshold to determine which of these are important dilations and which ones are not (a noisy outline will cause many local maxima).
Local minima are centroids of the erosions.
For example, I got the following output using the MATLAB code below.
Here is the code I used. It uses MATLAB with DIPimage 3, just as a quick proof of principle. This should be straightforward to translate to Python with whatever image processing library you like to use.
% Read in image and remove the red markup:
img = readim('https://i.stack.imgur.com/bNOTn.jpg');
img = img{3}>100;
img = closing(img,5);
% This is the algorithm described above:
img = fillholes(img); % Get rid of holes
radius = dt(img); % Distance transform
m = bskeleton(img); % Medial axis
radius(~m) = 0; % Ignore all pixels outside the medial axis
detection = dilation(radius,25)==radius & radius>25; % Local maxima with radius > 25
pos = findcoord(detection); % Coordinates of detections
radius = double(radius(detection)); % Radii of detections
% This is just to make the markup:
detection = newim(img,'bin');
for ii=1:numel(radius)
detection = drawshape(detection,2*radius(ii),pos(ii,:),'disk');
end
overlay(img,detection)
Related
For a project I want to detect braille dots on a plate. I make a picture on which I make my detection thanks to the connectedComponentsWithStats function. Despite my attempts I can never get a threshold value where all the dots and only them are detected, I have the same problem if I try to use the circle detection. I'm trying to use template matching on the advice of a teacher but I'm also having problems with my detection since the only factor that influences it is the threshold.
import matplotlib.pyplot as plt
img1 = cv.imread(r"traitement\prod.png")
plt.figure(figsize=(40,40))
plt.subplot(3,1,1)
gray_img = cv.cvtColor(img1, cv.COLOR_BGR2GRAY)
test = cv.adaptiveThreshold(gray_img, 255, cv.ADAPTIVE_THRESH_MEAN_C, cv.THRESH_BINARY_INV, 11, 6)
_, _, boxes, _ = cv.connectedComponentsWithStats(test)
boxes = boxes[1:]
filtered_boxes = []
for x,y,w,h,pixels in boxes:
if pixels < 1000 and h < 35 and w < 35 and h > 14 and w > 14 and x > 15 and y > 15:
filtered_boxes.append((x,y,w,h))
for x,y,w,h in filtered_boxes:
W = int(w)/2
H = int(h)/2
#print(w)
cv.circle(img1,(x+int(W),y+int(H)),2,(0,255,0),20)
cv.imwrite("gray.png",gray_img)
cv.imwrite("test.png",test)
plt.imshow(test)
plt.subplot(3,1,2)
plt.imshow(img1)
import cv2 as cv
import numpy as np
from imutils.object_detection import non_max_suppression
import matplotlib.pyplot as plt
img = cv.imread('traitement/prod.png')
temp_gray = cv.imread('dot.png',0)
W, H = temp.shape[:2]
thresh = 0.6
img_gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
match = cv.matchTemplate(image=img_gray, templ=temp_gray, method=cv.TM_CCOEFF_NORMED)
(y_points, x_points) = np.where(match >= thresh)
boxes = list()
for (x, y) in zip(x_points, y_points):
# update our list of rectangles
boxes.append((x, y, x + W, y + H))
boxes = non_max_suppression(np.array(boxes))
# loop over the final bounding boxes
for (x1, y1, x2, y2) in boxes:
cv.circle(img,(x1+int(W/2),y1+int(H/2)),2,(255,0,0),15)
plt.figure(figsize=(40,40))
plt.subplot(3,1,1)
plt.imshow(img)
Image with adaptive threshold:
Image with template detection:
I found a solution that may not be better than your solutions, because I had to overfit few parameters for the given input...
The problem is challenging because the input image was taken under non-uniform illumination conditions (the center part is brighter than the top). Consider taking a better snapshot...
Point of thought:
The dots are ordered in rows, and we are not using that information.
We may get better results if we were using the fact that the dots are ordered in rows.
For overcoming the brightness differences we may subtract the median of the surrounding pixels from each pixel (using large filter radius), and compute the absolute difference:
bg = cv2.medianBlur(gray, 151) # Background
fg = cv2.absdiff(gray, bg) # Foreground (use absdiff because the dost are dark but bright at the center).
Apply binary threshold (use THRESH_OTSU for automatic threshold level):
_, thresh = cv2.threshold(fg, 0, 255, cv2.THRESH_OTSU)
The result of thresh is not good enough for finding the dots.
We may use the fact that the dots are dark with bright center.
That fact makes an high edges around and inside the dots.
Apply Canny edge detection:
edges = cv2.Canny(gray, threshold1=50, threshold2=100)
Merge edges with thresh (use binary or):
thresh = cv2.bitwise_or(thresh, edges)
Find connected components and continue (filter the components by area).
Code sample:
import numpy as np
import cv2
img1 = cv2.imread('prod.jpg')
gray = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY) # Convert to grayscale
bg = cv2.medianBlur(gray, 151) # Compute the background (use a large filter radius for excluding the dots)
fg = cv2.absdiff(gray, bg) # Compute absolute difference
_, thresh = cv2.threshold(fg, 0, 255, cv2.THRESH_OTSU) # Apply binary threshold (THRESH_OTSU applies automatic threshold level)
edges = cv2.Canny(gray, threshold1=50, threshold2=100) # Apply Canny edge detection.
thresh = cv2.bitwise_or(thresh, edges) # Merge edges with thresh
_, _, boxes, _ = cv2.connectedComponentsWithStats(thresh)
boxes = boxes[1:]
filtered_boxes = []
for x, y, w, h, pixels in boxes:
#if pixels < 1000 and h < 35 and w < 35 and h > 14 and w > 14 and x > 15 and y > 15 and pixels > 100:
if pixels < 1000 and x > 15 and y > 15 and pixels > 200:
filtered_boxes.append((x, y, w, h))
for x, y, w, h in filtered_boxes:
W = int(w)/2
H = int(h)/2
cv2.circle(img1, (x+int(W), y+int(H)), 2, (0, 255, 0), 20)
# Show images for testing
cv2.imshow('bg', bg)
cv2.imshow('fg', fg)
cv2.imshow('gray', gray)
cv2.imshow('edges', edges)
cv2.imshow('thresh', thresh)
cv2.imshow('img1', img1)
cv2.waitKey()
cv2.destroyAllWindows()
Result:
There are few dots that are marked twice.
It is relatively simple to merge the overlapping circles into one circle.
Intermediate results:
thresh (before merging with edges):
edges:
thresh merged with edges:
Update:
As Jeru Luke commented we may use non-maximum suppression as done in question.
Here is a code sample:
import numpy as np
import cv2
from imutils.object_detection import non_max_suppression
img1 = cv2.imread('prod.jpg')
gray = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY) # Convert to grayscale
bg = cv2.medianBlur(gray, 151) # Compute the background (use a large filter radius for excluding the dots)
fg = cv2.absdiff(gray, bg) # Compute absolute difference
_, thresh = cv2.threshold(fg, 0, 255, cv2.THRESH_OTSU) # Apply binary threshold (THRESH_OTSU applies automatic threshold level)
edges = cv2.Canny(gray, threshold1=50, threshold2=100) # Apply Canny edge detection.
thresh = cv2.bitwise_or(thresh, edges) # Merge edges with thresh
_, _, boxes, _ = cv2.connectedComponentsWithStats(thresh)
boxes = boxes[1:]
filtered_boxes = []
for x, y, w, h, pixels in boxes:
if pixels < 1000 and x > 15 and y > 15 and pixels > 200:
filtered_boxes.append((x, y, x+w, y+h))
filtered_boxes = non_max_suppression(np.array(filtered_boxes), overlapThresh=0.2)
for x1, y1, x2, y2 in filtered_boxes:
cv2.circle(img1, ((x1+x2)//2, (y1+y2)//2), 2, (0, 255, 0), 20)
Result:
Approach: template matching.
Because the appearance of these dots can't be caught by just thresholding on brightness. These dots have both brighter and darker pixels than the flat surface. At best you'd get fractured components and given how close these dots are, any morphology operations to fix up the fractured components would run the risk of joining adjacent dots.
So here I do template matching. Works well enough, even though the appearance of these dots changes across the image. Brightness is uneven but that's not too much of a problem. TM_CCOEFF subtracts the mean for both patches before correlating.
imutils requires you to come up with bounding boxes for its NMS. Instead I'm using a simple and effective NMS formulation for raster data. Comparing floats for equality is okay here because dilate simply replicates the maximum value across the kernel size. It uses L-inf distance; for euclidean distance or an approximation, the kernel needs to be round. One downside: if there are multiple peaks of equal value, this will not remove either of them. To catch adjacent equal peaks, connectedComponents would work instead of findNonZero. For non-adjacent but equal peaks... let's just assume that doesn't happen because the data would make that situation impossible.
# Non-maximum suppression for raster data
def non_maximum_suppression(im, radius):
dilated = cv.dilate(im, kernel=None, iterations=radius)
return (im == dilated)
# Read image
im = cv.imread("fK2WOX.jpeg", cv.IMREAD_GRAYSCALE)
# Select template
# (x,y,w,h) = (880, 247, 44, 44)
(x,y,w,h) = cv.selectROI("ROI", 255-im, showCrosshair=False, fromCenter=True) # inverted to see the white rectangle...
cv.destroyWindow("ROI")
print((x,y,w,h))
template = im[y:y+h, x:x+w]
# find instances
scores = cv.matchTemplate(im, template, method=cv.TM_CCOEFF)
scores = scores / scores.max()
You see that the template creates additional peaks to the top and bottom of each dot. That can't be helped because that's the appearance of these dots, they consist of a bright dash surrounded by two darker blobs. These false peaks can be suppressed with NMS quite easily.
# find peaks of sufficient strength and NMS
threshold = 0.2
nmsradius = 20 # proportional to size of template/dot
nmsmask = (scores >= threshold) & non_maximum_suppression(scores, radius=nmsradius)
coords = cv.findNonZero(nmsmask.astype(np.uint8)).reshape((-1, 2))
coords += (w//2, h//2) # shift coordinates to be center of template
# draw result
canvas = cv.cvtColor(im, cv.COLOR_GRAY2BGR)
for pt in coords:
cv.circle(canvas, pt, radius=5, color=(0,0,255), thickness=cv.FILLED)
There are no overlapping/split detections. You'll have to excuse the false detections around the punched holes near the top. Just crop those out.
As for associating those dots into symbols, that's a new problem. I'd recommend using nearest-neighbor queries. When the spacing (in N/S/E/W direction) to other dots has been estimated, you can "probe" in those directions going from each dot and check if there is another dot there... and assemble symbols like that, marking dots as "associated" until there's nothing left. You would also want to associate symbols into strings, assuming a certain spacing between symbols. same approach there... and that gives you the added information of a baseline for the "text".
I'm following a simple approach to segment cells (microscopy images) using the Watershed algorithm in Python. I'm happy with the result 90% of the time, but I have two main problems: (i) the markers/contours are really "spiky" and (2) the algorithm sometimes fails when two cells are to close to each other (i.e they are segmented together). Can you give some tips in how to improve it?
Here's the code I'm using and an output image showing my 2 issues.
# Adjustable parameters for a future function
img_file = NP_file
sigma = 9 # size of gaussian blur kernel; has to be an even number
alpha = 0.2 #scalling factor distance transform
clear_border = False
remove_small_objects = True
# read image and covert to gray scale
im = cv2.imread(NP_file, 1)
im = enhanceContrast(im)
im_gray = cv2.cvtColor(im.copy(), cv2.COLOR_BGR2GRAY)
# Basic Median Filter
im_blur = cv2.medianBlur(im_gray, ksize = sigma)
# Threshold Image
th, im_seg = cv2.threshold(im_blur, im_blur.mean(), 255, cv2.THRESH_BINARY);
# filling holes in the segmented image
im_filled = binary_fill_holes(im_seg)
# discard cells touching the border
if clear_border == True:
im_filled = skimage.segmentation.clear_border(im_filled)
# filter small particles
if remove_small_objects == True:
im_filled = sk.morphology.remove_small_objects(im_filled, min_size = 5000)
# apply distance transform
# labels each pixel of the image with the distance to the nearest obstacle pixel.
# In this case, obstacle pixel is a boundary pixel in a binary image.
dist_transform = cv2.distanceTransform(img_as_ubyte(im_filled), cv2.DIST_L2, 3)
# get sure foreground area: region near to center of object
fg_val, sure_fg = cv2.threshold(dist_transform, alpha * dist_transform.max(), 255, 0)
# get sure background area: region much away from the object
sure_bg = cv2.dilate(img_as_ubyte(im_filled), np.ones((3,3),np.uint8), iterations = 6)
# The remaining regions (borders) are those which we don’t know if they are img or background
borders = cv2.subtract(sure_bg, np.uint8(sure_fg))
# use Connected Components labelling:
# scans an image and groups its pixels into components based on pixel connectivity
# label background of the image with 0 and other objects with integers starting from 1.
n_markers, markers1 = cv2.connectedComponents(np.uint8(sure_fg))
# filter small particles again! (bc of segmentation artifacts)
if remove_small_objects == True:
markers1 = sk.morphology.remove_small_objects(markers1, min_size = 1000)
# Make sure the background is 1 and not 0;
# and that borders are marked as 0
markers2 = markers1 + 1
markers2[borders == 255] = 0
# implement the watershed algorithm: connects markers with original image
# The label image will be modified and the marker in the border area will change to -1
im_out = im.copy()
markers3 = cv2.watershed(im_out, markers2)
# generate an extra image with color labels only for visuzalization
# color markers in BLUE (pixels = -1 after watershed algorithm)
im_out[markers3 == -1] = [0, 255, 255]
in case you want to try to reproduce my results you can find my .tif file here:
https://drive.google.com/file/d/13KfyUVyHodtEOP_yKAnfFCAhgyoY0BQL/view?usp=sharing
Thanks!
In the past, the best approach for me to apply the watershed algorithm is 'only when needed'. It is computationally intensive and not needed for the majority of cells in your image.
This is the code I have used with your image:
# Threshold your image
# This example worked very well with a threshold value of 1
tv, thresh = cv2.threshold(cv2.cvtColor(img, cv2.COLOR_BGR2GRAY), 1, 255, cv2.THRESH_BINARY)
# Minimize the holes in the cells to facilitate finding contours
for i in range(5):
thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, np.ones((3,3)))
thresh = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, np.ones((3,3)))
# Find contours and keep the ones big enough to be a cell
contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = [c for c in contours if cv2.contourArea(c) > 400]
output = np.zeros_like(thresh)
cv2.drawContours(output, contours, -1, 255, -1)
for i, contour in enumerate(contours):
x, y, w, h = cv2.boundingRect(contour)
cv2.putText(output, f"{i}", (x, y), cv2.FONT_HERSHEY_PLAIN, 1, 255, 2)
The output of this code is this image:
As you can see, only a pair of cells (contour #7) needs splitting using watershed algorithm.
Running the watershed algorithm on that cell is very fast (smaller image to work with) and this is the result:
EDIT
Some of the cell morphology calculations that can be used to assess whether the watershed algorithm should be run on an object in the image:
# area
area = cv2.contourArea(contour)
# perimeter, with the minimum value = 0.01 to avoid division by zero in other calculations
perimeter = max(0.01, cv2.arcLength(contour, True))
# circularity
circularity = (4 * math.pi * area) / (perimeter ** 2)
# Check if the cell is convex (not smoothly elliptical)
hull = cv2.convexHull(contour)
convexity = cv2.arcLength(hull, True) / perimeter
approx = cv2.approxPolyDP(contour, 0.1 * perimeter, True)
convex = cv2.isContourConvex(approx)
You will need to find the thresholds for each of the measurements in your project. In my project, cells were elliptic, and having a blob with a large area and convex usually means there are 2 or more cells lump together.
I want to detect objects in an image and measure the distance between them. This works as long as the objects do not come too close. Unfortunately the lighting of the image is not optimal, so it looks like objects are touching although they are not. I am trying to determine the distance with the help of a line, which represents the object. Problem is that as soon as the object contours join, I cannot determine the lines which represent the objects, so no distance can be calculated.
Input Image:
Code:
import cv2
import numpy as np
#import image
img = cv2.imread('img.png', 0)
#Thresh
_, thresh = cv2.threshold(img, 200, 255, cv2.THRESH_BINARY)
#Finding the contours in the image
_, contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
#Convert img to RGB and draw contour
img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)
cv2.drawContours(img, contours, -1, (0,0,255), 2)
#Object1
v = np.matrix([[0], [1]])
rect = cv2.minAreaRect(contours[0])
#determine angle
if rect[1][0] > rect[1][1]:
ang = (rect[2] + 90)* np.pi / 180
else:
ang = rect[2]* np.pi / 180
rot = np.matrix([[np.cos(ang), -np.sin(ang)],[np.sin(ang), np.cos(ang)]])
rv = rot*v
#draw angle line
lineSize = max(rect[1])*0.45 #length of line
p1 = tuple(np.array(rect[0] - lineSize*rv.T)[0].astype(int))
p2 = tuple(np.array(rect[0] + lineSize*rv.T)[0].astype(int))
cv2.line(img, p1, p2, (255,0,0), 2)
#Object2
if len(contours) > 1:
rect = cv2.minAreaRect(contours[1])
#determine angle
if rect[1][0] > rect[1][1]:
ang = (rect[2] + 90)* np.pi / 180
else:
ang = rect[2]* np.pi / 180
rot = np.matrix([[np.cos(ang), -np.sin(ang)],[np.sin(ang), np.cos(ang)]])
rv = rot*v
#draw angle line
lineSize = max(rect[1])*0.45 #length of line
p1 = tuple(np.array(rect[0] - lineSize*rv.T)[0].astype(int))
p2 = tuple(np.array(rect[0] + lineSize*rv.T)[0].astype(int))
cv2.line(img, p1, p2, (255,0,0), 2)
#save output img
cv2.imwrite('output_img.png', img)
Output Image:
This works fine but as soon as I use an image with joined contours this happens:
Is there a way to divide contours or maybe a workaround?
Edit
Thanks to the suggestion of B.M. I tried if erosion is a solution but unfortunately new problems come into place. It does not seem to be possible to find a balance between erosion and thresholding/contours.
Examples:
How about if you first search for contours and check if there are in fact two. If there is only one you could make a loop to erode and search for contours on the eroded image until you get two contours. When the event happens make a black bounding box that is bigger for the amount of kernel used on the eroded image and draw in on the "original image which will physically divide and create 2 contours. Then apply your code to the resulting image. Maybe you can upload the images you have the most dificulty with before processing? Hope it helps a bit or gives you a new idea. Cheers!
Example code:
import cv2
import numpy as np
img = cv2.imread('cont.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, threshold = cv2.threshold(gray, 200, 255, cv2.THRESH_BINARY)
_, contours, hierarchy = cv2.findContours(threshold,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
k = 2
if len(contours)==1:
for i in range (0,1000):
kernel = np.ones((1,k),np.uint8)
erosion = cv2.erode(threshold,kernel,iterations = 1)
_, contours, hierarchy = cv2.findContours(erosion,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
if len(contours) == 1:
k+=1
if len(contours) == 2:
break
if len(contours) > 2:
print('more than one contour')
x,y,w,h = cv2.boundingRect(contours[0])
cv2.rectangle(threshold,(x-k,y-k),(x+w+k,y+h+k), 0, 1)
_, contours, hierarchy = cv2.findContours(threshold,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
cv2.drawContours(img, contours, -1, (0,0,255), 2)
#Object1
v = np.matrix([[0], [1]])
rect = cv2.minAreaRect(contours[0])
#determine angle
if rect[1][0] > rect[1][1]:
ang = (rect[2] + 90)* np.pi / 180
else:
ang = rect[2]* np.pi / 180
rot = np.matrix([[np.cos(ang), -np.sin(ang)],[np.sin(ang), np.cos(ang)]])
rv = rot*v
#draw angle line
lineSize = max(rect[1])*0.45 #length of line
p1 = tuple(np.array(rect[0] - lineSize*rv.T)[0].astype(int))
p2 = tuple(np.array(rect[0] + lineSize*rv.T)[0].astype(int))
cv2.line(img, p1, p2, (255,0,0), 2)
#Object2
if len(contours) > 1:
rect = cv2.minAreaRect(contours[1])
#determine angle
if rect[1][0] > rect[1][1]:
ang = (rect[2] + 90)* np.pi / 180
else:
ang = rect[2]* np.pi / 180
rot = np.matrix([[np.cos(ang), -np.sin(ang)],[np.sin(ang), np.cos(ang)]])
rv = rot*v
#draw angle line
lineSize = max(rect[1])*0.45 #length of line
p1 = tuple(np.array(rect[0] - lineSize*rv.T)[0].astype(int))
p2 = tuple(np.array(rect[0] + lineSize*rv.T)[0].astype(int))
cv2.line(img, p1, p2, (255,0,0), 2)
#save output img
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
You can use erosion techniques like cv2.erode provides. After
from cv2 import erode
import numpy as np
kernel = np.ones((5,25),dtype=np.uint8) # this must be tuned
im1=erode(im0,kernel)
You obtain a image (im0 is your second image) where bright zones are shrank :
Now you will be able to measure a distance, even if the effect of erosion must be took in account.
I think you can adopt a watershed binary segmentation approach (I would suggest ITK for this).
A combination of :
distance map calculation
Reconstruction by dilation
Regional Maxima
Morphological Watershed From Markers
will lead to following separation :
Once separation done you can :
extract contour points
compute associated polygonal approximation
compute exact distance between two particules
(geometric part could use boost geometry library for example but this is a c++ library)
You could also use a pure geometric/morphological approach :
extract contour points
compute associated polygonal approximation
compute associated voronoi diagram (which can be considered as a skeleton for the shape)
compute thickness associated to each branch of previous skeleton
make cuts at min thickness sections
Regards,
I solved the problem like this:
take length of last image's objects (or make a factor if it is your first image)
build two thresholds for every object
erase everything below/above in the threshold (with help of 1.). Basically I started counting from lower edge of object up to its length (In threshold array).
Make a split and share it for both objects as border of contour.
Find new contours.
Counting the threshold array upwards (for second object) to find lowest point looks like this:
else:
del lineSize_list[-1], ang_list[-1] #delete wrong values from Size and Angle lists
z = 0
thresh2 = np.copy(thresh)
for x in thresh2[::-1]: #check threshold backwards for positive values
for e in x:
if e > 0:
break
z += 1
if e > 0:
- first positive value found in threshold (edge of object)
- add object length to this position
- make a cut in threshold (this will be the break for new contours)
- use threshold to find contours ....
This works for measuring the distance between both objects. The Result looks similar to kavko's. I will accept his answer because he invested time.
This is a simplified clunky solution. It only works because I can cut the threshold array in two. It would be awesome if there is a more professional way to divide contours. Anyway, thanks alot.
You can apply a cluster analysis (k-means for example) at all the points of the unique contour with 2 as number of clusters.
from sklearn.cluster import KMeans
array = np.vstack(contours)
all_points = array.reshape(array.shape[0], array.shape[2])
kmeans = KMeans(n_clusters=2, random_state=0, n_init = 50).fit(all_points)
new_contours = [all_points[kmeans.labels_==i] for i in range(2)]*
if the squares has connected region in image, how can I detect them.
I have tested the method mentioned in
OpenCV C++/Obj-C: Advanced square detection
It did not work well.
Any good ideas ?
import cv2
import numpy as np
def angle_cos(p0, p1, p2):
d1, d2 = (p0-p1).astype('float'), (p2-p1).astype('float')
return abs( np.dot(d1, d2) / np.sqrt( np.dot(d1, d1)*np.dot(d2, d2) ) )
def find_squares(img):
squares = []
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# cv2.imshow("gray", gray)
gaussian = cv2.GaussianBlur(gray, (5, 5), 0)
temp,bin = cv2.threshold(gaussian, 80, 255, cv2.THRESH_BINARY)
# cv2.imshow("bin", bin)
contours, hierarchy = cv2.findContours(bin, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours( gray, contours, -1, (0, 255, 0), 3 )
#cv2.imshow('contours', gray)
for cnt in contours:
cnt_len = cv2.arcLength(cnt, True)
cnt = cv2.approxPolyDP(cnt, 0.02*cnt_len, True)
if len(cnt) == 4 and cv2.contourArea(cnt) > 1000 and cv2.isContourConvex(cnt):
cnt = cnt.reshape(-1, 2)
max_cos = np.max([angle_cos( cnt[i], cnt[(i+1) % 4], cnt[(i+2) % 4] ) for i in xrange(4)])
if max_cos < 0.1:
squares.append(cnt)
return squares
if __name__ == '__main__':
img = cv2.imread('123.bmp')
#cv2.imshow("origin", img)
squares = find_squares(img)
print "Find %d squres" % len(squares)
cv2.drawContours( img, squares, -1, (0, 255, 0), 3 )
cv2.imshow('squares', img)
cv2.waitKey()
I use some method in the opencv example, but the result is not good.
Applying a Watershed Transform based on the Distance Transform will separate the objects:
Handling objects at the border is always problematic, and often discarded, so that pink rectangle at top left not separated is not a problem at all.
Given a binary image, we can apply the Distance Transform (DT) and from it obtain markers for the Watershed. Ideally there would be a ready function for finding regional minima/maxima, but since it isn't there, we can make a decent guess on how we can threshold DT. Based on the markers we can segment using Watershed, and the problem is solved. Now you can worry about distinguishing components that are rectangles from those that are not.
import sys
import cv2
import numpy
import random
from scipy.ndimage import label
def segment_on_dt(img):
dt = cv2.distanceTransform(img, 2, 3) # L2 norm, 3x3 mask
dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(numpy.uint8)
dt = cv2.threshold(dt, 100, 255, cv2.THRESH_BINARY)[1]
lbl, ncc = label(dt)
lbl[img == 0] = lbl.max() + 1
lbl = lbl.astype(numpy.int32)
cv2.watershed(cv2.cvtColor(img, cv2.COLOR_GRAY2BGR), lbl)
lbl[lbl == -1] = 0
return lbl
img = cv2.cvtColor(cv2.imread(sys.argv[1]), cv2.COLOR_BGR2GRAY)
img = cv2.threshold(img, 0, 255, cv2.THRESH_OTSU)[1]
img = 255 - img # White: objects; Black: background
ws_result = segment_on_dt(img)
# Colorize
height, width = ws_result.shape
ws_color = numpy.zeros((height, width, 3), dtype=numpy.uint8)
lbl, ncc = label(ws_result)
for l in xrange(1, ncc + 1):
a, b = numpy.nonzero(lbl == l)
if img[a[0], b[0]] == 0: # Do not color background.
continue
rgb = [random.randint(0, 255) for _ in xrange(3)]
ws_color[lbl == l] = tuple(rgb)
cv2.imwrite(sys.argv[2], ws_color)
From the above image you can consider fitting ellipses in each component to determine rectangles. Then you can use some measurement to define whether the component is a rectangle or not. This approach has a greater chance to work for rectangles that are fully visible, and will likely produce bad results for partially visible ones. The following image shows the result of such approach considering that a component is a rectangle if the rectangle from the fitted ellipse is within 10% of component's area.
# Fit ellipse to determine the rectangles.
wsbin = numpy.zeros((height, width), dtype=numpy.uint8)
wsbin[cv2.cvtColor(ws_color, cv2.COLOR_BGR2GRAY) != 0] = 255
ws_bincolor = cv2.cvtColor(255 - wsbin, cv2.COLOR_GRAY2BGR)
lbl, ncc = label(wsbin)
for l in xrange(1, ncc + 1):
yx = numpy.dstack(numpy.nonzero(lbl == l)).astype(numpy.int64)
xy = numpy.roll(numpy.swapaxes(yx, 0, 1), 1, 2)
if len(xy) < 100: # Too small.
continue
ellipse = cv2.fitEllipse(xy)
center, axes, angle = ellipse
rect_area = axes[0] * axes[1]
if 0.9 < rect_area / float(len(xy)) < 1.1:
rect = numpy.round(numpy.float64(
cv2.cv.BoxPoints(ellipse))).astype(numpy.int64)
color = [random.randint(60, 255) for _ in xrange(3)]
cv2.drawContours(ws_bincolor, [rect], 0, color, 2)
cv2.imwrite(sys.argv[3], ws_bincolor)
Solution 1:
Dilate your image to delete connected components.
Find contours of detected components. Eliminate contours which are not rectangles by introducing some measure (ex. ratio perimeter / area).
This solution will not detect rectangles connected to borders.
Solution 2:
Dilate to delete connected components.
Find contours.
Approximate contours to reduce their points (for rectangle contour should be 4 points).
Check if angle between contour lines is 90 degrees.
Eliminate contours which have no 90 degrees.
This should solve problem with rectangles connected to borders.
You have three problems:
The rectangles are not very strict rectangles (the edges are often somewhat curved)
There are a lot of them.
They are often connected.
It seems that all your rects are essentially the same size(?), and do not greatly overlap, but the pre-processing has connected them.
For this situation the approach I would try is:
dilate your image a few times (as also suggested by #krzych) - this will remove the connections, but result in slightly smaller rects.
Use scipy to label and find_objects - You now know the position and slice for every remaining blob in the image.
Use minAreaRect to find the center, orientation, width and height of each rectangle.
You can use step 3. to test whether the blob is a valid rectangle or not, by its area, dimension ratio or proximity to the edge..
This is quite a nice approach, as we assume each blob is a rectangle, so minAreaRect will find the parameters for our minimum enclosing rectangle. Further we could test each blob using something like humoments if absolutely neccessary.
Here is what I was suggesting in action, boundary collision matches shown in red.
Code:
import numpy as np
import cv2
from cv2 import cv
import scipy
from scipy import ndimage
im_col = cv2.imread('jdjAf.jpg')
im = cv2.imread('jdjAf.jpg',cv2.CV_LOAD_IMAGE_GRAYSCALE)
im = np.where(im>100,0,255).astype(np.uint8)
im = cv2.erode(im, None,iterations=8)
im_label, num = ndimage.label(im)
for label in xrange(1, num+1):
points = np.array(np.where(im_label==label)[::-1]).T.reshape(-1,1,2).copy()
rect = cv2.minAreaRect(points)
lines = np.array(cv2.cv.BoxPoints(rect)).astype(np.int)
if any([np.any(lines[:,0]<=0), np.any(lines[:,0]>=im.shape[1]-1), np.any(lines[:,1]<=0), np.any(lines[:,1]>=im.shape[0]-1)]):
cv2.drawContours(im_col,[lines],0,(0,0,255),1)
else:
cv2.drawContours(im_col,[lines],0,(255,0,0),1)
cv2.imshow('im',im_col)
cv2.imwrite('rects.png',im_col)
cv2.waitKey()
I think the Watershed and distanceTransform approach demonstrated by #mmgp is clearly superior for segmenting the image, but this simple approach can be effective depending upon your needs.
Image I'm working with:
I'm trying to find each of the boxes in this image. The results don't have to be 100% accurate, just as long as the boxes found are approximately correct in position/size. From playing with the example for square detection, I've managed to get contours, bounding boxes, corners and the centers of boxes.
There are a few issues I'm running into here:
bounding rectangles are detected for both the inside and the outside of the drawn lines.
some extraneous corners/centers are detected.
I'm not sure how to match corners/centers with the related contours/bounding boxes, especially when taking nested boxes into account.
Image resulting from code:
Here's the code I'm using to generate the image above:
import numpy as np
import cv2
from operator import itemgetter
from glob import glob
def angle_cos(p0, p1, p2):
d1, d2 = (p0-p1).astype('float'), (p2-p1).astype('float')
return abs( np.dot(d1, d2) / np.sqrt( np.dot(d1, d1)*np.dot(d2, d2) ) )
def makebin(gray):
bin = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 5, 2)
return cv2.bitwise_not(bin)
def find_squares(img):
img = cv2.GaussianBlur(img, (11, 11), 0)
squares = []
points = []`
for gray in cv2.split(img):
bin = makebin(gray)
contours, hierarchy = cv2.findContours(bin, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
corners = cv2.goodFeaturesToTrack(gray,len(contours)*4,0.2,15)
cv2.cornerSubPix(gray,corners,(6,6),(-1,-1),(cv2.TERM_CRITERIA_MAX_ITER | cv2.TERM_CRITERIA_EPS,10, 0.1))
for cnt in contours:
cnt_len = cv2.arcLength(cnt, True)
if len(cnt) >= 4 and cv2.contourArea(cnt) > 200:
rect = cv2.boundingRect(cnt)
if rect not in squares:
squares.append(rect)
return squares, corners, contours
if __name__ == '__main__':
for fn in glob('../1 (Small).jpg'):
img = cv2.imread(fn)
squares, corners, contours = find_squares(img)
for p in corners:
cv2.circle(img, (p[0][0],p[0][3]), 3, (0,0,255),2)
squares = sorted(squares,key=itemgetter(1,0,2,3))
areas = []
moments = []
centers = []
for s in squares:
areas.append(s[2]*s[3])
cv2.rectangle( img, (s[0],s[1]),(s[0]+s[2],s[1]+s[3]),(0,255,0),1)
for c in contours:
moments.append(cv2.moments(np.array(c)))
for m in moments:
centers.append((int(m["m10"] // m["m00"]), int(m["m01"] // m["m00"])))
for cent in centers:
print cent
cv2.circle(img, (cent[0],cent[1]), 3, (0,255,0),2)
cv2.imshow('squares', img)
ch = 0xFF & cv2.waitKey()
if ch == 27:
break
cv2.destroyAllWindows()
I suggest a simpler approach as a starting point. For instance, morphological gradient can serve as a good local detector of strong edges, and threshold on it tends to be simple. Then, you can remove too small components, which is relatively easy for your problem too. In your example, each remaining connected component is a single box, so the problem is solved in this instance.
Here is what you would obtain with this simple procedure:
The red points represent the centroid of the component, so you could grow another box from there that is contained in the yellow one if the yellow ones are bad for you.
Here is the code for achieving that:
import sys
import numpy
from PIL import Image, ImageOps, ImageDraw
from scipy.ndimage import morphology, label
def boxes(orig):
img = ImageOps.grayscale(orig)
im = numpy.array(img)
# Inner morphological gradient.
im = morphology.grey_dilation(im, (3, 3)) - im
# Binarize.
mean, std = im.mean(), im.std()
t = mean + std
im[im < t] = 0
im[im >= t] = 1
# Connected components.
lbl, numcc = label(im)
# Size threshold.
min_size = 200 # pixels
box = []
for i in range(1, numcc + 1):
py, px = numpy.nonzero(lbl == i)
if len(py) < min_size:
im[lbl == i] = 0
continue
xmin, xmax, ymin, ymax = px.min(), px.max(), py.min(), py.max()
# Four corners and centroid.
box.append([
[(xmin, ymin), (xmax, ymin), (xmax, ymax), (xmin, ymax)],
(numpy.mean(px), numpy.mean(py))])
return im.astype(numpy.uint8) * 255, box
orig = Image.open(sys.argv[1])
im, box = boxes(orig)
# Boxes found.
Image.fromarray(im).save(sys.argv[2])
# Draw perfect rectangles and the component centroid.
img = Image.fromarray(im)
visual = img.convert('RGB')
draw = ImageDraw.Draw(visual)
for b, centroid in box:
draw.line(b + [b[0]], fill='yellow')
cx, cy = centroid
draw.ellipse((cx - 2, cy - 2, cx + 2, cy + 2), fill='red')
visual.save(sys.argv[3])
I see you have already got the answer. But I think there is a much more simpler,shorter and better method available in OpenCV to resolve this problem.
While finding contours, you are also finding the hierarchy of the contours. Hierarchy of the contours is the relation between different contours.
So the flag you used in your code, cv2.RETR_TREE provides all the hierarchical relationship.
cv2.RETR_LIST provides no hierarchy while cv2.RETR_EXTERNAL gives you only external contours.
The best one for you is cv2.RETR_CCOMP which provides you all the contour, and a two-level hierarchical relationship. ie outer contour is always parent and inner hole contour always is child.
Please read following article for more information on hierarchy : Contour - 5 : Hierarchy
So hierarchy of a contour is a 4 element array in which last element is the index pointer to its parent. If a contour has no parent, it is external contour and it has a value -1. If it is a inner contour, it is a child and it will have some value which points to its parent. We are going to exploit this feature in your problem.
import cv2
import numpy as np
# Normal routines
img = cv2.imread('square.JPG')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,50,255,1)
# Remove some small noise if any.
dilate = cv2.dilate(thresh,None)
erode = cv2.erode(dilate,None)
# Find contours with cv2.RETR_CCOMP
contours,hierarchy = cv2.findContours(erode,cv2.RETR_CCOMP,cv2.CHAIN_APPROX_SIMPLE)
for i,cnt in enumerate(contours):
# Check if it is an external contour and its area is more than 100
if hierarchy[0,i,3] == -1 and cv2.contourArea(cnt)>100:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
m = cv2.moments(cnt)
cx,cy = m['m10']/m['m00'],m['m01']/m['m00']
cv2.circle(img,(int(cx),int(cy)),3,255,-1)
cv2.imshow('img',img)
cv2.imwrite('sofsqure.png',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result :
This question is related to python image recognition. A solution is given in squres.py demo.