Develop Reference

How to detect circle defects?

Is there any way to tell if a circle has such defects? Roundness does not work. Or is there a way to eliminate them?
perimeter = cv2.arcLength(cnts[0],True)
area = cv2.contourArea(cnts[0])
roundness = 4*pi*area/(perimeter*perimeter)
print("Roundness:", roundness)

The "roundness" measure is sensitive to a precise estimate of the perimeter. What cv2.arcLength() does is add the lengths of each of the polygon edges, which severely overestimates the length of outlines. I think this is the main reason that this measure hasn't worked for you. With a better perimeter estimator you would get useful results.
An alternative measure that might be more useful is "circularity", defined as the coefficient of variation of the radius. In short, you compute the distance of each polygon vertex (i.e. outline point) to the centroid, then determine the coefficient of variation of these distances (== std / mean).
I wrote a quick Python script to compute this starting from an OpenCV contour:
import cv2
import numpy as np
# read in OP's example image, making sure we ignore the red arrow
img = cv2.imread('jGssp.png')[:, :, 1]
_, img = cv2.threshold(img, 127, 255, 0)
# get the contour of the shape
contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
contour = contours[0][:, 0, :]
# add the first point as the last, to close it
contour = np.concatenate((contour, contour[0, None, :]))
# compute centroid
def cross_product(v1, v2):
"""2D cross product."""
return v1[0] * v2[1] - v1[1] * v2[0]
sum = 0.0
xsum = 0.0
ysum = 0.0
for ii in range(1, contour.shape[0]):
v = cross_product(contour[ii - 1, :], contour[ii, :])
sum += v
xsum += (contour[ii - 1, 0] + contour[ii, 0]) * v
ysum += (contour[ii - 1, 1] + contour[ii, 1]) * v
centroid = np.array([ xsum, ysum ]) / (3 * sum)
# Compute coefficient of variation of distances to centroid (==circularity)
d = np.sqrt(np.sum((contour - centroid) ** 2, axis=1))
circularity = np.std(d) / np.mean(d)

This make me think of a similar problem that I had. You could compute the signature of the shape. The signature can be defined as, for each pixel of the border of the shape, the distance between this pixel and the center of the shape.
For a perfect circle, the distance from the border to the center should be constant (in an ideal continuous world). When defects are visible on the edge of the circle (either dents or excesses), the ideal constant line changes to a wiggly curve, with huge variation when on the defects.
It's fairly easy to detect those variation with FFT for example, which allows to quantify the defect significance.
You can expand this solution to any given shape. If your ideal shape is a square, you can compute the signature, which will give you some kind of sinusoidal curve. Defects will appear in a same way on the curve, and would be detectable with the same logic as with a circle.
I can't give you an code example, as the project was for a company project, but the idea is still here.

Here is one way to do that in Python/OpenCV.
Read the input
Threshold on white (to remove the red arrow)
Apply Hough Circle
Draw the circle on the thresholded image for comparison
Draw a white filled circle on black background from the circle parameters.
Get the difference between the thresholded image and the drawn circle image
Apply morphology open to remove the ring from the irregular boundary of the original circle
Count the number of white pixels in the previous image as the amount off defect
Input:
import cv2
import numpy as np
# Read image
img = cv2.imread('circle_defect.png')
hh, ww = img.shape[:2]
# threshold on white to remove red arrow
lower = (255,255,255)
upper = (255,255,255)
thresh = cv2.inRange(img, lower, upper)
# get Hough circles
min_dist = int(ww/5)
circles = cv2.HoughCircles(thresh, cv2.HOUGH_GRADIENT, 1, minDist=min_dist, param1=150, param2=15, minRadius=0, maxRadius=0)
print(circles)
# draw circles on input thresh (without red arrow)
circle_img = thresh.copy()
circle_img = cv2.merge([circle_img,circle_img,circle_img])
for circle in circles[0]:
# draw the circle in the output image, then draw a rectangle
# corresponding to the center of the circle
(x,y,r) = circle
x = int(x)
y = int(y)
r = int(r)
cv2.circle(circle_img, (x, y), r, (0, 0, 255), 1)
# draw filled circle on black background
circle_filled = np.zeros_like(thresh)
cv2.circle(circle_filled, (x,y), r, 255, -1)
# get difference between the thresh image and the circle_filled image
diff = cv2.absdiff(thresh, circle_filled)
# apply morphology to remove ring
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
result = cv2.morphologyEx(diff, cv2.MORPH_OPEN, kernel)
# count non-zero pixels
defect_count = np.count_nonzero(result)
print("defect count:", defect_count)
# save results
cv2.imwrite('circle_defect_thresh.jpg', thresh)
cv2.imwrite('circle_defect_circle.jpg', circle_img)
cv2.imwrite('circle_defect_circle_diff.jpg', diff)
cv2.imwrite('circle_defect_detected.png', result)
# show images
cv2.imshow('thresh', thresh)
cv2.imshow('circle_filled', circle_filled)
cv2.imshow('diff', diff)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Input without Red Arrow:
Red Circle Drawn on Input:
Circle from HoughCircle:
Difference:
Difference Cleaned Up:
Textual Result:
defect count: 500

Braille Dot detection on image using OpenCV

For a project I want to detect braille dots on a plate. I make a picture on which I make my detection thanks to the connectedComponentsWithStats function. Despite my attempts I can never get a threshold value where all the dots and only them are detected, I have the same problem if I try to use the circle detection. I'm trying to use template matching on the advice of a teacher but I'm also having problems with my detection since the only factor that influences it is the threshold.
import matplotlib.pyplot as plt
img1 = cv.imread(r"traitement\prod.png")
plt.figure(figsize=(40,40))
plt.subplot(3,1,1)
gray_img = cv.cvtColor(img1, cv.COLOR_BGR2GRAY)
test = cv.adaptiveThreshold(gray_img, 255, cv.ADAPTIVE_THRESH_MEAN_C, cv.THRESH_BINARY_INV, 11, 6)
_, _, boxes, _ = cv.connectedComponentsWithStats(test)
boxes = boxes[1:]
filtered_boxes = []
for x,y,w,h,pixels in boxes:
if pixels < 1000 and h < 35 and w < 35 and h > 14 and w > 14 and x > 15 and y > 15:
filtered_boxes.append((x,y,w,h))
for x,y,w,h in filtered_boxes:
W = int(w)/2
H = int(h)/2
#print(w)
cv.circle(img1,(x+int(W),y+int(H)),2,(0,255,0),20)
cv.imwrite("gray.png",gray_img)
cv.imwrite("test.png",test)
plt.imshow(test)
plt.subplot(3,1,2)
plt.imshow(img1)
import cv2 as cv
import numpy as np
from imutils.object_detection import non_max_suppression
import matplotlib.pyplot as plt
img = cv.imread('traitement/prod.png')
temp_gray = cv.imread('dot.png',0)
W, H = temp.shape[:2]
thresh = 0.6
img_gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
match = cv.matchTemplate(image=img_gray, templ=temp_gray, method=cv.TM_CCOEFF_NORMED)
(y_points, x_points) = np.where(match >= thresh)
boxes = list()
for (x, y) in zip(x_points, y_points):
# update our list of rectangles
boxes.append((x, y, x + W, y + H))
boxes = non_max_suppression(np.array(boxes))
# loop over the final bounding boxes
for (x1, y1, x2, y2) in boxes:
cv.circle(img,(x1+int(W/2),y1+int(H/2)),2,(255,0,0),15)
plt.figure(figsize=(40,40))
plt.subplot(3,1,1)
plt.imshow(img)
Image with adaptive threshold:
Image with template detection:

I found a solution that may not be better than your solutions, because I had to overfit few parameters for the given input...
The problem is challenging because the input image was taken under non-uniform illumination conditions (the center part is brighter than the top). Consider taking a better snapshot...
Point of thought:
The dots are ordered in rows, and we are not using that information.
We may get better results if we were using the fact that the dots are ordered in rows.
For overcoming the brightness differences we may subtract the median of the surrounding pixels from each pixel (using large filter radius), and compute the absolute difference:
bg = cv2.medianBlur(gray, 151) # Background
fg = cv2.absdiff(gray, bg) # Foreground (use absdiff because the dost are dark but bright at the center).
Apply binary threshold (use THRESH_OTSU for automatic threshold level):
_, thresh = cv2.threshold(fg, 0, 255, cv2.THRESH_OTSU)
The result of thresh is not good enough for finding the dots.
We may use the fact that the dots are dark with bright center.
That fact makes an high edges around and inside the dots.
Apply Canny edge detection:
edges = cv2.Canny(gray, threshold1=50, threshold2=100)
Merge edges with thresh (use binary or):
thresh = cv2.bitwise_or(thresh, edges)
Find connected components and continue (filter the components by area).
Code sample:
import numpy as np
import cv2
img1 = cv2.imread('prod.jpg')
gray = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY) # Convert to grayscale
bg = cv2.medianBlur(gray, 151) # Compute the background (use a large filter radius for excluding the dots)
fg = cv2.absdiff(gray, bg) # Compute absolute difference
_, thresh = cv2.threshold(fg, 0, 255, cv2.THRESH_OTSU) # Apply binary threshold (THRESH_OTSU applies automatic threshold level)
edges = cv2.Canny(gray, threshold1=50, threshold2=100) # Apply Canny edge detection.
thresh = cv2.bitwise_or(thresh, edges) # Merge edges with thresh
_, _, boxes, _ = cv2.connectedComponentsWithStats(thresh)
boxes = boxes[1:]
filtered_boxes = []
for x, y, w, h, pixels in boxes:
#if pixels < 1000 and h < 35 and w < 35 and h > 14 and w > 14 and x > 15 and y > 15 and pixels > 100:
if pixels < 1000 and x > 15 and y > 15 and pixels > 200:
filtered_boxes.append((x, y, w, h))
for x, y, w, h in filtered_boxes:
W = int(w)/2
H = int(h)/2
cv2.circle(img1, (x+int(W), y+int(H)), 2, (0, 255, 0), 20)
# Show images for testing
cv2.imshow('bg', bg)
cv2.imshow('fg', fg)
cv2.imshow('gray', gray)
cv2.imshow('edges', edges)
cv2.imshow('thresh', thresh)
cv2.imshow('img1', img1)
cv2.waitKey()
cv2.destroyAllWindows()
Result:
There are few dots that are marked twice.
It is relatively simple to merge the overlapping circles into one circle.
Intermediate results:
thresh (before merging with edges):
edges:
thresh merged with edges:
Update:
As Jeru Luke commented we may use non-maximum suppression as done in question.
Here is a code sample:
import numpy as np
import cv2
from imutils.object_detection import non_max_suppression
img1 = cv2.imread('prod.jpg')
gray = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY) # Convert to grayscale
bg = cv2.medianBlur(gray, 151) # Compute the background (use a large filter radius for excluding the dots)
fg = cv2.absdiff(gray, bg) # Compute absolute difference
_, thresh = cv2.threshold(fg, 0, 255, cv2.THRESH_OTSU) # Apply binary threshold (THRESH_OTSU applies automatic threshold level)
edges = cv2.Canny(gray, threshold1=50, threshold2=100) # Apply Canny edge detection.
thresh = cv2.bitwise_or(thresh, edges) # Merge edges with thresh
_, _, boxes, _ = cv2.connectedComponentsWithStats(thresh)
boxes = boxes[1:]
filtered_boxes = []
for x, y, w, h, pixels in boxes:
if pixels < 1000 and x > 15 and y > 15 and pixels > 200:
filtered_boxes.append((x, y, x+w, y+h))
filtered_boxes = non_max_suppression(np.array(filtered_boxes), overlapThresh=0.2)
for x1, y1, x2, y2 in filtered_boxes:
cv2.circle(img1, ((x1+x2)//2, (y1+y2)//2), 2, (0, 255, 0), 20)
Result:

Approach: template matching.
Because the appearance of these dots can't be caught by just thresholding on brightness. These dots have both brighter and darker pixels than the flat surface. At best you'd get fractured components and given how close these dots are, any morphology operations to fix up the fractured components would run the risk of joining adjacent dots.
So here I do template matching. Works well enough, even though the appearance of these dots changes across the image. Brightness is uneven but that's not too much of a problem. TM_CCOEFF subtracts the mean for both patches before correlating.
imutils requires you to come up with bounding boxes for its NMS. Instead I'm using a simple and effective NMS formulation for raster data. Comparing floats for equality is okay here because dilate simply replicates the maximum value across the kernel size. It uses L-inf distance; for euclidean distance or an approximation, the kernel needs to be round. One downside: if there are multiple peaks of equal value, this will not remove either of them. To catch adjacent equal peaks, connectedComponents would work instead of findNonZero. For non-adjacent but equal peaks... let's just assume that doesn't happen because the data would make that situation impossible.
# Non-maximum suppression for raster data
def non_maximum_suppression(im, radius):
dilated = cv.dilate(im, kernel=None, iterations=radius)
return (im == dilated)
# Read image
im = cv.imread("fK2WOX.jpeg", cv.IMREAD_GRAYSCALE)
# Select template
# (x,y,w,h) = (880, 247, 44, 44)
(x,y,w,h) = cv.selectROI("ROI", 255-im, showCrosshair=False, fromCenter=True) # inverted to see the white rectangle...
cv.destroyWindow("ROI")
print((x,y,w,h))
template = im[y:y+h, x:x+w]
# find instances
scores = cv.matchTemplate(im, template, method=cv.TM_CCOEFF)
scores = scores / scores.max()
You see that the template creates additional peaks to the top and bottom of each dot. That can't be helped because that's the appearance of these dots, they consist of a bright dash surrounded by two darker blobs. These false peaks can be suppressed with NMS quite easily.
# find peaks of sufficient strength and NMS
threshold = 0.2
nmsradius = 20 # proportional to size of template/dot
nmsmask = (scores >= threshold) & non_maximum_suppression(scores, radius=nmsradius)
coords = cv.findNonZero(nmsmask.astype(np.uint8)).reshape((-1, 2))
coords += (w//2, h//2) # shift coordinates to be center of template
# draw result
canvas = cv.cvtColor(im, cv.COLOR_GRAY2BGR)
for pt in coords:
cv.circle(canvas, pt, radius=5, color=(0,0,255), thickness=cv.FILLED)
There are no overlapping/split detections. You'll have to excuse the false detections around the punched holes near the top. Just crop those out.
As for associating those dots into symbols, that's a new problem. I'd recommend using nearest-neighbor queries. When the spacing (in N/S/E/W direction) to other dots has been estimated, you can "probe" in those directions going from each dot and check if there is another dot there... and assemble symbols like that, marking dots as "associated" until there's nothing left. You would also want to associate symbols into strings, assuming a certain spacing between symbols. same approach there... and that gives you the added information of a baseline for the "text".

Counting the number of shapes in an image

I am trying to count the number of shapes in an image of a board game. For some reason, the count is completely off.
I have been using code from the web (for example, https://www.pyimagesearch.com/2016/02/01/opencv-center-of-contour/)
I only need opencv for this, has anyone else had this problem, and how do I limit this to only the squares that I am interested in?
This is the same image as below but the background is now black. Now there are more center-dots than necessary all around the edge of the image.
I want the center dot to appear in every square on the board

I used paint to remove the edges of the numbers that were interrupting the white lines of each box. I assume that those numbers and dots were things that you added and not part of the original image.
Here is the image after my modifications
I made a mask by looking for white in the image. I dilated the results to get clearer separation and inverted so that each box was white.
I used findContours to get the contour of each white object in the mask. I filtered the contours by size to get rid of the arrows, letter bits, and the background of the image. From there I drew the centroid of each remaining contour.
import cv2
import numpy as np
# load image
img = cv2.imread("chutes_n_ladders.png");
# mask
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY);
mask = cv2.inRange(gray, 240, 255);
# dilate and invert
kernel = np.ones((3,3), np.uint8);
mask = cv2.dilate(mask, kernel, iterations = 1);
mask = cv2.bitwise_not(mask);
# contours
_, contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE);
# remove very large and very small contours
filtered = [];
low = 1000;
high = 100000;
for con in contours:
area = cv2.contourArea(con);
if low < area and area < high:
filtered.append(con);
# draw centers of each
print("Shapes: " + str(len(filtered)));
for con in filtered:
M = cv2.moments(con);
cx = int(M['m10']/M['m00']);
cy = int(M['m01']/M['m00']);
cv2.circle(img, (cx, cy), 10, (0, 200, 100), -1);
# show
cv2.imshow("Image", img);
cv2.imshow("Mask", mask);
cv2.waitKey(0);

How to identify the shape of a floorplan?

I'm trying to differentiate between two different styles of houses using a floorplan. I'm very new to cv2, so I'm struggling a bit here. I'm able to identify the exterior of the house using contours using the code below, that is from another Stack Overflow response.
import cv2
import numpy as np
def find_rooms(img, noise_removal_threshold=25, corners_threshold=0.1,
room_closing_max_length=100, gap_in_wall_threshold=500):
assert 0 <= corners_threshold <= 1
# Remove noise left from door removal
img[img < 128] = 0
img[img > 128] = 255
contours, _ = cv2.findContours(~img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
mask = np.zeros_like(img)
for contour in contours:
area = cv2.contourArea(contour)
if area > noise_removal_threshold:
cv2.fillPoly(mask, [contour], 255)
img = ~mask
# Detect corners (you can play with the parameters here)
dst = cv2.cornerHarris(img ,2,3,0.04)
dst = cv2.dilate(dst,None)
corners = dst > corners_threshold * dst.max()
# Draw lines to close the rooms off by adding a line between corners on the same x or y coordinate
# This gets some false positives.
# You could try to disallow drawing through other existing lines for example.
for y,row in enumerate(corners):
x_same_y = np.argwhere(row)
for x1, x2 in zip(x_same_y[:-1], x_same_y[1:]):
if x2[0] - x1[0] < room_closing_max_length:
color = 0
cv2.line(img, (x1, y), (x2, y), color, 1)
for x,col in enumerate(corners.T):
y_same_x = np.argwhere(col)
for y1, y2 in zip(y_same_x[:-1], y_same_x[1:]):
if y2[0] - y1[0] < room_closing_max_length:
color = 0
cv2.line(img, (x, y1), (x, y2), color, 1)
# Mark the outside of the house as black
contours, _ = cv2.findContours(~img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contour_sizes = [(cv2.contourArea(contour), contour) for contour in contours]
biggest_contour = max(contour_sizes, key=lambda x: x[0])[1]
mask = np.zeros_like(mask)
cv2.fillPoly(mask, [biggest_contour], 255)
img[mask == 0] = 0
return biggest_contour, mask
#Read gray image
img = cv2.imread("/content/51626-7-floorplan-2.jpg", cv2.IMREAD_GRAYSCALE)
ext_contour, mask = find_rooms(img.copy())
cv2_imshow(mask)
print('exterior')
epsilon = 0.01*cv2.arcLength(ext_contour,True)
approx = cv2.approxPolyDP(ext_contour,epsilon,True)
final = cv2.drawContours(img, [approx], -1, (0, 255, 0), 2)
cv2_imshow(final)
These floorplans will only have one of two shapes, a 6 sided shape and a 4 sided shape. Below are the two styles:
I need to ignore any bay windows or small extrusions.
I believe the next step is to only have a contour for the main walls, have that contour be smooth, and then count the edges in the array. I'm stuck as to how to do this. Any assistance would be greatly appreciated!

If you really just need the decision, whether it's a four or six sided house, you can simply do the following: Grayscale image, and inverse binary threshold everything, which is not nearly white. Then, just calculate the ratio between that mask and the total number of pixels. That ratio must be larger for four sided houses than for six sided houses. The exact cut-off depends on your data. For the two given examples, one could set the cut-off to 0.9.
Here's some code:
import cv2
from skimage import io # Only needed for web grabbing images
def house_analysis(image):
# Grayscale image
mask = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
# Inverse binary threshold everything, which is not nearly white
mask = cv2.threshold(mask, 248, 255, cv2.THRESH_BINARY_INV)[1]
# Calculate ratio between mask and total number of pixels
ratio = cv2.countNonZero(mask) / (mask.shape[0] * mask.shape[1])
print(ratio)
# Decide with respect to cut-off, if house is four or six sided
cutoff = 0.9
if ratio > cutoff:
print('Four sided house')
else:
print('Six sided house')
cv2.imshow('image', image)
cv2.imshow('mask', mask)
cv2.waitKey(0)
house_4 = cv2.cvtColor(io.imread('https://i.stack.imgur.com/vqzZB.jpg'), cv2.COLOR_RGB2BGR)
house_6 = cv2.cvtColor(io.imread('https://i.stack.imgur.com/ZpkQW.jpg'), cv2.COLOR_RGB2BGR)
house_analysis(house_4)
house_analysis(house_6)
cv2.destroyAllWindows()
The print outputs:
0.9533036597428289
Four sided house
0.789531416400426
Six sided house
If you have larger white space around the main walls, one could crop that part to get more robust ratios.
Hope that helps!
----------------------------------------
System information
----------------------------------------
Platform: Windows-10-10.0.16299-SP0
Python: 3.8.1
OpenCV: 4.1.2
----------------------------------------

Simple contour finding is unlikely to give you a robust solution.
however your current approach can be improved by first calculating a mask of the white background.
Using the shape of this mask you can determine the layout.
lower_color_bounds = cv.Scalar(255, 255, 255)
upper_color_bounds = cv.Scalar(220, 220, 220)
mask = cv2.inRange(frame,lower_color_bounds,upper_color_bounds )
mask_rgb = cv2.cvtColor(mask,cv2.COLOR_GRAY2BGR)

Advanced square detection (with connected region)

if the squares has connected region in image, how can I detect them.
I have tested the method mentioned in
OpenCV C++/Obj-C: Advanced square detection
It did not work well.
Any good ideas ?
import cv2
import numpy as np
def angle_cos(p0, p1, p2):
d1, d2 = (p0-p1).astype('float'), (p2-p1).astype('float')
return abs( np.dot(d1, d2) / np.sqrt( np.dot(d1, d1)*np.dot(d2, d2) ) )
def find_squares(img):
squares = []
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# cv2.imshow("gray", gray)
gaussian = cv2.GaussianBlur(gray, (5, 5), 0)
temp,bin = cv2.threshold(gaussian, 80, 255, cv2.THRESH_BINARY)
# cv2.imshow("bin", bin)
contours, hierarchy = cv2.findContours(bin, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours( gray, contours, -1, (0, 255, 0), 3 )
#cv2.imshow('contours', gray)
for cnt in contours:
cnt_len = cv2.arcLength(cnt, True)
cnt = cv2.approxPolyDP(cnt, 0.02*cnt_len, True)
if len(cnt) == 4 and cv2.contourArea(cnt) > 1000 and cv2.isContourConvex(cnt):
cnt = cnt.reshape(-1, 2)
max_cos = np.max([angle_cos( cnt[i], cnt[(i+1) % 4], cnt[(i+2) % 4] ) for i in xrange(4)])
if max_cos < 0.1:
squares.append(cnt)
return squares
if __name__ == '__main__':
img = cv2.imread('123.bmp')
#cv2.imshow("origin", img)
squares = find_squares(img)
print "Find %d squres" % len(squares)
cv2.drawContours( img, squares, -1, (0, 255, 0), 3 )
cv2.imshow('squares', img)
cv2.waitKey()
I use some method in the opencv example, but the result is not good.

Applying a Watershed Transform based on the Distance Transform will separate the objects:
Handling objects at the border is always problematic, and often discarded, so that pink rectangle at top left not separated is not a problem at all.
Given a binary image, we can apply the Distance Transform (DT) and from it obtain markers for the Watershed. Ideally there would be a ready function for finding regional minima/maxima, but since it isn't there, we can make a decent guess on how we can threshold DT. Based on the markers we can segment using Watershed, and the problem is solved. Now you can worry about distinguishing components that are rectangles from those that are not.
import sys
import cv2
import numpy
import random
from scipy.ndimage import label
def segment_on_dt(img):
dt = cv2.distanceTransform(img, 2, 3) # L2 norm, 3x3 mask
dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(numpy.uint8)
dt = cv2.threshold(dt, 100, 255, cv2.THRESH_BINARY)[1]
lbl, ncc = label(dt)
lbl[img == 0] = lbl.max() + 1
lbl = lbl.astype(numpy.int32)
cv2.watershed(cv2.cvtColor(img, cv2.COLOR_GRAY2BGR), lbl)
lbl[lbl == -1] = 0
return lbl
img = cv2.cvtColor(cv2.imread(sys.argv[1]), cv2.COLOR_BGR2GRAY)
img = cv2.threshold(img, 0, 255, cv2.THRESH_OTSU)[1]
img = 255 - img # White: objects; Black: background
ws_result = segment_on_dt(img)
# Colorize
height, width = ws_result.shape
ws_color = numpy.zeros((height, width, 3), dtype=numpy.uint8)
lbl, ncc = label(ws_result)
for l in xrange(1, ncc + 1):
a, b = numpy.nonzero(lbl == l)
if img[a[0], b[0]] == 0: # Do not color background.
continue
rgb = [random.randint(0, 255) for _ in xrange(3)]
ws_color[lbl == l] = tuple(rgb)
cv2.imwrite(sys.argv[2], ws_color)
From the above image you can consider fitting ellipses in each component to determine rectangles. Then you can use some measurement to define whether the component is a rectangle or not. This approach has a greater chance to work for rectangles that are fully visible, and will likely produce bad results for partially visible ones. The following image shows the result of such approach considering that a component is a rectangle if the rectangle from the fitted ellipse is within 10% of component's area.
# Fit ellipse to determine the rectangles.
wsbin = numpy.zeros((height, width), dtype=numpy.uint8)
wsbin[cv2.cvtColor(ws_color, cv2.COLOR_BGR2GRAY) != 0] = 255
ws_bincolor = cv2.cvtColor(255 - wsbin, cv2.COLOR_GRAY2BGR)
lbl, ncc = label(wsbin)
for l in xrange(1, ncc + 1):
yx = numpy.dstack(numpy.nonzero(lbl == l)).astype(numpy.int64)
xy = numpy.roll(numpy.swapaxes(yx, 0, 1), 1, 2)
if len(xy) < 100: # Too small.
continue
ellipse = cv2.fitEllipse(xy)
center, axes, angle = ellipse
rect_area = axes[0] * axes[1]
if 0.9 < rect_area / float(len(xy)) < 1.1:
rect = numpy.round(numpy.float64(
cv2.cv.BoxPoints(ellipse))).astype(numpy.int64)
color = [random.randint(60, 255) for _ in xrange(3)]
cv2.drawContours(ws_bincolor, [rect], 0, color, 2)
cv2.imwrite(sys.argv[3], ws_bincolor)

Solution 1:
Dilate your image to delete connected components.
Find contours of detected components. Eliminate contours which are not rectangles by introducing some measure (ex. ratio perimeter / area).
This solution will not detect rectangles connected to borders.
Solution 2:
Dilate to delete connected components.
Find contours.
Approximate contours to reduce their points (for rectangle contour should be 4 points).
Check if angle between contour lines is 90 degrees.
Eliminate contours which have no 90 degrees.
This should solve problem with rectangles connected to borders.

You have three problems:
The rectangles are not very strict rectangles (the edges are often somewhat curved)
There are a lot of them.
They are often connected.
It seems that all your rects are essentially the same size(?), and do not greatly overlap, but the pre-processing has connected them.
For this situation the approach I would try is:
dilate your image a few times (as also suggested by #krzych) - this will remove the connections, but result in slightly smaller rects.
Use scipy to label and find_objects - You now know the position and slice for every remaining blob in the image.
Use minAreaRect to find the center, orientation, width and height of each rectangle.
You can use step 3. to test whether the blob is a valid rectangle or not, by its area, dimension ratio or proximity to the edge..
This is quite a nice approach, as we assume each blob is a rectangle, so minAreaRect will find the parameters for our minimum enclosing rectangle. Further we could test each blob using something like humoments if absolutely neccessary.
Here is what I was suggesting in action, boundary collision matches shown in red.
Code:
import numpy as np
import cv2
from cv2 import cv
import scipy
from scipy import ndimage
im_col = cv2.imread('jdjAf.jpg')
im = cv2.imread('jdjAf.jpg',cv2.CV_LOAD_IMAGE_GRAYSCALE)
im = np.where(im>100,0,255).astype(np.uint8)
im = cv2.erode(im, None,iterations=8)
im_label, num = ndimage.label(im)
for label in xrange(1, num+1):
points = np.array(np.where(im_label==label)[::-1]).T.reshape(-1,1,2).copy()
rect = cv2.minAreaRect(points)
lines = np.array(cv2.cv.BoxPoints(rect)).astype(np.int)
if any([np.any(lines[:,0]<=0), np.any(lines[:,0]>=im.shape[1]-1), np.any(lines[:,1]<=0), np.any(lines[:,1]>=im.shape[0]-1)]):
cv2.drawContours(im_col,[lines],0,(0,0,255),1)
else:
cv2.drawContours(im_col,[lines],0,(255,0,0),1)
cv2.imshow('im',im_col)
cv2.imwrite('rects.png',im_col)
cv2.waitKey()
I think the Watershed and distanceTransform approach demonstrated by #mmgp is clearly superior for segmenting the image, but this simple approach can be effective depending upon your needs.