Develop Reference

Extract most central area in a Binary Image

I am processing binary images, and was previously using this code to find the largest area in the binary image:
# Use the hue value to convert to binary
thresh = 20
thresh, thresh_img = cv2.threshold(h, thresh, 255, cv2.THRESH_BINARY)
cv2.imshow('thresh', thresh_img)
cv2.waitKey(0)
cv2.destroyAllWindows()
# Finding Contours
# Use a copy of the image since findContours alters the image
contours, _ = cv2.findContours(thresh_img.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
#Extract the largest area
c = max(contours, key=cv2.contourArea)
This code isn't really doing what I need it to do, now I think it would better to extract the most central area in the binary image.
Binary Image
Largest Image
This is currently what the code is extracting, but I am hoping to get the central circle in the first binary image extracted.

OpenCV comes with a point-polygon test function (for contours). It even gives a signed distance, if you ask for that.
I'll find the contour that is closest to the center of the picture. That may be a contour actually overlapping the center of the picture.
Timings, on my quadcore from 2012, give or take a millisecond:
findContours: ~1 millisecond
all pointPolygonTests and argmax: ~1 millisecond
mask = cv.imread("fkljm.png", cv.IMREAD_GRAYSCALE)
(height, width) = mask.shape
ret, mask = cv.threshold(mask, 128, 255, cv.THRESH_BINARY) # required because the sample picture isn't exactly clean
# get contours
contours, hierarchy = cv.findContours(mask, cv.RETR_LIST | cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
center = (np.array([width, height]) - 1) / 2
# find contour closest to center of picture
distances = [
cv.pointPolygonTest(contour, center, True) # looking for most positive (inside); negative is outside
for contour in contours
]
iclosest = np.argmax(distances)
print("closest contour is", iclosest, "with distance", distances[iclosest])
# draw closest contour
canvas = cv.cvtColor(mask, cv.COLOR_GRAY2BGR)
cv.drawContours(image=canvas, contours=[contours[iclosest]], contourIdx=-1, color=(0, 255, 0), thickness=5)
closest contour is 45 with distance 65.19202405202648
a cv.floodFill() on the center point can also quickly yield a labeling on that blob... assuming the mask is positive there. Otherwise, there needs to be search.
(cx, cy) = center.astype(int)
assert mask[cy,cx], "floodFill not applicable"
# trying cv.floodFill on the image center
mask2 = mask >> 1 # turns everything else gray
cv.floodFill(image=mask2, mask=None, seedPoint=center.astype(int), newVal=255)
# use (mask2 == 255) to identify that blob
This also takes less than a millisecond.
Some practically faster approaches might involve a pyramid scheme (low-res versions of the mask) to quickly identify areas of the picture that are candidates for an exact test (distance/intersection).
Test target pixel. Hit (positive)? Done.
Calculate low-res mask. Per block, if any pixel is positive, block is positive.
Find positive blocks, sort by distance, examine closer all those that are within sqrt(2) * blocksize of the best distance.

There are several ways you define "most central." I chose to define it as the region with the closest distance to the point you're searching for. If the point is inside the region, then that distance will be zero.
I also chose to do this with a pixel-based approach rather than a polygon-based approach, like you're doing with findContours().
Here's a step-by-step breakdown of what this code is doing.
Load the image, put it into grayscale, and threshold it. You're already doing these things.
Identify connected components of the image. Connected components are places where there are white pixels which are directly connected to other white pixels. This breaks up the image into regions.
Using np.argwhere(), convert a true/false mask into an array of coordinates.
For each coordinate, compute the Euclidean distance between that point and search_point.
Find the minimum within each region.
Across all regions, find the smallest distance.
import cv2
import numpy as np
img = cv2.imread('test197_img.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, thresh_img = cv2.threshold(gray,127,255,cv2.THRESH_BINARY)
n_groups, comp_grouped = cv2.connectedComponents(thresh_img)
components = []
search_point = [600, 150]
for i in range(1, n_groups):
mask = (comp_grouped == i)
component_coords = np.argwhere(mask)[:, ::-1]
min_distance = np.sqrt(((component_coords - search_point) ** 2).sum(axis=1)).min()
components.append({
'mask': mask,
'min_distance': min_distance,
})
closest = min(components, key=lambda x: x['min_distance'])['mask']
Output:

How to detect a precise rectangle in a photo opencv

I was using this code to detect the rectangle on the photo, at first it was working well, untill i realized that i would have an object that is also a square in the middle :
Question:
How can i properly detect the 4 corners like on the first result picture without detecting the corner of the thing in the middle of the square. Thanks a lot.
Code:
import numpy as np
import cv2
img = cv2.imread('Photos/lastBoard.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
canny = cv2.Canny(gray, 100, 200)
corners = cv2.goodFeaturesToTrack(gray, 25, 0.01, 50)
corner_list = []
for corner in corners:
x, y = corner.ravel()
if(y < 700 and (50 < x < 800 )):
corner_list.append([int(x), int(y)])
cv2.circle(img, (x, y), 5, (36, 255, 12), -1)
cv2.imshow("yo", img)
cv2.waitKey(0)

My man, it breaks my heart you aren't using the techniques and processing we covered in your last question. You have already plenty of functions you could re-use. The rectangle you are trying to segment has a unique color (kind of green) and has a defined area and aspect ratio! Look all the things you have on the table, they are smaller than the rectangle! Plus, the rectangle is almost a square! That means that its aspect ratio is close to 1.0. If you somehow segment the rectangle, approximating its corners should be relativity easy.
This is valuable info, because it allows you to trace your action plan. I see you are using cv2.goodFeaturesToTrack to detect the corners of everything. That's OK, but it could be simplified. I propose a plan of action very similar to last time:
Try to segment the rectangle using its color, let's compute an
HSV-based mask
Let's clean the mask from noise using an area filter and some morphology
Find contours - we are looking for the biggest green contour, the rectangle.
The contour of interest has defined features. Use the area and aspect ratio to filter garbage contours.
Once you have the contour/blob of interest, approximate its corners.
Let's see the code:
# imports:
import numpy as np
import cv2
# image path
path = "D://opencvImages//"
fileName = "table1.jpg"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
inputCopy = inputImage.copy()
# The HSV mask values:
lowerValues = np.array([58, 151, 25])
upperValues = np.array([86, 255, 75])
# Convert the image to HSV:
hsvImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2HSV)
# Create the HSV mask
mask = cv2.inRange(hsvImage, lowerValues, upperValues)
The first steps aim to create the HSV mask. Very similar to last time, I've defined the HSV range of interest already and applied exactly the same stuff as before. You could (and should) explore more exotic techniques latter, but let's stick with what we know works for the time being, as the project surely is due soon. This is the result:
You see how the mask is pretty nice already? Only the green puck and the rectangle survived the thresholding. It doesn't matter that the rectangle is not complete, because we're gonna approximate its contour with a bounding rectangle! Alright, let's clean this bad boy a little bit better. Use a filterArea (this is exactly the same function we saw last time) and then a closing (dilate followed by erode) just to get a nice mask:
# Run a minimum area filter:
minArea = 50
mask = areaFilter(minArea, mask)
# Pre-process mask:
kernelSize = 3
structuringElement = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
iterations = 2
mask = cv2.morphologyEx(mask, cv2.MORPH_DILATE, structuringElement, None, None, iterations, cv2.BORDER_REFLECT101)
mask = cv2.morphologyEx(mask, cv2.MORPH_ERODE, structuringElement, None, None, iterations, cv2.BORDER_REFLECT101)
This is the filtered mask, the noise is mostly gone:
Now, let's find contours and filtered based on area and aspect ratio, just like last time. The parameters, however, are different, because our target is not the plucks, but the rectangle:
# Find the big contours/blobs on the filtered image:
contours, hierarchy = cv2.findContours(mask, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# Store the poly approximation and bound
contoursPoly = [None] * len(contours)
# Store the corners of the square here:
detectedCorners = []
# Look for the outer bounding boxes:
for _, c in enumerate(contours):
# Approximate the contour to a polygon:
contoursPoly = cv2.approxPolyDP(c, 3, True)
# Convert the polygon to a bounding rectangle:
boundRect = cv2.boundingRect(contoursPoly)
# Get the bounding rect's data:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Calculate the rect's area:
rectArea = rectWidth * rectHeight
# Calculate the aspect ratio:
aspectRatio = rectWidth / rectHeight
delta = abs(1.0 - aspectRatio)
# Set the min threshold values to identify the
# blob of interest:
minArea = 2500
epsilon = 0.2
Alright, so far so good, I hope. As you see I approximated the contour to a 4-vertex polygon and then computed its bounding rectangle. This approximation should fit very nicely to our blob of interest. Now, apply the contour filter and use the bounding rectangle data to approximate the corners. I approximate each corner, one by one, and store them in the
detectedCorners array. Then, we can draw 'em. Here, still inside the for loop:
# Is this bounding rectangle we
# are looking for?
if rectArea > minArea and delta < epsilon:
# Compute the corners/vertices:
# Corner 1 (top left)
corner1 = (rectX, rectY)
detectedCorners.append(corner1)
# Corner 2 (top right)
corner2 = (rectX + rectWidth, rectY)
detectedCorners.append(corner2)
# Corner 3 (bottom left)
corner3 = (rectX, rectY + rectHeight)
detectedCorners.append(corner3)
# Corner 4 (bottom right)
corner4 = (rectX + rectWidth, rectY + rectHeight)
detectedCorners.append(corner4)
# Draw the corner points:
for p in detectedCorners:
color = (0, 0, 255)
cv2.circle(inputCopy, (p[0], p[1]), 5, color, -1)
cv2.imshow("Square Corners", inputCopy)
cv2.waitKey(0)
Here are the results for both images. The approximated corners are the red dots:
Here's the definition and implementation of the areaFilter function:
def areaFilter(minArea, inputImage):
# Perform an area filter on the binary blobs:
componentsNumber, labeledImage, componentStats, componentCentroids = \
cv2.connectedComponentsWithStats(inputImage, connectivity=4)
# Get the indices/labels of the remaining components based on the area stat
# (skip the background component at index 0)
remainingComponentLabels = [i for i in range(1, componentsNumber) if componentStats[i][4] >= minArea]
# Filter the labeled pixels based on the remaining labels,
# assign pixel intensity to 255 (uint8) for the remaining pixels
filteredImage = np.where(np.isin(labeledImage, remainingComponentLabels) == True, 255, 0).astype('uint8')
return filteredImage

remove demarcation from text image - image processing

Hi I need to write a program that remove demarcation from gray scale image(image with text in it)
i read about thresholding and blurring but still i dont see how can i do it.
my image is an image of a hebrew text like that:
and i need to remove the demarcation(assuming that the demarcation is the smallest element in the image) the output need to be something like that
I want to write the code in python using opencv, what topics do i need to learn to be able to do that, and how?
thank you.
Edit:
I can use only cv2 functions

The symbols you want to remove are significantly smaller than all other shapes, you can use that to determine witch ones to remove.
First use threshold to convert the image to binary. Next, you can use findContours to detect the shapes and then contourArea to determine if the shape is larger than a threshold.
Finally you can can create a mask to remove the unwanted shapes, draw the larger symbols on a new image or draw the smaller symbols in white over the original symbols in the original image - making them disappear. I used that last technique in the code below.
Result:
Code:
import cv2
# load image as grayscale
img = cv2.imread('1MioS.png',0)
# convert to binary. Inverted, so you get white symbols on black background
_ , thres = cv2.threshold(img, 200, 255, cv2.THRESH_BINARY_INV)
# find contours in the thresholded image (this gives all symbols)
contours, hierarchy = cv2.findContours(thres, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# loop through the contours, if the size of the contour is below a threshold,
# draw a white shape over it in the input image
for cnt in contours:
if cv2.contourArea(cnt) < 250:
cv2.drawContours(img,[cnt],0,(255),-1)
# display result
cv2.imshow('res', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Update
To find the largest contour, you can loop through them and keep track of the largest value:
maxArea = 0
for cnt in contours:
currArea = cv2.contourArea(cnt)
if currArea > maxArea:
maxArea = currArea
print(maxArea)
I also whipped up a little more complex version, that creates a sorted list of the indexes and sizes of the contours. Then it looks for the largest relative difference in size of all contours, so you know which contours are 'small' and 'large'. I do not know if this works for all letters / fonts.
# create a list of the indexes of the contours and their sizes
contour_sizes = []
for index,cnt in enumerate(contours):
contour_sizes.append([index,cv2.contourArea(cnt)])
# sort the list based on the contour size.
# this changes the order of the elements in the list
contour_sizes.sort(key=lambda x:x[1])
# loop through the list and determine the largest relative distance
indexOfMaxDifference = 0
currentMaxDifference = 0
for i in range(1,len(contour_sizes)):
sizeDifference = contour_sizes[i][1] / contour_sizes[i-1][1]
if sizeDifference > currentMaxDifference:
currentMaxDifference = sizeDifference
indexOfMaxDifference = i
# loop through the list again, ending (or starting) at the indexOfMaxDifference, to draw the contour
for i in range(0, indexOfMaxDifference):
cv2.drawContours(img,contours,contour_sizes[i][0] ,(255),-1)
To get the background color you can do use minMaxLoc. This returns the lowest color value and it's position of an image (also the max value, but you don't need that). If you apply it to the thresholded image - where the background is black -, it will return the location of a background pixel (big odds it will be (0,0) ). You can then look up this pixel in the original color image.
# get the location of a pixel with background color
min_val, _, min_loc, _ = cv2.minMaxLoc(thres)
# load color image
img_color = cv2.imread('1MioS.png')
# get bgr values of background
b,g,r = img_color[min_loc]
# convert from numpy object
background_color = (int(b),int(g),int(r))
and then to draw the contours
cv2.drawContours(img_color,contours,contour_sizes[i][0],background_color,-1)
and of course
cv2.imshow('res', img_color)

This looks like a problem for template matching since you have what looks like a known font and can easily understand what the characters and/or demarcations are. Check out https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_template_matching/py_template_matching.html
Admittedly, the tutorial talks about finding the match; modification is up to you. In that case, you know the exact shape of the template itself, so using that information along with the location of the match, just overwrite the image data with the appropriate background color (based on the examples above, 255).

You can solve it by removing all the small clusters.
I found a Python solution (using OpenCV) here.
For supporting smaller fonts, I added the following heuristic:
"The largest size of the demarcation cluster is 1/500 of the largest letter cluster".
The heuristic can be refined, by statistical analysts (or improved by other heuristics, such as demarcation locations relative to the letters).
import numpy as np
import cv2
I = cv2.imread('Goodluck.png', cv2.IMREAD_GRAYSCALE)
J = 255 - I # Invert I
img = cv2.threshold(J, 127, 255, cv2.THRESH_BINARY)[1] # Convert to binary
# https://answers.opencv.org/question/194566/removing-noise-using-connected-components/
nlabel,labels,stats,centroids = cv2.connectedComponentsWithStats(img, connectivity=8)
labels_small = []
areas_small = []
# Find largest cluster:
max_size = np.max(stats[:, cv2.CC_STAT_AREA])
thresh_size = max_size / 500 # Set the threshold to maximum cluster size divided by 500.
for i in range(1, nlabel):
if stats[i, cv2.CC_STAT_AREA] < thresh_size:
labels_small.append(i)
areas_small.append(stats[i, cv2.CC_STAT_AREA])
mask = np.ones_like(labels, dtype=np.uint8)
for i in labels_small:
I[labels == i] = 255
cv2.imshow('I', I)
cv2.waitKey(0)
Here is a MATLAB code sample (kept threshold = 200):
clear
I = imbinarize(rgb2gray(imread('בהצלחה.png')));
figure;imshow(I);
J = ~I;
%Clustering
CC = bwconncomp(J);
%Cover all small clusters with zewros.
for i = 1:CC.NumObjects
C = CC.PixelIdxList{i}; %Cluster coordinates.
%Fill small clusters with zeros.
if numel(C) < 200
J(C) = 0;
end
end
J = ~J;
figure;imshow(J);
Result:

Speeding up per pixel loop in opencv python

So i am trying to find pixels which are not white and create a bounding box around the image by examining the colors. I want to get the topmost, bottom most, leftmost and rightmost non white pixels, and use them to create a bounding box. I have used four loops to travel through each sides.also What i want to do is remove the background color (the background color is mostly grey) and change it to pure white. I have implemented all the functionality but now since i am using a lot of loops the code runs too slow. I need to optimize the loops while still having the functionality of finding the topmost, bottom most, leftmost and rightmost non white pixels and removing the colors. How can i do it?
The code below shows what i am doing to get the bounding box along with background removal at the same time. The mask is a black and white version of the image. If it is mask[i][j]==0 then it is a different color and hence i need to take the value and compare it with the values stored at p. It helps me find the bounding box. And if the mask[i][j]!=0 then i am changing the values of the image to white.
//for bounding box
p = []
p.append(5000)
p.append(0)
p.append(5000)
p.append(0)
for i in range(0, height):
for j in range(0, width):
if mask[i][j] == 0:
if j < p[0]:
p[0] = j
break
else:
img[i, j] = [255, 255, 255]
for i in range(0, height):
for j in reversed(range(0, width)):
if mask[i][j] == 0:
if j > p[1]:
p[1] = j
break
else:
img[i, j] = [255, 255, 255]
//topdown
for i in range(0, width):
for j in range(0, height):
if mask[j][i] == 0:
if j < p[2]:
p[2] = j
break
else:
img[j, i] = [255, 255, 255]
for i in reversed(range(0, width)):
for j in reversed(range(0, height)):
if mask[j][i] == 0:
if j > p[3]:
p[3] = j
break
else:
img[j, i] = [255, 255, 255]
So how can i optimize these loops while still getting the same functionality of getting pixel values and being able to change the color of some other image?

Background
To make the background white you can use a bitwise operation with the mask. To automate the creation of a mask read here.
Example:
import cv2
import numpy as np
# load image and mask
img = cv2.imread('image.png')
mask = cv2.imread('mask.png')
# combine images
res = cv2.bitwise_or(img,mask)
cv2.imshow("result", res)
cv2.waitKey(0)
cv2.destroyAllWindows()
The mask needs to have the equal number of color-channels as the image. All white areas in the mask will also become white in the image. Black areas in the mask will remain unaffected in the image.
Boundingbox
To get the boundingbox you could use findContours. It takes a binary mask as input and returns a list of contours. You can use the contour to find the boundingbox, rotated boundingbox or minimum enclosing circle. The result may not be perfect depending on your input, but you can use it to increase performance as it greatly narrows the search.
Note: the input to findContours should have a black background. You can modify your mask using inverted_mask = cv2.bitwise_not(mask). Or, if you obtained you mask using thresholding, you can choose an inverted threshold type.
Result:
Code:
import cv2
import numpy as np
# load image // use your mask instead
mask = cv2.imread('mask.png',0)
# find contours
contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
# get the boundingrect and draw a red line over it
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(mask2,(x,y),(x+w,y+h),(0,0,255),3)
# get the minumum enclosing rectangle and draw it in blue
rect = cv2.minAreaRect(cnt)
box = cv2.boxPoints(rect)
box = np.int0(box)
cv2.drawContours(mask2,[box],0,(255,0,0),3)
# diplay result
cv2.imshow("img", mask)
cv2.waitKey(0)
cv2.destroyAllWindows()
Maths
If you'd rather stick to checking array values, you can boost performance by first summing the rows and the columns. Summing is fast (and baked into numpy) and now you can discard rows/columns by checking one value. You can see an example of this process in this answer. I would suggest using the mask with background black for this, as you can compare the sum with zero. This will essentially result in the red boundingbox above. Of course, when a non zero row/col is found you will still have to loop that one to find the exact coordinate.
Pfiew, that turned out much longer than intended...

#JohnColeman has a point. Nested Python loops will be relatively slow even with the best algorithms and there are libraries that can optimize such operations.
The algorithm itself could be sped up by using the results of each loop to limit the range of following loop. For example, if when looking for the top non-white pixel, you scanned from top to bottom as the outer loop, and left to right as the inner loop, and found a pixel (a, b) (with a is the distance from the top), then in the next section you went looking for the left pixel, you know that you can start scanning from a+1 in the top down outer loop, and no further than b - 1 in the left-right inner loop. Let's call the result (c, d).
Similarly the bottom pixel can be no less than c in the vertical and d in the horizontal.

Merge MSER detected objetcs (OpenCV, Python)

I am working on this image as source:
Applying the next code...
import cv2
import numpy as np
mser = cv2.MSER_create()
img = cv2.imread('C:\\Users\\Link\\Desktop\\test2.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
vis = img.copy()
regions, _ = mser.detectRegions(gray)
hulls = [cv2.convexHull(p.reshape(-1, 1, 2)) for p in regions]
cv2.polylines(vis, hulls, 1, (0, 255, 0))
mask = np.zeros((img.shape[0], img.shape[1], 1), dtype=np.uint8)
for contour in hulls:
cv2.drawContours(mask, [contour], -1, (255, 255, 255), -1)
text_only = cv2.bitwise_and(img, img, mask=mask)
cv2.imshow('img', vis)
cv2.waitKey(0)
cv2.imshow('img', mask)
cv2.waitKey(0)
cv2.imshow('img', text_only)
cv2.waitKey(0)
cv2.imwrite('C:\\Users\\Link\\Desktop\\test_o\\1.png', text_only)
...I am obtaining this as result (mask):
The question is this:
how to merge into a single object the number 5 in the number series (157661546) as long as it is divided in the mask image ?
Thanks

Have a look here, it seems like the exact answer.
Here instead there is my version of the above code fine tuned for text extraction (with masking too).
Below there is the original code from the previous article, "ported" to python 3, opencv 3, added mser and bounding boxes. The main difference with my version is how the grouping distance is defined: mine is text-oriented while the one below is a free geometrical distance.
import sys
import cv2
import numpy as np
def find_if_close(cnt1,cnt2):
row1,row2 = cnt1.shape[0],cnt2.shape[0]
for i in range(row1):
for j in range(row2):
dist = np.linalg.norm(cnt1[i]-cnt2[j])
if abs(dist) < 25: # <-- threshold
return True
elif i==row1-1 and j==row2-1:
return False
img = cv2.imread(sys.argv[1])
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
cv2.imshow('input', img)
ret,thresh = cv2.threshold(gray,127,255,0)
mser=False
if mser:
mser = cv2.MSER_create()
regions = mser.detectRegions(thresh)
hulls = [cv2.convexHull(p.reshape(-1, 1, 2)) for p in regions[0]]
contours = hulls
else:
thresh = cv2.bitwise_not(thresh) # wants black bg
im2,contours,hier = cv2.findContours(thresh,cv2.RETR_EXTERNAL,2)
cv2.drawContours(img, contours, -1, (0,0,255), 1)
cv2.imshow('base contours', img)
LENGTH = len(contours)
status = np.zeros((LENGTH,1))
print("Elements:", len(contours))
for i,cnt1 in enumerate(contours):
x = i
if i != LENGTH-1:
for j,cnt2 in enumerate(contours[i+1:]):
x = x+1
dist = find_if_close(cnt1,cnt2)
if dist == True:
val = min(status[i],status[x])
status[x] = status[i] = val
else:
if status[x]==status[i]:
status[x] = i+1
unified = []
maximum = int(status.max())+1
for i in range(maximum):
pos = np.where(status==i)[0]
if pos.size != 0:
cont = np.vstack(contours[i] for i in pos)
hull = cv2.convexHull(cont)
unified.append(hull)
cv2.drawContours(img,contours,-1,(0,0,255),1)
cv2.drawContours(img,unified,-1,(0,255,0),2)
#cv2.drawContours(thresh,unified,-1,255,-1)
for c in unified:
(x,y,w,h) = cv2.boundingRect(c)
cv2.rectangle(img, (x,y), (x+w,y+h), (255, 0, 0), 2)
cv2.imshow('result', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Sample output (the yellow blob is below the binary threshold conversion so it's ignored). Red: original contours, green: unified ones, blue: bounding boxes.
Probably there is no need to use MSER as a simple findContours may work fine.
------------------------
Starting from here there is my old answer, before I found the above code. I'm leaving it anyway as it describe a couple of different approaches that may be easier/more appropriate for some scenarios.
A quick and dirty trick is to add a small gaussian blur and a high threshold before the MSER (or some dilute/erode if you prefer fancy things). In practice you just make the text bolder so that it fills small gaps. Obviously you can later discard this version and crop from the original one.
Otherwise, if your text is in lines, you may try to detect the average line center (make an histogram of Y coordinates and find the peaks for example). Then, for each line, look for fragments with a close average X. Quite fragile if text is noisy/complex.
If you do not need to split each letter, getting the bounding box for the whole word, may be easier: just split in groups based on a maximum horizontal distance between fragments (using the leftmost/rightmost points of the contour). Then use the leftmost and rightmost boxes within each group to find the whole bounding box. For multiline text first group by centroids Y coordinate.
Implementation notes:
Opencv allows you to create histograms but you probably can get away with something like this (worked for me on a similar task):
def histogram(vals, th=4, bins=400):
hist = np.zeros(bins)
for y_center in vals:
bucket = int(round(y_center / 2.)) <-- change this "2."
hist[bucket-1] += 1
print("hist: ", hist)
hist = np.where(hist > th, hist, 0)
return hist
Here my histogram is just an array with 400 buckets (my image was 800px high so each bucket catches two pixels, that is where the "2." comes from). Vals are the Y coordinates of the centroids of each fragment (you may want to ignore very small elements when you build this list). The th threshold is there just to remove some noise. You should get something like this:
0,0,0,5,22,0,0,0,0,43,7,0,0,0
This list describes, moving top to bottom, how many fragments are at each location.
Now I ran another pass to merge the peaks into a single value (just scan the array and sum while it is non-zero and reset the count on first zero) getting something like this {y:count}:
{9:27, 20:50}
Now I know I have two text rows at y=9 and y=20. Now, or before, you assign each fragment to on line (with again an 8px threshold in my case). Now you can process each line on its own, finding "words". BTW, I have your identical problem with broken letters that's why I came here looking for MSER :). Notice that if you find the whole bounding box for the word this problem happens only on the first/last letters: the other broken letters just falls inside the word box anyway.
Here is a reference for the erode/dilate thing, but gaussian blur/th worked for me.
UPDATE: I've noticed that there is something wrong in this line:
regions = mser.detectRegions(thresh)
I pass in the already thresholded image(!?). This is not relevant for the aggregation part but keep in mind that the mser part is not being used as expected.