Develop Reference

bounding boxes on handwritten digits with opencv

I tried the code provided bellow to segment each digit in this image and put a contour around it then crop it out but it's giving me bad results, I'm not sure what I need to change or work on.
The best idea I can think of right now is filtering the 4 largest contours in the image except the image contour itself.
The code I'm working with:
import sys
import numpy as np
import cv2
im = cv2.imread('marks/mark28.png')
im3 = im.copy()
gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5, 5), 0)
thresh = cv2.adaptiveThreshold(blur, 255, 1, 1, 11, 2)
################# Now finding Contours ###################
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
samples = np.empty((0, 100))
responses = []
keys = [i for i in range(48, 58)]
for cnt in contours:
if cv2.contourArea(cnt) > 50:
[x, y, w, h] = cv2.boundingRect(cnt)
if h > 28:
cv2.rectangle(im, (x, y), (x + w, y + h), (0, 0, 255), 2)
roi = thresh[y:y + h, x:x + w]
roismall = cv2.resize(roi, (10, 10))
cv2.imshow('norm', im)
key = cv2.waitKey(0)
if key == 27: # (escape to quit)
sys.exit()
elif key in keys:
responses.append(int(chr(key)))
sample = roismall.reshape((1, 100))
samples = np.append(samples, sample, 0)
responses = np.array(responses, np.float32)
responses = responses.reshape((responses.size, 1))
print
"training complete"
np.savetxt('generalsamples.data', samples)
np.savetxt('generalresponses.data', responses)
I need to change the if condition on height probably but more importantly I need if conditions to get the 4 largest contours on the image. Sadly, I haven't managed to find what I'm supposed to be filtering.
This is the kind of results I get, I'm trying to escape getting those inner contours on the digit "zero"
Unprocessed images as requested: example 1 example 2
All I need is an idea on what I should filter for, don't write code please. Thank you community.

You almost have it. You have multiple bounding rectangles on each digit because you are retrieving every contour (external and internal). You are using cv2.findContours in RETR_LIST mode, which retrieves all the contours, but doesn't create any parent-child relationship. The parent-child relationship is what discriminates between inner (child) and outter (parent) contours, OpenCV calls this "Contour Hierarchy". Check out the docs for an overview of all hierarchy modes. Of particular interest is RETR_EXTERNAL mode. This mode fetches only external contours - so you don't get multiple contours and (by extension) multiple bounding boxes for each digit!
Also, it seems that your images have a red border. This will introduce noise while thresholding the image, and this border might be recognized as the top-level outer contour - thus, every other contour (the children of this parent contour) will not be fetched in RETR_EXTERNAL mode. Fortunately, the border position seems constant and we can eliminate it with a simple flood-fill, which pretty much fills a blob of a target color with a substitute color.
Let's check out the reworked code:
# Imports:
import cv2
import numpy as np
# Set image path
path = "D://opencvImages//"
fileName = "rhWM3.png"
# Read Input image
inputImage = cv2.imread(path+fileName)
# Deep copy for results:
inputImageCopy = inputImage.copy()
# Convert BGR to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Threshold via Otsu:
threshValue, binaryImage = cv2.threshold(grayscaleImage, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
The first step is to get the binary image with all the target blobs/contours. This is the result so far:
Notice the border is white. We have to delete this, a simple flood-filling at position (x=0,y=0) with black color will suffice:
# Flood-fill border, seed at (0,0) and use black (0) color:
cv2.floodFill(binaryImage, None, (0, 0), 0)
This is the filled image, no more border!
Now we can retrieve the external, outermost contours in RETR_EXTERNAL mode:
# Get each bounding box
# Find the big contours/blobs on the filtered image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
Notice you also get each contour's hierarchy as second return value. This is useful if you want to check out if the current contour is a parent or a child. Alright, let's loop through the contours and get their bounding boxes. If you want to ignore contours below a minimum area threshold, you can also implement an area filter:
# Look for the outer bounding boxes (no children):
for _, c in enumerate(contours):
# Get the bounding rectangle of the current contour:
boundRect = cv2.boundingRect(c)
# Get the bounding rectangle data:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Estimate the bounding rect area:
rectArea = rectWidth * rectHeight
# Set a min area threshold
minArea = 10
# Filter blobs by area:
if rectArea > minArea:
# Draw bounding box:
color = (0, 255, 0)
cv2.rectangle(inputImageCopy, (int(rectX), int(rectY)),
(int(rectX + rectWidth), int(rectY + rectHeight)), color, 2)
cv2.imshow("Bounding Boxes", inputImageCopy)
# Crop bounding box:
currentCrop = inputImage[rectY:rectY+rectHeight,rectX:rectX+rectWidth]
cv2.imshow("Current Crop", currentCrop)
cv2.waitKey(0)
The last three lines of the above snippet crop and show the current digit. This is the result of detected bounding boxes for both of your images (the bounding boxes are colored in green, the red border is part of the input images):

cv2.inRange() make it work for all inputs

I'm using OpenCv (4.x) on Anime Sketch dataset from Kaggle to get the image's silhouette. What I found to be the hardest part was to detect that empty areas inside that silhouette, areas between arm-body, legs and hair. The tutorials I followed always use "full filled" objects, like a ball, head or cars and I ended up tunning that code to make it work, but it is too specific so that tunning just work ok on one image.
Playing around in online-image-editor.com I've noticed that I can use the tool called Trans-parency to change one color, just like cv2.inRange() does.
Original image
The code:
image = cv2.imread("2.png",cv2.IMREAD_UNCHANGED)
crop_img = image[:, 0:512]
fuzz_factor = 0.97
maxColor = (crop_img[1,1] * 1).astype(int)
minColor = (maxColor * fuzz_factor).astype(int)
mask = cv2.inRange(crop_img, minColor, maxColor)
cv2.imshow("mask", mask)
cv2.waitKey()
and outputs this (not that bad..)
BUT then trying with another image it doesn't work anymore, output:
So, question(s):
There is some "magic rule" where I can extract a specific fuzz_factor for each image?
How could I use the image's right half to get that silhouette/contour?
Thanks guys

I post to close this question.
Thanks to Micka I made some progress, there are two variables that have high impact on output's quality:
fuzz_factor: which sets the color range for cv2.inRange()
max_contours: number of contours to draw (sorted by size)
High numbers are better until there are white zones that are not background, so next thing could be discard that ones.
import numpy as np
import cv2
# constants
fuzz_factor = 1
max_contours = -10
image_path = "9.png"
image = cv2.imread(image_path)
image = image[:, 0:512]
# background color boundaries
color = image[3,3]
upper = (color).astype(int)
lower = (color * (100 - fuzz_factor/2.0)/100).astype(int)
# create mask with specific colors
mask = cv2.inRange(image, lower, upper)
# get all contours
contours, _ = cv2.findContours(mask, mode = cv2.RETR_EXTERNAL, method = cv2.CHAIN_APPROX_NONE)
if(len(contours) > 1):
# get the [max_contours] biggest areas
contours = sorted(contours, key=cv2.contourArea)[max_contours:]
# mask where contours are filled
mask = np.zeros_like(image)
# draw contours and fill
cv2.drawContours(mask, contours, -1, color=[255,255,255], thickness= -1)
cv2.drawContours(image, contours, -1, 255, 2)
cv2.imshow("Result", np.hstack([image, mask]))
cv2.waitKey(0)

Efficient way to get ROI from original image using contours found in mask

the task that I'm trying to accomplish is isolating certain objects in an image through finding contours in the mask of the image, then taking each contour (based on area) and isolating it , and then using this contour to crop the same region in the original image, in order to get the pixel values of the region,
e.g.:
the code I wrote in order to get just one contour and then isolating it with the original pixel value:
import cv2
import matplotlib.pyplot as plt
import numpy as np
image = cv2.imread("./xxxx/xx.png")
mask = cv2.imread("./xxxx/xxx.png")
# making them the same size (function I wrote)
image, mask = resize_two_images(image,mask)
#grayscalling the mask (using cv2.cvtCOLOR)
mask = to_gray(mask)
# a function I wrote to display images using plt
display(image,"image: original image")
display(mask,"mask: mask of the image")
th, mask = cv2.threshold(mask, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)
contours, hierarchy = cv2.findContours(
mask, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
for i in range(len(contours)):
hier = hierarchy[0][i][3]
cnt = contours[i]
cntArea = cv2.contourArea(cnt)
if 1000 < cntArea < 2000:
break
#breaking because I'm just keeping the first contour that fills the condtion
# Creating two zero numpy array
img1 = np.zeros(image.shape, dtype=np.uint8)
img2 = img1.copy()
# drawing the contour which will basiclly give the edges of the object
cv2.drawContours(img1, [cnt], -1, (255,255,255), 2)
# drawing indise the the edges
cv2.fillPoly(img2, [cnt], (255,255,255))
# adding the filled poly with the drawn contour gives bigger object that
# contains the both the edges and the insides of the object
img1 = img1 + img2
display(img1,"img1: final")
res = np.bitwise_and(img1,image)
display(res,"res : the ROI with original pixel values")
#cropping the ROI (the object we want)
x,y,w,h = cv2.boundingRect(cnt)
# (de)increased values in order to get nonzero borders or lost pixels
res1 = res[y-1:y+h+1,x-1:x+w+1]
display(res1,"res1: cropped ROI")
The problem is that yes I found a way to do it for just one contour, but is there another way where I can do it more efficiently because per image there could be hundreds of contours.

It's not clear if you want to have just one image with all selected contours as the output, or one individual image per selected contour.
You could get one image with all selected contours in a efficient manner.
First select all the contours you want to work with, then, plot all the contours filling them with white color so you can use this as a mask, and then mask the original image:
selected_contours = [c for c in contours if cv2.contourArea(c) >= 2000]
# the last parameter, negative line thickness, fills the contour
mask = cv2.drawContours(img1, selected_contours, -1, (255,255,255), -1)
res = np.bitwise_and(mask,image)

cropping x-ray image to remove background

I have many x-ray scans and need to crop the scanned object from its background noise.
The files are in .png format and I am planning to use OpenCV Python for this task. I have seen some works with FindContours() but unsure that thresholding will work for this case.
Before Image:
After/Cropped Image:
Any suggested solution/code is appreciated.

Here is one way to do that in Python/OpenCV. It assumes you have the same excess border in all your images so that one can sort contours by area and skip the largest contour to get the second largest one.
Input:
import cv2
import numpy as np
# load image
img = cv2.imread("table_xray.jpg")
hh, ww = img.shape[:2]
# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# median filter
filt = cv2.medianBlur(gray, 15)
# threshold the filtered image and invert
thresh = cv2.threshold(filt, 64, 255, cv2.THRESH_BINARY)[1]
thresh = 255 - thresh
# find contours and store index with area in list
cntrs_info = []
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
index=0
for cntr in contours:
area = cv2.contourArea(cntr)
print(index, area)
cntrs_info.append((index,area))
index = index + 1
# sort contours by area
def takeSecond(elem):
return elem[1]
cntrs_info.sort(key=takeSecond, reverse=True)
# get bounding box of second largest contour skipping large border
index_second = cntrs_info[1][0]
x,y,w,h = cv2.boundingRect(contours[index_second])
print(index_second,x,y,w,h)
# crop input image
results = img[y:y+h,x:x+w]
# write result to disk
cv2.imwrite("table_xray_thresholded.png", thresh)
cv2.imwrite("table_xray_extracted.png", results)
cv2.imshow("THRESH", thresh)
cv2.imshow("RESULTS", results)
cv2.waitKey(0)
cv2.destroyAllWindows()
Filtered and Thresholded Image:
Cropped Result:

This is another possible solution. It uses the K-Channel of your input image, once converted to the CMYK color-space. The K (or Key) channel has most of the information of the black color, so it should be useful for segmenting the input image. After that, you can apply a heavy morphological chain to produce a good mask of the object. After that, cropping the object is very straightforward. Let's see the code:
# Imports
import cv2
import numpy as np
# Read image
imagePath = "D://opencvImages//"
inputImage = cv2.imread(imagePath+"jU6QA.jpg")
# Convert to float and divide by 255:
imgFloat = inputImage.astype(np.float) / 255.
# Calculate channel K:
kChannel = 1 - np.max(imgFloat, axis=2)
# Convert back to uint 8:
kChannel = (255*kChannel).astype(np.uint8)
The first bit of the program converts your image to the CMYK color-space and extracts the K channel. OpenCV has no direct conversion to this color-space, so a manual conversion is necessary. We need to be careful with the data types because there are float operations involved. The resulting image is this:
Pixels with black information are assigned an intensity close to 255. Now, let's threshold this image to get a binary mask. The threshold level is fixed:
# Threshold the image with a fixed thresh level
thresholdLevel = 200
_, binaryImage = cv2.threshold(kChannel, thresholdLevel, 255, cv2.THRESH_BINARY)
This produces the following binary image:
Alright. We need to isolate the object, however we have both the lines of the background and the "frame" around the image. Let's get rid of the lines first. We will apply a morphological Erosion. Then, we will remove the frame Flood-Filling with black color at two locations: upper left and bottom right of the image. After that, we will apply a Dilation to restore the object's original size. I wrapped these OpenCV functions inside custom functions that save me the typing of a couple of lines - These helper functions are presented at the end of the post. This is the approach:
# Perform Small Erosion:
binaryImage = morphoOperation(binaryImage, 3, 5, "Erode")
# Flood-Fill at two locations: Top left corner and bottom right:
(imageHeight, imageWidth) = binaryImage.shape[:2]
floodPositions = [(0, 0),(imageWidth-1, imageHeight-1)]
binaryImage = floodFill(binaryImage, floodPositions, 0)
# Perform Small Dilate:
binaryImage = morphoOperation(binaryImage, 3, 5, "Dilate")
This is the result:
Nice. We can improve the mask by applying a second morphological chain, this time with more iterations. Let's apply a Dilation to try and join the "holes" of the object, followed with a Erosion to, once again, restore the object's original size:
# Perform Big Dilate:
binaryImage = morphoOperation(binaryImage, 3, 10, "Dilate")
# Perform Big Erode:
binaryImage = morphoOperation(binaryImage, 3, 10, "Erode")
This yields the following result:
The gaps inside the object have been filled. Now, let's retrieve the contours on this mask to find the object's contour. I've additionally included an area filter. The mask is pretty clean by this point, so maybe this filter is not too necessary. Once the contour is located, we can crop the object from the original image:
# Find the contours on the binary image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# BGR image for drawing results:
binaryBGR = cv2.cvtColor(binaryImage, cv2.COLOR_GRAY2BGR)
# Look for the outer bounding boxes (no children):
for _, c in enumerate(contours):
# Get blob area:
currentArea = cv2.contourArea(c)
# Set a min area value:
minArea = 10000
if minArea < currentArea:
# Get the contour's bounding rectangle:
boundRect = cv2.boundingRect(c)
# Get the dimensions of the bounding rect:
rectX = boundRect[0]
rectY = boundRect[1]
rectWidth = boundRect[2]
rectHeight = boundRect[3]
# Set bounding rect:
color = (0, 255, 0)
cv2.rectangle( binaryBGR, (int(rectX), int(rectY)),
(int(rectX + rectWidth), int(rectY + rectHeight)), color, 5 )
cv2.imshow("Rects", binaryBGR)
# Crop original input:
currentCrop = inputImage[rectY:rectY + rectHeight, rectX:rectX + rectWidth]
cv2.imshow("Cropped", currentCrop)
cv2.waitKey(0)
The last step produces the following two images. The first is the object enclosed by a rectangle, the second one is the actual crop:
I also tested the algorithm with your second image, these are the final results:
Wow. Somebody brought a gun to the airport? That's not OK. These are the helper functions used earlier. This first function performs the morphological operations:
def morphoOperation(binaryImage, kernelSize, opIterations, opString):
# Get the structuring element:
morphKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Perform Operation:
if opString == "Dilate":
op = cv2.MORPH_DILATE
else:
if opString == "Erode":
op = cv2.MORPH_ERODE
outImage = cv2.morphologyEx(binaryImage, op, morphKernel, None, None, opIterations,
cv2.BORDER_REFLECT101)
return outImage
The second function performs Flood-Filling given a list of seed-points:
def floodFill(binaryImage, positions, color):
# Loop thru the positions list of tuples:
for p in range(len(positions)):
currentSeed = positions[p]
x = int(currentSeed[0])
y = int(currentSeed[1])
# Apply flood-fill:
cv2.floodFill(binaryImage, mask=None, seedPoint=(x, y), newVal=(color))
return binaryImage

Opencv not finding all contours

I'm trying to find the contours of this image, but the method findContours only returns 1 contour, the contour is highlighted in image 2. I'm trying to find all external contours like these circles where the numbers are inside. What am i doing wrong? What can i do to accomplish it?
image 1:
image 2:
Below is the relevant portion of my code.
thresh = cv2.threshold(image, 0, 255,
cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
When i change cv2.RETR_EXTERNAL to cv2.RETR_LIST it seems to detect the same contour twice or something like this. Image 3 shows when the border of circle is first detected and then it is detected again as shows image 4. I'm trying to find only outer borders of these circles. How can i accomplish that?
image 3
image 4

The problem is the flag cv2.RETR_EXTERNAL that you used in the function call. As described in the OpenCV documentation, this only returns the external contour.
Using the flag cv2.RETR_LIST you get all contours in the image. Since you try to detect rings, this list will contain the inner and the outer contour of these rings.
To filter the outer boundary of the circles, you could use cv2.contourArea() to find the larger of two overlapping contours.

I am not sure this is really what you expect nevertheless in case like this there is many way to help findContours to do its job.
Here is a way I use frequently.
Convert your image to gray
Ig = cv2.cvtColor(I,cv2.COLOR_BGR2GRAY)
Thresholding
The background and foreground values looklike quite uniform in term of colours but locally they are not so I apply an thresholding based on Otsu's method in order to binarise the intensities.
_,It = cv2.threshold(Ig,0,255,cv2.THRESH_OTSU)
Sobel magnitude
In order to extract only the contours I process the magnitude of the Sobel edges detector.
sx = cv2.Sobel(It,cv2.CV_32F,1,0)
sy = cv2.Sobel(It,cv2.CV_32F,0,1)
m = cv2.magnitude(sx,sy)
m = cv2.normalize(m,None,0.,255.,cv2.NORM_MINMAX,cv2.CV_8U)
thinning (optional)
I use the thinning function which is implemented in ximgproc.
The interest of the thining is to reduce the contours thickness to as less pixels as possible.
m = cv2.ximgproc.thinning(m,None,cv2.ximgproc.THINNING_GUOHALL)
Final Step findContours
_,contours,hierarchy = cv2.findContours(m,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
disp = cv2.merge((m,m,m)
disp = cv2.drawContours(disp,contours,-1,hierarchy=hierarchy,color=(255,0,0))
Hope it help.
I think an approach based on SVM or a CNN might be more robust.
You can find an example here.
This one may also be interesting.
-EDIT-
I found a let say easier way to reach your goal.
Like previously after loading the image applying a threshold ensure that the image is binary.
By reversing the image using a bitwise not operation the contours become white over a black background.
Applying cv2.connectedComponentsWithStats return (among others) a label matrix in which each connected white region in the source has been assign a unique label.
Then applying findContours based on the labels it is possible give the external contours for every areas.
import numpy as np
import cv2
from matplotlib import pyplot as plt
I = cv2.imread('/home/smile/Downloads/ext_contours.png',cv2.IMREAD_GRAYSCALE)
_,I = cv2.threshold(I,0.,255.,cv2.THRESH_OTSU)
I = cv2.bitwise_not(I)
_,labels,stats,centroid = cv2.connectedComponentsWithStats(I)
result = np.zeros((I.shape[0],I.shape[1],3),np.uint8)
for i in range(0,labels.max()+1):
mask = cv2.compare(labels,i,cv2.CMP_EQ)
_,ctrs,_ = cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
result = cv2.drawContours(result,ctrs,-1,(0xFF,0,0))
plt.figure()
plt.imshow(result)
P.S. Among the outputs return by the function findContours there is a hierachy matrix.
It is possible to reach the same result by analyzing that matrix however it is a little bit more complex as explain here.

Instead of finding contours, I would suggest applying the Hough circle transform using the appropriate parameters.
Finding contours poses a challenge. Once you invert the binary image the circles are in white. OpenCV finds contours both along the outside and the inside of the circle. Moreover since there are letters such as 'A' and 'B', contours will again be found along the outside of the letters and within the holes. You can find contours using the appropriate hierarchy criterion but it is still tedious.
Here is what I tried by finding contours and using hierarchy:
Code:
#--- read the image, convert to gray and obtain inverse binary image ---
img = cv2.imread('keypad.png', 1)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV|cv2.THRESH_OTSU)
#--- find contours ---
_, contours, hierarchy = cv2.findContours(binary, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
#--- copy of original image ---
img2 = img.copy()
#--- select contours having a parent contour and append them to a list ---
l = []
for h in hierarchy[0]:
if h[0] > -1 and h[2] > -1:
l.append(h[2])
#--- draw those contours ---
for cnt in l:
if cnt > 0:
cv2.drawContours(img2, [contours[cnt]], 0, (0,255,0), 2)
cv2.imshow('img2', img2)
For more info on contours and their hierarchical relationship please refer this
UPDATE
I have a rather crude way to ignore unwanted contours. Find the average area of all the contours in list l and draw those that are above the average:
Code:
img3 = img.copy()
a = 0
for j, i in enumerate(l):
a = a + cv2.contourArea(contours[i])
mean_area = int(a/len(l))
for cnt in l:
if (cnt > 0) & (cv2.contourArea(contours[cnt]) > mean_area):
cv2.drawContours(img3, [contours[cnt]], 0, (0,255,0), 2)
cv2.imshow('img3', img3)

You can select only the outer borders by this function:
def _select_contours(contours, hierarchy):
"""select contours of the second level"""
# find the border of the image, which has no father
father_i = None
for i, h in enumerate(hierarchy):
if h[3] == -1:
father_i = i
break
# collect its sons
new_contours = []
for c, h in zip(contours, hierarchy):
if h[3] == father_i:
new_contours.append(c)
return new_contours
Note that you should use cv2.RETR_TREE in cv2.findContours() to get the contours and hierarchy.