Develop Reference

how do I get the color that surrounds the Contour from outside, and how to fill the inside of a contour with a color?

I wrote a program that takes a picture as an input and detects the written text
I need to edit my program to add two more functionalities
First
since I already was able to get the Contour coordinates(done), I want to know how do I get the color of the background that surrounds the contour line of each character ( at multiple points because I want to calculate the average of RGB color values surrounding that character )
I just don't know if there is already a function to do what I want to do or if there is a specific approach that I should follow
second,
I tried to fill the inside of the Contour but it did not work, can someone help me to know why
I did try both thickness=-1 and thickness=cv2.FILLE in my cv2.drawContours, nothing worked, clearly only the control line is blue but what is inside is not blue
I know that the code is not perfect and there are so many comments but it is because I'm Kim trying new stuff
thank you in advance to everyone who's going to help
import cv2
import pytesseract
import numpy as np
from PIL import ImageGrab
import pyautogui
def Contour(img):
Contours,hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for cnt in Contours:
area = cv2.contourArea(cnt)
# print(area)
# if area<150: could be useded to be cpecific
# we need a copy of the image so we dont draw on the original or thickness=-1
cv2.drawContours(imgcpy, cnt, -1, (255,0,0), thickness=-1)
#pytesseract.pytesseract.tesseract_cmd= 'C:\\Program Files\\Tesseract-OCR\\tesseract.exe'
cap= cv2
img = cv2.imread('D:\\opencv\\R\\img\\lec\\3.png')
imgcpy = img.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# for japanese txt = pytesseract.image_to_string(gray, lang='jpn')
txt = pytesseract.image_to_string(gray)
imgblur = cv2.GaussianBlur(img, (7, 7), 0)
imgcanny = cv2.Canny(img, 200, 200)
#imgDialation = cv2.dilate(imgcanny, kernel, iterations=1)
#imgeroded = cv2.erode(imgDialation, kernel, iterations=1)
imBlank = np.zeros_like(img)
# waste of time only the same picture work -_-
# imstack = np.hstack((gray,gray,gray))
Contour(imgcanny)
cv2.imshow('Contour', imgcpy)
#img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
print(txt)
print(Contour(imgcanny))
#cv2.imshow('test', img)
cv2.waitKey()

The "first" issue is solved by
getting a mask, either
using connectedComponents or
by filling a mask using drawContours into np.zeros((height, width), dtype=np.uint8)
dilate the mask, then
subtract the mask from the dilated version (literal subtraction will work, but so do bitwise operations)
The result is a mask that contains pixels just outside of the contour.
Use this mask to get those pixel values:
# have `mask` somehow, which should be dtype uint8
dilated = cv.dilate(mask)
surrounding = dilated - mask
values = img[surrounding != 0] # numpy indexing using a boolean array
meancolor = values.mean(axis=(0,1)) # sum/average over both axes
The "second" issue is that you pass a single contour to drawContours, which expects a list of contours.
Change your code to pass [cnt] instead of just cnt:
cv2.drawContours(imgcpy, [cnt], -1, (255,0,0), thickness=-1)
If you only pass cnt, it will interpret the list of points (a contour is a list of points) as a list of contours, hence it will interpret each point as a contour... and that's just a point, not a contour.

How to remove small contours attached to another big one

I'm doing cell segmentation, so I'm trying to code a function that removes all minor contours around the main one in order to do a mask.
That happens because I load an image with some color markers:
The problem is when I do threshold, it assumes that "box" between the color markers as a part of the main contour.
As you may see in my code, I don't directly pass color image to grays because the red turns black but there are other colors too, at least 8, and always different in each image. I've got thousands of images like this where just one cell is displayed, but in most of it, there are always outsiders contours attached. My goal is to come to a function that gives a binary image of a single cell for each image input like this. So I'm starting with this code:
import cv2 as cv
cell1 = cv.imread(image_cell, 0)
imgray = cv.cvtColor(cell1,cv.COLOR_BGR2HSV)
imgray = cv.cvtColor(imgray,cv.COLOR_BGR2GRAY)
ret,thresh_binary = cv.threshold(imgray,107,255,cv.THRESH_BINARY)
cnts= cv.findContours(image =cv.convertScaleAbs(thresh_binary) , mode =
cv.RETR_TREE,method = cv.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv.drawContours(thresh_binary,[c], 0, (255,255,255), -1)
kernel = cv.getStructuringElement(cv.MORPH_RECT, (3,3))
opening = cv.morphologyEx(thresh_binary, cv.MORPH_OPEN, kernel,
iterations=2) # erosion followed by dilation
Summing up, how do I get just the red contour from image 1?

So another approach, without color ranges.
A couple of things are not going right in your code I think. First, you are drawing the contours on thresh_binary, but that already has the outer lines of the other cells as well - the lines you are trying to get rid off. I think that is why you use opening(?) while in this case you shouldn't.
To fix things, first a little information on how findContours works. findContours starts looking for white shapes on a black background and then looks for black shapes inside that white contour and so on. That means that the white outline of the cells in the thresh_binary are detected as a contour. Inside of it are other contours, including the one you want. docs with examples
What you should do is first look only for contours that have no contours inside of them. The findContours also returns a hierarchy of contours. It indicates whether a contour has 'childeren'. If it has none (value: -1) then you look at the size of the contour and disregard the ones that are to small. You could also just look for the largest, as that is probably the one you want. Finally you draw the contour on a black mask.
Result:
Code:
import cv2 as cv
import numpy as np
# load image as grayscale
cell1 = cv.imread("PjMQR.png",0)
# threshold image
ret,thresh_binary = cv.threshold(cell1,107,255,cv.THRESH_BINARY)
# findcontours
contours, hierarchy = cv.findContours(image =thresh_binary , mode = cv.RETR_TREE,method = cv.CHAIN_APPROX_SIMPLE)
# create an empty mask
mask = np.zeros(cell1.shape[:2],dtype=np.uint8)
# loop through the contours
for i,cnt in enumerate(contours):
# if the contour has no other contours inside of it
if hierarchy[0][i][2] == -1 :
# if the size of the contour is greater than a threshold
if cv2.contourArea(cnt) > 10000:
cv.drawContours(mask,[cnt], 0, (255), -1)
# display result
cv2.imshow("Mask", mask)
cv2.imshow("Img", cell1)
cv2.waitKey(0)
cv2.destroyAllWindows()
Note: I used the image you uploaded, your image probably has far fewer pixels, so a smaller contourArea
Note2: enumerate loops through the contours, and returns both a contour and an index for each loop

Actually, in your code the 'box' is a legitimate extra contour. And you draw all contours on the final image, so that includes the 'box'. This could cause issues if any of the other colored cells are fully in the image.
A better approach is to separate out the color you want. The code below creates a binary mask that only displays the pixels that are in the defined range of red colors. You can use this mask with findContours.
Result:
Code:
import cv2
# load image
img = cv2.imread("PjMQR.png")
# Convert HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# define range of red color in HSV
lower_val = np.array([0,20,0])
upper_val = np.array([15,255,255])
# Threshold the HSV image to get only red colors
mask = cv2.inRange(hsv, lower_val, upper_val)
# display image
cv2.imshow("Mask", mask)
cv2.waitKey(0)
cv2.destroyAllWindows()
This code can help you understand how the different values in this process (HSV with inRange) works. inRange docs

Get the average color inside a contour with Open CV

So I decided to get started learning Open CV and Python together!
My first project is to detect moving objects on a relatively still background and then detect their average color to sort them. There are at least 10 objects to detect and I am processing a colored video.
So far I managed to remove the background, identify the contours (optionally get the center of each contour) but now I am struggling getting the average or mean color inside of each contour. There are some topics about this kind of question but most of them are written in C. Apparently I could use cv.mean() but I can't get a working mask to feed in this function. I guess it's not so difficult but I am stuck there... Cheers!
import numpy as np
import cv2
video_path = 'test.h264'
cap = cv2.VideoCapture(video_path)
fgbg = cv2.createBackgroundSubtractorMOG2()
while (cap.isOpened):
ret, frame = cap.read()
if ret==True:
fgmask = fgbg.apply(frame)
(contours, hierarchy) = cv2.findContours(fgmask, cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
for c in contours:
if cv2.contourArea(c) > 2000:
cv2.drawContours(frame, c, -1, (255,0,0), 3)
cv2.imshow('foreground and background',fgmask)
cv2.imshow('rgb',frame)
key = cv2.waitKey(1) & 0xFF
if key == ord("q"):
break
cap.release()
cv2.destroyAllWindows()

You can create a mask by first creating a new image with the same dimensions as your input image and pixel values set to zero.
You then draw the contour(s) onto this image with pixel value 255. The resulting image can be used as a mask.
mask = np.zeros(frame.shape, np.uint8)
cv2.drawContours(mask, c, -1, 255, -1)
mask can then be used as a parameter to cv.mean like
mean = cv.mean(frame, mask=mask)
Just one word of caution, the mean of RGB colors does not always make sense. Maybe try converting to HSV color space and solely use the H channel for detecting the color of your objects.

Solution on an image
1) find contour (in this case rectangle, contour that is not rectangle is much harder to make)
2) find coordiantes of contour
3) cut the image from contour
4) sum individual channels and divide them by number of pixels in it ( or with mean function)
import numpy as np
import cv2
img = cv2.imread('my_image.jpg',1)
cp = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(cp,150,255,0)
cv2.imshow('img',thresh)
cv2.waitKey(0)
im2,contours,hierarchy = cv2.findContours(thresh.astype(np.uint8), 1, 2)
cnts = contours
for cnt in cnts:
if cv2.contourArea(cnt) >800: # filter small contours
x,y,w,h = cv2.boundingRect(cnt) # offsets - with this you get 'mask'
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
cv2.imshow('cutted contour',img[y:y+h,x:x+w])
print('Average color (BGR): ',np.array(cv2.mean(img[y:y+h,x:x+w])).astype(np.uint8))
cv2.waitKey(0)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
To remove noise, you can just take center of the contour, and take smaller rectangle to examin.
For non rectangular contour, look at cv2.fillPoly function -> Cropping non rectangular contours. But its a bit slow algorithm (but nothing limiting)
If you are interested in non rectangular contour, you will have to be careful about doing mean, because you will need mask and the mask/background is always rectangular so you will be doing mean on something you dont want

Opencv not finding all contours

I'm trying to find the contours of this image, but the method findContours only returns 1 contour, the contour is highlighted in image 2. I'm trying to find all external contours like these circles where the numbers are inside. What am i doing wrong? What can i do to accomplish it?
image 1:
image 2:
Below is the relevant portion of my code.
thresh = cv2.threshold(image, 0, 255,
cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
When i change cv2.RETR_EXTERNAL to cv2.RETR_LIST it seems to detect the same contour twice or something like this. Image 3 shows when the border of circle is first detected and then it is detected again as shows image 4. I'm trying to find only outer borders of these circles. How can i accomplish that?
image 3
image 4

The problem is the flag cv2.RETR_EXTERNAL that you used in the function call. As described in the OpenCV documentation, this only returns the external contour.
Using the flag cv2.RETR_LIST you get all contours in the image. Since you try to detect rings, this list will contain the inner and the outer contour of these rings.
To filter the outer boundary of the circles, you could use cv2.contourArea() to find the larger of two overlapping contours.

I am not sure this is really what you expect nevertheless in case like this there is many way to help findContours to do its job.
Here is a way I use frequently.
Convert your image to gray
Ig = cv2.cvtColor(I,cv2.COLOR_BGR2GRAY)
Thresholding
The background and foreground values looklike quite uniform in term of colours but locally they are not so I apply an thresholding based on Otsu's method in order to binarise the intensities.
_,It = cv2.threshold(Ig,0,255,cv2.THRESH_OTSU)
Sobel magnitude
In order to extract only the contours I process the magnitude of the Sobel edges detector.
sx = cv2.Sobel(It,cv2.CV_32F,1,0)
sy = cv2.Sobel(It,cv2.CV_32F,0,1)
m = cv2.magnitude(sx,sy)
m = cv2.normalize(m,None,0.,255.,cv2.NORM_MINMAX,cv2.CV_8U)
thinning (optional)
I use the thinning function which is implemented in ximgproc.
The interest of the thining is to reduce the contours thickness to as less pixels as possible.
m = cv2.ximgproc.thinning(m,None,cv2.ximgproc.THINNING_GUOHALL)
Final Step findContours
_,contours,hierarchy = cv2.findContours(m,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
disp = cv2.merge((m,m,m)
disp = cv2.drawContours(disp,contours,-1,hierarchy=hierarchy,color=(255,0,0))
Hope it help.
I think an approach based on SVM or a CNN might be more robust.
You can find an example here.
This one may also be interesting.
-EDIT-
I found a let say easier way to reach your goal.
Like previously after loading the image applying a threshold ensure that the image is binary.
By reversing the image using a bitwise not operation the contours become white over a black background.
Applying cv2.connectedComponentsWithStats return (among others) a label matrix in which each connected white region in the source has been assign a unique label.
Then applying findContours based on the labels it is possible give the external contours for every areas.
import numpy as np
import cv2
from matplotlib import pyplot as plt
I = cv2.imread('/home/smile/Downloads/ext_contours.png',cv2.IMREAD_GRAYSCALE)
_,I = cv2.threshold(I,0.,255.,cv2.THRESH_OTSU)
I = cv2.bitwise_not(I)
_,labels,stats,centroid = cv2.connectedComponentsWithStats(I)
result = np.zeros((I.shape[0],I.shape[1],3),np.uint8)
for i in range(0,labels.max()+1):
mask = cv2.compare(labels,i,cv2.CMP_EQ)
_,ctrs,_ = cv2.findContours(mask,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
result = cv2.drawContours(result,ctrs,-1,(0xFF,0,0))
plt.figure()
plt.imshow(result)
P.S. Among the outputs return by the function findContours there is a hierachy matrix.
It is possible to reach the same result by analyzing that matrix however it is a little bit more complex as explain here.

Instead of finding contours, I would suggest applying the Hough circle transform using the appropriate parameters.
Finding contours poses a challenge. Once you invert the binary image the circles are in white. OpenCV finds contours both along the outside and the inside of the circle. Moreover since there are letters such as 'A' and 'B', contours will again be found along the outside of the letters and within the holes. You can find contours using the appropriate hierarchy criterion but it is still tedious.
Here is what I tried by finding contours and using hierarchy:
Code:
#--- read the image, convert to gray and obtain inverse binary image ---
img = cv2.imread('keypad.png', 1)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
_, binary = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV|cv2.THRESH_OTSU)
#--- find contours ---
_, contours, hierarchy = cv2.findContours(binary, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)
#--- copy of original image ---
img2 = img.copy()
#--- select contours having a parent contour and append them to a list ---
l = []
for h in hierarchy[0]:
if h[0] > -1 and h[2] > -1:
l.append(h[2])
#--- draw those contours ---
for cnt in l:
if cnt > 0:
cv2.drawContours(img2, [contours[cnt]], 0, (0,255,0), 2)
cv2.imshow('img2', img2)
For more info on contours and their hierarchical relationship please refer this
UPDATE
I have a rather crude way to ignore unwanted contours. Find the average area of all the contours in list l and draw those that are above the average:
Code:
img3 = img.copy()
a = 0
for j, i in enumerate(l):
a = a + cv2.contourArea(contours[i])
mean_area = int(a/len(l))
for cnt in l:
if (cnt > 0) & (cv2.contourArea(contours[cnt]) > mean_area):
cv2.drawContours(img3, [contours[cnt]], 0, (0,255,0), 2)
cv2.imshow('img3', img3)

You can select only the outer borders by this function:
def _select_contours(contours, hierarchy):
"""select contours of the second level"""
# find the border of the image, which has no father
father_i = None
for i, h in enumerate(hierarchy):
if h[3] == -1:
father_i = i
break
# collect its sons
new_contours = []
for c, h in zip(contours, hierarchy):
if h[3] == father_i:
new_contours.append(c)
return new_contours
Note that you should use cv2.RETR_TREE in cv2.findContours() to get the contours and hierarchy.

Extracting text from inside a circular border

I'm trying to develop a script using Python and OpenCV to detect some highlighted regions on a scanned instrumentation diagram and output text using Tesseract's OCR function. My workflow is first to detect the general vicinity of the region of interest, and then apply processing steps to remove everything aside from the blocks of text (lines, borders, noise). The processed image is then feed into Tesseract's OCR engine.
This workflow is works on about half of the images, but fails on the rest due to the text touching the borders. I'll show some examples of what I mean below:
Step 1: Find regions of interest by creating a mask using InRange with the color range of the highlighter.
Step 2: Contour regions of interest, crop and save to file.
--- Referenced code begins here ---
Step 3: Threshold image and apply Canny Edge Detection
Step 4: Contour the edges and filter them into circular shape using cv2.approxPolyDP and looking at ones with vertices greater than 8. Taking the first or second largest contour usually corresponds to the inner edge.
Step 5: Using masks and bitwise operations, everything inside contour is transferred to a white background image. Dilation and erosion is applied to de-noise the image and create the final image that gets fed into the OCR engine.
import cv2
import numpy as np
import pytesseract
pytesseract.pytesseract.tesseract_cmd = 'C:/Program Files (x86)/Tesseract-OCR/tesseract'
d_path = "Test images\\"
img_name = "cropped_12.jpg"
img = cv2.imread(d_path + img_name) # Reads the image
## Resize image before calculating contour
height, width = img.shape[:2]
img = cv2.resize(img,(2*width,2*height),interpolation = cv2.INTER_CUBIC)
img_orig = img.copy() # Makes copy of original image
img = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) # Convert to grayscale
# Apply threshold to get binary image and write to file
_, img = cv2.threshold(img,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Edge detection
edges = cv2.Canny(img,100,200)
# Find contours of mask threshold
_, contours, hierarchy = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
# Find contours associated w/ polygons with 8 sides or more
cnt_list = []
area_list = [cv2.contourArea(c) for c in contours]
for j in contours:
poly_pts = cv2.approxPolyDP(j,0.01*cv2.arcLength(j,True),True)
area = cv2.contourArea(j)
if (len(poly_pts) > 8) & (area == max(area_list)):
cnt_list.append(j)
cv2.drawContours(img_orig, cnt_list, -1, (255,0,0), 2)
# Show contours
cv2.namedWindow('Show',cv2.WINDOW_NORMAL)
cv2.imshow("Show",img_orig)
cv2.waitKey()
cv2.destroyAllWindows()
# Zero pixels outside circle
mask = np.zeros(img.shape).astype(img.dtype)
cv2.fillPoly(mask, cnt_list, (255,255,255))
mask_inv = cv2.bitwise_not(mask)
a = cv2.bitwise_and(img,img,mask = mask)
wh_back = np.ones(img.shape).astype(img.dtype)*255
b = cv2.bitwise_and(wh_back,wh_back,mask = mask_inv)
res = cv2.add(a,b)
# Get rid of noise
kernel = np.ones((2, 2), np.uint8)
res = cv2.dilate(res, kernel, iterations=1)
res = cv2.erode(res, kernel, iterations=1)
# Show final image
cv2.namedWindow('result',cv2.WINDOW_NORMAL)
cv2.imshow("result",res)
cv2.waitKey()
cv2.destroyAllWindows()
When code works, these are the images that get outputted:
Working
However, in the instances where the text touches the circular border, the code assumes part of the text is part of the larger contour and ignores the last letter. For example:
Not working
Are there any processing steps that can help me bypass this problem? Or perhaps a different approach? I've tried using Hough Circle Transforms to try to detect the borders, but they're quite finicky and doesn't work as well as contouring.
I'm quite new to OpenCV and Python so any help would be appreciated.

If the Hough circle transform didn't work for you I think you're best option will be to approximate the boarder shape. The best method I know for that is: Douglas-Peucker algorithm which will make your contour simpler by reducing the perimeter on pics.
You can check this reference file from OpenCV to see the type of post processing you can apply to your boarder. They also mention Douglas-Peucker:
OpenCV boarder processing

Just a hunch. After OTSU thresholding. Erode and dilate the image. This will result in vanishing of very thin joints. The code for the same is below.
kernel = np.ones((5,5),np.uint8)
th3 = cv2.erode(th3, kernel,iterations=1)
th3 = cv2.dilate(th3, kernel,iterations=1)
Let me know how it goes. I have couple more idea if this did not work.