Develop Reference

Detecting Grid blocks using Computer vision in realtime (Open CV, Python)

I want to detect the centroid of individual blocks in the following grids for path planning. The idea is that a central navigation system like overhead camera will detect the blocks of grids along with the bot and help in navigation. Till now I have tried Hough lines Probabilistic and Harris corner detection but both of them either detect extra points or fail in real world scenario. I want to detect the blocks in real time and number them. Those numbering should not change or the whole path planning will be messed up.
Is there any solution to this problem that I missed.
thanks in advance

You need to learn how to eliminate noise. This is not a complete answer. The more time you spend and learn, the better your results will be.
import cv2
import numpy as np
import sys
# Load source as grayscale
im = cv2.imread(sys.path[0]+'/im.jpg', cv2.IMREAD_GRAYSCALE)
H, W = im.shape[:2]
# Convert im to black and white
im = cv2.adaptiveThreshold(
im, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 21, 2)
# Remove noise
im = cv2.medianBlur(im, 11)
im = cv2.erode(im, np.ones((15, 15)))
# Fill the area around the shape
im = ~im
mask = np.zeros((H+2, W+2), np.uint8)
cv2.floodFill(im, mask, (0, 0), 255)
cv2.floodFill(im, mask, (W-1, 0), 255)
cv2.floodFill(im, mask, (0, H-1), 255)
cv2.floodFill(im, mask, (W-1, H-1), 255)
# Remove noise again
im = cv2.dilate(im, np.ones((15, 15)))
# Find the final blocks
cnts, _ = cv2.findContours(~im, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for c in cnts:
x, y, w, h = cv2.boundingRect(c)
cv2.circle(im, (x+w//2, y+h//2), max(w, h)//2, 127, 5)
print("Found any: ", len(cnts) > 0)
# Save the output
cv2.imwrite(sys.path[0]+'/im_.jpg', im)

How can I create bounding boxes/contour around the outer object only - Python OpenCV

So I've been trying to make bounding boxes around a couple of fruits that I made in paint. I'm a total beginner to opencv so I watched a couple tutorials and the code that I typed made, makes contours around the object and using that I create bounding boxes. However it makes way too many tiny contours and bounding boxes. For example, heres the initial picture:
and heres the picture after my code runs:
However this is what I want:
Heres my code:
import cv2
import numpy as np
img = cv2.imread(r'C:\Users\bob\Desktop\coursera\coursera-2021\project2\fruits.png')
grayscaled = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(grayscaled, (7,7), 1)
canny = cv2.Canny(blur, 50, 50)
img1 = img.copy()
all_pics = []
contours, Hierarchy = cv2.findContours(canny, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
peri = cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, 0.02*peri, True)
x, y, w, h = cv2.boundingRect(approx)
bob = img [y:y+h, x:x+w]
all_pics.append((x, bob))
cv2.rectangle(img1, (x, y), (x+w, y+h), (0, 255, 0), 2)
cv2.imshow("title", img1)
cv2.waitKey(0)

I think FloodFill in combination with FindContours can help you:
import os
import cv2
import numpy as np
# Read original image
dir = os.path.abspath(os.path.dirname(__file__))
im = cv2.imread(dir+'/'+'im.png')
# convert image to Grayscale and Black/White
gry = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
cv2.imwrite(dir+'/im_1_grayscale.png',gry)
bw=cv2.threshold(gry, 127, 255, cv2.THRESH_BINARY)[1]
cv2.imwrite(dir+'/im_2_black_white.png',bw)
# Use floodfill to identify outer shape of objects
imFlood = bw.copy()
h, w = bw.shape[:2]
mask = np.zeros((h+2, w+2), np.uint8)
cv2.floodFill(imFlood, mask, (0,0), 0)
cv2.imwrite(dir+'/im_3_floodfill.png',imFlood)
# Combine flood filled image with original objects
imFlood[np.where(bw==0)]=255
cv2.imwrite(dir+'/im_4_mixed_floodfill.png',imFlood)
# Invert output colors
imFlood=~imFlood
cv2.imwrite(dir+'/im_5_inverted_floodfill.png',imFlood)
# Find objects and draw bounding box
cnts, _ = cv2.findContours(imFlood, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
for cnt in cnts:
peri = cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, 0.02*peri, True)
x, y, w, h = cv2.boundingRect(approx)
bob = im[y:y+h, x:x+w]
cv2.rectangle(im, (x, y), (x+w, y+h), (0, 255, 0), 2)
# Save final image
cv2.imwrite(dir+'/im_6_output.png',im)
Grayscale and black/white:
Add flood fill:
Blend original objects with the flood filled version:
Invert the colors of blended version:
Final output:

his guys answer was just what I was looking for. If anyone runs through this problem try this:
Draw contours around objects with OpenCV

Detecting the edge of a colorful picture OpenCV

I am new to CV and I just learned how to detect the edge of a paper. I want to try something more complicated. So I make a screenshot from a movie website and want to detect the poster from the website. It works well if the background color is different from the poster. But when they are similar in color, I can't find the edge of the picture by
cv2.findContours()
The original Picture is:
Poster
And what I do is:
img = cv2.imread('pic5.jpg')
orig = img.copy()
image = orig
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
binary = cv2.medianBlur(gray,3)
# blur = cv2.GaussianBlur(binary, (5, 5), 0)
# ret, binary = cv2.threshold(blur,127,255,cv2.THRESH_TRUNC)
edged = cv2.Canny(binary, 3, 30)
show(edged)
# detect edge
contours, hierarchy = cv2.findContours(edged.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cnts = sorted(contours, key=cv2.contourArea, reverse=True)[:5]
#
for c in cnts:
# approx
peri = cv2.arcLength(c, True)
eps = 0.02
approx = cv2.approxPolyDP(c, eps*peri, True)
# detect square (4 points)
if len(approx) == 4:
screenCnt = approx
break
res = cv2.drawContours(image, [screenCnt], -1, (0, 255, 0), 2)
show(orig)
And the result is:
after preprocess
What I detect
I don't know if this method works. Is it possible to detect the square part based on the background color (regardless of the poster's color)?

You may continue with the edged result, and use closing morphological operation for closing small gaps.
Instead of searching for a rectangle using approxPolyDP, I suggest you to find the bounding rectangle of the largest connected component (or largest contour).
In my code sample, I replaced findContours with connectedComponentsWithStats due to the external boundary line.
You may use opening morphological operation to get rid of the external line (and use continue using findContours).
You may also use approxPolyDP for refining the result.
Here is the code sample:
import numpy as np
import cv2
img = cv2.imread('pic5.png')
orig = img.copy()
image = orig
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
binary = cv2.medianBlur(gray, 3)
edged = cv2.Canny(binary, 3, 30)
edged = cv2.morphologyEx(edged, cv2.MORPH_CLOSE, np.ones((5,5))) # Close small gaps
#contours, hierarchy = cv2.findContours(edged, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
#c = max(contours, key=cv2.contourArea) # Get the largest contour
#x, y, w, h = cv2.boundingRect(c) # Find bounding rectangle.
nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(edged, 8) # finding components
# https://stackoverflow.com/a/61662694/4926757
# Find the largest non background component.
# Note: range() starts from 1 since 0 is the background label.
max_label, max_size = max([(i, stats[i, cv2.CC_STAT_AREA]) for i in range(1, nb_components)], key=lambda x: x[1])
# Find bounding rectangle of largest connected component.
x = stats[max_label, cv2.CC_STAT_LEFT]
y = stats[max_label, cv2.CC_STAT_TOP]
w = stats[max_label, cv2.CC_STAT_WIDTH]
h = stats[max_label, cv2.CC_STAT_HEIGHT]
res = image.copy()
cv2.rectangle(res, (x, y), (x+w, y+h), (0, 255, 0), 2) # Draw a rectangle
cv2.imshow('edged', edged)
cv2.imshow('res', res)
cv2.waitKey()
cv2.destroyAllWindows()
Results:
edged:
res:

How to scale down symbols in an image using Open CV

I'm trying to use Open CV to scale down numbers in an image. I currently am able to identify the contours but I am having trouble figuring out how to scale down the numbers once I have identified them.
Here is an example image:
Here are the contours I have identified:
Here is the code I am using to achieve this:
import cv2
image = cv2.imread("numbers.png")
edged = cv2.Canny(image, 10, 250)
# applying closing function
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7, 7))
closed = cv2.morphologyEx(edged, cv2.MORPH_CLOSE, kernel)
_, cnts,_ = cv2.findContours(closed.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
contours = []
for c in cnts:
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.02 * peri, True)
contours.append(approx)
cv2.drawContours(image, [approx], -1, (0, 255, 0), 2)
cv2.imshow("Output", image)
cv2.waitKey(0)
I want to be able to use the contours to scale down the numbers without affecting the size of the image. Is this possible? Thanks!

Assuming you have an input image named "numbers.png".
First of all, import useful libraries and load the input image:
import cv2
import numpy as np
img = cv2.imread("./numbers.png", 1)
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
Secondly, you need to binarize the input image and find the external contours of the numbers:
_, im_th = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
_, contours, _ = cv2.findContours(255-im_th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
So you can see the detected contours will be around the numbers.
Thirdly, find the relative bounding boxes around the numbers and find the middle point coordinates of the boxes (I assume the numbers should be resized and put in the center of the bottom line):
number_imgs = []
number_btm_mid_pos = []
for cnt in contours:
(x, y, w, h) = cv2.boundingRect(cnt)
number_imgs.append(img[y:y+h, x:x+w])
number_btm_mid_pos.append((int(x+w/2), y+h))
Finally, resize the numbers, put them back to the image, and display the result:
# resize images and put it back
output_img = np.ones_like(img) * 255
resize_ratio = 0.5
for (i, num_im) in enumerate(number_imgs):
num_im = cv2.resize(num_im, (0,0), fx=resize_ratio, fy=resize_ratio)
(img_h, img_w) = num_im.shape[:2]
# x1, y1, x2, y2
btm_x, btm_y = number_btm_mid_pos[i]
x1 = btm_x - int(img_w / 2)
y1 = btm_y - img_h
x2 = x1 + img_w
y2 = y1 + img_h
output_img[y1:y2, x1:x2] = num_im
cv2.imshow("Output Image", output_img)
cv2.imshow("Original Input", img)
cv2.waitKey()
You can adjust the variable "resize_ratio" to make sure the ratio is what you expected. The result should be something like this image here:
You may notice the last number "10" is splitting apart. It is because the "1 0" was recognized as two separate digits. To make it perfect, it is possible to write some code to test the gap/distance between every two digits. However, that would be not closely relevant, and a bit hard to generalize the solution based on the limited testing input. So I stop here.
Anyway, good luck and have fun.

Recognizing handwritten digits off a scanned image

I'm trying to read all of the handwritten digits in the scanned image here
I tried looking through pixel-by-pixel using PIL, cropping the sub-images, then feeding them through a neural network, but the regions that were being cropped never quite lined up and led to a lot of inaccuracy.
I've also tried using OpenCV to find all the grey squares, then crop the images and feed them through a neural network, but I couldn't seem to get it to find all or even only miss a few; it would miss ~30% of squares. (I'm not very experienced with OpenCV, so I could be messing something up)
So I'm just looking for a potential idea/solution for this problem, so any suggestions would be appreciated, thanks in advance!

I assume the input image name is "sqaures.jpg"
First of all, import required libraries and load image in both RGB and Gray format:
import cv2
import numpy as np
image = cv2.imread("squares.jpg", 1)
image_gray = cv2.imread("squares.jpg", 0)
Then, we perform a simple operation to clean some noise from the input image using np.where() function:
image_gray = np.where(image_gray > 240, 255, image_gray)
image_gray = np.where(image_gray <= 240, 0, image_gray)
Because we want to grab the whole square regions from the image. We need to blur the image a little bit before performing the adaptive thresholding method:
image_gray = cv2.blur(image_gray, (5, 5))
im_th = cv2.adaptiveThreshold(image_gray, 255,
cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
cv2.THRESH_BINARY, 115, 1)
kernal = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3))
im_th = cv2.morphologyEx(im_th, cv2.MORPH_OPEN, kernal, iterations=3)
Use contour detection in OpenCV to find all possible regions:
_, contours, _ = cv2.findContours(im_th.copy(), cv2.RETR_LIST,
cv2.CHAIN_APPROX_SIMPLE)
contours = sorted(contours, key=cv2.contourArea, reverse=True)
contours.remove(contours[0]) #remove the biggest contour
Finally, try to find the potential square regions based on the ratio of height and width:
square_rects = []
square_areas = []
for i, cnt in enumerate(contours):
(x, y, w, h) = cv2.boundingRect(cnt)
ar = w / float(h)
if 0.9 < ar < 1.1:
square_rects.append(((x,y), (x+w, y+h)))
square_areas.append(w*h) #store area information
We need to remove anything that is too small from the list by doing the follows:
import statistics
median_size_limit= statistics.median(square_areas) * 0.8
square_rects = [rect for i, rect in enumerate(square_rects)
if square_areas[i] > median_size_limit]
You can visually check the output by drawing all the rectangles on the original image:
for rect in square_rects:
cv2.rectangle(image, rect[0], rect[1], (0,255,0), 2)
cv2.imwrite("_output_image.png", image)
cv2.imshow("image", image)
cv2.waitKey()
You can use "square_rects" to locate all the squares and crop them from the original image.
The following is the preview of the final result.
Cheers.