I'm trying to separate the lines and circles, instead of lines some circles are connected with curves.
I tried to use contours to find circles but it is however including the lines inside the contour, so I also tried skeletoning the image so that to see if the connection between the circles and lines might break, but it is unsuccessful.
Hough_circles is not detecting circles in all cases, so the only option I've to find the circles using contours once the lines around it are eliminated.
EDIT
Example 2
Input
Output : Not desired output
In the output image, I got circles weren't got separated and lines got merged with circles and the contour gave a different shape.
Please find some way to split the circles and lines. Please try to answer it in Python instead of C++. C++ answers are allowed too.
Thanks in advance!
Here's a simple approach using morphological operations. The idea is to fill the contours, create an elliptical shaped structuring element, then morph open to remove the lines. From here we simply find the external contours and draw the circles. Here's the process visualized:
Filled thresholded image
Morph open
Result
Code
import cv2
# Load iamge, grayscale, Otsu's threshold
image = cv2.imread('1.png')
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Fill contours
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(thresh, [c], -1, (255,255,255), -1)
# Morph open
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=3)
# Draw circles
cnts = cv2.findContours(opening, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
cv2.drawContours(image, [c], -1, (36,255,12), 3)
cv2.imshow('thresh', thresh)
cv2.imshow('opening', opening)
cv2.imshow('image', image)
cv2.waitKey()
For the other image using contour hierarchy
import cv2
import numpy as np
# Load image, grayscale, Otsu's threshold
image = cv2.imread('1.png')
mask = np.zeros(image.shape, dtype=np.uint8)
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Filter using contour hierarchy
cnts, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
hierarchy = hierarchy[0]
for component in zip(cnts, hierarchy):
currentContour = component[0]
currentHierarchy = component[1]
x,y,w,h = cv2.boundingRect(currentContour)
# Only select inner contours
if currentHierarchy[3] > 0:
cv2.drawContours(mask, [currentContour], -1, (255,255,255), -1)
# Filter contours on mask using contour approximation
mask = cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
cnts = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.05 * peri, True)
if len(approx) > 5:
cv2.drawContours(mask, [c], -1, (0,0,0), -1)
else:
cv2.drawContours(image, [c], -1, (36,255,12), 2)
cv2.imshow('thresh', thresh)
cv2.imshow('image', image)
cv2.imshow('mask', mask)
cv2.waitKey()
Note: For a through explanation on contour hierarchy, take a look at understanding contour hierarchy and retrieval modes
Related
In the image below, I am using OpenCV harris corner detector to detect only the corners for the squares (and the smaller squares within the outer squares). However, I am also getting corners detected for the numbers on the side of the image. How do I get this to focus only on the squares and not the numbers? I need a method to ignore the numbers when performing OpenCV corner detection. The code, input image and output image are below:
import cv2 as cv
img = cv.imread(filename)
gray = cv.cvtColor(img,cv.COLOR_BGR2GRAY)
gray = np.float32(gray)
dst = cv.cornerHarris(gray, 2, 3, 0.04)
dst = cv.dilate(dst,None)
# Threshold for an optimal value, it may vary depending on the image.
img[dst>0.01*dst.max()]=[0,0,255]
cv.imshow('dst', img)
Input image
Output from Harris corner detector
Here's a potential approach using traditional image processing:
Obtain binary image. We load the image, convert to grayscale, Gaussian blur, then adaptive threshold to obtain a black/white binary image. We then remove small noise using contour area filtering. At this stage we also create two blank masks.
Detect horizontal and vertical lines. Now we isolate horizontal lines by creating a horizontal shaped kernel and perform morphological operations. To detect vertical lines, we do the same but with a vertical shaped kernel. We draw the detected lines onto separate masks.
Find intersection points. The idea is that if we combine the horizontal and vertical masks, the intersection points will be the corners. We can perform a bitwise-and operation on the two masks. Finally we find the centroid of each intersection point and highlight corners by drawing a circle.
Here's a visualization of the pipeline
Input image -> binary image
Detected horizontal lines -> horizontal mask
Detected vertical lines -> vertical mask
Bitwise-and both masks -> detected intersection points -> corners -> cleaned up corners
The results aren't perfect but it's pretty close. The problem comes from the noise on the vertical mask due to the slanted image. If the image was centered without an angle, the results would be ideal. You can probably fine tune the kernel sizes or iterations to get better results.
Code
import cv2
import numpy as np
# Load image, create horizontal/vertical masks, Gaussian blur, Adaptive threshold
image = cv2.imread('1.png')
original = image.copy()
horizontal_mask = np.zeros(image.shape, dtype=np.uint8)
vertical_mask = np.zeros(image.shape, dtype=np.uint8)
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.adaptiveThreshold(blur, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 23, 7)
# Remove small noise on thresholded image
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area < 150:
cv2.drawContours(thresh, [c], -1, 0, -1)
# Detect horizontal lines
dilate_horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10,1))
dilate_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, dilate_horizontal_kernel, iterations=1)
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (40,1))
detected_lines = cv2.morphologyEx(dilate_horizontal, cv2.MORPH_OPEN, horizontal_kernel, iterations=1)
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(image, [c], -1, (36,255,12), 2)
cv2.drawContours(horizontal_mask, [c], -1, (255,255,255), 2)
# Remove extra horizontal lines using contour area filtering
horizontal_mask = cv2.cvtColor(horizontal_mask,cv2.COLOR_BGR2GRAY)
cnts = cv2.findContours(horizontal_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area > 1000 or area < 100:
cv2.drawContours(horizontal_mask, [c], -1, 0, -1)
# Detect vertical
dilate_vertical_kernel = cv2.getStructuringElement(cv2.MORPH_CROSS, (1,7))
dilate_vertical = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, dilate_vertical_kernel, iterations=1)
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (1,2))
detected_lines = cv2.morphologyEx(dilate_vertical, cv2.MORPH_OPEN, vertical_kernel, iterations=4)
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(image, [c], -1, (36,255,12), 2)
cv2.drawContours(vertical_mask, [c], -1, (255,255,255), 2)
# Find intersection points
vertical_mask = cv2.cvtColor(vertical_mask,cv2.COLOR_BGR2GRAY)
combined = cv2.bitwise_and(horizontal_mask, vertical_mask)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2,2))
combined = cv2.morphologyEx(combined, cv2.MORPH_OPEN, kernel, iterations=1)
# Highlight corners
cnts = cv2.findContours(combined, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
# Find centroid and draw center point
try:
M = cv2.moments(c)
cx = int(M['m10']/M['m00'])
cy = int(M['m01']/M['m00'])
cv2.circle(original, (cx, cy), 3, (36,255,12), -1)
except ZeroDivisionError:
pass
cv2.imshow('thresh', thresh)
cv2.imshow('horizontal_mask', horizontal_mask)
cv2.imshow('vertical_mask', vertical_mask)
cv2.imshow('combined', combined)
cv2.imshow('original', original)
cv2.imshow('image', image)
cv2.waitKey()
I am trying to locate the X&Y of all horizontal lines in a PDF document.
I was using the code here:
code to detect horizontal lines
This code marks the horizontal lines verywell but I am not able to extract their coordinates in the document.
This is my code:
def DetectLine(pageNum):
# Convert to grayscale and adaptive threshold to obtain a binary image
img = cv2.imread(outpath + 'page_' + str(pageNum) + '.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Then we create a kernel and perform morphological transformations to isolate horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15,1))
detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
# detect lines find contours and draw the result
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(img, [c], -1, (36,255,12), 3)
cv2.imshow('image_' + str(pageNum), img)
This function gets the pagenumber and reads a pre-prepared JPG of the specific page.
How can I return the Xs & Ys?
If only need the points:
you can extract it with:
Point 1: c[0][0] or cnts[num]c[0][0]
Point 2: c[1][0] or cnts[num]c[1][0]
where num is the index of the contour
Middle point
The solution will be:
(cnts[0][1][0][0]+cnts[0][0][0][0])//2,cnts[0][0][0][1]
Since each line or countour for get has two points, you can calculate the middle point with the average formula.
e.g:
x1=10 and x2=90, the middle point then is (10+90)/2
Here is the complete code:
import cv2
image = cv2.imread('2.png')
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15,1))
detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
x,y=(c[1][0][0]+c[0][0][0])//2,c[0][0][1]
print(f'The middle point is: x: {x}, y: {y}')
cv2.drawContours(image, [c], -1, (36,255,12), 3)
cv2.circle(image, (x,y), radius=5, color=(0, 0, 255), thickness=-1)
cv2.imshow('thresh', thresh)
cv2.imshow('detected_lines', detected_lines)
cv2.imshow('image', image)
cv2.waitKey()
The result image is the following:
Here is my and I want to do mask and get this as
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY)[1]
# use morphology to close figure
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (35,35))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, )
# find contours and bounding boxes
mask = np.zeros_like(thresh)
cv2.waitKey(0)
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for count in contours:
cv2.drawContours(mask, [count], 0, 255, -1)
So..Please tell how can I get this output?
how do I remove this black patch ?
I am trying to run OCR (using Google's Tesseract) on a document with the following format:
However, Tesseract assumes the short bars in between to be letters/numbers (l or i or 1).
As a pre-processing measure I tried to remove vertical and horizontal lines using the following code:
import cv2
from pdf2image import convert_from_path
pages = convert_from_path('..\\app\\1.pdf', 500)
for page in pages:
page.save('..\\app\\out.jpg', 'JPEG')
image = cv2.imread('out.jpg')
result = image.copy()
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Remove horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (40,1))
remove_horizontal = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(remove_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(result, [c], -1, (255,255,255), 5)
# Remove vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,40))
remove_vertical = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(remove_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(result, [c], -1, (255,255,255), 5)
cv2.imshow('thresh', thresh)
cv2.imshow('result', result)
cv2.imwrite('result.png', result)
cv2.waitKey()
I run into an issue where the output of this document removes most of the vertical and horizontal lines in the document even the start and the finish line on the left and right side of the image below but not the small bars in between.
I'm wondering if I am going about this wrong by trying to pre-process and remove lines. Is there a better way to pre-process or another way to solve this problem?
With the observation that the form fields are separate from the characters, you can simply filter using contour area to isolate the text characters. The idea is to Gaussian blur, then Otsu's threshold to obtain a binary image. From here we find contours and filter using contour area with some predetermined threshold value. We can effectively remove the lines by drawing in the contours with cv2.drawContours.
Binary image
Removed lines
Invert ready for OCR
OCR result using Pytesseract
HELLO
Code
import cv2
import pytesseract
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"
# Load image, grayscale, blur, Otsu's threshold
image = cv2.imread('1.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Find contours and filter using contour area
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
if area > 500:
cv2.drawContours(thresh, [c], -1, 0, -1)
# Invert image and OCR
invert = 255 - thresh
data = pytesseract.image_to_string(invert, lang='eng',config='--psm 6')
print(data)
cv2.imshow('thresh', thresh)
cv2.imshow('invert', invert)
cv2.waitKey()
Note: If you still want to go with the remove horizontal/vertical lines approach, you need to modify the vertical kernel size. For instance, change (1,40) to (1,10). This will help to remove smaller lines but it may also remove some of the vertical lines in the text such as in L.
I'm trying to detect main structures of many floor plan pictures by detecting straight lines and edges, with reference from here.
The example above is one example I need to deal with, is it possible to get main structure by detecting lines with opencv HoughLinesP from it? Thanks for your help at advance.
import cv2
import numpy as np
def get_lines(lines_in):
if cv2.__version__ < '3.0':
return lines_in[0]
return [l[0] for l in lines]
img = cv2.imread('./test.jpg', 1)
img_gray = gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cannied = cv2.Canny(img_gray, threshold1=50, threshold2=200, apertureSize=3)
lines = cv2.HoughLinesP(cannied, rho=1, theta=np.pi / 180, threshold=80, minLineLength=30, maxLineGap=10)
for line in get_lines(lines):
leftx, boty, rightx, topy = line
cv2.line(img, (leftx, boty), (rightx,topy), (255, 255, 0), 2)
cv2.imwrite('./lines.png', img)
cv2.imwrite('./canniedHouse.png', cannied)
cv2.waitKey(0)
cv2.destroyAllWindows()
Results:
lines.png
canniedHouse.png
Other references:
How to get the external contour of a floorplan in python?
Floor Plan Edge Detection - Image Processing?
Here's an approach
Convert image to grayscale
Adaptive threshold to obtain binary image
Perform morphological transformations to smooth image
Create horizontal kernel and detect horizontal lines
Create vertical kernel and detect vertical lines
After converting to grayscale, we adaptive threshold to obtain a binary image
image = cv2.imread('1.jpg')
original = image.copy()
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
From here we perform morphological transformations to smooth the image
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))
close = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=2)
Now we create a horizontal kernel with cv2.getStructuringElement() and detect horizontal lines
# Find horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (35,2))
detect_horizontal = cv2.morphologyEx(close, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(original, [c], -1, (36,255,12), -1)
Similarly, we create a vertical kernel and detect vertical lines
# Find vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2,35))
detect_vertical = cv2.morphologyEx(close, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(original, [c], -1, (36,255,12), -1)
Here's the result
import cv2
import numpy as np
image = cv2.imread('1.jpg')
original = image.copy()
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))
close = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=2)
# Find horizontal lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (35,2))
detect_horizontal = cv2.morphologyEx(close, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
cnts = cv2.findContours(detect_horizontal, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(original, [c], -1, (36,255,12), -1)
# Find vertical lines
vertical_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (2,35))
detect_vertical = cv2.morphologyEx(close, cv2.MORPH_OPEN, vertical_kernel, iterations=2)
cnts = cv2.findContours(detect_vertical, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(original, [c], -1, (36,255,12), -1)
cv2.imshow('thresh', thresh)
cv2.imshow('close', close)
cv2.imshow('original', original)
cv2.waitKey()
If you wanted to just find the external contour you can find contours after the morphological close operation
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
cv2.drawContours(original, [c], -1, (36,255,12), 3)