I have an input image of a paragraph of text in single line spacing. I'm trying to implement something like the line spacing option to increase/decrease space between text lines in Microsoft Word. The current image is in single space, how can I convert the text into double space? Or say .5 space? Essentially I'm trying to dynamically restructure the spacing between text lines, preferably with an adjustable parameter. Something like this:
Input image
Desired result
My current attempt looks like this. I've been able to increase the spacing slightly but the text detail seems to be eroded and there is random noise in between lines.
Any ideas on how to improve the code or any better approaches?
import numpy as np
import cv2
img = cv2.imread('text.png')
H, W = img.shape[:2]
grey = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
threshed = cv2.threshold(grey, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
hist = cv2.reduce(threshed, 1, cv2.REDUCE_AVG).reshape(-1)
spacing = 2
delimeter = [y for y in range(H - 1) if hist[y] <= spacing < hist[y + 1]]
arr = []
y_prev, y_curr = 0, 0
for y in delimeter:
y_prev = y_curr
y_curr = y
arr.append(threshed[y_prev:y_curr, 0:W])
arr.append(threshed[y_curr:H, 0:W])
space_array = np.zeros((10, W))
result = np.zeros((1, W))
for im in arr:
v = np.concatenate((space_array, im), axis=0)
result = np.concatenate((result, v), axis=0)
result = (255 - result).astype(np.uint8)
cv2.imshow('result', result)
cv2.waitKey()
Approach #1: Pixel analysis
Obtain binary image. Load the image, convert to grayscale, and Otsu's threshold
Sum row pixels. The idea is that the pixel sum of a row can be used to determine if it corresponds to text or white space
Create new image and add additional white space. We iterate through the pixel array and add additional white space
Binary image
# Load image, grayscale, Otsu's threshold
image = cv2.imread('1.png')
h, w = image.shape[:2]
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
Now we iterate through each row and sum the white pixels to generate a pixel array.
We can profile a column of data generated from the sum of all the pixels in each row to determine which rows correspond to text. Sections of the data that equal 0 represents rows of the image that are composed of white space. Here's a visualization of the data array:
# Sum white pixels in each row
# Create blank space array and and final image
pixels = np.sum(thresh, axis=1).tolist()
space = np.ones((2, w), dtype=np.uint8) * 255
result = np.zeros((1, w), dtype=np.uint8)
We convert the data to a list and iterate through the data to build the final image. If a row is determined to be white space then we concatenate an empty space array to the final image. By adjusting the size of the empty array, we can change the amount of space to add to the image.
# Iterate through each row and add space if entire row is empty
# otherwise add original section of image to final image
for index, value in enumerate(pixels):
if value == 0:
result = np.concatenate((result, space), axis=0)
row = gray[index:index+1, 0:w]
result = np.concatenate((result, row), axis=0)
Here's the result
Code
import cv2
import numpy as np
import matplotlib.pyplot as plt
# import pandas as pd
# Load image, grayscale, Otsu's threshold
image = cv2.imread('1.png')
h, w = image.shape[:2]
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Sum white pixels in each row
# Create blank space array and and final image
pixels = np.sum(thresh, axis=1).tolist()
space = np.ones((1, w), dtype=np.uint8) * 255
result = np.zeros((0, w), dtype=np.uint8)
# Iterate through each row and add space if entire row is empty
# otherwise add original section of image to final image
for index, value in enumerate(pixels):
if value == 0:
result = np.concatenate((result, space), axis=0)
row = gray[index:index+1, 0:w]
result = np.concatenate((result, row), axis=0)
# Uncomment for plot visualization
'''
x = range(len(pixels))[::-1]
df = pd.DataFrame({'y': x, 'x': pixels})
df.plot(x='x', y='y', xlim=(-2000,max(pixels) + 2000), legend=None, color='teal')
'''
cv2.imshow('result', result)
cv2.imshow('thresh', thresh)
plt.show()
cv2.waitKey()
Approach #2: Individual line extraction
For a more dynamic approach, we can find the contours of each line and then add space in between each contour. We use the same method of appending extra white space as the 1st approach.
Obtain binary image. Load image, grayscale, Gaussian blur, and Otsu's threshold
Connect text contours. We create a horizontal shaped kernel and dilate to connect the words of each line into a single contour
Extract each line contour. We find contours, sort from top-to-bottom using imtuils.contours.sort_contours() and extract each line ROI
Append white space in between each line. We create a empty array and build the new image by appending white space between each line contour
Binary image
# Load image, grayscale, blur, Otsu's threshold
image = cv2.imread('1.png')
original = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
invert = 255 - thresh
height, width = image.shape[:2]
Create horizontal kernel and dilate
# Dilate with a horizontal kernel to connect text contours
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10,2))
dilate = cv2.dilate(thresh, kernel, iterations=2)
Extracted individual line contour highlighted in green
# Extract each line contour
lines = []
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
(cnts, _) = contours.sort_contours(cnts, method="top-to-bottom")
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (0, y), (width, y+h), (36,255,12), 2)
line = original[y:y+h, 0:width]
line = cv2.cvtColor(line, cv2.COLOR_BGR2GRAY)
lines.append(line)
Append white space in between each line. Here's the result with a 1 pixel wide space array
Result with a 5 pixel wide space array
# Append white space in between each line
space = np.ones((1, width), dtype=np.uint8) * 255
result = np.zeros((0, width), dtype=np.uint8)
result = np.concatenate((result, space), axis=0)
for line in lines:
result = np.concatenate((result, line), axis=0)
result = np.concatenate((result, space), axis=0)
Full code
import cv2
import numpy as np
from imutils import contours
# Load image, grayscale, blur, Otsu's threshold
image = cv2.imread('1.png')
original = image.copy()
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (3,3), 0)
thresh = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
invert = 255 - thresh
height, width = image.shape[:2]
# Dilate with a horizontal kernel to connect text contours
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10,2))
dilate = cv2.dilate(thresh, kernel, iterations=2)
# Extract each line contour
lines = []
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
(cnts, _) = contours.sort_contours(cnts, method="top-to-bottom")
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (0, y), (width, y+h), (36,255,12), 2)
line = original[y:y+h, 0:width]
line = cv2.cvtColor(line, cv2.COLOR_BGR2GRAY)
lines.append(line)
# Append white space in between each line
space = np.ones((1, width), dtype=np.uint8) * 255
result = np.zeros((0, width), dtype=np.uint8)
result = np.concatenate((result, space), axis=0)
for line in lines:
result = np.concatenate((result, line), axis=0)
result = np.concatenate((result, space), axis=0)
cv2.imshow('result', result)
cv2.imshow('image', image)
cv2.imshow('dilate', dilate)
cv2.waitKey()
Related
I have an image that contains a table, the table can be in many sizes and the image too, and the table can be fully gridded (with only some blank spot that needs to be filled), it can be with only vertical grid lines and can be only with horizontal grid lines.
I've searched the web for a long time and found no solution that worked for me.
I found the following questions that seem to be suitable for me:
Python & OpenCV: How to add lines to the gridless table
Draw a line on a gridless image Python Opencv
How to repair incomplete grid cells and fix missing sections in image
Python & OpenCV: How to crop half-formed bounding boxes
My code is taken from the answers to the above questions and the "best" result I got from the above question codes is that it drew 2 lines one at the rightmost part and one on the leftmost part.
I'm kind of new to OpenCV and the image processing field so I am not sure how can I fix the above questions codes to suit my needs or how to accomplish my needs exactly, I would appreciate any help you can provide.
Example of an image table:
Update:
To remove the horizontal lines I use exactly the code you can find in here, but the result I get on the example image is this:
as you can see it removed most of them but not all of them, and then when I try to apply the same for the vertical ones (I tried the same code with rotation, or flipping the kernel) it does not work at all...
I also tried this code but it didn't work at all also.
Update 2:
I was able to remove the lines using this code:
def removeLines(result, axis) -> np.ndarray:
img = result.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
if axis == "horizontal":
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, 25))
elif axis == "vertical":
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25, 1))
else:
raise ValueError("Axis must be either 'horizontal' or 'vertical'")
detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=2)
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
result = img.copy()
for c in cnts:
cv2.drawContours(result, [c], -1, (255, 255, 255), 2)
return result
gridless = removeLines(removeLines(cv2.imread(image_path), 'horizontal'), 'vertical')
Result:
Problem:
After I remove lines, when I try to draw the vertical lines using this code:
# read image
img = old_image.copy() # cv2.imread(image_path1)
hh, ww = img.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# average gray image to one column
column = cv2.resize(gray, (ww,1), interpolation = cv2.INTER_AREA)
# threshold on white
thresh = cv2.threshold(column, 248, 255, cv2.THRESH_BINARY)[1]
# get contours
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# Draw vertical
for cntr in contours_v:
x,y,w,h = cv2.boundingRect(cntr)
xcenter = x+w//2
cv2.line(original_image, (xcenter,0), (xcenter,hh-1), (0, 0, 0), 1)
I get this result:
Update 3:
when I try even thresh = cv2.threshold(column, 254, 255, cv2.THRESH_BINARY)[1] (I tried lowering it 1 by 1 until 245, for both the max value and the threshold value, each time I get a different or similar result but always too much lines or too less lines) I get the following:
Input:
Output:
It's putting too many lines instead of just 1 line in each column
Code:
# read image
img = old_image.copy() # cv2.imread(image_path1)
hh, ww = img.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# average gray image to one column
column = cv2.resize(gray, (ww, 1), interpolation = cv2.INTER_AREA)
# threshold on white
thresh = cv2.threshold(column, 254, 255, cv2.THRESH_BINARY)[1]
# get contours
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
x, y, w, h = cv2.boundingRect(cntr)
xcenter = x + w // 2
cv2.line(original_image, (xcenter,0), (xcenter, hh_-1), (0, 0, 0), 1)
Here is one way to get the lines in Python/OpenCV. Average the image down to 1 column. Then threshold and get the contours. Then get the bounding boxes and find the vertical centers. Draw lines at those places.
If you do not want the extra lines, crop your image first to get the inside of the table.
Input:
import cv2
import numpy as np
# read image
img = cv2.imread("table4.png")
hh, ww = img.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# average gray image to one column
column = cv2.resize(gray, (1,hh), interpolation = cv2.INTER_AREA)
# threshold on white
thresh = cv2.threshold(column, 248, 255, cv2.THRESH_BINARY)[1]
# get contours
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# loop over contours and get bounding boxes and ycenter and draw horizontal line at ycenter
result = img.copy()
for cntr in contours:
x,y,w,h = cv2.boundingRect(cntr)
ycenter = y+h//2
cv2.line(result, (0,ycenter), (ww-1,ycenter), (0, 0, 255), 1)
# write results
cv2.imwrite("table4_lines3.png", result)
# display results
cv2.imshow("RESULT", result)
cv2.waitKey(0)
Result:
You wrote that you tried to remove the lines using the code, but it did not work.
It works fine for me in Python/OpenCV.
Read the input
Convert to grayscale
Threshold to show the horizontal lines
Apply morphology open with a horizontal kernel to isolate the horizontal lines
Get their contours
Draw the contours on a copy of the input as white to cover over the black horizontal lines
Save the results
Input:
import cv2
import numpy as np
# read the input
img = cv2.imread('table4.png')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# do morphology to detect lines
horizontal_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (25,1))
detected_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, horizontal_kernel, iterations=2)
# get contours
cnts = cv2.findContours(detected_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
# draw contours as white on copy of input
result = img.copy()
for c in cnts:
cv2.drawContours(result, [c], -1, (255,255,255), 2)
# save results
cv2.imwrite('table4_horizontal_lines_threshold.png', thresh)
cv2.imwrite('table4_horizontal_lines_detected.png', detected_lines)
cv2.imwrite('table4_horizontal_lines_removed.png', result)
# show results
cv2.imshow('thresh', thresh)
cv2.imshow('morphology', detected_lines)
cv2.imshow('result', result)
cv2.waitKey(0)
Threshold Image:
Morphology Detected Lines Image:
Result:
I have an image like this below. [![enter image description here][2]][2]
I try to add both vertical and horizontal lines between rows and columns. I succeeded in adding the horizontal ones using the code provided [here][1]. But I failed to add the vertical ones. Can anyone help to point out what went wrong with my code?
import cv2
import numpy as np
import matplotlib.pyplot as plt
file = r"C:/Users/gaojia/Dropbox/Projects/Parking_lot/Community_Group_Buying/scripts/test_2.png"
# read image
img = cv2.imread(file)
hh, ww = img.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold gray image
thresh = cv2.threshold(gray, 254, 255, cv2.THRESH_BINARY)[1]
# count number of non-zero pixels in each row
count = np.count_nonzero(thresh, axis=0)
# threshold count at ww (width of image)
count_thresh = count.copy()
count_thresh[count==hh] = 255
count_thresh[count<hh] = 0
count_thresh = count_thresh.astype(np.uint8)
# get contours
contours = cv2.findContours(count_thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# loop over contours and get bounding boxes and ycenter and draw horizontal line at ycenter
result = img.copy()
for cntr in contours:
x,y,w,h = cv2.boundingRect(cntr)
xcenter = x+w//2
cv2.line(result, (xcenter,0), (xcenter, hh-1), (0, 0, 0), 2)
# display results
cv2.imshow("THRESHOLD", thresh)
cv2.imshow("RESULT", result)
cv2.waitKey(0)
The code above draws only one vertical line at the far left of the image. I removed the vertical line in the original image, but the result remains the same.
-----------------------EDIT-----------------------------
As is in the first answer, I realized that the input for findContours() should not be a one-dimensional array. I thus replaced the following code:
# count number of non-zero pixels in each row
count = np.count_nonzero(thresh, axis=0)
# threshold count at ww (width of image)
count_thresh = count.copy()
count_thresh[count==hh] = 255
count_thresh[count<hh] = 0
count_thresh = count_thresh.astype(np.uint8)
with:
# find row index with any value equals 0
row_zero = np.nonzero(np.any(thresh == 0, axis=1))[0]
# replace values in column with any 0 with 0.
thresh[row_zero, :] = 0
This adds the vertical lines between columns of text.
[1]: Python & OpenCV: How to add lines to gridless table
[2]: https://i.stack.imgur.com/YnPns.png
Your issue is that count is one dimensional no matter whether along axis 0 or 1. So you have to transpose the x,y and w,h in your bounding boxes to use as you did in Python/OpenCV.
Input:
import cv2
import numpy as np
# read image
img = cv2.imread("gridless_table.png")
hh, ww = img.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold gray image
thresh = cv2.threshold(gray, 254, 255, cv2.THRESH_BINARY)[1]
# count number of non-zero pixels in each column
count = np.count_nonzero(thresh, axis=0)
# threshold count at hh (height of image)
count_thresh = count.copy()
count_thresh[count==hh] = 255
count_thresh[count<hh] = 0
count_thresh = count_thresh.astype(np.uint8)
# get contours
contours = cv2.findContours(count_thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# loop over contours and get bounding boxes and xcenter and draw vertical line at ycenter
result = img.copy()
for cntr in contours:
# must transpose x,y and w,h since count is one-dimensional but represents each column
y,x,h,w = cv2.boundingRect(cntr)
print(x,y,w,h)
xcenter = x+w//2
cv2.line(result, (xcenter,0), (xcenter, hh-1), (0, 0, 0), 2)
# save results
cv2.imwrite("gridless_table_lines.png", result)
# display results
cv2.imshow("THRESHOLD", thresh)
cv2.imshow("RESULT", result)
cv2.waitKey(0)
Result:
I want to remove the text on the edged in the image
I have used the following code but it does not work it also remove the text in the center
Input:
Output:
import cv2
import matplotlib.pyplot as plt
import glob
import os
import numpy as np
def crop_buttom_text(img):
""" Remove the text from the bottom edge of img """
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#blur = cv2.GaussianBlur(gray, (9,9), 0) # No need for blurring
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Create rectangular structuring element and dilate
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (30, 1)) # Use horizontal line as kernel - dilate horizontally.
dilate = cv2.dilate(thresh, kernel, iterations=1)
#kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (10,10))
#dilate = cv2.morphologyEx(dilate, cv2.MORPH_OPEN, kernel) # No need for opening
# Find contours and draw rectangle
cnts = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2] # [-2] indexing takes return value before last (due to OpenCV compatibility issues).
#cnts = cnts[0] if len(cnts) == 2 else cnts[1] # [-2] is shorter....
res_img = img.copy() # Copy img to res_img - in case there is no edges text.
for c in cnts:
x, y, w, h = cv2.boundingRect(c)
y2 = y + h # Bottom y coordinate of the bounding rectangle
if (y2 >= img.shape[0]):
# If the rectangle touches the bottom of the img
res_img = res_img[0:y-1, :].copy() # Crop rows from first row to row y-1
return res_img
def remove_lines(image_path,outdir):
image = cv2.imread(image_path)
img1 = crop_buttom_text(image)
img2 = crop_buttom_text(np.rot90(img1)) # Rotate by 90 degrees and crop.
img3 = crop_buttom_text(np.rot90(img2)) # Rotate by 90 degrees and crop.
img4 = crop_buttom_text(np.rot90(img3)) # Rotate by 90 degrees and crop.
output_img = np.rot90(img4)
cv2.imwrite(os.path.join(outdir,os.path.basename(image_path)), output_img)
for jpgfile in glob.glob(r'/content/Dataset/*'):
print(jpgfile)
remove_lines(jpgfile,r'/content/output')
How can i modify the above code to remove the text around the edge
How do you remove the text which is at the edge? Here is a rough way to attack the problem: remove every connected component which touches the border.
To solve this, you can add a 1 pixel border, extract the connected components and then use the one corresponding to the border as a binary mask.
The mask becomes
The result is
Notice how a 7 and few commas get removed as well.
from cv2 import cv2
import numpy as np
# Load original image
img = cv2.imread('kShDc.jpg', cv2.IMREAD_GRAYSCALE)
# Save original dimensions
h, w = img.shape[:2]
# Ensure only bilevel image with white as foreground
_, bimg = cv2.threshold(img, 127, 255, cv2.THRESH_BINARY_INV)
# Add one pixel border
bimg = cv2.copyMakeBorder(bimg, 1, 1, 1, 1, cv2.BORDER_CONSTANT, None, 255)
# Extract connected components (Spaghetti is the fastest algorithm)
nlabels, label_image = cv2.connectedComponentsWithAlgorithm(bimg, 8, cv2.CV_32S, cv2.CCL_SPAGHETTI)
# Make a mask with the edge label (0 is background, 1 is the first encountered label, i.e. the border)
ccedge = np.uint8((label_image != 1)*255)
cv2.imwrite("mask.png", ccedge, [cv2.IMWRITE_PNG_BILEVEL, 1])
# Zero every pixel touching the border
bimg = cv2.bitwise_and(bimg, ccedge)
# Remove border and invert again
bimg = 255 - bimg[1:h+1, 1:w+1]
# Save result
cv2.imwrite("result.png", bimg, [cv2.IMWRITE_PNG_BILEVEL, 1])
I have the following table:
I want to write a script that creates lines based on the natural breakages on the table text. The result would look like this:
Is there an OpenCV implementation that makes drawing these lines possible? I looked at the answers to the questions here and here, but neither worked. What is the best approach to solving this problem?
Here is one way to get the horizontal lines in Python/OpenCV by counting the number of white pixels in each row of the image to find their center y values. The vertical lines can be added by a similar process.
Input:
import cv2
import numpy as np
# read image
img = cv2.imread("table.png")
hh, ww = img.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# threshold gray image
thresh = cv2.threshold(gray, 254, 255, cv2.THRESH_BINARY)[1]
# count number of non-zero pixels in each row
count = np.count_nonzero(thresh, axis=1)
# threshold count at ww (width of image)
count_thresh = count.copy()
count_thresh[count==ww] = 255
count_thresh[count<ww] = 0
count_thresh = count_thresh.astype(np.uint8)
# get contours
contours = cv2.findContours(count_thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# loop over contours and get bounding boxes and ycenter and draw horizontal line at ycenter
result = img.copy()
for cntr in contours:
x,y,w,h = cv2.boundingRect(cntr)
ycenter = y+h//2
cv2.line(result, (0,ycenter), (ww-1,ycenter), (0, 0, 0), 2)
# write results
cv2.imwrite("table_thresh.png", thresh)
cv2.imwrite("table_lines.png", result)
# display results
cv2.imshow("THRESHOLD", thresh)
cv2.imshow("RESULT", result)
cv2.waitKey(0)
Threshold Image:
Result with lines:
ADDITION
Here is an alternate method that is slightly simpler. It averages the image down to one column rather than counting white pixels.
import cv2
import numpy as np
# read image
img = cv2.imread("table.png")
hh, ww = img.shape[:2]
# convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# average gray image to one column
column = cv2.resize(gray, (1,hh), interpolation = cv2.INTER_AREA)
# threshold on white
thresh = cv2.threshold(column, 254, 255, cv2.THRESH_BINARY)[1]
# get contours
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# loop over contours and get bounding boxes and ycenter and draw horizontal line at ycenter
result = img.copy()
for cntr in contours:
x,y,w,h = cv2.boundingRect(cntr)
ycenter = y+h//2
cv2.line(result, (0,ycenter), (ww-1,ycenter), (0, 0, 0), 2)
# write results
cv2.imwrite("table_lines2.png", result)
# display results
cv2.imshow("RESULT", result)
cv2.waitKey(0)
Result:
I would like to get from the image in the groups that are on the image
I have managed to remove first contour (as described below), but issue is that when I try to read the text, I have some missing text, I expect that this is because of other contours that have stayed on the image, but while I try to remove them, I loose the grouping or part of text...
for i in range(len(contours)):
if 800 < cv2.contourArea(contours[i]) < 2000:
x, y, width, height = cv2.boundingRect(contours[i])
roi = img[y:y + height, x:x + width]
roi_h = roi.shape[0]
roi_w = roi.shape[1]
resize_roi = cv2.resize(roi,(int(roi_w*6),int(roi_h*6)), interpolation=cv2.INTER_LINEAR)
afterd = cv2.cvtColor(resize_roi, cv2.COLOR_BGR2GRAY)
retim, threshm = cv2.threshold(afterd, 210, 225, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
contoursm, hierarchym = cv2.findContours(threshm, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
mask = np.ones(resize_roi.shape[:2], dtype="uint8") * 255
for m in range(len(contoursm)):
if 10000 < cv2.contourArea(contoursm[m]) < 33000:
cv2.drawContours(mask, contoursm, m, 0, 7)
afterd = cv2.bitwise_not(afterd)
afterd = cv2.bitwise_and(afterd, afterd, mask=mask)
afterd = cv2.bitwise_not(afterd)
print(pytesseract.image_to_string(afterd, lang='eng', config='--psm 3'))
Instead of dealing with all the boxes, I suggest deleting them by finding connected components, and filling the large clusters with background color.
You may use the following stages:
Convert image to Grayscale, apply threshold, and invert polarity.
Delete all clusters having more than 100 pixels (assume letters are smaller).
Dilate thresh for uniting text areas to single "blocks".
Find contours on the dilated thresh image.
Find bounding rectangles, and apply OCR to the rectangle.
Here is the complete code sample:
import numpy as np
import cv2
import pytesseract
pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe' # I am using Windows
img = cv2.imread('img.png') # Read input image
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Convert to Grayscale.
ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU) # Convert to binary and invert polarity
nlabel,labels,stats,centroids = cv2.connectedComponentsWithStats(thresh, connectivity=8)
thresh_size = 100
# Delete all lines by filling large clusters with zeros.
for i in range(1, nlabel):
if stats[i, cv2.CC_STAT_AREA] > thresh_size:
thresh[labels == i] = 0
# Dilate thresh for uniting text areas to single blocks.
dilated_thresh = cv2.dilate(thresh, np.ones((5,5)))
# Find contours on dilated thresh
contours, hierarchy = cv2.findContours(dilated_thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Iterate contours, find bounding rectangles
for c in contours:
# Get bounding rectangle
x, y, w, h = cv2.boundingRect(c)
# Draw green rectangle for testing
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), thickness = 1)
# Get the slice with the text (slice with margins).
afterd = thresh[y-3:y+h+3, x-3:x+w+3]
# Show afterd as image for testing
# cv2.imshow('afterd', afterd)
# cv2.waitKey(100)
# The OCR works only when image is enlarged and black text?
resized_afterd = cv2.resize(afterd, (afterd.shape[1]*5, afterd.shape[0]*5), interpolation=cv2.INTER_LANCZOS4)
print(pytesseract.image_to_string(255 - resized_afterd, lang='eng', config='--psm 3'))
cv2.imshow('thresh', thresh)
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result strings after OCR:
DF6DF645
RFFTW
2345
2277
AABBA
DF1267
ABCET5456
Input image with green boxes around the text:
Update:
Grouping contours:
For contours contours you may use the hierarchy result of cv2.findContours with cv2.RETR_TREE.
See Contours Hierarchy documentation.
You may use the parent-child relationship for grouping contours.
Here is an incomplete sample code for using the hierarchy:
img = cv2.imread('img.png') # Read input image
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Convert to Grayscale.
ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU) # Convert to binary and invert polarity
nlabel,labels,stats,centroids = cv2.connectedComponentsWithStats(thresh, connectivity=8)
thresh_boxes = np.zeros_like(thresh)
thresh_size = 100
# Delete all lines by filling large clusters with zeros.
# Make new image that contains only boxes - without text
for i in range(1, nlabel):
if stats[i, cv2.CC_STAT_AREA] > thresh_size:
thresh[labels == i] = 0
thresh_boxes[labels == i] = 255
# Find contours on thresh_boxes, use cv2.RETR_TREE to build tree with hierarchy
contours, hierarchy = cv2.findContours(thresh_boxes, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
# Iterate contours, and hierarchy
for c, i in zip(contours, range(len(contours))):
h = hierarchy[0, i, :]
h_child = h[2]
# if contours has no child (last level)
if h_child == -1:
h_parent = h[3]
x, y, w, h = cv2.boundingRect(c)
cv2.putText(img, str(h_parent), (x+w//2-4, y+h//2+8), fontFace=cv2.FONT_HERSHEY_SIMPLEX, fontScale=1, color=(0, 0, 255), thickness=2)
cv2.imshow('thresh', thresh)
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result: