The objective is to remove large irregular area and maintained only character in the image.
For example, given the following
and the expected masked output
I have the impression this can be achieved as below
import cv2
import numpy as np
from matplotlib import pyplot as plt
dpath='remove_bg1.jpg'
img = cv2.imread(dpath)
img_fh=img.copy()
cv2.bitwise_not(img_fh,img_fh)
ksize=10
kernel = np.ones((ksize,ksize),np.uint8)
erosion = cv2.erode(img_fh,kernel,iterations = 3)
invertx = cv2.bitwise_not(erosion)
masked = cv2.bitwise_not(cv2.bitwise_and(img_fh,invertx))
all_image=[img,invertx,masked]
ncol=len(all_image)
for idx, i in enumerate(all_image):
plt.subplot(int(f'1{ncol}{idx+1}')),plt.imshow(i)
plt.show()
which produce
Clearly, the code above did not produced the expected result.
May I know how to address this issue properly?
To remove the unwanted blob, we must create a mask such that it encloses it completely.
Flow:
Inversely binarize the image (such that you have a white foreground against dark background)
Dilate the image (since the blob makes contact with letter 'A', it has to be isolated )
Find contour with the largest area
Draw the contour on an another 1-channel image and thicken it (dilation)
Pixel Assignment: Pixels containing the dilated blob are made white on the original image
Code:
im = cv2.imread('stained_text.jpg')
im2 = im.copy()
gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
# inverse binaraization
th = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
Notice the blob region touching the letter 'A'. Hence to isolate it we perform erosion using an elliptical kernel
# erosion
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
erode = cv2.erode(th, kernel, iterations=2)
# find contours
contours, hierarchy = cv2.findContours(erode, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Contour of maximum area
c = max(contours, key = cv2.contourArea)
# create 1-channel image in black
black = np.zeros((im.shape[0], im.shape[1]), np.uint8)
# draw the contour on it
black = cv2.drawContours(black, [c], 0, 255, -1)
# perform dilation to have clean border
# we are using the same kernel
dilate = cv2.dilate(black, kernel, iterations = 3)
# assign the dilated area in white over the original image
im2[dilate == 255] = (255,255,255)
This was just one of the many possible ways on how to proceed. The key thing to note is how to isolate the blob.
Related
I want to retrieve all contours of the image below, but ignore text.
Image:
When I try to find the contours of the current image I get the following:
I have no idea how to go about this as I am new to using OpenCV and image processing. I want to get ignore the text, how can I achieve this? If ignoring is not possible but making a single bounding box surrounding the text is, than that would be good too.
Edit:
Criteria that I need to match:
The contours may very in size and shape.
The colors from the image may differ.
The colors and size of the text inside the image may differ.
Here is one way to do that in Python/OpenCV.
Read the input
Convert to grayscale
Get Canny edges
Apply morphology close to ensure they are closed
Get all contour hierarchy
Filter contours to keep only those above threshold in perimeter
Draw contours on input
Draw each contour on a black background
Save results
Input:
import numpy as np
import cv2
# read input
img = cv2.imread('short_title.png')
# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# get canny edges
edges = cv2.Canny(gray, 1, 50)
# apply morphology close to ensure they are closed
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
edges = cv2.morphologyEx(edges, cv2.MORPH_CLOSE, kernel)
# get contours
contours = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
contours = contours[0] if len(contours) == 2 else contours[1]
# filter contours to keep only large ones
result = img.copy()
i = 1
for c in contours:
perimeter = cv2.arcLength(c, True)
if perimeter > 500:
cv2.drawContours(result, c, -1, (0,0,255), 1)
contour_img = np.zeros_like(img, dtype=np.uint8)
cv2.drawContours(contour_img, c, -1, (0,0,255), 1)
cv2.imwrite("short_title_contour_{0}.jpg".format(i),contour_img)
i = i + 1
# save results
cv2.imwrite("short_title_gray.jpg", gray)
cv2.imwrite("short_title_edges.jpg", edges)
cv2.imwrite("short_title_contours.jpg", result)
# show images
cv2.imshow("gray", gray)
cv2.imshow("edges", edges)
cv2.imshow("result", result)
cv2.waitKey(0)
Grayscale:
Edges:
All contours on input:
Contour 1:
Contour 2:
Contour 3:
Contour 4:
Here are two options for erasing the text:
Using pytesseract OCR.
Finding white (and small) connected components.
Both solution build a mask, dilate the mask and use cv2.inpaint for erasing the text.
Using pytesseract:
Find text boxes using pytesseract.image_to_boxes.
Fill the boxes in the mask with 255.
Code sample:
import cv2
import numpy as np
from pytesseract import pytesseract, Output
# Tesseract path
pytesseract.tesseract_cmd = "C:\\Program Files\\Tesseract-OCR\\tesseract.exe"
img = cv2.imread('ShortAndInteresting.png')
# https://stackoverflow.com/questions/20831612/getting-the-bounding-box-of-the-recognized-words-using-python-tesseract
boxes = pytesseract.image_to_boxes(img, lang='eng', config=' --psm 6') # Run tesseract, returning the bounding boxes
h, w, _ = img.shape # assumes color image
mask = np.zeros((h, w), np.uint8)
# Fill the bounding boxes on the image
for b in boxes.splitlines():
b = b.split(' ')
mask = cv2.rectangle(mask, (int(b[1]), h - int(b[2])), (int(b[3]), h - int(b[4])), 255, -1)
mask = cv2.dilate(mask, np.ones((5, 5), np.uint8)) # Dilate the boxes in the mask
clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS) # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).
# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()
Mask:
Finding white (and small) connected components:
Use mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255)) for finding the text (assume the text is white).
Finding connected components in the mask using cv2.connectedComponentsWithStats(mask, 4)
Remove large components from the mask - fill components with large area with zeros.
Code sample:
import cv2
import numpy as np
img = cv2.imread('ShortAndInteresting.png')
mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255))
nlabel, labels, stats, centroids = cv2.connectedComponentsWithStats(mask, 4) # Finding connected components with statistics
# Remove large components from the mask (fill components with large area with zeros).
for i in range(1, nlabel):
area = stats[i, cv2.CC_STAT_AREA] # Get area
if area > 1000:
mask[labels == i] = 0 # Remove large connected components from the mask (fill with zero)
mask = cv2.dilate(mask, np.ones((5, 5), np.uint8)) # Dilate the text in the maks
cv2.imwrite('mask2.png', mask)
clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS) # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).
# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()
Mask:
Clean image:
Note:
My assumption is that you know how to split the image into contours, and the only issue is the present of the text.
I would recommend using flood fill, find the seed point for each color region, flood fill it to ignore the text values within. Hope that helps!
Refer to example of using floodfill here: https://www.programcreek.com/python/example/89425/cv2.floodFill
Example below copied from link above
def fillhole(input_image):
'''
input gray binary image get the filled image by floodfill method
Note: only holes surrounded in the connected regions will be filled.
:param input_image:
:return:
'''
im_flood_fill = input_image.copy()
h, w = input_image.shape[:2]
mask = np.zeros((h + 2, w + 2), np.uint8)
im_flood_fill = im_flood_fill.astype("uint8")
cv.floodFill(im_flood_fill, mask, (0, 0), 255)
im_flood_fill_inv = cv.bitwise_not(im_flood_fill)
img_out = input_image | im_flood_fill_inv
return img_out
I have an image like this:
And I want to crop the image anywhere there is red.
So with this image I would be looking to produce 4 crops:
Obviously I first need to detect anywhere there is red in the image. I can do the following:
import cv2
import numpy as np
from google.colab.patches import cv2_imshow
## (1) Read and convert to HSV
img = cv2.imread("my_image_with_red.png")
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
## (2) Find the target red region in HSV
hsv_lower = np.array([0,50,50])
hsv_upper = np.array([10,255,255])
mask = cv2.inRange(hsv, hsv_lower, hsv_upper)
## (3) morph-op to remove horizone lines
kernel = np.ones((5,1), np.uint8)
mask2 = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)
## (4) crop the region
ys, xs = np.nonzero(mask2)
ymin, ymax = ys.min(), ys.max()
xmin, xmax = xs.min(), xs.max()
croped = img[ymin:ymax, xmin:xmax]
pts = np.int32([[xmin, ymin],[xmin,ymax],[xmax,ymax],[xmax,ymin]])
cv2.drawContours(img, [pts], -1, (0,255,0), 1, cv2.LINE_AA)
cv2_imshow(croped)
cv2_imshow(img)
cv2.waitKey()
Which gives the following result:
The bounding box covers the entire area containing red.
How can I get bounding boxes around each red piece of the image? I have looked into multiple masks but this doesn't seem to work.
What I am looking for is:
detect each red spot in the image;
return boundaries on each red dot;
use those boundaries to produce 4 individual crops as new images.
There are currently several problems:
If you look at your mask image, you will see that all traces of red are captured on the mask including the small noise. You're currently using np.nonzero() which captures all white pixels. This is what causes the bounding box to cover the entire area. To fix this, we can tighten up the lower hsv threshold to get this resulting mask:
Note there are still a lot of small blobs. Your question should be rephrased to
How can I crop the large red regions?
If you want to capture all red regions, you will obtain much more then 4 crops. So to remedy this, we will perform morphological operations to remove the small noise and keep only the large pronounced red regions. This results in a mask image that contains the large regions
You do not require multiple masks
How can I get bounding boxes around each red piece of the image?
You can do this using cv2.findContours() on the mask image to return the bounding rectangles of each red dot.
Oh? This is not your desired result. Since your desired result has some space surrounding each red dot, we also need to include a offset to the bounding rectangle. After adding an offset, here's our result
Since we have the bounding rectangles, we can simply use Numpy slicing to extract and save each ROI. Here's the saved ROIs
So to recap, to detect each red spot in the image, we can use HSV color thresholding. Note this will return all pixels which match this threshold which may be different from what you expect so it is necessary to perform morphological operations to filter the resulting mask. To obtain the bounding rectangles on each red blob, we can use cv2.findContours() which will give us the ROIs using cv2.boundingRect(). Once we have the ROI, we add a offset and extract the ROI using Numpy slicing.
import cv2
import numpy as np
image = cv2.imread("1.png")
original = image.copy()
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
hsv_lower = np.array([0,150,50])
hsv_upper = np.array([10,255,255])
mask = cv2.inRange(hsv, hsv_lower, hsv_upper)
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
opening = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel, iterations=1)
close = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, kernel, iterations=1)
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
offset = 20
ROI_number = 0
for c in cnts:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (x - offset, y - offset), (x + w + offset, y + h + offset), (36,255,12), 2)
ROI = original[y-offset:y+h+offset, x-offset:x+w+offset]
cv2.imwrite('ROI_{}.png'.format(ROI_number), ROI)
ROI_number += 1
cv2.imshow('mask', mask)
cv2.imshow('close', close)
cv2.imshow('image', image)
cv2.waitKey()
I am new to OpenCV so I really need your help. I have a bunch of images like this one:
I need to detect the rectangle on the image, extract the text part from it and save it as a new image.
Can you please help me with this?
Thank you!
Just to add to Danyals answer I have added an example code with steps written in comments. For this image you don't even need to perform morphological opening on the image. But usually for this kind of noise in the image it is recomended. Cheers!
import cv2
import numpy as np
# Read the image and create a blank mask
img = cv2.imread('napis.jpg')
h,w = img.shape[:2]
mask = np.zeros((h,w), np.uint8)
# Transform to gray colorspace and invert Otsu threshold the image
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
_, thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# ***OPTIONAL FOR THIS IMAGE
### Perform opening (erosion followed by dilation)
#kernel = np.ones((2,2),np.uint8)
#opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
# ***
# Search for contours, select the biggest and draw it on the mask
_, contours, hierarchy = cv2.findContours(thresh, # if you use opening then change "thresh" to "opening"
cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
cnt = max(contours, key=cv2.contourArea)
cv2.drawContours(mask, [cnt], 0, 255, -1)
# Perform a bitwise operation
res = cv2.bitwise_and(img, img, mask=mask)
########### The result is a ROI with some noise
########### Clearing the noise
# Create a new mask
mask = np.zeros((h,w), np.uint8)
# Transform the resulting image to gray colorspace and Otsu threshold the image
gray = cv2.cvtColor(res,cv2.COLOR_BGR2GRAY)
_, thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Search for contours and select the biggest one again
_, contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
cnt = max(contours, key=cv2.contourArea)
# Draw it on the new mask and perform a bitwise operation again
cv2.drawContours(mask, [cnt], 0, 255, -1)
res = cv2.bitwise_and(img, img, mask=mask)
# If you will use pytesseract it is wise to make an aditional white border
# so that the letters arent on the borders
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(res,(x,y),(x+w,y+h),(255,255,255),1)
# Crop the result
final_image = res[y:y+h+1, x:x+w+1]
# Display the result
cv2.imshow('img', final_image)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
One way to do this (if the rectangle sizes are somewhat predictable) is:
Convert the image to black and white
Invert the image
Perform morphological opening on the image from (2) with a horizontal line / rectangle (I tried with 2x30).
Perform morphological opening on the image from (2) with a vertical line (I tried it with 15x2).
Add the images from (3) and (4). You should only have a white rectangle now. Now can remove all corresponding rows and columns in the original image that are entirely zero in this image.
I am trying to have the circle detected in the following image.
So I did color thresholding and finally got this result.
Because of the lines in the center being removed, the circle is split into many small parts, so if I do contour detection on this, it can only give me each contour separately.
But is there a way I can somehow combine the contours so I could get a circle instead of just pieces of it?
Here is my code for color thresholding:
blurred = cv2.GaussianBlur(img, (9,9), 9)
ORANGE_MIN = np.array((12, 182, 221),np.uint8)
ORANGE_MAX = np.array((16, 227, 255),np.uint8)
hsv_disk = cv2.cvtColor(blurred,cv2.COLOR_BGR2HSV)
disk_threshed = cv2.inRange(hsv_disk, ORANGE_MIN, ORANGE_MAX)
The task is much easier when performed with the red plane only.
I guess there was problem with the thresholds for color segmentation, So the idea here was to generate a binary mask. By inspection your region of interest seems to be brighter than the other regions of input image, so thresholding can simply be done on a grayScale image to simplify the context. Note: You may change this step as per your requirement. After satisfying with the threshold output, you may use cv2.convexHull() to get the convex shape of your contour.
Also keep in mind to select the largest contour and ignore the small contours. The following code can be used to generate the required output:
import cv2
import numpy as np
# Loading the input_image
img = cv2.imread("/Users/anmoluppal/Downloads/3xGG4.jpg")
# Converting the input image to grayScale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Thresholding the image to get binary mask.
ret, img_thresh = cv2.threshold(img_gray, 145, 255, cv2.THRESH_BINARY)
# Dilating the mask image
kernel = np.ones((3,3),np.uint8)
dilation = cv2.dilate(img_thresh,kernel,iterations = 3)
# Getting all the contours
_, contours, __ = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Finding the largest contour Id
largest_contour_area = 0
largest_contour_area_idx = 0
for i in xrange(len(contours)):
if (cv2.contourArea(contours[i]) > largest_contour_area):
largest_contour_area = cv2.contourArea(contours[i])
largest_contour_area_idx = i
# Get the convex Hull for the largest contour
hull = cv2.convexHull(contours[largest_contour_area_idx])
# Drawing the contours for debugging purposes.
img = cv2.drawContours(img, [hull], 0, [0, 255, 0])
cv2.imwrite("./garbage.png", img)
I have used an adaptive thresholding technique to create a picture like the one below:
The code I used was:
image = cv2.adaptiveThreshold(image, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 45, 0)
Then, I use this code to get contours:
cnt = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
My goal is to generate a mask using all the pixels within the outer contour, so I want to fill in all pixels within the object to be white. How can I do this?
I have tried the code below to create a mask, but the resulting mask seems no different then the image after applying adaptive threshold
mask = np.zeros(image.shape[:2], np.uint8)
cv2.drawContours(mask, cnt, -1, 255, -1)
What you have is almost correct. If you take a look at your thresholded image, the reason why it isn't working is because your shoe object has gaps in the image. Specifically, what you're after is that you expect that the shoe has its perimeter to be all connected. If this were to happen, then if you extract the most external contour (which is what your code is doing), you should only have one contour which represents the outer perimeter of the object. Once you fill in the contour, then your shoe should be completely solid.
Because the perimeter of your shoe is not complete and broken, this results in disconnected white regions. Should you use findContours to find all of the contours, it will only find the contours of each of the white shapes and not the most outer perimeter. As such, if you try and use findContours, it'll give you the same result as the original image, because you're simply finding the perimeter of each white region inside the image, then filling in these regions with findContours.
What you need to do is ensure that the image is completely closed. What I would recommend you do is use morphology to close all of the disconnected regions together, then run a findContours call on this new image. Specifically, perform a binary morphological closing. What this does is that it takes disconnected white regions that are close together and ensures that they're connected. Use a morphological closing, and perhaps use something like a 7 x 7 square structuring element to close the shoe. This structuring element you can think of as the minimum separation between white regions to consider them as being connected.
As such, do something like this:
import numpy as np
import cv2
image = cv2.imread('...') # Load your image in here
# Your code to threshold
image = cv2.adaptiveThreshold(image, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 45, 0)
# Perform morphology
se = np.ones((7,7), dtype='uint8')
image_close = cv2.morphologyEx(image, cv2.MORPH_CLOSE, se)
# Your code now applied to the closed image
cnt = cv2.findContours(image_close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
mask = np.zeros(image.shape[:2], np.uint8)
cv2.drawContours(mask, cnt, -1, 255, -1)
This code essentially takes your thresholded image, and applies morphological closing to this image. After, we find the external contours of this image, and fill them in with white. FWIW, I downloaded your thresholded image, and tried this on my own. This is what I get with your image:
A simple approach would be to close the holes in the foreground to form a single contour with cv2.morphologyEx() and cv2.MORPH_CLOSE
Now that the external contour is filled, we can find the outer contour with cv2.findContours() and use cv2.fillPoly() to fill in all pixels with white
import cv2
# Load in image, convert to grayscale, and threshold
image = cv2.imread('1.png')
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Close contour
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7,7))
close = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=1)
# Find outer contour and fill with white
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cv2.fillPoly(close, cnts, [255,255,255])
cv2.imshow('close', close)
cv2.waitKey()