I have an image that has an object which I cropped out of the image using Canny filter
import cv2
import numpy as np
from matplotlib import pyplot as plt
from PIL import Image
# load image
img = cv2.imread('dataset/example.png')
#.
#.
#.
#.
#.
# canny edge detection then find the non-zero min-max coords of canny
#.
#.
#.
#.
#.
# ROI
roi = img[y1:y2, x1:x2]
## crop ROI
cropped = np.array(img)
cropped[y1:y2, x1:x2] = (0, 0, 0)
bg = Image.fromarray(cropped)
This is the result I get:
Is there a way to select the region outside the crop area (black box)? Basically selecting the inverse of cropped[y1:y2, x1:x2] and then getting the average colour of that background?
You cannot crop non-4 vertex polygons - remember you are working with matrices. If you want to get the contours of the non-black region, you can first get a binary mask using a threshold value of 0. This will render everything above that value in white. Then get the contours of that binary mask, like this:
# importing cv2 & numpy
import numpy as np
import cv2
# image path
path = "C://opencvImages//"
fileName = "squareTest.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Grayscale conversion:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
# Fixed Thresholding:
thresholdValue = 0
_, binaryImage = cv2.threshold(grayscaleImage, thresholdValue, 255, cv2.THRESH_BINARY)
This is the mask you obtain:
Now, simple get the contours:
# Find the contours on the mask image:
contours, hierarchy = cv2.findContours(binaryImage, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# Draw the contours on the mask image:
cv2.drawContours(inputCopy, contours, -1, (255, 0, 0), 3)
This is the result:
Now, if you want the mean BGR(A) value of the non-black region, use the binary mask we got and pass it to cv2.mean as mask, like this:
means = cv2.mean(inputImage, mask=binaryImage)
You get:
(130.01283431634118, 223.66963836747732, 121.75817119126356, 0.0)
You can use cv2.mean with a mask:
# create a mask from coordinages
mask = cv2.rectangle(np.zeros(img.shape[:2],'uint8'), (y1,x1), (y2,x2), 255, -1)
# out
means = cv2.mean(img, mask=mask)
Related
The objective is to remove large irregular area and maintained only character in the image.
For example, given the following
and the expected masked output
I have the impression this can be achieved as below
import cv2
import numpy as np
from matplotlib import pyplot as plt
dpath='remove_bg1.jpg'
img = cv2.imread(dpath)
img_fh=img.copy()
cv2.bitwise_not(img_fh,img_fh)
ksize=10
kernel = np.ones((ksize,ksize),np.uint8)
erosion = cv2.erode(img_fh,kernel,iterations = 3)
invertx = cv2.bitwise_not(erosion)
masked = cv2.bitwise_not(cv2.bitwise_and(img_fh,invertx))
all_image=[img,invertx,masked]
ncol=len(all_image)
for idx, i in enumerate(all_image):
plt.subplot(int(f'1{ncol}{idx+1}')),plt.imshow(i)
plt.show()
which produce
Clearly, the code above did not produced the expected result.
May I know how to address this issue properly?
To remove the unwanted blob, we must create a mask such that it encloses it completely.
Flow:
Inversely binarize the image (such that you have a white foreground against dark background)
Dilate the image (since the blob makes contact with letter 'A', it has to be isolated )
Find contour with the largest area
Draw the contour on an another 1-channel image and thicken it (dilation)
Pixel Assignment: Pixels containing the dilated blob are made white on the original image
Code:
im = cv2.imread('stained_text.jpg')
im2 = im.copy()
gray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
# inverse binaraization
th = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
Notice the blob region touching the letter 'A'. Hence to isolate it we perform erosion using an elliptical kernel
# erosion
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
erode = cv2.erode(th, kernel, iterations=2)
# find contours
contours, hierarchy = cv2.findContours(erode, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Contour of maximum area
c = max(contours, key = cv2.contourArea)
# create 1-channel image in black
black = np.zeros((im.shape[0], im.shape[1]), np.uint8)
# draw the contour on it
black = cv2.drawContours(black, [c], 0, 255, -1)
# perform dilation to have clean border
# we are using the same kernel
dilate = cv2.dilate(black, kernel, iterations = 3)
# assign the dilated area in white over the original image
im2[dilate == 255] = (255,255,255)
This was just one of the many possible ways on how to proceed. The key thing to note is how to isolate the blob.
I want to retrieve all contours of the image below, but ignore text.
Image:
When I try to find the contours of the current image I get the following:
I have no idea how to go about this as I am new to using OpenCV and image processing. I want to get ignore the text, how can I achieve this? If ignoring is not possible but making a single bounding box surrounding the text is, than that would be good too.
Edit:
Criteria that I need to match:
The contours may very in size and shape.
The colors from the image may differ.
The colors and size of the text inside the image may differ.
Here is one way to do that in Python/OpenCV.
Read the input
Convert to grayscale
Get Canny edges
Apply morphology close to ensure they are closed
Get all contour hierarchy
Filter contours to keep only those above threshold in perimeter
Draw contours on input
Draw each contour on a black background
Save results
Input:
import numpy as np
import cv2
# read input
img = cv2.imread('short_title.png')
# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# get canny edges
edges = cv2.Canny(gray, 1, 50)
# apply morphology close to ensure they are closed
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
edges = cv2.morphologyEx(edges, cv2.MORPH_CLOSE, kernel)
# get contours
contours = cv2.findContours(edges, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
contours = contours[0] if len(contours) == 2 else contours[1]
# filter contours to keep only large ones
result = img.copy()
i = 1
for c in contours:
perimeter = cv2.arcLength(c, True)
if perimeter > 500:
cv2.drawContours(result, c, -1, (0,0,255), 1)
contour_img = np.zeros_like(img, dtype=np.uint8)
cv2.drawContours(contour_img, c, -1, (0,0,255), 1)
cv2.imwrite("short_title_contour_{0}.jpg".format(i),contour_img)
i = i + 1
# save results
cv2.imwrite("short_title_gray.jpg", gray)
cv2.imwrite("short_title_edges.jpg", edges)
cv2.imwrite("short_title_contours.jpg", result)
# show images
cv2.imshow("gray", gray)
cv2.imshow("edges", edges)
cv2.imshow("result", result)
cv2.waitKey(0)
Grayscale:
Edges:
All contours on input:
Contour 1:
Contour 2:
Contour 3:
Contour 4:
Here are two options for erasing the text:
Using pytesseract OCR.
Finding white (and small) connected components.
Both solution build a mask, dilate the mask and use cv2.inpaint for erasing the text.
Using pytesseract:
Find text boxes using pytesseract.image_to_boxes.
Fill the boxes in the mask with 255.
Code sample:
import cv2
import numpy as np
from pytesseract import pytesseract, Output
# Tesseract path
pytesseract.tesseract_cmd = "C:\\Program Files\\Tesseract-OCR\\tesseract.exe"
img = cv2.imread('ShortAndInteresting.png')
# https://stackoverflow.com/questions/20831612/getting-the-bounding-box-of-the-recognized-words-using-python-tesseract
boxes = pytesseract.image_to_boxes(img, lang='eng', config=' --psm 6') # Run tesseract, returning the bounding boxes
h, w, _ = img.shape # assumes color image
mask = np.zeros((h, w), np.uint8)
# Fill the bounding boxes on the image
for b in boxes.splitlines():
b = b.split(' ')
mask = cv2.rectangle(mask, (int(b[1]), h - int(b[2])), (int(b[3]), h - int(b[4])), 255, -1)
mask = cv2.dilate(mask, np.ones((5, 5), np.uint8)) # Dilate the boxes in the mask
clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS) # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).
# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()
Mask:
Finding white (and small) connected components:
Use mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255)) for finding the text (assume the text is white).
Finding connected components in the mask using cv2.connectedComponentsWithStats(mask, 4)
Remove large components from the mask - fill components with large area with zeros.
Code sample:
import cv2
import numpy as np
img = cv2.imread('ShortAndInteresting.png')
mask = cv2.inRange(img, (230, 230, 230), (255, 255, 255))
nlabel, labels, stats, centroids = cv2.connectedComponentsWithStats(mask, 4) # Finding connected components with statistics
# Remove large components from the mask (fill components with large area with zeros).
for i in range(1, nlabel):
area = stats[i, cv2.CC_STAT_AREA] # Get area
if area > 1000:
mask[labels == i] = 0 # Remove large connected components from the mask (fill with zero)
mask = cv2.dilate(mask, np.ones((5, 5), np.uint8)) # Dilate the text in the maks
cv2.imwrite('mask2.png', mask)
clean_img = cv2.inpaint(img, mask, 2, cv2.INPAINT_NS) # Remove the text using inpaint (replace the masked pixels with the neighbor pixels).
# Show mask and clean_img for testing
cv2.imshow('mask', mask)
cv2.imshow('clean_img', clean_img)
cv2.waitKey()
cv2.destroyAllWindows()
Mask:
Clean image:
Note:
My assumption is that you know how to split the image into contours, and the only issue is the present of the text.
I would recommend using flood fill, find the seed point for each color region, flood fill it to ignore the text values within. Hope that helps!
Refer to example of using floodfill here: https://www.programcreek.com/python/example/89425/cv2.floodFill
Example below copied from link above
def fillhole(input_image):
'''
input gray binary image get the filled image by floodfill method
Note: only holes surrounded in the connected regions will be filled.
:param input_image:
:return:
'''
im_flood_fill = input_image.copy()
h, w = input_image.shape[:2]
mask = np.zeros((h + 2, w + 2), np.uint8)
im_flood_fill = im_flood_fill.astype("uint8")
cv.floodFill(im_flood_fill, mask, (0, 0), 255)
im_flood_fill_inv = cv.bitwise_not(im_flood_fill)
img_out = input_image | im_flood_fill_inv
return img_out
I am trying to digitize the kid's drawing into SVG or transparent png file format so that they can be used in Scratch. The white paper should be replaced by transparent background and all the drawing part should be preserved.
My plan is to get the outest contour of the drawing and generate a mask, then use the mask to get the drawing part without paper background.
The problem is the drawing may not consecutive which means there may have some small holes leading to break the entire drawing contour to many many small contours.
Now I want to concatenate the near outest contours to form a big outest contour for masking.
The original drawing and the processed result is attached.
Code:
from __future__ import print_function
import cv2 as cv
import numpy as np
import argparse
import random as rng
rng.seed(12345)
def thresh_callback(val):
threshold = val
# Detect edges using Canny
canny_output = cv.Canny(src_gray, threshold, threshold * 2)
# Find contours
contours, hierarchy = cv.findContours(canny_output, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
# Draw contours
drawing = np.zeros((canny_output.shape[0], canny_output.shape[1], 3), dtype=np.uint8)
for i in range(len(contours)):
color = (rng.randint(0,256), rng.randint(0,256), rng.randint(0,256))
cv.drawContours(drawing, contours, i, color, 2, cv.LINE_8, hierarchy, 0)
# Show in a window
cv.imshow('Contours', drawing)
# Load source image
parser = argparse.ArgumentParser(description='Code for Finding contours in your image tutorial.')
parser.add_argument('--input', help='Path to input image.', default='IMG_4446.jpg')
args = parser.parse_args()
src = cv.imread(cv.samples.findFile(args.input))
if src is None:
print('Could not open or find the image:', args.input)
exit(0)
# Convert image to gray and blur it
src_gray = cv.cvtColor(src, cv.COLOR_BGR2GRAY)
src_gray = cv.blur(src_gray, (3,3))
# Create Window
source_window = 'Source'
cv.namedWindow(source_window)
cv.imshow(source_window, src)
max_thresh = 255
thresh = 100 # initial threshold
cv.createTrackbar('Canny Thresh:', source_window, thresh, max_thresh, thresh_callback)
thresh_callback(thresh)
cv.waitKey()
import cv2, numpy as np
# Read Image
img = cv2.imread('/home/stephen/Desktop/test_img.png')
img =cv2.resize(img, (750,1000))
# Find the gray image
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Gray
gray = cv2.blur(gray, (2,2))
cv2.imwrite('/home/stephen/Desktop/gray.png',gray)
# Find the canny image
canny = cv2.Canny(gray, 30, 150) # Canny
# Find contours
contours, _ = cv2.findContours(canny,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
# Draw contours on canny (this connects the contours)
cv2.drawContours(canny, contours, -1, 255, 6)
cv2.imwrite('/home/stephen/Desktop/contours.png',canny)
# Get mask for floodfill
h, w = canny.shape[:2]
mask = np.zeros((h+2, w+2), np.uint8)
# Floodfill from point (0, 0)
cv2.floodFill(canny, mask, (0,0), 123)
cv2.imwrite('/home/stephen/Desktop/floodfill.png',canny)
# Exclude everying but the floodfill region
canny = cv2.inRange(canny, 122, 124)
cv2.imwrite('/home/stephen/Desktop/inrange.png',canny)
I wrote a program that takes a picture as an input and detects the written text
I need to edit my program to add two more functionalities
First
since I already was able to get the Contour coordinates(done), I want to know how do I get the color of the background that surrounds the contour line of each character ( at multiple points because I want to calculate the average of RGB color values surrounding that character )
I just don't know if there is already a function to do what I want to do or if there is a specific approach that I should follow
second,
I tried to fill the inside of the Contour but it did not work, can someone help me to know why
I did try both thickness=-1 and thickness=cv2.FILLE in my cv2.drawContours, nothing worked, clearly only the control line is blue but what is inside is not blue
I know that the code is not perfect and there are so many comments but it is because I'm Kim trying new stuff
thank you in advance to everyone who's going to help
import cv2
import pytesseract
import numpy as np
from PIL import ImageGrab
import pyautogui
def Contour(img):
Contours,hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for cnt in Contours:
area = cv2.contourArea(cnt)
# print(area)
# if area<150: could be useded to be cpecific
# we need a copy of the image so we dont draw on the original or thickness=-1
cv2.drawContours(imgcpy, cnt, -1, (255,0,0), thickness=-1)
#pytesseract.pytesseract.tesseract_cmd= 'C:\\Program Files\\Tesseract-OCR\\tesseract.exe'
cap= cv2
img = cv2.imread('D:\\opencv\\R\\img\\lec\\3.png')
imgcpy = img.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# for japanese txt = pytesseract.image_to_string(gray, lang='jpn')
txt = pytesseract.image_to_string(gray)
imgblur = cv2.GaussianBlur(img, (7, 7), 0)
imgcanny = cv2.Canny(img, 200, 200)
#imgDialation = cv2.dilate(imgcanny, kernel, iterations=1)
#imgeroded = cv2.erode(imgDialation, kernel, iterations=1)
imBlank = np.zeros_like(img)
# waste of time only the same picture work -_-
# imstack = np.hstack((gray,gray,gray))
Contour(imgcanny)
cv2.imshow('Contour', imgcpy)
#img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
print(txt)
print(Contour(imgcanny))
#cv2.imshow('test', img)
cv2.waitKey()
The "first" issue is solved by
getting a mask, either
using connectedComponents or
by filling a mask using drawContours into np.zeros((height, width), dtype=np.uint8)
dilate the mask, then
subtract the mask from the dilated version (literal subtraction will work, but so do bitwise operations)
The result is a mask that contains pixels just outside of the contour.
Use this mask to get those pixel values:
# have `mask` somehow, which should be dtype uint8
dilated = cv.dilate(mask)
surrounding = dilated - mask
values = img[surrounding != 0] # numpy indexing using a boolean array
meancolor = values.mean(axis=(0,1)) # sum/average over both axes
The "second" issue is that you pass a single contour to drawContours, which expects a list of contours.
Change your code to pass [cnt] instead of just cnt:
cv2.drawContours(imgcpy, [cnt], -1, (255,0,0), thickness=-1)
If you only pass cnt, it will interpret the list of points (a contour is a list of points) as a list of contours, hence it will interpret each point as a contour... and that's just a point, not a contour.
I have a question about python and opencv. I would like to change the part of the picture which is black to some other color (no matter what). After changing, I would like to get the pixel values, these 8 points marked with a red circle. How to do it?
https://imgur.com/2E1Wwqg
import cv2
import numpy as np
img = cv2.imread("image.jpg");
img[np.where((img == [0,0,0]).all(axis = 2))] = [50,150,166]
cv2.imwrite('output.png', img)
cv2.imshow("shapes", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
You can do that using OpenCV findContours() and minAreaRect() like this:
#!/usr/bin/env python3
import numpy as np
import cv2
# Load image
im = cv2.imread('start.png')
# Convert to grayscale
imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
# Theshold inverse so the black comes out white because findContours() looks for white objects
ret,thresh = cv2.threshold(imgray,16,255,cv2.THRESH_BINARY_INV)
cv2.imwrite('thresh.png',thresh)
# Remove noise specks
thresh = cv2.medianBlur(thresh,5)
cv2.imwrite('thresh-m.png',thresh)
# Find contours, draw on image and save
im2, contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(im, contours, -1, (0,255,0), 3)
# Show user what we found
i=0
for cnt in contours:
rect = cv2.minAreaRect(cnt)
box = cv2.boxPoints(rect)
box = np.int0(box)
cv2.drawContours(im,[box],0,(255,0,0),2)
print('Contour:{}\n{}'.format(i,box))
i = i+1
cv2.imwrite('result.png',im)
The thresholded image looks like this:
And the result image look like this:
The program output is the 4 corner points of the 4 minimum rectangles each one containing one of your lines.
Contour:0
[[416 776]
[410 767]
[659 607]
[664 616]]
Contour:1
[[297 780]
[ 77 599]
[ 83 592]
[303 773]]
Contour:2
[[518 695]
[507 694]
[519 176]
[530 177]]
Contour:3
[[226 688]
[224 174]
[233 173]
[235 687]]