What is the efficient way to constract bounding boxes for an objeject given the encoded pixel of that object. I am trying to convert a task from segmentation to object detection in yolo. Any suggestions are welcomed.
If you have the points of the segmentation then you can find the extents of the points by iterating through and recording the lowest and highest values of [x,y]. This will give you the the top left and bottom right corners.
If you're trying to work backwards from a colored, segmented image then you can use opencv to threshold the colors and get boxes from the blobs like this:
import cv2
import numpy as np
# load image
img = cv2.imread("mask.png");
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY);
# threshold
thresh = cv2.inRange(gray, 100, 255);
# contour
_, contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE);
# draw rectangle
x,y,w,h = cv2.boundingRect(contours[0]);
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2);
# show
cv2.imshow("box", img);
cv2.imshow("Thresh", thresh);
cv2.waitKey(0);
# save
cv2.imwrite("boxed.png", img);
Related
I have a program that uses PIL to automatically crop a series of images to a certain area. However, when I run the program with screenshots from a different sized display, the area cropped is in the wrong place. Is there a way to use OpenCV or PIL to automatically find the rectangle that I want to crop to (for example the main viewer of a Youtube video) and crop to it, while leaving the image in color and then saving the image to a new folder?
My code for cropping images:
import os, random
from PIL import Image
files = []
for x in os.listdir():
if '.jpg' in f'{x}' or '.png' in f'{x}' or '.jpeg' in f'{x}':
files.append(x)
else:
print(f'Passed by {x}! It is not an image!')
for x in files:
y = hex(random.randint(100000,500000))
image = Image.open(f'{x}')
newimage = image.crop((48,367,1626,1256))
newimage.save(f'newdir/{y}.png')
The example image (this works with the PIL cropper):
The image I want:
Another image from another computer that needs to be cropped to the same viewer:
Here is one way to do that using Python/OpenCV.
Basically, threshold the image, then get contours, then get the bounding box of the largest contour and crop using the bounding box.
Input:
import cv2
import numpy as np
# load image
img = cv2.imread("screen.jpg")
# get color bounds of white region
lower =(180,180,180) # lower bound for each channel
upper = (255,255,255) # upper bound for each channel
# threshold
threshold = cv2.inRange(img, lower, upper)
# get the largest contour
contours = cv2.findContours(threshold, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)
# get bounding box
x,y,w,h = cv2.boundingRect(big_contour)
print(x,y,w,h)
# crop the image at the bounds
crop = img[y:y+h, x:x+w]
# write result to disk
cv2.imwrite("screen_threshold.jpg", threshold)
cv2.imwrite("screen_cropped.jpg", crop)
# display it
cv2.imshow("threshold", threshold)
cv2.imshow("crop", crop)
cv2.waitKey(0)
Threshold Image:
Cropped Result:
Cropped (x,y,w,h):
48 368 1578 801
Opencv vers: 4.5
I'm trying to re-create the dimensions of an object by setting it up on a grid and taking as close to a top-down photo I can which I will then get the contours of the largest bounding rectangle and then perspective warp.
I'm currently unable to get the contour for a large bounding square however, it continually only finds smaller rectangles/squares which I'm assuming would not be large enough to properly fix the perspective.
First image: Original
Second image: What I get with my code using openCV
Third image: Close to what I'd ideally get
My code:
import imutils
import numpy as np
import cv2 as cv
# load the query image
image = cv.imread("path/to/image")
# make image greyscale, blur, find edges
grayscale_image = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
thresh = cv.adaptiveThreshold(grayscale_image, 255, cv.ADAPTIVE_THRESH_GAUSSIAN_C,
cv.THRESH_BINARY, 11, 2)
# find contours in the threshed image, keep only the largest
# ones
cnts = cv.findContours(
thresh.copy(), cv.RETR_LIST, cv.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
cnts = sorted(cnts, key=cv.contourArea, reverse=True)[:5]
# draw contours for reference
cv.drawContours(image, cnts, -1, (0, 255, 0), 3)
Instead of adaptive thresholding for pre-processing I've tried using bilateral filter or gaussian blur into canny edge detection but the outcome still doesn't find large rectangles.
Any help would be greatly appreciated as I'm at a loss on why it can't detect larger squares. Also, if people think there's a better method for fixing the perspective so that I can accurately recreate the board dimensions please let me know.
You may apply the following stages:
Apply threshold using cv2.threshold (instead of cv2.adaptiveThreshold).
Apply opening with long column vector for keeping only the vertical lines.
Find contours in vert_lines.
Sort contours left to right.
Draw most left and most right contours on a sketch (black) image.
Apply opening with long row vector for keeping only the horizontal lines, find contours, sort top to bottom, and draw top and bottom contours.
Find inner contours in the sketch image (with the left, right, top and bottom lines).
The inner contour is the smallest one.
Here is a code sample:
import imutils
import numpy as np
import cv2
# load the query image
image = cv2.imread("image.png")
# make image greyscale, blur, find edges
grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
#thresh = cv2.adaptiveThreshold(grayscale_image, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)
thresh = cv2.threshold(grayscale_image, 0, 255, cv2.THRESH_OTSU)[1] # Apply automatic threshold (use THRESH_OTSU).
rect_im = np.zeros_like(thresh) # Sketch image
# Apply opening with long column vector for keeping only the vertical lines.
vert_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, np.ones(50))
# Apply opening with long row vector for keeping only the horizontal lines.
horz_lines = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, np.ones((1,50)))
# Find contours in vert_lines
cnts = imutils.grab_contours(cv2.findContours(vert_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE))
# Sort contours left to right.
cnts = sorted(cnts, key=lambda c: cv2.boundingRect(c)[0]) # cv2.boundingRect(c)[0] is the left side x coordinate.
cv2.drawContours(rect_im, [cnts[0], cnts[-1]], -1, 255, -1) # Draw left and right contours
# Find contours in horz_lines
cnts = imutils.grab_contours(cv2.findContours(horz_lines, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE))
# Sort contours top to bottom.
cnts = sorted(cnts, key=lambda c: cv2.boundingRect(c)[1]) # cv2.boundingRect(c)[1] is the top y coordinate.
cv2.drawContours(rect_im, [cnts[0], cnts[-1]], -1, 255, -1) # Draw top and bottom contours
# Find contours in rect_im
cnts = imutils.grab_contours(cv2.findContours(rect_im, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)) # Note: use RETR_TREE for getting inner contour.
c = min(cnts, key=cv2.contourArea) # Get the smallest contour
# Draw contour for reference
cv2.drawContours(image, [c], -1, (0, 255, 0), 3)
Results:
thresh:
vert_lines:
horz_lines:
Left and right lines:
rect_im:
image (output):
I'm a beginner in image processing with Python so I need help.
I'm trying to remove areas of connected pixels from my pictures with the code posted below. Actually, it works but not well.
What I desire is the removing of areas of pixels, such as those marked in red in the pictures reported below, from my images, so as to obtain a cleaned picture.
Would be also great to set a minimum and a maximum limit for the dimensions of the detected areas of connected pixels.
Example of a picture with marked areas 1
Example of a picture with marked areas 2
This is my currently code:
### LOAD MODULES ###
import numpy as np
import imutils
import cv2
def is_contour_bad(c): # Decide what I want to find and its features
peri=cv2.contourArea(c, True) # Find areas
approx=cv2.approxPolyDP(c, 0.3*peri, True) # Set areas approximation
return not len(approx)>2 # Threshold to decide if add an area to the mask for its removing (if>2 remove)
### DATA PROCESSING ###
image=cv2.imread("025.jpg") # Load a picture
gray=cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # Convert to grayscale
cv2.imshow("Original image", image) # Plot
edged=cv2.Canny(gray, 50, 200, 3) # Edges of areas detection
cnts=cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) # Find contours: a curve joining all the continuous points (along the boundary), having same color or intensity
cnts=imutils.grab_contours(cnts)
mask=np.ones(image.shape[:2], dtype="uint8")*255 # Setup the mask with white background
# Loop over the detected contours
for c in cnts:
# If the contour satisfies "is_contour_bad", draw it on the mask
if is_contour_bad(c):
cv2.drawContours(mask, [c], -1, 0, -1) # (source image, list of contours, with -1 all contours in [c] pass, 0 is the intensity, -1 the thickness)
image_cleaned=cv2.bitwise_and(image, image, mask=mask) # Remove the contours from the original image
cv2.imshow("Adopted mask", mask) # Plot
cv2.imshow("Cleaned image", image_cleaned) # Plot
cv2.imwrite("cleaned_025.jpg", image_cleaned) # Write in a file
You may execute the following processing steps:
Threshold the image to binary image using cv2.threshold.
It's not a must, but in your case it looks like shades of gray are not important.
Use closing morphological operation, for closing small gaps in the binary image.
Use cv2.findContours with cv2.RETR_EXTERNAL parameter, for getting the contours (perimeter) surrounding the white clusters.
Modify the logic of "bad contour", to return true, only if area is large (assuming you only want to clean the large three contour).
Here is the updated code:
### LOAD MODULES ###
import numpy as np
import imutils
import cv2
def is_contour_bad(c): # Decide what I want to find and its features
peri = cv2.contourArea(c) # Find areas
return peri > 50 # Large area is considered "bad"
### DATA PROCESSING ###
image = cv2.imread("025.jpg") # Load a picture
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # Convert to grayscale
# Convert to binary image (all values above 20 are converted to 1 and below to 0)
ret, thresh_gray = cv2.threshold(gray, 20, 255, cv2.THRESH_BINARY)
# Use "close" morphological operation to close the gaps between contours
# https://stackoverflow.com/questions/18339988/implementing-imcloseim-se-in-opencv
thresh_gray = cv2.morphologyEx(thresh_gray, cv2.MORPH_CLOSE, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5)));
#Find contours on thresh_gray, use cv2.RETR_EXTERNAL to get external perimeter
_, cnts, _ = cv2.findContours(thresh_gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # Find contours: a curve joining all the continuous points (along the boundary), having same color or intensity
image_cleaned = gray
# Loop over the detected contours
for c in cnts:
# If the contour satisfies "is_contour_bad", draw it on the mask
if is_contour_bad(c):
# Draw black contour on gray image, instead of using a mask
cv2.drawContours(image_cleaned, [c], -1, 0, -1)
#cv2.imshow("Adopted mask", mask) # Plot
cv2.imshow("Cleaned image", image_cleaned) # Plot
cv2.imwrite("cleaned_025.jpg", image_cleaned) # Write in a file
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
Marking contours found for testing:
for c in cnts:
if is_contour_bad(c):
# Draw green line for marking the contour
cv2.drawContours(image, [c], 0, (0, 255, 0), 1)
Result:
There is still work to be done...
Update
Two iterations approach:
First iteration - remove the large contour.
Second iteration - remove small but bright contours.
Here is the code:
import numpy as np
import imutils
import cv2
def is_contour_bad(c, thrs): # Decide what I want to find and its features
peri = cv2.contourArea(c) # Find areas
return peri > thrs # Large area is considered "bad"
image = cv2.imread("025.jpg") # Load a picture
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # Convert to grayscale
# First iteration - remove the large contour
###########################################################################
# Convert to binary image (all values above 20 are converted to 1 and below to 0)
ret, thresh_gray = cv2.threshold(gray, 20, 255, cv2.THRESH_BINARY)
# Use "close" morphological operation to close the gaps between contours
# https://stackoverflow.com/questions/18339988/implementing-imcloseim-se-in-opencv
thresh_gray = cv2.morphologyEx(thresh_gray, cv2.MORPH_CLOSE, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5)));
#Find contours on thresh_gray, use cv2.RETR_EXTERNAL to get external perimeter
_, cnts, _ = cv2.findContours(thresh_gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # Find contours: a curve joining all the continuous points (along the boundary), having same color or intensity
image_cleaned = gray
# Loop over the detected contours
for c in cnts:
# If the contour satisfies "is_contour_bad", draw it on the mask
if is_contour_bad(c, 1000):
# Draw black contour on gray image, instead of using a mask
cv2.drawContours(image_cleaned, [c], -1, 0, -1)
###########################################################################
# Second iteration - remove small but bright contours
###########################################################################
# In the second iteration, use high threshold
ret, thresh_gray = cv2.threshold(image_cleaned, 150, 255, cv2.THRESH_BINARY)
# Use "dilate" with small radius
thresh_gray = cv2.morphologyEx(thresh_gray, cv2.MORPH_DILATE, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2,2)));
#Find contours on thresh_gray, use cv2.RETR_EXTERNAL to get external perimeter
_, cnts, _ = cv2.findContours(thresh_gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # Find contours: a curve joining all the continuous points (along the boundary), having same color or intensity
# Loop over the detected contours
for c in cnts:
# If the contour satisfies "is_contour_bad", draw it on the mask
# Remove contour if area is above 20 pixels
if is_contour_bad(c, 20):
# Draw black contour on gray image, instead of using a mask
cv2.drawContours(image_cleaned, [c], -1, 0, -1)
###########################################################################
Marked contours:
I am trying to extract the account number from an image of a cheque. The logic that I have is that, I am trying to find the rectangle that contains the account number, slice the bounding rectangle and then feed the slice into an OCR to get the text out of it.
The problem I am facing is when the rectangle is not very prominent and light colour, I am not able to get the rectangle contour since the edges are not connected totally.
How to overcome this?
Things I tried, but did not work are
I cannot increase the erosion iteration, to erode it more, because then the edges connect with the surrounding black pixels and form a different shape.
Reducing the threshold offset might help, but, it seems inefficient. Since the code has to work with several types of images. I can start with offset 10 and keep incrementing the offset and checking if I found the rectangle or not. This will increase the time a lot for cheques with prominent rectangles that work well at offset 20 or more. And since I don't have a condition to check if the edges of the rectangle are prominent or not, the loop has to be applied in all the cheques.
Keeping the above points in mind. Can someone help me out with a solution to this problem?
Libraries used and versions
scikit-image==0.13.1
opencv-python==3.3.0.10
Code
from skimage.filters import threshold_adaptive, threshold_local
import cv2
Step 1:
image = cv2.imread('cropped.png')
Step 2:
Using adaptive threshold from skimage to remove the background, so that I can get the account number rectangle box. This works fine for the cheques where the rectangle is more pronounced, but when the rectangle edges are thin, or are lighter in colour, the threshold results in
unconnected edges, because of which I am not able to find the contours. I have attached examples of this further down in the question.
account_number_block = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
account_number_block = threshold_adaptive(account_number_block, 251, offset=20)
account_number_block = account_number_block.astype("uint8") * 255
Step 3:
Erode the image a bit to try to connect small disconnections in the edges
kernel = np.ones((3,3), np.uint8)
account_number_block = cv2.erode(account_number_block, kernel, iterations=5)
Find the contours
(_, cnts, _) = cv2.findContours(account_number_block.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
# cnts = sorted(cnts, key=cv2.contourArea)[:3]
rect_cnts = [] # Rectangular contours
for cnt in cnts:
approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
if len(approx) == 4:
rect_cnts.append(cnt)
rect_cnts = sorted(rect_cnts, key=cv2.contourArea, reverse=True)[:1]
Working Example
Step 1: Original Image
Step 2: After thresholding to remove the background.
Step 3: Finding contours to find rectangle box of the account number.
Failure Working example - Light rectangular boundary.
Step 1: Read original image
Step 2: After thresholding to remove the background. Notice that the edges of the rectangle are not connected, because of which I am not able to get the contour out of it.
Step 3: Finding contours to find rectangle box of the account number.
import numpy as np
import cv2
import pytesseract as pt
from PIL import Image
#Run Main
if __name__ == "__main__" :
image = cv2.imread("image.jpg", -1)
# resize image to speed up computation
rows,cols,_ = image.shape
image = cv2.resize(image, (np.int32(cols/2),np.int32(rows/2)))
# convert to gray and binarize
gray_img = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
binary_img = cv2.adaptiveThreshold(gray_img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 9, 9)
# note: erosion and dilation works on white forground
binary_img = cv2.bitwise_not(binary_img)
# dilate the image to fill the gaps
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
dilated_img = cv2.morphologyEx(binary_img, cv2.MORPH_DILATE, kernel,iterations=2)
# find contours, discard contours which do not belong to a rectangle
(_, cnts, _) = cv2.findContours(dilated_img, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
rect_cnts = [] # Rectangular contours
for cnt in cnts:
approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
if len(approx) == 4:
rect_cnts.append(cnt)
# sort contours based on area
rect_cnts = sorted(rect_cnts, key=cv2.contourArea, reverse=True)[:1]
# find bounding rectangle of biggest contour
box = cv2.boundingRect(rect_cnts[0])
x,y,w,h = box[:]
# extract rectangle from the original image
newimg = image[y:y+h,x:x+w]
# use 'pytesseract' to get the text in the new image
text = pt.image_to_string(Image.fromarray(newimg))
print(text)
cv2.namedWindow('Image', cv2.WINDOW_NORMAL)
cv2.imshow('Image', newimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
result: 03541140011724
result: 34785736216
I am trying to have the circle detected in the following image.
So I did color thresholding and finally got this result.
Because of the lines in the center being removed, the circle is split into many small parts, so if I do contour detection on this, it can only give me each contour separately.
But is there a way I can somehow combine the contours so I could get a circle instead of just pieces of it?
Here is my code for color thresholding:
blurred = cv2.GaussianBlur(img, (9,9), 9)
ORANGE_MIN = np.array((12, 182, 221),np.uint8)
ORANGE_MAX = np.array((16, 227, 255),np.uint8)
hsv_disk = cv2.cvtColor(blurred,cv2.COLOR_BGR2HSV)
disk_threshed = cv2.inRange(hsv_disk, ORANGE_MIN, ORANGE_MAX)
The task is much easier when performed with the red plane only.
I guess there was problem with the thresholds for color segmentation, So the idea here was to generate a binary mask. By inspection your region of interest seems to be brighter than the other regions of input image, so thresholding can simply be done on a grayScale image to simplify the context. Note: You may change this step as per your requirement. After satisfying with the threshold output, you may use cv2.convexHull() to get the convex shape of your contour.
Also keep in mind to select the largest contour and ignore the small contours. The following code can be used to generate the required output:
import cv2
import numpy as np
# Loading the input_image
img = cv2.imread("/Users/anmoluppal/Downloads/3xGG4.jpg")
# Converting the input image to grayScale
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Thresholding the image to get binary mask.
ret, img_thresh = cv2.threshold(img_gray, 145, 255, cv2.THRESH_BINARY)
# Dilating the mask image
kernel = np.ones((3,3),np.uint8)
dilation = cv2.dilate(img_thresh,kernel,iterations = 3)
# Getting all the contours
_, contours, __ = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Finding the largest contour Id
largest_contour_area = 0
largest_contour_area_idx = 0
for i in xrange(len(contours)):
if (cv2.contourArea(contours[i]) > largest_contour_area):
largest_contour_area = cv2.contourArea(contours[i])
largest_contour_area_idx = i
# Get the convex Hull for the largest contour
hull = cv2.convexHull(contours[largest_contour_area_idx])
# Drawing the contours for debugging purposes.
img = cv2.drawContours(img, [hull], 0, [0, 255, 0])
cv2.imwrite("./garbage.png", img)