opencv bounding box issue - python

I have an image I need to draw a bounding box around and I'm trying to use the code at bottom of this post.
The issue I have is that I have tried to blur the blue box shape to remove its details e.g.
cv2.blur(img,(20,20))
but the blurred image doesnt seem to have a good enough edge to produce a bounding box.
I have found that if I use my code below with a plain blue square with sharp edges of the same size as the image below, it works just fine. It seems the detail within the blue area stops a boundary box being drawn.
I was hoping someone could tell me how to get this to work please.
import cv2
# Load the image - container completely blurred out so no hinges,locking bars , writing etc are visible, just a blank blue square
img = cv2.imread('blue_object.jpg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
edged = cv2.Canny(img, 120,890)
# Apply adaptive threshold
thresh = cv2.adaptiveThreshold(edged, 255, 1, 1, 11, 2)
thresh_color = cv2.cvtColor(thresh, cv2.COLOR_GRAY2BGR)
# apply some dilation and erosion to join the gaps - change iteration to detect more or less area's
thresh = cv2.dilate(thresh,None,iterations = 50)
thresh = cv2.erode(thresh,None,iterations = 50)
# Find the contours
contours,hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
# For each contour, find the bounding rectangle and draw it
for cnt in contours:
area = cv2.contourArea(cnt)
if area > 50000:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
The image below is the yellow container after lower HSV mask range of 8,0,0 and upper range of 78,255,255 . Trees are above and to top right of the container, so hard to separate the tress from the container to put a proper bounding box around it.Happy to move to chat if that helps.

You're converting to gray, throwing all that valuable color information away. You're also Canny-ing, which is generally a bad idea. Beginners don't have the judgment to apply Canny sensibly. Best stay away from it.
This can be solved with the usual approach:
transform colorspace into something useful, say HSV
inRange on well-saturated blue
some morphology operations to clean up debris
bounding box
That is assuming you are looking for a blue container, or any well-saturated color really.
im = cv.imread("i4nPk.jpg")
hsv = cv.cvtColor(im, cv.COLOR_BGR2HSV)
lower = (90, 84, 0)
upper = (180, 255, 255)
mask1 = cv.inRange(hsv, lower, upper)
mask2 = cv.erode(mask1, kernel=None, iterations=2)
(x, y, w, h) = cv.boundingRect(mask2) # yes that works on masks too
canvas = cv.cvtColor(mask2, cv.COLOR_GRAY2BGR)
cv.rectangle(canvas, (x,y), (x+w, y+h), color=(0,0,255), thickness=3)

Related

How to accurately detect the positions of hollow circles (rings) with OpenCV Python?

I would like to determine the center positions of the tips of the syringes in this (video still) image. The tips are nominally round and of known size and quantity.
I am currently putting red ink on the tips to make them easier to detect. It would be nice to not to have to do this but I think without it, detection would be very difficult. Anyone like a challenge?
I started off trying SimpleBlobDetector as it has some nice filtering. One thing I couldn't figure out was how to get SimpleBlobDetector to detect the hollow circles (rings)?
I then tried canny + hough but the circle detection was too unstable, the positions jumped around.
I am currently using findContours + minEnclosingCircle which works OK but still quite unstable.
The mask looks like this. The result. You can see the accuracy is not great:
I briefly looked at RANSAC but I couldn't find a Python example that would detect multiple circles plus the edge detection is tricky.
My current code:
# https://stackoverflow.com/questions/32522989/opencv-better-detection-of-red-color
frame_inv = ~frame0
# Convert BGR to HSV
hsv = cv2.cvtColor(frame_inv, cv2.COLOR_BGR2HSV)
blur = cv2.GaussianBlur(hsv, (5, 5), 0)
# define range of color in HSV
lower_red = np.array([90 - 10, 70, 50])
upper_red = np.array([90 + 10, 255, 255])
# Threshold the HSV image to get only red colors
mask = cv2.inRange(hsv, lower_red, upper_red)
# cv2.imshow('Mask', mask)
kernel = np.ones((5, 5), np.uint8)
dilate = cv2.dilate(mask, kernel)
# cv2.imshow('Dilate', dilate)
contours = cv2.findContours(dilate, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[0]
tipXY = []
for c in contours:
area = cv2.contourArea(c)
if area > 200:
(x, y), r = cv2.minEnclosingCircle(c)
center = (int(x), int(y))
r = int(r)
shift = 2
factor = 2 ** shift
cv2.circle(frame0, (int(round((x) * factor)), int(round((y) * factor))),
int(round(10 * factor)), (0, 255, 0), 2, shift=shift)
tipXY.append(center)
Any suggestions to improve the position detection accuracy/stability?
Here is a better way to segment red color using the second image as input.
Idea:
Since the red color is prominent, I tried converting to other known color spaces (LAB and YCrCb) and viewed their individual channels. The Cr from YCrCb expressed the red color more prominently. According to this link, Cr channel represents the difference between red and luminance, enabling the color to stand out.
Code:
img = cv2.imread('stacked_rings.jpg')
ycc = cv2.cvtColor(img, cv2.COLOR_BGR2YCrCb)
cr_channel = ycc[:,:,1]
Though the hollow rings can be seen, the pixel intensity range is limited to the range [109 - 194]. Let's stretch the range:
dst = cv2.normalize(cr_channel, dst=None, alpha=0, beta=255,norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)
The circles a more prominent. Hope this pre-processing step helps you.

Remove everything of a specific color (with a color variation tolerance) from an image with Python

I have some text in blue #00a2e8, and some text in black on a PNG image (white background).
How to remove everything in blue (including text in blue) on an image with Python PIL or OpenCV, with a certain tolerance for the variations of color?
Indeed, every pixel of the text is not perfectly of the same color, there are variations, shades of blue.
Here is what I was thinking:
convert from RGB to HSV
find the Hue h0 for the blue
do a Numpy mask for Hue in the interval [h0-10, h0+10]
set these pixels to white
Before coding this, is there a more standard way to do this with PIL or OpenCV Python?
Example PNG file: foo and bar blocks should be removed
Your image has some issues. Firstly, it has a completely superfluous alpha channel which can be ignored. Secondly, the colours around your blues are quite a long way from blue!
I used your planned approach and found the removal was pretty poor:
#!/usr/bin/env python3
import cv2
import numpy as np
# Load image
im = cv2.imread('nwP8M.png')
# Define lower and upper limits of our blue
BlueMin = np.array([90, 200, 200],np.uint8)
BlueMax = np.array([100, 255, 255],np.uint8)
# Go to HSV colourspace and get mask of blue pixels
HSV = cv2.cvtColor(im,cv2.COLOR_BGR2HSV)
mask = cv2.inRange(HSV, BlueMin, BlueMax)
# Make all pixels in mask white
im[mask>0] = [255,255,255]
cv2.imwrite('DEBUG-plainMask.png', im)
That gives this:
If you broaden the range, to get the rough edges, you start to affect the green letters, so instead I dilated the mask so that pixels spatially near the blues are made white as well as pixels chromatically near the blues:
# Try dilating (enlarging) mask with 3x3 structuring element
SE = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
mask = cv2.dilate(mask, kernel, iterations=1)
# Make all pixels in mask white
im[mask>0] = [255,255,255]
cv2.imwrite('result.png', im)
That gets you this:
You may wish to diddle with the actual values for your other images, but the principle is the same.
I would like to chime in with a different approach. My basic idea is convert the image from BGR to LAB color space and figure out if I can isolate the regions in blue. This can be done by focusing on the b-component of LAB, since it represents the color from yellow to blue.
Code
img = cv2.imread('image_path', cv2.IMREAD_UNCHANGED)
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
b_component = lab[:,:,2]
(Note: The blue regions are actually quite darker such that it can be isolated easily.)
th = cv2.threshold(b_component,127,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)[1]
But after applying threshold, the image contains some unwanted white pixels around the regions containing numeric text, which we do not want to consider.
To avoid the unwanted regions I tried out the following:
Find contours above a certain area and draw each of them on 2-channel mask
Mask out rectangular bounding box area for each contour.
Locate pixels within that bounding box area that are 255 (white) on the threshold image
Change those pixel values to white on the original PNG image.
In code below:
# finding contours
contours = cv2.findContours(th, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# initialize a mask of image shape and make copy of original image
black = np.zeros((img.shape[0], img.shape[1]), np.uint8)
res = img.copy()
# draw only contours above certain area on the mask
for c in contours:
area = cv2.contourArea(c)
if int(area) > 200:
cv2.drawContours(black, [c], 0, 255, -1)
If you see the following mask, it has enclosed all pixels within the contour in white. However, the pixels within the word "bar" should not be considered.
To isolate only the region with blue pixels, we perform "AND" operation with the threshold image th
mask = cv2.bitwise_and(th, th, mask = black)
We got the mask we actually want. The regions that are white in mask are made white in the copy of the original image res:
res[mask == 255] = (255, 255, 255, 255)
But the above image is not perfect. There are some regions still visible around the edges of the word foo.
In the following we dilate mask and repeat.
res = img.copy()
kernel_ellipse = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
dilate = cv2.dilate(mask, kernel_ellipse, iterations=1)
res[dilate == 255] = (255, 255, 255, 255)
Note: Using the A and B components of LAB color space you can isolate different colors quite easily, without having to spend time searching for the range. Colors with nearby shading and saturation can also be segmented.
I think you are looking for the function inRange:
thresh = 5
bgr = [255 - thresh, thresh , thresh ]
minBGR = np.array([bgr[0] - thresh, bgr[1] - thresh, bgr[2] - thresh])
maxBGR = np.array([bgr[0] + thresh, bgr[1] + thresh, bgr[2] + thresh])
maskBGR = cv2.inRange(image, minBGR, maxBGR)
resultBGR = cv2.bitwise_or(image, maskBGR)

How to crop an image along major axis of the blob/contour with a rectangular bounding box using Python

I would like to crop an image which has a hand drawn highlighted area in orange as shown below,
The result should be a cropped image along the major axis of the blob or contour with a rectangular bounding box, as shown below,
Here's what i have tried,
import numpy as np
import cv2
# load the image
image = cv2.imread("frame50.jpg", 1)
#color boundaries [B, G, R]
lower = [0, 3, 30]
upper = [30, 117, 253]
# create NumPy arrays from the boundaries
lower = np.array(lower, dtype="uint8")
upper = np.array(upper, dtype="uint8")
# find the colors within the specified boundaries and apply
# the mask
mask = cv2.inRange(image, lower, upper)
output = cv2.bitwise_and(image, image, mask=mask)
ret,thresh = cv2.threshold(mask, 50, 255, 0)
if (int(cv2.__version__[0]) > 3):
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
else:
im2, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
if len(contours) != 0:
# find the biggest countour (c) by the area
c = max(contours, key = cv2.contourArea)
x,y,w,h = cv2.boundingRect(c)
ROI = image[y:y+h, x:x+w]
cv2.imshow('ROI',ROI)
cv2.imwrite('ROI.png',ROI)
cv2.waitKey(0)
This does not seem to work most of the time.For some images, the following happens,
I would like to know if there is better way to go about this or how i can fix what i have right now.Note that the highlighted area is hand drawn and can be of any shape but it is closed and not left open and the colour of the highlight is that shade of orange itself in all cases.
And is there a way to only retain content inside the circle and blackout everything outside it?
EDIT1:
I was able to fix the wrong clipping by varying the threshold more. But my main query now is: is there a way to only retain content inside the circle and blackout everything outside it? I can see the mask as show below,
How do I fill this mask and retain content inside the circle and blackout everything outside it, with the same rectangular bounding box?
Have you tried
image[x:x+w, y:y+h]
And could you check bbox with below code
cv2.rectangle(thresh,(x,y),(x+w,y+h),(255,0,0),2)
First of all, it is always better to use an HSV image instead of BGR image for masking(extracting a color). You can do this by the following code.
HSV_Image = cv2.cvtColor(Image, cv2.COLOR_BGR2HSV)
ThreshImage = cv2.inRange(HSV_Image, np.array([0, 28, 191]), np.array([24, 255, 255]))
The range numbers here are found for orange color in this case.
Image is the input image and ThreshImage is the output image with the orange region colored as white and everything else as black.
Now finding the contour in ThreshImage with cv2.RETR_EXTERNAL flag will give only one contour that is the outer boundary of the orange region.
Contours, Hierarchy = cv2.findContours(ThreshImage, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
To crop the orange region:
BoundingRect = cv2.boundingRect(Contours[0])
(x, y, w, h) = BoundingRect
CroppedImage = Image[y:y+h, x:x+w].copy()
"CroppedImage" will store the cropped orange region as desired.
To get contents of only inside the contour:
Bitwise AND operation will be useful here as we already have detected the contour.
First, we have to create a black image of shape the same as that of input image and draw the contour with white color filled in it.
ContourFilledImage = np.zeros(Image.shape, dtype=np.uint8)
cv2.drawContours(ContourFilledImage, Contours, -1, (255, 255, 255), -1)
Now perform a bitwise AND operation on Input Image and "ContourFilledImage"
OnlyInnerData = cv2.bitwise_and(ContourFilledImage, Image)
"OnlyInnerData" image is the desired output image having only the content of inside the circle.

Refine OpenCV mask edge

I'm in the process of scanning old photographs, and I would like to automate the process of extracting the photograph from the (noisy) solid white background of the scanner so that I have a transparent photograph. This part of the program now works, but I have one more small problem with this.
The photograph can now be accurately detected (and extracted), but it leaves a small and sharp black border from the background around the entire photograph. I've tried to apply a gaussian blur to the transparency mask, but this wouldn't smooth the blackness away (and it made the border of the photograph look 'smudged').
This is the code that I have to extract the photo and generate the transparency mask:
# Load the scan, and convert it to RGBA.
original = cv2.imread('input.jpg')
original = cv2.cvtColor(original, cv2.COLOR_BGR2BGRA)
# Make the scan grayscale, and apply a blur.
image = cv2.cvtColor(original, cv2.COLOR_BGR2GRAY)
image = cv2.GaussianBlur(image, (25, 25), 0)
# Binarize the scan.
retval, threshold = cv2.threshold(image, 50, 255, cv2.THRESH_BINARY)
# Find the contour of the object.
contours, hierarchy = cv2.findContours(threshold, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
largestContourArea = -1
largestContour = -1
for contour in contours:
area = cv2.contourArea(contour)
if area > largestContourArea:
largestContourArea = area
largestContour = contour
# Generate the transparency mask.
mask = numpy.zeros(original.shape, numpy.uint8)
cv2.drawContours(mask, [ largestContour ], -1, (255, 255, 255, 255), -1)
# Apply the transparency mask.
original = cv2.multiply(mask.astype(float) / 255.0, original.astype(float))
cv2.imwrite('output.png', original)
I have a sample scan and the result of the code above using the sample scan. As you can see, there is a slight black border all around the photograph, which I would like to remove.
By using the erode method, you could shrink the contour (mask), effectively removing the black edge.
Since this method supports in-place operation, the code would look something like this: cv2.erode(mask, mask, kernel), where kernel is a kernel acquired using cv2.getStructuringElement. By playing around with the kernel and the iteration count, you can control the amount of shrinking.
This page explains everything in great detail and also provides nice examples: https://docs.opencv.org/2.4/doc/tutorials/imgproc/erosion_dilatation/erosion_dilatation.html

Removing partial borders with openCV

I'm using OpenCV to find tabular data within images so that I can use an OCR on it. So far I have been able to find the table in the image, find the columns of the table, then find each cell within each column. It works pretty well, but I'm having an issue with the cell walls getting stuck in my images and I'm unable to remove them reliably.This is one example that I'm having difficulty with.This would be another example.
I have tried several approaches to get these images better. I have been having the most luck with finding the contours.
img2gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, mask = cv2.threshold(img2gray, 180, 255, cv2.THRESH_BINARY)
image_final = cv2.bitwise_and(img2gray, img2gray, mask=mask)
ret, new_img = cv2.threshold(image_final, 180, 255, cv2.THRESH_BINARY_INV)
kernel = cv2.getStructuringElement(cv2.MORPH_CROSS, (5, 3))
# dilate , more the iteration more the dilation
dilated = cv2.dilate(new_img, kernel, iterations=3)
cv2.imwrite('../test_images/find_contours_dilated.png', dilated)
I have been toying with the kernel size and dilation iterations and have found this to be the best configuration.
Another approach I used was with PIL, but it is only really good if the border is uniform around the whole image, which in my cases it is not.
copy = Image.fromarray(img)
try:
bg = Image.new(copy.mode, copy.size, copy.getpixel((0, 0)))
except:
return None
diff = ImageChops.difference(copy, bg)
diff = ImageChops.add(diff, diff, 2.0, -100)
bbox = diff.getbbox()
if bbox:
return np.array(copy.crop(bbox))
There were a few other ideas that I tried, but none got me very far. Any help would be appreciated.
You could try with finding contours and "drawing" them out. Meaning you can draw a border that will be connected with the "walls" mannualy with cv2.rectangle on the borders of your image (that will combine all contours - walls). The biggest two contours will be outer and inner line of your walls and you can draw the contour white to remove the border. Then apply threshold again to remove the rest of the noise. Cheers!
Example:
import cv2
import numpy as np
# Read the image
img = cv2.imread('borders2.png')
# Get image shape
h, w, channels = img.shape
# Draw a rectangle on the border to combine the wall to one contour
cv2.rectangle(img,(0,0),(w,h),(0,0,0),2)
# Convert to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Apply binary threshold
_, threshold = cv2.threshold(gray, 180, 255, cv2.THRESH_BINARY_INV)
# Search for contours and sort them by size
_, contours, hierarchy = cv2.findContours(threshold,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
area = sorted(contours, key=cv2.contourArea, reverse=True)
# Draw it out with white color from biggest to second biggest contour
cv2.drawContours(img, ((contours[0]),(contours[1])), -1, (255,255,255), -1)
# Apply binary threshold again to the new image to remove little noises
_, img = cv2.threshold(img, 180, 255, cv2.THRESH_BINARY)
# Display results
cv2.imshow('img', img)
Result:

Categories

Resources