Refine OpenCV mask edge - python

I'm in the process of scanning old photographs, and I would like to automate the process of extracting the photograph from the (noisy) solid white background of the scanner so that I have a transparent photograph. This part of the program now works, but I have one more small problem with this.
The photograph can now be accurately detected (and extracted), but it leaves a small and sharp black border from the background around the entire photograph. I've tried to apply a gaussian blur to the transparency mask, but this wouldn't smooth the blackness away (and it made the border of the photograph look 'smudged').
This is the code that I have to extract the photo and generate the transparency mask:
# Load the scan, and convert it to RGBA.
original = cv2.imread('input.jpg')
original = cv2.cvtColor(original, cv2.COLOR_BGR2BGRA)
# Make the scan grayscale, and apply a blur.
image = cv2.cvtColor(original, cv2.COLOR_BGR2GRAY)
image = cv2.GaussianBlur(image, (25, 25), 0)
# Binarize the scan.
retval, threshold = cv2.threshold(image, 50, 255, cv2.THRESH_BINARY)
# Find the contour of the object.
contours, hierarchy = cv2.findContours(threshold, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
largestContourArea = -1
largestContour = -1
for contour in contours:
area = cv2.contourArea(contour)
if area > largestContourArea:
largestContourArea = area
largestContour = contour
# Generate the transparency mask.
mask = numpy.zeros(original.shape, numpy.uint8)
cv2.drawContours(mask, [ largestContour ], -1, (255, 255, 255, 255), -1)
# Apply the transparency mask.
original = cv2.multiply(mask.astype(float) / 255.0, original.astype(float))
cv2.imwrite('output.png', original)
I have a sample scan and the result of the code above using the sample scan. As you can see, there is a slight black border all around the photograph, which I would like to remove.

By using the erode method, you could shrink the contour (mask), effectively removing the black edge.
Since this method supports in-place operation, the code would look something like this: cv2.erode(mask, mask, kernel), where kernel is a kernel acquired using cv2.getStructuringElement. By playing around with the kernel and the iteration count, you can control the amount of shrinking.
This page explains everything in great detail and also provides nice examples: https://docs.opencv.org/2.4/doc/tutorials/imgproc/erosion_dilatation/erosion_dilatation.html

Related

opencv bounding box issue

I have an image I need to draw a bounding box around and I'm trying to use the code at bottom of this post.
The issue I have is that I have tried to blur the blue box shape to remove its details e.g.
cv2.blur(img,(20,20))
but the blurred image doesnt seem to have a good enough edge to produce a bounding box.
I have found that if I use my code below with a plain blue square with sharp edges of the same size as the image below, it works just fine. It seems the detail within the blue area stops a boundary box being drawn.
I was hoping someone could tell me how to get this to work please.
import cv2
# Load the image - container completely blurred out so no hinges,locking bars , writing etc are visible, just a blank blue square
img = cv2.imread('blue_object.jpg')
# convert to grayscale
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
edged = cv2.Canny(img, 120,890)
# Apply adaptive threshold
thresh = cv2.adaptiveThreshold(edged, 255, 1, 1, 11, 2)
thresh_color = cv2.cvtColor(thresh, cv2.COLOR_GRAY2BGR)
# apply some dilation and erosion to join the gaps - change iteration to detect more or less area's
thresh = cv2.dilate(thresh,None,iterations = 50)
thresh = cv2.erode(thresh,None,iterations = 50)
# Find the contours
contours,hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
# For each contour, find the bounding rectangle and draw it
for cnt in contours:
area = cv2.contourArea(cnt)
if area > 50000:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
The image below is the yellow container after lower HSV mask range of 8,0,0 and upper range of 78,255,255 . Trees are above and to top right of the container, so hard to separate the tress from the container to put a proper bounding box around it.Happy to move to chat if that helps.
You're converting to gray, throwing all that valuable color information away. You're also Canny-ing, which is generally a bad idea. Beginners don't have the judgment to apply Canny sensibly. Best stay away from it.
This can be solved with the usual approach:
transform colorspace into something useful, say HSV
inRange on well-saturated blue
some morphology operations to clean up debris
bounding box
That is assuming you are looking for a blue container, or any well-saturated color really.
im = cv.imread("i4nPk.jpg")
hsv = cv.cvtColor(im, cv.COLOR_BGR2HSV)
lower = (90, 84, 0)
upper = (180, 255, 255)
mask1 = cv.inRange(hsv, lower, upper)
mask2 = cv.erode(mask1, kernel=None, iterations=2)
(x, y, w, h) = cv.boundingRect(mask2) # yes that works on masks too
canvas = cv.cvtColor(mask2, cv.COLOR_GRAY2BGR)
cv.rectangle(canvas, (x,y), (x+w, y+h), color=(0,0,255), thickness=3)

Remove everything of a specific color (with a color variation tolerance) from an image with Python

I have some text in blue #00a2e8, and some text in black on a PNG image (white background).
How to remove everything in blue (including text in blue) on an image with Python PIL or OpenCV, with a certain tolerance for the variations of color?
Indeed, every pixel of the text is not perfectly of the same color, there are variations, shades of blue.
Here is what I was thinking:
convert from RGB to HSV
find the Hue h0 for the blue
do a Numpy mask for Hue in the interval [h0-10, h0+10]
set these pixels to white
Before coding this, is there a more standard way to do this with PIL or OpenCV Python?
Example PNG file: foo and bar blocks should be removed
Your image has some issues. Firstly, it has a completely superfluous alpha channel which can be ignored. Secondly, the colours around your blues are quite a long way from blue!
I used your planned approach and found the removal was pretty poor:
#!/usr/bin/env python3
import cv2
import numpy as np
# Load image
im = cv2.imread('nwP8M.png')
# Define lower and upper limits of our blue
BlueMin = np.array([90, 200, 200],np.uint8)
BlueMax = np.array([100, 255, 255],np.uint8)
# Go to HSV colourspace and get mask of blue pixels
HSV = cv2.cvtColor(im,cv2.COLOR_BGR2HSV)
mask = cv2.inRange(HSV, BlueMin, BlueMax)
# Make all pixels in mask white
im[mask>0] = [255,255,255]
cv2.imwrite('DEBUG-plainMask.png', im)
That gives this:
If you broaden the range, to get the rough edges, you start to affect the green letters, so instead I dilated the mask so that pixels spatially near the blues are made white as well as pixels chromatically near the blues:
# Try dilating (enlarging) mask with 3x3 structuring element
SE = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
mask = cv2.dilate(mask, kernel, iterations=1)
# Make all pixels in mask white
im[mask>0] = [255,255,255]
cv2.imwrite('result.png', im)
That gets you this:
You may wish to diddle with the actual values for your other images, but the principle is the same.
I would like to chime in with a different approach. My basic idea is convert the image from BGR to LAB color space and figure out if I can isolate the regions in blue. This can be done by focusing on the b-component of LAB, since it represents the color from yellow to blue.
Code
img = cv2.imread('image_path', cv2.IMREAD_UNCHANGED)
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
b_component = lab[:,:,2]
(Note: The blue regions are actually quite darker such that it can be isolated easily.)
th = cv2.threshold(b_component,127,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)[1]
But after applying threshold, the image contains some unwanted white pixels around the regions containing numeric text, which we do not want to consider.
To avoid the unwanted regions I tried out the following:
Find contours above a certain area and draw each of them on 2-channel mask
Mask out rectangular bounding box area for each contour.
Locate pixels within that bounding box area that are 255 (white) on the threshold image
Change those pixel values to white on the original PNG image.
In code below:
# finding contours
contours = cv2.findContours(th, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
# initialize a mask of image shape and make copy of original image
black = np.zeros((img.shape[0], img.shape[1]), np.uint8)
res = img.copy()
# draw only contours above certain area on the mask
for c in contours:
area = cv2.contourArea(c)
if int(area) > 200:
cv2.drawContours(black, [c], 0, 255, -1)
If you see the following mask, it has enclosed all pixels within the contour in white. However, the pixels within the word "bar" should not be considered.
To isolate only the region with blue pixels, we perform "AND" operation with the threshold image th
mask = cv2.bitwise_and(th, th, mask = black)
We got the mask we actually want. The regions that are white in mask are made white in the copy of the original image res:
res[mask == 255] = (255, 255, 255, 255)
But the above image is not perfect. There are some regions still visible around the edges of the word foo.
In the following we dilate mask and repeat.
res = img.copy()
kernel_ellipse = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
dilate = cv2.dilate(mask, kernel_ellipse, iterations=1)
res[dilate == 255] = (255, 255, 255, 255)
Note: Using the A and B components of LAB color space you can isolate different colors quite easily, without having to spend time searching for the range. Colors with nearby shading and saturation can also be segmented.
I think you are looking for the function inRange:
thresh = 5
bgr = [255 - thresh, thresh , thresh ]
minBGR = np.array([bgr[0] - thresh, bgr[1] - thresh, bgr[2] - thresh])
maxBGR = np.array([bgr[0] + thresh, bgr[1] + thresh, bgr[2] + thresh])
maskBGR = cv2.inRange(image, minBGR, maxBGR)
resultBGR = cv2.bitwise_or(image, maskBGR)

Identify region of image python

I have a microscopy image and need to calculate the area shown in red. The idea is to build a function that returns the area inside the red line on the right photo (float value, X mm²).
Since I have almost no experience in image processing, I don't know how to approach this (maybe silly) problem and so I'm looking for help. Other image examples are pretty similar with just 1 aglomerated "interest area" close to the center.
I'm comfortable coding in python and have used the software ImageJ for some time.
Any python package, software, bibliography, etc. should help.
Thanks!
EDIT:
The example in red I made manually just to make people understand what I want. Detecting the "interest area" must be done inside the code.
Canny, morphological transformation and contours can provide a decent result.
Although it might need some fine-tuning depending on the input images.
import numpy as np
import cv2
# Change this with your filename
image = cv2.imread('test.png', cv2.IMREAD_GRAYSCALE)
# You can fine-tune this, or try with simple threshold
canny = cv2.Canny(image, 50, 580)
# Morphological Transformations
se = np.ones((7,7), dtype='uint8')
image_close = cv2.morphologyEx(canny, cv2.MORPH_CLOSE, se)
contours, _ = cv2.findContours(image_close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Create black canvas to draw area on
mask = np.zeros(image.shape[:2], np.uint8)
biggest_contour = max(contours, key = cv2.contourArea)
cv2.drawContours(mask, biggest_contour, -1, 255, -1)
area = cv2.contourArea(biggest_contour)
print(f"Total area image: {image.shape[0] * image.shape[1]} pixels")
print(f"Total area contour: {area} pixels")
cv2.imwrite('mask.png', mask)
cv2.imshow('img', mask)
cv2.waitKey(0)
# Draw the contour on the original image for demonstration purposes
original = cv2.imread('test.png')
cv2.drawContours(original, biggest_contour, -1, (0, 0, 255), -1)
cv2.imwrite('result.png', original)
cv2.imshow('result', original)
cv2.waitKey(0)
The code produces the following output:
Total area image: 332628 pixels
Total area contour: 85894.5 pixels
Only thing left to do is convert the pixels to your preferred measurement.
Two images for demonstration below.
Result
Mask
I don't know much about this topic, but it seems like a very similar question to this one, which has what look like two great answers:
Python: calculate an area within an irregular contour line

How to remove noise in binary image? [duplicate]

This question already has answers here:
Filling holes inside a binary object
(7 answers)
Closed 11 months ago.
This is my code, I am trying to delete the mask (noise) from the binary image. What I am getting is white lines left around the noise. I am aware that there is a contour around that noise creating the final white line in the results. any help?
Original Image
Mask and results
Code
import numpy as np
import cv2
from skimage import util
img = cv2.imread('11_otsu.png')
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(imgray, 127, 255, 0, cv2.THRESH_BINARY)
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
#cv2.drawContours(img, contours, -1, (0,255,0), 2)
# create an empty mask
mask = np.zeros(img.shape[:2], dtype=np.uint8)
# loop through the contours
for i, cnt in enumerate(contours):
# if the contour has no other contours inside of it
if hierarchy[0][i][2] == -1:
# if the size of the contour is greater than a threshold
if cv2.contourArea(cnt) <70:
cv2.drawContours(mask, [cnt], 0, (255), -1)
# display result
cv2.imshow("Mask", mask)
cv2.imshow("Img", img)
image = cv2.bitwise_not(img, img, mask=mask)
cv2.imshow("Mask", mask)
cv2.imshow("After", image)
cv2.waitKey()
cv2.destroyAllWindows()
Your code is perfectly fine just make these adjustments and it should work:
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE) # Use cv2.CCOMP for two level hierarchy
if hierarchy[0][i][3] != -1: # basically look for holes
# if the size of the contour is less than a threshold (noise)
if cv2.contourArea(cnt) < 70:
# Fill the holes in the original image
cv2.drawContours(img, [cnt], 0, (255), -1)
Instead of trying to find inner contours and filling those in, may I suggest using cv2.floodFill instead? The flood fill operation is commonly used to fill in holes inside closed objects. Specifically, if you set the seed pixel to be the top left corner of the image then flood fill the image, what will get filled is the background while closed objects are left alone. If you invert this image, you will find all of the pixels that are interior to the closed objects that have "holes". If you take this inverted image and use the non-zero locations to directly set the original image, you will thus fill in the holes.
Therefore:
im = cv2.imread('8AdUp.png', 0)
h, w = im.shape[:2]
mask = np.zeros((h+2, w+2), dtype=np.uint8)
holes = cv2.floodFill(im.copy(), mask, (0, 0), 255)[1]
holes = ~holes
im[holes == 255] = 255
cv2.imshow('Holes Filled', im)
cv2.waitKey(0)
cv2.destroyAllWindows()
First we read in the image that you've provided which is thresholded and before the "noise filtering", then get the height and width of it. We also use an input mask to tell us which pixels to operate on the flood fill. Using a mask of all zeroes means you will operate on the entire image. It's also important to note that the mask needs to have a 1 pixel border surrounding it before using it. We flood fill the image using the top left corner as the initial point, invert it, set any "hole" pixels to 255 and show it. Take note that the input image is mutated once the method finishes so you need to pass in a copy to leave the input image untouched. Also, cv2.floodFill (using OpenCV 4) returns a tuple of four elements. I'll let you look at the documentation but you need the second element of this tuple, which is the filled in image.
We thus get:
I think using cv2.GaussianBlur() method might help you. After you convert the image to gray-scale, blur it using this method (as the name suggests, this is a Gaussian filter). Here is the documentation:
https://docs.opencv.org/4.3.0/d4/d86/group__imgproc__filter.html

Get area within contours Opencv Python?

I have used an adaptive thresholding technique to create a picture like the one below:
The code I used was:
image = cv2.adaptiveThreshold(image, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 45, 0)
Then, I use this code to get contours:
cnt = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
My goal is to generate a mask using all the pixels within the outer contour, so I want to fill in all pixels within the object to be white. How can I do this?
I have tried the code below to create a mask, but the resulting mask seems no different then the image after applying adaptive threshold
mask = np.zeros(image.shape[:2], np.uint8)
cv2.drawContours(mask, cnt, -1, 255, -1)
What you have is almost correct. If you take a look at your thresholded image, the reason why it isn't working is because your shoe object has gaps in the image. Specifically, what you're after is that you expect that the shoe has its perimeter to be all connected. If this were to happen, then if you extract the most external contour (which is what your code is doing), you should only have one contour which represents the outer perimeter of the object. Once you fill in the contour, then your shoe should be completely solid.
Because the perimeter of your shoe is not complete and broken, this results in disconnected white regions. Should you use findContours to find all of the contours, it will only find the contours of each of the white shapes and not the most outer perimeter. As such, if you try and use findContours, it'll give you the same result as the original image, because you're simply finding the perimeter of each white region inside the image, then filling in these regions with findContours.
What you need to do is ensure that the image is completely closed. What I would recommend you do is use morphology to close all of the disconnected regions together, then run a findContours call on this new image. Specifically, perform a binary morphological closing. What this does is that it takes disconnected white regions that are close together and ensures that they're connected. Use a morphological closing, and perhaps use something like a 7 x 7 square structuring element to close the shoe. This structuring element you can think of as the minimum separation between white regions to consider them as being connected.
As such, do something like this:
import numpy as np
import cv2
image = cv2.imread('...') # Load your image in here
# Your code to threshold
image = cv2.adaptiveThreshold(image, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV, 45, 0)
# Perform morphology
se = np.ones((7,7), dtype='uint8')
image_close = cv2.morphologyEx(image, cv2.MORPH_CLOSE, se)
# Your code now applied to the closed image
cnt = cv2.findContours(image_close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)[0]
mask = np.zeros(image.shape[:2], np.uint8)
cv2.drawContours(mask, cnt, -1, 255, -1)
This code essentially takes your thresholded image, and applies morphological closing to this image. After, we find the external contours of this image, and fill them in with white. FWIW, I downloaded your thresholded image, and tried this on my own. This is what I get with your image:
A simple approach would be to close the holes in the foreground to form a single contour with cv2.morphologyEx() and cv2.MORPH_CLOSE
Now that the external contour is filled, we can find the outer contour with cv2.findContours() and use cv2.fillPoly() to fill in all pixels with white
import cv2
# Load in image, convert to grayscale, and threshold
image = cv2.imread('1.png')
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
# Close contour
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (7,7))
close = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel, iterations=1)
# Find outer contour and fill with white
cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cv2.fillPoly(close, cnts, [255,255,255])
cv2.imshow('close', close)
cv2.waitKey()

Categories

Resources