I'm trying to remove the background of an image using a mask where the alpha value of a pixel is proportional to the black intensity. For instance, given the following input image and mask, the result contains "faded" areas:
Result:
Note the faded areas. Basically I'm trying to imitate the layer mask function in Photoshop.
I'm able to turn the mask into alpha using binary threshold, but I wonder how to make the alpha proportional. The code for binary threshold is as follows:
mask = cv2.imread(mask_path, 0)
mask2 = np.where(mask<50, 0, 1).astype('uint8')
img = img * mask2[:, :, np.newaxis]
_, alpha = cv2.threshold(mask2, 0, 255, cv2.THRESH_BINARY)
png = np.dstack((img, alpha))
cv2.imwrite(dest_path, png)
I suppose it may perhaps be irrelevant as thresholds are probably not needed for layer masking.
I'm not sure if this is what you want, but you can get the proportional effect by subtracting the values of the mask from the image. That means you have to invert the mask, so the amount of alpha you want to remove is white. For subtract(), the input arrays need to have the same size, so convert the inverted mask to 3 color channels. If the size of the mask is not equal to the background image, you'll first have to create a subimage.
import cv2
import numpy as np
# load background image
img = cv2.imread('grass.jpg')
# load alpha mask as grayscale
mask = cv2.imread('a_mask.jpg',0)
# invert mask and convert to 3 color channels
mask = cv2.bitwise_not(mask)
fullmask = cv2.cvtColor(mask,cv2.COLOR_GRAY2BGR)
# create a subimage with the size of the mask
xOffset = 30
yOffset = 30
height, width = mask.shape[:2]
subimg = img[yOffset:yOffset+height,xOffset:xOffset+width]
#subtract mask values from subimage
res = cv2.subtract(subimg,fullmask)
# put subimage back in background
img[yOffset:yOffset+height,xOffset:xOffset+width] = res
#display result
cv2.imshow('Result',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Related
I'm trying to identify a list of objects that appeared newly in a photo. The plan is to get multiple cropped images from the original image and feed them to a neural network for object detection. Right now, I'm having trouble in extracting objects that appeared in a frame.
import cv2 as cv
import matplotlib.pyplot as plt
def mdisp(image):
plt.imshow(image)
plt.show()
im1 = cv.imread('images/litter-before.jpg')
mdisp(im1)
print(im1.shape)
im2 = cv.imread('images/litter-after.jpg')
mdisp(im2)
print(im2.shape)
backsub1=cv.createBackgroundSubtractorMOG2()
backsub2=cv.createBackgroundSubtractorKNN()
fgmask = backsub1.apply(im1)
fgmask = backsub1.apply(im2)
print(fgmask.shape)
mdisp(fgmask)
new_image = im2 * (fgmask[:,:,None].astype(im2.dtype))
mdisp(new_image)
Ideally, I would like to get a cropped picture of the item within red circle. How can I do it with OpenCv
Here's an approach, subtracting the two frames directly. The idea is that you first convert your images to grayscale, then blur a little bit to ignore the noise. Subtract the two frames, threshold the difference and look for the largest blob that is above a certain area threshold value.
Let's see:
import cv2
import numpy as np
# image path
path = "C:/opencvImages/"
fileName01 = "01.jpg"
fileName02 = "02.jpg"
# Read the2 images in default mode:
image01 = cv2.imread(path + fileName01)
image02 = cv2.imread(path + fileName02)
# Store a copy of the last frame for results drawing:
inputCopy = image02.copy()
# Convert RGB images to grayscale:
grayscaleImage01 = cv2.cvtColor(image01, cv2.COLOR_BGR2GRAY)
grayscaleImage02 = cv2.cvtColor(image02, cv2.COLOR_BGR2GRAY)
# Convert RGB images to grayscale:
filterSize = 5
imageMedian01 = cv2.medianBlur(grayscaleImage01, filterSize)
imageMedian02 = cv2.medianBlur(grayscaleImage02, filterSize)
Now you have the grayscale, blurred frames. Next, we need to calculate the difference between these frames. I don't wanna loose data, so I have to be careful with the data type here. Remember that these are grayscale, uint8 matrices, but the difference could potentially yield negative values. Let's convert the matrices to floats, take the difference, and convert this matrix to uint8:
# uint8 to float32 conversion:
imageMedian01 = imageMedian01.astype('float32')
imageMedian02 = imageMedian02.astype('float32')
# Take the difference and convert back to uint8
imageDifference = np.clip(imageMedian01 - imageMedian02, 0, 255)
imageDifference = imageDifference.astype('uint8')
This gives you the frames difference:
Let's threshold this to get a binary image. I'm using a threshold value of 127, as it is a the center of the 8-bit range:
threshValue = 127
_, binaryImage = cv2.threshold(imageDifference, threshValue, 255, cv2.THRESH_BINARY)
This is the binary image:
We are looking for the biggest blob here, let's find blob/contours and filter the small ones. Let's set a minimum area of 10 pixels:
# Perform an area filter on the binary blobs:
componentsNumber, labeledImage, componentStats, componentCentroids = \
cv2.connectedComponentsWithStats(binaryImage, connectivity=4)
# Set the minimum pixels for the area filter:
minArea = 10
# Get the indices/labels of the remaining components based on the area stat
# (skip the background component at index 0)
remainingComponentLabels = [i for i in range(1, componentsNumber) if componentStats[i][4] >= minArea]
# Filter the labeled pixels based on the remaining labels,
# assign pixel intensity to 255 (uint8) for the remaining pixels
filteredImage = np.where(np.isin(labeledImage, remainingComponentLabels) == True, 255, 0).astype('uint8')
# Find the big contours/blobs on the filtered image:
contours, hierarchy = cv2.findContours(filteredImage, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
contours_poly = [None] * len(contours)
boundRect = []
# Alright, just look for the outer bounding boxes:
for i, c in enumerate(contours):
if hierarchy[0][i][3] == -1:
contours_poly[i] = cv2.approxPolyDP(c, 3, True)
boundRect.append(cv2.boundingRect(contours_poly[i]))
# Draw the bounding boxes on the (copied) input image:
for i in range(len(boundRect)):
print(boundRect[i])
color = (0, 255, 0)
cv2.rectangle(inputCopy, (int(boundRect[i][0]), int(boundRect[i][1])), \
(int(boundRect[i][0] + boundRect[i][2]), int(boundRect[i][1] + boundRect[i][3])), color, 1)
Check out the results:
I'm doing a project with depth image. But I have problems with noise and failed pixel reading with my depth camera. There are some spots and contours (especially edges) that have zero value. How to just ignore this zero value and blend it with surrounding values?
I have tried dilation and erosion (morph image processing), but I still can't get the right combination. It indeed removed some of the noise, but I just need to get rid of zeros at all points
Image Example:
The zero value is the darkest blue (I'm using colormap)
To illustrate what I want to do, please refer to this poor paint drawing:
I want to get rid the black spot (for example black value is 0 or certain value), and blend it with its surround.
Yes, I'm able to localized the spot using np.where or the similar function, but I have no idea how to blend it. Maybe a filter to be applied? I need to do this in a stream, so I need a fairly fast process, maybe 10-20 fps will do. Thank you in advance!
Update :
Is there a way other than inpaint? I've looked for various inpaints, but I don't need as sophisticated as impainting. I just need to blend it with simple line, curve, or shape and 1D. I think inpaint is an overkill. Besides, I need them to be fast enough to be used for video stream 10-20 fps, or even better.
Here is one way to do that in Python/OpenCV.
Use median filtering to fill the holes.
Read the input
Convert to gray
Threshold to make a mask (spots are black)
Invert the mask (spots are white)
Find the largest spot contour perimeter from the inverted mask and use half of that value as a median filter size
Apply median filtering to the image
Apply the mask to the input
Apply the inverse mask to the median filtered image
Add the two together to form the result
Save the results
Input:
import cv2
import numpy as np
import math
# read image
img = cv2.imread('spots.png')
# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# threshold
mask = cv2.threshold(gray,0,255,cv2.THRESH_BINARY)[1]
# erode mask to make black regions slightly larger
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
mask = cv2.morphologyEx(mask, cv2.MORPH_ERODE, kernel)
# make mask 3 channel
mask = cv2.merge([mask,mask,mask])
# invert mask
mask_inv = 255 - mask
# get area of largest contour
contours = cv2.findContours(mask_inv[:,:,0], cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
contours = contours[0] if len(contours) == 2 else contours[1]
perimeter_max = 0
for c in contours:
perimeter = cv2.arcLength(c, True)
if perimeter > perimeter_max:
perimeter_max = perimeter
# approx radius from largest area
radius = int(perimeter_max/2) + 1
if radius % 2 == 0:
radius = radius + 1
print(radius)
# median filter input image
median = cv2.medianBlur(img, radius)
# apply mask to image
img_masked = cv2.bitwise_and(img, mask)
# apply inverse mask to median
median_masked = cv2.bitwise_and(median, mask_inv)
# add together
result = cv2.add(img_masked,median_masked)
# save results
cv2.imwrite('spots_mask.png', mask)
cv2.imwrite('spots_mask_inv.png', mask_inv)
cv2.imwrite('spots_median.png', median)
cv2.imwrite('spots_masked.png', img_masked)
cv2.imwrite('spots_median_masked.png', median_masked)
cv2.imwrite('spots_removed.png', result)
cv2.imshow('mask', mask)
cv2.imshow('mask_inv', mask_inv )
cv2.imshow('median', median)
cv2.imshow('img_masked', img_masked)
cv2.imshow('median_masked', median_masked)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
Threshold image as mask:
Inverted mask:
Median filtered image:
Masked image:
Masked median filtered image:
Result:
Perhaps using a NaN-adjusted Gaussian filter is good and fast enough? When you consider your zeros/black spots as NaNs, this approach also works for larger black areas.
# import modules
import matplotlib.pyplot as plt
import numpy as np
import skimage
import skimage.filters
# set seed
np.random.seed(42)
# create dummy image
# (smooth for more realisitc appearance)
size = 50
img = np.random.rand(size, size)
img = skimage.filters.gaussian(img, sigma=5)
# create dummy missing/NaN spots
mask = np.random.rand(size, size) < 0.02
img[mask] = np.nan
# define and apply NaN-adjusted Gaussian filter
# (https://stackoverflow.com/a/36307291/5350621)
def nangaussian(U, sigma=1, truncate=4.0):
V = U.copy()
V[np.isnan(U)] = 0
VV = skimage.filters.gaussian(V, sigma=sigma, truncate=truncate)
W = 0*U.copy()+1
W[np.isnan(U)] = 0
WW = skimage.filters.gaussian(W, sigma=sigma, truncate=truncate)
return VV/WW
smooth = nangaussian(img, sigma=1, truncate=4.0)
# do not smooth full image but only copy smoothed NaN spots
fill = img.copy()
fill[mask] = smooth[mask]
# plot results
vmin, vmax = np.nanmin(img), np.nanmax(img)
aspect = 'auto'
plt.subplot(121)
plt.title('original image (white = NaN)')
plt.imshow(img, aspect=aspect, vmin=vmin, vmax=vmax)
plt.axis('off')
plt.subplot(122)
plt.title('filled image')
plt.imshow(fill, aspect=aspect, vmin=vmin, vmax=vmax)
plt.axis('off')
I am trying to get the outline of an album cover and edge detectors (Canny, Laplace) pick up too much noise. I don't fully understand how image masking works and would like put a white mask over the image so I see only the black pixels
I have applied a GaussianBlur 5x5 and converted the image to hsv values. I have a range of values which are black, and I have filtered these out.
# imported image and processing (shorthand here)
image = cv2.imread(args["image"])
blur = cv2.GaussianBlur(image, (5,5), 0)
blur_hsv = cv2.cvtColor(blur, cv2.COLOR_BGR2HSV)
# set regions of color
boundaries = [
# black
([0,0,0],[180, 255, 40])
#pink
#([151, 80, 50], [174, 255, 255])
]
# loop over the boundaries
for (lower, upper) in boundaries:
# create NumPy arrays from the boundaries
lower = np.array(lower, dtype = "uint8")
upper = np.array(upper, dtype = "uint8")
# find the colors within the specified boundaries and apply
mask = cv2.inRange(blur_hsv, lower, upper)
output = cv2.bitwise_and(image, image, mask = mask)
# show the images
cv2.imshow("images", np.hstack([image, output]))
I was hoping for some distinction in the final output, but the window is just black. How can I create a different color mask?
Edit:
Not the exact image, but a sample LEFT: original; RIGHT: processed
From my understanding, you want to obtain a mask where all colored pixels (non-black) are white. When we use cv2.inRange(), we give it a lower/upper threshold that returns all pixels within the bounds in white. Then when we use cv2.bitwise_and() with the mask and the original image, the resulting image will be the areas where both the mask and original image are white. Essentially any pixels in white are the areas that we want to keep.
Your current output shows all areas in the original picture that have pixels between the lower/upper threshold. But if your goal is to display all non-black pixels, then you can simply invert the mask. Here's a visualization:
Here's your current mask and represents all pixels within the thresholds in the original image as white.
We can simply invert the mask or use cv2.bitwise_not() to get your desired mask. This new mask represents all colored pixels not within your lower/upper threshold as white. Therefore this mask is all the colored pixels.
final_mask = 255 - mask
Remember, any pixels that we want to keep, we should make it white whereas any pixels we want to throw away, we make it black. So if we cv2.bitwise_and() this new mask with the original image, we get this
Here's a good tutorial on bitwise operations and masking
import cv2
import numpy as np
image = cv2.imread('1.png')
blur = cv2.GaussianBlur(image, (5,5), 0)
blur_hsv = cv2.cvtColor(blur, cv2.COLOR_BGR2HSV)
# create NumPy arrays from the boundaries
lower = np.array([0,0,0], dtype = "uint8")
upper = np.array([180,255,40], dtype = "uint8")
# find the colors within the specified boundaries and apply
mask = cv2.inRange(blur_hsv, lower, upper)
mask = 255 - mask
output = cv2.bitwise_and(image, image, mask = mask)
# show the images
cv2.imshow("output", output)
cv2.imshow("mask", mask)
cv2.waitKey()
I have a set of points that make a shape (closed polyline). Now I want to copy/crop all pixels from some image inside this shape, leaving the rest black/transparent. How do I do this?
For example, I have this:
and I want to get this:
*edit - updated to work with images that have an alpha channel.
This worked for me:
Make a mask with all black (all masked)
Fill a polygon with white in the shape of your ROI
combine the mask and your image to get the ROI with black everywhere else
You probably just want to keep the image and mask separate for functions that accept masks. However, I believe this does what you specifically asked for:
import cv2
import numpy as np
# original image
# -1 loads as-is so if it will be 3 or 4 channel as the original
image = cv2.imread('image.png', -1)
# mask defaulting to black for 3-channel and transparent for 4-channel
# (of course replace corners with yours)
mask = np.zeros(image.shape, dtype=np.uint8)
roi_corners = np.array([[(10,10), (300,300), (10,300)]], dtype=np.int32)
# fill the ROI so it doesn't get wiped out when the mask is applied
channel_count = image.shape[2] # i.e. 3 or 4 depending on your image
ignore_mask_color = (255,)*channel_count
cv2.fillPoly(mask, roi_corners, ignore_mask_color)
# from Masterfool: use cv2.fillConvexPoly if you know it's convex
# apply the mask
masked_image = cv2.bitwise_and(image, mask)
# save the result
cv2.imwrite('image_masked.png', masked_image)
The following code would be helpful for cropping the images and get them in a white background.
import cv2
import numpy as np
# load the image
image_path = 'input image path'
image = cv2.imread(image_path)
# create a mask with white pixels
mask = np.ones(image.shape, dtype=np.uint8)
mask.fill(255)
# points to be cropped
roi_corners = np.array([[(0, 300), (1880, 300), (1880, 400), (0, 400)]], dtype=np.int32)
# fill the ROI into the mask
cv2.fillPoly(mask, roi_corners, 0)
# The mask image
cv2.imwrite('image_masked.png', mask)
# applying th mask to original image
masked_image = cv2.bitwise_or(image, mask)
# The resultant image
cv2.imwrite('new_masked_image.png', masked_image)
Input Image:
Mask Image:
Resultant output image:
How can I apply mask to a color image in latest python binding (cv2)? In previous python binding the simplest way was to use cv.Copy e.g.
cv.Copy(dst, src, mask)
But this function is not available in cv2 binding. Is there any workaround without using boilerplate code?
Here, you could use cv2.bitwise_and function if you already have the mask image.
For check the below code:
img = cv2.imread('lena.jpg')
mask = cv2.imread('mask.png',0)
res = cv2.bitwise_and(img,img,mask = mask)
The output will be as follows for a lena image, and for rectangular mask.
Well, here is a solution if you want the background to be other than a solid black color. We only need to invert the mask and apply it in a background image of the same size and then combine both background and foreground. A pro of this solution is that the background could be anything (even other image).
This example is modified from Hough Circle Transform. First image is the OpenCV logo, second the original mask, third the background + foreground combined.
# http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_houghcircles/py_houghcircles.html
import cv2
import numpy as np
# load the image
img = cv2.imread('E:\\FOTOS\\opencv\\opencv_logo.png')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# detect circles
gray = cv2.medianBlur(cv2.cvtColor(img, cv2.COLOR_RGB2GRAY), 5)
circles = cv2.HoughCircles(gray, cv2.HOUGH_GRADIENT, 1, 20, param1=50, param2=50, minRadius=0, maxRadius=0)
circles = np.uint16(np.around(circles))
# draw mask
mask = np.full((img.shape[0], img.shape[1]), 0, dtype=np.uint8) # mask is only
for i in circles[0, :]:
cv2.circle(mask, (i[0], i[1]), i[2], (255, 255, 255), -1)
# get first masked value (foreground)
fg = cv2.bitwise_or(img, img, mask=mask)
# get second masked value (background) mask must be inverted
mask = cv2.bitwise_not(mask)
background = np.full(img.shape, 255, dtype=np.uint8)
bk = cv2.bitwise_or(background, background, mask=mask)
# combine foreground+background
final = cv2.bitwise_or(fg, bk)
Note: It is better to use the opencv methods because they are optimized.
import cv2 as cv
im_color = cv.imread("lena.png", cv.IMREAD_COLOR)
im_gray = cv.cvtColor(im_color, cv.COLOR_BGR2GRAY)
At this point you have a color and a gray image. We are dealing with 8-bit, uint8 images here. That means the images can have pixel values in the range of [0, 255] and the values have to be integers.
Let's do a binary thresholding operation. It creates a black and white masked image. The black regions have value 0 and the white regions 255
_, mask = cv.threshold(im_gray, thresh=180, maxval=255, type=cv.THRESH_BINARY)
im_thresh_gray = cv.bitwise_and(im_gray, mask)
The mask can be seen below on the left. The image on its right is the result of applying bitwise_and operation between the gray image and the mask. What happened is, the spatial locations where the mask had a pixel value zero (black), became pixel value zero in the result image. The locations where the mask had pixel value 255 (white), the resulting image retained its original gray value.
To apply this mask to our original color image, we need to convert the mask into a 3 channel image as the original color image is a 3 channel image.
mask3 = cv.cvtColor(mask, cv.COLOR_GRAY2BGR) # 3 channel mask
Then, we can apply this 3 channel mask to our color image using the same bitwise_and function.
im_thresh_color = cv.bitwise_and(im_color, mask3)
mask3 from the code is the image below on the left, and im_thresh_color is on its right.
You can plot the results and see for yourself.
cv.imshow("original image", im_color)
cv.imshow("binary mask", mask)
cv.imshow("3 channel mask", mask3)
cv.imshow("im_thresh_gray", im_thresh_gray)
cv.imshow("im_thresh_color", im_thresh_color)
cv.waitKey(0)
The original image is lenacolor.png that I found here.
Answer given by Abid Rahman K is not completely correct. I also tried it and found very helpful but got stuck.
This is how I copy image with a given mask.
x, y = np.where(mask!=0)
pts = zip(x, y)
# Assuming dst and src are of same sizes
for pt in pts:
dst[pt] = src[pt]
This is a bit slow but gives correct results.
EDIT:
Pythonic way.
idx = (mask!=0)
dst[idx] = src[idx]
The other methods described assume a binary mask. If you want to use a real-valued single-channel grayscale image as a mask (e.g. from an alpha channel), you can expand it to three channels and then use it for interpolation:
assert len(mask.shape) == 2 and issubclass(mask.dtype.type, np.floating)
assert len(foreground_rgb.shape) == 3
assert len(background_rgb.shape) == 3
alpha3 = np.stack([mask]*3, axis=2)
blended = alpha3 * foreground_rgb + (1. - alpha3) * background_rgb
Note that mask needs to be in range 0..1 for the operation to succeed. It is also assumed that 1.0 encodes keeping the foreground only, while 0.0 means keeping only the background.
If the mask may have the shape (h, w, 1), this helps:
alpha3 = np.squeeze(np.stack([np.atleast_3d(mask)]*3, axis=2))
Here np.atleast_3d(mask) makes the mask (h, w, 1) if it is (h, w) and np.squeeze(...) reshapes the result from (h, w, 3, 1) to (h, w, 3).