Related
Original Image
Expected Output.
I am using this code for translating a specific part into the same image, but output is not changing,
import numpy as np
import cv2 as cv
img = cv.imread('eye0.jpg', 0)
rows, cols = img.shape
roi = img[200: 300, 360: 450]
M = np.float32([[1, 0, 100], [0, 1, 50]])
dst = cv.warpAffine(roi, M, roi.shape)
cv.imshow('img', img)
cv.imshow('img', dst)
cv.waitKey(0)
cv.destroyAllWindows()
I see no changes from original image. How can I do so? Moreover, as an openCV newbie I would like to know which function should I use/explore here to get my purpose served?
Copy() function can help you instead of warpAffine(). You can check here also:
Here is output and code:
import numpy as np
import cv2 as cv
img = cv.imread('eye.jpg', 1)
#rows, cols = img.shape
roi = img[80: 100, 140: 160]
img2 = img.copy()
img2[95:115, 140:160]=roi
cv.imshow('img', img)
cv.imshow('imaag', img2)
cv.waitKey(0)
cv.destroyAllWindows()
**Image after warp affine tranformation... but for circling the part it seem difficult..
**
import numpy as np
import cv2 as cv
img = cv.imread('eye.jpg')
roi = img[78: 100, 130: 160]
M = np.float32([[1, 0, 6], [0, 1, 4]])
dst = cv.warpAffine(roi, M, (30, 22))
img[80:102, 132:162] = dst
cv.imwrite("newimage.jpg",img)
cv.imshow('img', img)
cv.imshow('img1',dst)
cv.waitKey(0)
cv.destroyAllWindows()
I have tried several methods of image segmentation, while they're not working.
I don't want to use Deep-learning method to solve this problem, OpenCV-Python is what I am currently learning.
Since considering the color of the background, the large white tray, is too similar with the foreground, especially the small white dishes, I have followed this link, aiming to get a satisfied segmentation result.
OpenCV sharpen the edges (edges with no holes)
While the result is still disappointing......
import cv2
import numpy as np
img = cv2.imread('2.jpg')
img_hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
img_h, img_s, img_v = cv2.split(img_hsv)
img_s = cv2.GaussianBlur(img_s, (9, 9), 2, 2)
img_s = cv2.convertScaleAbs(img_s, 0.5, 0.5)
img_s = np.array(img_s, np.float32)
img_new_1 = cv2.Sobel(img_s, -1, 1, 0)
img_new_2 = cv2.Sobel(img_s, -1, 0, 1)
img_new_1 = cv2.multiply(img_new_1, img_new_1)
img_new_2 = cv2.multiply(img_new_2, img_new_2)
img_grad_abs_val_approx = cv2.pow((img_new_1 + img_new_2), 0.5)
filtered = cv2.GaussianBlur(img_grad_abs_val_approx, (9, 9), 2, 2)
mean, std = cv2.meanStdDev(filtered)
_, filtered_1 = cv2.threshold(filtered, mean[0] + std[0], 1.0, cv2.THRESH_TOZERO)
_, filtered_2 = cv2.threshold(filtered, mean[0] + std[0], 1.0, cv2.THRESH_BINARY)
filtered_2 = np.array(filtered_2, np.uint8)
contour = cv2.findContours(filtered_2, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
contour_img = np.zeros((filtered_2.shape[0], filtered_2.shape[1]), np.uint8)
cv2.drawContours(contour_img, contour[1], -1, 255)
cv2.imshow('image', filtered_1 / 50)
cv2.imshow('image_contour', contour_img)
cv2.waitKey(0)
2.jpg
I am totally confused about how to use traditional method(opencv) to solve this problem, the white dishes and the candies in them are so annoying.
Problem
Using this answer to create a segmentation program, it is counting the objects incorrectly. I noticed that alone objects are being ignored or poor imaging acquisition.
I counted 123 objects and the program returns 117, as can be seen, bellow. The objects circled in red seem to be missing:
Using the following image from a 720p webcam:
Code
import cv2
import numpy as np
import matplotlib.pyplot as plt
from scipy.ndimage import label
import urllib.request
# https://stackoverflow.com/a/14617359/7690982
def segment_on_dt(a, img):
border = cv2.dilate(img, None, iterations=5)
border = border - cv2.erode(border, None)
dt = cv2.distanceTransform(img, cv2.DIST_L2, 3)
plt.imshow(dt)
plt.show()
dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(np.uint8)
_, dt = cv2.threshold(dt, 140, 255, cv2.THRESH_BINARY)
lbl, ncc = label(dt)
lbl = lbl * (255 / (ncc + 1))
# Completing the markers now.
lbl[border == 255] = 255
lbl = lbl.astype(np.int32)
cv2.watershed(a, lbl)
print("[INFO] {} unique segments found".format(len(np.unique(lbl)) - 1))
lbl[lbl == -1] = 0
lbl = lbl.astype(np.uint8)
return 255 - lbl
# Open Image
resp = urllib.request.urlopen("https://i.stack.imgur.com/YUgob.jpg")
img = np.asarray(bytearray(resp.read()), dtype="uint8")
img = cv2.imdecode(img, cv2.IMREAD_COLOR)
## Yellow slicer
mask = cv2.inRange(img, (0, 0, 0), (55, 255, 255))
imask = mask > 0
slicer = np.zeros_like(img, np.uint8)
slicer[imask] = img[imask]
# Image Binarization
img_gray = cv2.cvtColor(slicer, cv2.COLOR_BGR2GRAY)
_, img_bin = cv2.threshold(img_gray, 140, 255,
cv2.THRESH_BINARY)
# Morphological Gradient
img_bin = cv2.morphologyEx(img_bin, cv2.MORPH_OPEN,
np.ones((3, 3), dtype=int))
# Segmentation
result = segment_on_dt(img, img_bin)
plt.imshow(np.hstack([result, img_gray]), cmap='Set3')
plt.show()
# Final Picture
result[result != 255] = 0
result = cv2.dilate(result, None)
img[result == 255] = (0, 0, 255)
plt.imshow(result)
plt.show()
Question
How to count the missing objects?
Answering your main question, watershed does not remove single objects. Watershed was functioning fine in your algorithm. It receives the predefined labels and perform segmentation accordingly.
The problem was the threshold you set for the distance transform was too high and it removed the weak signal from the single objects, thus preventing the objects from being labeled and sent to the watershed algorithm.
The reason for the weak distance transform signal was due to the improper segmentation during the color segmentation stage and the difficulty of setting a single threshold to remove noise and extract signal.
To remedy this, we need to perform proper color segmentation and use adaptive threshold instead of the single threshold when segmenting the distance transform signal.
Here is the code i modified. I have incorporated color segmentation method by #user1269942 in the code. Extra explanation is in the code.
import cv2
import numpy as np
import matplotlib.pyplot as plt
from scipy.ndimage import label
import urllib.request
# https://stackoverflow.com/a/14617359/7690982
def segment_on_dt(a, img, img_gray):
# Added several elliptical structuring element for better morphology process
struct_big = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5))
struct_small = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
# increase border size
border = cv2.dilate(img, struct_big, iterations=5)
border = border - cv2.erode(img, struct_small)
dt = cv2.distanceTransform(img, cv2.DIST_L2, 3)
dt = ((dt - dt.min()) / (dt.max() - dt.min()) * 255).astype(np.uint8)
# blur the signal lighty to remove noise
dt = cv2.GaussianBlur(dt,(7,7),-1)
# Adaptive threshold to extract local maxima of distance trasnform signal
dt = cv2.adaptiveThreshold(dt, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 21, -9)
#_ , dt = cv2.threshold(dt, 2, 255, cv2.THRESH_BINARY)
# Morphology operation to clean the thresholded signal
dt = cv2.erode(dt,struct_small,iterations = 1)
dt = cv2.dilate(dt,struct_big,iterations = 10)
plt.imshow(dt)
plt.show()
# Labeling
lbl, ncc = label(dt)
lbl = lbl * (255 / (ncc + 1))
# Completing the markers now.
lbl[border == 255] = 255
plt.imshow(lbl)
plt.show()
lbl = lbl.astype(np.int32)
cv2.watershed(a, lbl)
print("[INFO] {} unique segments found".format(len(np.unique(lbl)) - 1))
lbl[lbl == -1] = 0
lbl = lbl.astype(np.uint8)
return 255 - lbl
# Open Image
resp = urllib.request.urlopen("https://i.stack.imgur.com/YUgob.jpg")
img = np.asarray(bytearray(resp.read()), dtype="uint8")
img = cv2.imdecode(img, cv2.IMREAD_COLOR)
## Yellow slicer
# blur to remove noise
img = cv2.blur(img, (9,9))
# proper color segmentation
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv, (0, 140, 160), (35, 255, 255))
#mask = cv2.inRange(img, (0, 0, 0), (55, 255, 255))
imask = mask > 0
slicer = np.zeros_like(img, np.uint8)
slicer[imask] = img[imask]
# Image Binarization
img_gray = cv2.cvtColor(slicer, cv2.COLOR_BGR2GRAY)
_, img_bin = cv2.threshold(img_gray, 140, 255,
cv2.THRESH_BINARY)
plt.imshow(img_bin)
plt.show()
# Morphological Gradient
# added
cv2.morphologyEx(img_bin, cv2.MORPH_OPEN,cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3)),img_bin,(-1,-1),10)
cv2.morphologyEx(img_bin, cv2.MORPH_ERODE,cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3)),img_bin,(-1,-1),3)
plt.imshow(img_bin)
plt.show()
# Segmentation
result = segment_on_dt(img, img_bin, img_gray)
plt.imshow(np.hstack([result, img_gray]), cmap='Set3')
plt.show()
# Final Picture
result[result != 255] = 0
result = cv2.dilate(result, None)
img[result == 255] = (0, 0, 255)
plt.imshow(result)
plt.show()
Final results :
124 Unique items found.
An extra item was found because one of the object was divided to 2.
With proper parameter tuning, you might get the exact number you are looking. But i would suggest getting a better camera.
Looking at your code, it is completely reasonable so I'm just going to make one small suggestion and that is to do your "inRange" using HSV color space.
opencv docs on color spaces:
https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_colorspaces/py_colorspaces.html
another SO example using inRange with HSV:
How to detect two different colors using `cv2.inRange` in Python-OpenCV?
and a small code edits for you:
img = cv2.blur(img, (5,5)) #new addition just before "##yellow slicer"
## Yellow slicer
#mask = cv2.inRange(img, (0, 0, 0), (55, 255, 255)) #your line: comment out.
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) #new addition...convert to hsv
mask = cv2.inRange(hsv, (0, 120, 120), (35, 255, 255)) #new addition use hsv for inRange and an adjustment to the values.
Improving Accuracy
Detecting missing objects
im_1, im_2, im_3
I've count 12 missing objects: 2, 7, 8, 11, 65, 77, 78, 84, 92, 95, 96. edit: 85 too
117 found, 12 missing, 6 wrong
1° Attempt: Decrease Mask Sensibility
#mask = cv2.inRange(img, (0, 0, 0), (55, 255, 255)) #Current
mask = cv2.inRange(img, (0, 0, 0), (80, 255, 255)) #1' Attempt
inRange documentaion
im_4, im_5, im_6, im_7
[INFO] 120 unique segments found
120 found, 9 missing, 6 wrong
I have an image. Like this:
I detect a subject(which is a person in this case) & it masks the image like this:
I want the background of the subject to be blurrred. Like this:
Below is the code I have tried. the following code only blurs
import cv2
import numpy as np
from matplotlib import pyplot as plt
import os
path = 'selfies\\'
selfImgs = os.listdir(path)
for image in selfImgs:
img = cv2.imread(path+image)
img=cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
blur = cv2.blur(img,(10,10))
#canny = cv2.Canny(blur, 10, 30)
#plt.imshow(canny)
plt.imshow(blur)
j=cv2.cvtColor(blur, cv2.COLOR_BGR2RGB)
print(image)
cv2.imwrite('blurred\\'+image+".jpg",j)
Is there any way by which I can blur only specific part/parts of the image.
This project is based on https://github.com/matterport/Mask_RCNN
I can provide more information if required.
I have an approach in numpy :-
final_image = original * mask + blurred * (1-mask)
You may use np.where() method to select the pixels where you want blurred values and then replace them as:
import cv2
import numpy as np
img = cv2.imread("/home/user/Downloads/lena.png")
blurred_img = cv2.GaussianBlur(img, (21, 21), 0)
mask = np.zeros((512, 512, 3), dtype=np.uint8)
mask = cv2.circle(mask, (258, 258), 100, np.array([255, 255, 255]), -1)
out = np.where(mask==np.array([255, 255, 255]), img, blurred_img)
cv2.imwrite("./out.png", out)
As ZdaR said:
import cv2
import numpy as np
img = cv2.imread("/home/user/Downloads/lena.png")
blurred_img = cv2.GaussianBlur(img, (21, 21), 0)
mask = np.zeros((512, 512, 3), dtype=np.uint8)
mask = cv2.circle(mask, (258, 258), 100, np.array([255, 255, 255]), -1)
out = np.where(mask==np.array([255, 255, 255]), img, blurred_img)
cv2.imwrite("./out.png", out)
This is good idea but I've got same error as penta:
#ZdaR I get TypeError: Scalar value for argument 'color' is not numeric
A simple solution is to modify color value when you create Circle:
mask = cv2.circle(mask, (258, 258), 100, (255, 255,255), -1)
just CHANGE np.array([255,255,255]) to (255,255,255).
I don't know what tools you use for subject detection, but if you have a way to copy the subject, you can first copy the whole image and then blur it. finally, you can copy the subject on the blurred image.
if it gives false and true on pixels in oppose to giving borders, you can just bitwise it.
I have two images, both with alpha channels. I want to put one image over the other, resulting in a new image with an alpha channel, just as would occur if they were rendered in layers. I would like to do this with the Python Imaging Library, but recommendations in other systems would be fantastic, even the raw math would be a boon; I could use NumPy.
This appears to do the trick:
from PIL import Image
bottom = Image.open("a.png")
top = Image.open("b.png")
r, g, b, a = top.split()
top = Image.merge("RGB", (r, g, b))
mask = Image.merge("L", (a,))
bottom.paste(top, (0, 0), mask)
bottom.save("over.png")
Pillow 2.0 now contains an alpha_composite function that does this.
img3 = Image.alpha_composite(img1, img2)
I couldn't find an alpha composite function in PIL, so here is my attempt at implementing it with numpy:
import numpy as np
from PIL import Image
def alpha_composite(src, dst):
'''
Return the alpha composite of src and dst.
Parameters:
src -- PIL RGBA Image object
dst -- PIL RGBA Image object
The algorithm comes from http://en.wikipedia.org/wiki/Alpha_compositing
'''
# http://stackoverflow.com/a/3375291/190597
# http://stackoverflow.com/a/9166671/190597
src = np.asarray(src)
dst = np.asarray(dst)
out = np.empty(src.shape, dtype = 'float')
alpha = np.index_exp[:, :, 3:]
rgb = np.index_exp[:, :, :3]
src_a = src[alpha]/255.0
dst_a = dst[alpha]/255.0
out[alpha] = src_a+dst_a*(1-src_a)
old_setting = np.seterr(invalid = 'ignore')
out[rgb] = (src[rgb]*src_a + dst[rgb]*dst_a*(1-src_a))/out[alpha]
np.seterr(**old_setting)
out[alpha] *= 255
np.clip(out,0,255)
# astype('uint8') maps np.nan (and np.inf) to 0
out = out.astype('uint8')
out = Image.fromarray(out, 'RGBA')
return out
For example given these two images,
img1 = Image.new('RGBA', size = (100, 100), color = (255, 0, 0, 255))
draw = ImageDraw.Draw(img1)
draw.rectangle((33, 0, 66, 100), fill = (255, 0, 0, 128))
draw.rectangle((67, 0, 100, 100), fill = (255, 0, 0, 0))
img1.save('/tmp/img1.png')
img2 = Image.new('RGBA', size = (100, 100), color = (0, 255, 0, 255))
draw = ImageDraw.Draw(img2)
draw.rectangle((0, 33, 100, 66), fill = (0, 255, 0, 128))
draw.rectangle((0, 67, 100, 100), fill = (0, 255, 0, 0))
img2.save('/tmp/img2.png')
alpha_composite produces:
img3 = alpha_composite(img1, img2)
img3.save('/tmp/img3.png')