Related
I am trying to estimate the area of vegetation in square meters on satellite photos, from the colors. I don't have a training dataset, and therefore cannot do machine learning. So I know the results will not be very good, but I try anyway.
To do this, I apply a filter on the colors thanks to cv2.inRange.
import numpy as np
import cv2
img = cv2.imread('staticmap.png')
upperbound = np.array([70, 255,255])
lowerbound = np.array([40, 40,40])
mask = cv2.inRange(img, lowerbound, upperbound)
imask = mask>0
white = np.full_like(img, [255,255,255], np.uint8)
result = np.zeros_like(img, np.uint8)
result[imask] = white[imask]
cv2.imshow(winname = 'satellite image', mat = img)
cv2.imshow('vegetation detection', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
This gives the following results
So it seems that the detection is not too bad.
Now, I would like, from the density of white pixels, detect the areas where there is vegetation and areas where there is not. I imagine an output like this :
Are there any open cv functions that can do this?
You could consider using a Gaussian blur followed by Otsu thresholding like this:
import cv2
# Load image as greyscale
im = cv2.imread('veg.jpg', cv2.IMREAD_GRAYSCALE)
# Apply blur
blur = cv2.GaussianBlur(im,(19,19),0)
# Otsu threshold
_,thr = cv2.threshold(blur,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
I need to create masks for 100.000 images, this code runs on cpu and creates ~500 masks a hour. Is there a way I can speed this up either by parallelising or running code on gpu? I'm okay with solutions that make me heavily rewrite code as long as it speeds up the process.
I tried compiling opencv library myself with cuda support, however I couldn't get most of cv2 methods I use here to run on gpu.
This is my code
Edit #1
Added import list and comments to code.
Added input and output images.
import cv2
import matplotlib.pyplot as plt
import numpy as np
from PIL import Image
import glob
import sys
import os
import skimage.color
import skimage.filters
import skimage.io
import skimage.viewer
grayScale = cv2.imread(filename,cv2.IMREAD_REDUCED_GRAYSCALE_4)#read image as grayscale with size reduction
kernel = cv2.getStructuringElement(1,(17,17))
blackhat = cv2.morphologyEx(grayScale, cv2.MORPH_BLACKHAT, kernel)
ret,thresh2 = cv2.threshold(blackhat,10,255,cv2.THRESH_BINARY)
dst = cv2.inpaint(newimg,thresh2,1,cv2.INPAINT_TELEA) #4 lines above are used to remove hair from image
mask = np.zeros(dst.shape[:2],np.uint8)
h,w,c = dst.shape
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)
rect = (int(0.1*w),int(0.1*h),int(0.8*w),int(0.8*h))
cv2.grabCut(dst,mask,rect,bgdModel,fgdModel,1,cv2.GC_INIT_WITH_RECT) #removes some background from image
#code for k means clustering starts here
mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
dst = dst*mask2[:,:,np.newaxis]
vectorized = dst.reshape((-1,3))
vectorized = np.float32(vectorized)
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0) #11 lines above are used to remove some background from image
K = 4
attempts=1
ret,label,center=cv2.kmeans(vectorized,K,None,criteria,attempts,cv2.KMEANS_PP_CENTERS)
center = np.uint8(center)
labels = label.flatten()
res = center[label.flatten()]
result_image = res.reshape((dst.shape)) #k means clustering ends here
gray = cv2.cvtColor(result_image, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(gray, 10, 20, cv2.THRESH_BINARY)
result_image[thresh == 0] = 255
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
erosion = cv2.erode(result_image, kernel, iterations = 1)
blur = skimage.color.rgb2gray(erosion)
blur = skimage.filters.gaussian(blur, sigma=float(1))
histogram, bin_edges = np.histogram(blur, bins=256, range=(0, 1))
index = next((i for i, x in enumerate(histogram) if x), None)
mask = blur > bin_edges[index+1] #10 lines above are used to create mask
mask = abs(mask-255) #inverts mask
array = np.array(mask, dtype='uint8')
finimg = cv2.resize(array,None,fx=4.0,fy=4.0) #returns image to original size
plt.imsave("Masks/"+filename, finimg, cmap = plt.cm.gray) #saves result image
input image - skin mole image
output image - mask of skin mole
You might try using kmeans processing in Python/Opencv as a first step. Then get the inner contour and use that for your mask. Draw the inner contour as white filled on a black background. You may need to use morphology to clean the kmeans results first
Input:
Kmeans 2:
Kmeans 3:
Kmeans 4:
I am doing OCR to extract information from the ID card. However, accuracy is quite low.
My assumption is that removing the background will make OCR more accurate.
I use the ID scanner machine (link) to obtain the grey image below. It seems that the machine uses IR instead of image processing.
Does anyone knows how to get the same result by using Opencv or tools (photoshop, gimp, etc)?
Thanks in advance.
Here are two more methods: adaptive thresholding and division normalization.
Input:
import cv2
import numpy as np
# read image
img = cv2.imread("green_card.jpg")
# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# do adaptive threshold on gray image
thresh1 = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 51, 25)
# write results to disk
cv2.imwrite("green_card_thresh1.jpg", thresh1)
# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_RECT , (11,11))
morph = cv2.morphologyEx(gray, cv2.MORPH_DILATE, kernel)
# divide gray by morphology image
division = cv2.divide(gray, morph, scale=255)
# threshold
thresh2 = cv2.threshold(division, 0, 255, cv2.THRESH_OTSU )[1]
# write results to disk
cv2.imwrite("green_card_thresh2.jpg", thresh2)
# display it
cv2.imshow("thresh1", thresh1)
cv2.imshow("thresh2", thresh2)
cv2.waitKey(0)
Adaptive Thresholding Result:
Division Normalization Result:
EDIT:
since there are different lighting conditions, contrast adjustment is added here.
The simple approache in my mind to solve your issue is that: since the undesired background colours are Green and Red, and the desired font colour is Black, simply suppress the Red and green colours as following:
import numpy as np
import matplotlib.pyplot as plt
from skimage.io import imread, imsave
from skimage.color import rgb2gray
from skimage.filters import threshold_otsu
from skimage import exposure
def adjustContrast(img):
p2, p98 = np.percentile(img, (2, 98))
img_rescale = exposure.rescale_intensity(img, in_range=(p2, p98))
return img_rescale
# Read the image
img = imread('ID_OCR.jpg')
# Contrast Adjustment for each channel
img[:,:,0] = adjustContrast(img[:,:,0]) # R
img[:,:,1] = adjustContrast(img[:,:,1]) # G
img[:,:,2] = adjustContrast(img[:,:,2]) # B
# # Supress unwanted colors
img[img[...,0] > 100] = 255 # R
img[img[...,1] > 100] = 255 # B
# Convert the image to graylevel
img = rgb2gray(img)
# Rescale into 0-255
img = 255*img.astype(np.uint8)
# Save the results
imsave('Result.png', img)
The image will look like:
The Results are not optimal, because also your image resolution isn't high.
At the end, there are many solutions, and improvements, also you can use Morphology to make it look nicer, this is just a simple proposal to solve the problem.
I'm trying to crop an object from an image, and paste it on another image. Examining the method in this answer, I've successfully managed to do that. For example:
The code (show_mask_applied.py):
import sys
from pathlib import Path
from helpers_cv2 import *
import cv2
import numpy
img_path = Path(sys.argv[1])
img = cmyk_to_bgr(str(img_path))
threshed = threshold(img, 240, type=cv2.THRESH_BINARY_INV)
contours = find_contours(threshed)
mask = mask_from_contours(img, contours)
mask = dilate_mask(mask, 50)
crop = cv2.bitwise_or(img, img, mask=mask)
bg = cv2.imread("bg.jpg")
bg_mask = cv2.bitwise_not(mask)
bg_crop = cv2.bitwise_or(bg, bg, mask=bg_mask)
final = cv2.bitwise_or(crop, bg_crop)
cv2.imshow("debug", final)
cv2.waitKey(0)
cv2.destroyAllWindows()
helpers_cv2.py:
from pathlib import Path
import cv2
import numpy
from PIL import Image
from PIL import ImageCms
from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True
def cmyk_to_bgr(cmyk_img):
img = Image.open(cmyk_img)
if img.mode == "CMYK":
img = ImageCms.profileToProfile(img, "Color Profiles\\USWebCoatedSWOP.icc", "Color Profiles\\sRGB_Color_Space_Profile.icm", outputMode="RGB")
return cv2.cvtColor(numpy.array(img), cv2.COLOR_RGB2BGR)
def threshold(img, thresh=128, maxval=255, type=cv2.THRESH_BINARY):
if len(img.shape) == 3:
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
threshed = cv2.threshold(img, thresh, maxval, type)[1]
return threshed
def find_contours(img):
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (11,11))
morphed = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)
contours = cv2.findContours(morphed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
return contours[-2]
def mask_from_contours(ref_img, contours):
mask = numpy.zeros(ref_img.shape, numpy.uint8)
mask = cv2.drawContours(mask, contours, -1, (255,255,255), -1)
return cv2.cvtColor(mask, cv2.COLOR_BGR2GRAY)
def dilate_mask(mask, kernel_size=11):
kernel = numpy.ones((kernel_size,kernel_size), numpy.uint8)
dilated = cv2.dilate(mask, kernel, iterations=1)
return dilated
Now, instead of sharp edges, I want to crop with feathered/smooth edges. For example (the right one; created in Photoshop):
How can I do that?
All images and codes can be found that at this repository.
You are using a mask to select parts of the overlay image. The mask currently looks like this:
Let's first add a Gaussian blur to this mask.
mask_blurred = cv2.GaussianBlur(mask,(99,99),0)
We get to this:
Now, the remaining task it to blend the images using the alpha value in the mask, rather than using it as a logical operator like you do currently.
mask_blurred_3chan = cv2.cvtColor(mask_blurred, cv2.COLOR_GRAY2BGR).astype('float') / 255.
img = img.astype('float') / 255.
bg = bg.astype('float') / 255.
out = bg * (1 - mask_blurred_3chan) + img * mask_blurred_3chan
The above snippet is quite simple. First, transform the mask into a 3 channel image (since we want to mask all the channels). Then transform the images to float, since the masking is done in floating point. The last line does the actual work: for each pixel, blends the bg and img images according to the value in the mask. The result looks like this:
The amount of feathering is controlled by the size of the kernel in the Gaussian blur. Note that it has to be an odd number.
After this, out (the final image) is still in floating point. It can be converted back to int using:
out = (out * 255).astype('uint8')
While Paul92's answer is more than enough, I wanted to post my code anyway for any future visitor.
I'm doing this cropping to get rid of white background in some product photos. So, the main goal is to get rid of the whites while keeping the product intact. Most of the product photos have shadows on the ground. They are either the ground itself (faded), or the product's shadow, or both.
While the object detection works fine, these shadows also count as part of the object. Differentiating the shadows from the objects is not really necessary, but it results in some images that are not so desired. For example, examine the left and bottom sides of the image (shadow). The cut/crop is obviously visible, and doesn't look all that nice.
To get around this problem, I wanted to do non-rectangular crops. Using masks seems to do the job just fine. The next problem was to do the cropping with feathered/blurred edges so that I can get rid of these visible shadow cuts. With the help of Paul92, I've managed to do that. Example output (notice the missing shadow cuts, the edges are softer):
Operations on the image(s):
The code (show_mask_feathered.py, helpers_cv2.py)
import sys
from pathlib import Path
import cv2
import numpy
from helpers_cv2 import *
img_path = Path(sys.argv[1])
img = cmyk_to_bgr(str(img_path))
threshed = threshold(img, 240, type=cv2.THRESH_BINARY_INV)
contours = find_contours(threshed)
dilation_length = 51
blur_length = 51
mask = mask_from_contours(img, contours)
mask_dilated = dilate_mask(mask, dilation_length)
mask_smooth = smooth_mask(mask_dilated, odd(dilation_length * 1.5))
mask_blurred = cv2.GaussianBlur(mask_smooth, (blur_length, blur_length), 0)
mask_blurred = cv2.cvtColor(mask_blurred, cv2.COLOR_GRAY2BGR)
mask_threshed = threshold(mask_blurred, 1)
mask_contours = find_contours(mask_threshed)
mask_contour = max_contour(mask_contours)
x, y, w, h = cv2.boundingRect(mask_contour)
img_cropped = img[y:y+h, x:x+w]
mask_cropped = mask_blurred[y:y+h, x:x+w]
background = numpy.full(img_cropped.shape, (200,240,200), dtype=numpy.uint8)
output = alpha_blend(background, img_cropped, mask_cropped)
I performed a foreground extraction using grabcut on an image. Now, I want to calculate the area of the resulted foreground portion of image.
One way i thought i could do this is by thresholding image, that is foreground becomes white and background becomes black. And then by calculating the number of white pixels -
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('image.jpg')
mask = np.zeros(img.shape[:2],np.uint8)
bgdModel = np.zeros((1,65),np.float64)
fgdModel = np.zeros((1,65),np.float64)
rect = (1200,1050,1950,1250)
#applying grabcut
cv2.grabCut(img,mask,rect,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_RECT)
mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
img = img*mask2[:,:,np.newaxis]
#thresholding
cvimg = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(cvimg,0,255,cv2.THRESH_BINARY)
count = cv2.countNonZero(thresh)
print("White " , count)
cv2.imshow("image",thresh)
plt.imshow(img)
plt.colorbar()
plt.show()
But i doubt the accuracy as no. of pixels change when resolution of image changes. How can i find the area of foreground of images with a common unit of measurement for all images? Is there any other simple and appropriate method exists?
Thanks for helping...