Writing CNN to classify pictures. I encountered a problem with garbage pixels. image The resulting network gives ~90% quality, it seems that it can be improved by averaging these pixels.
Is there a ready algorithm in numpy, opencv, etc. that allows to do this? Not normally smoothing, but specifically for these pixels. Or do I have to do it manually?
I agree that if you are using a CNN for some kind of classification you should train the network to handle this kind of noisy images. Maybe augment your dataset with some salt and pepper noise. Anyway, here's a possible solution for filtering out the outliers. It builds on the idea proposed by fmw42. These are the steps:
Apply a median blur with a large kernel
Convert the original (unprocessed) image to grayscale
(Invert) Threshold the grayscale image with a low threshold value (e.g, 5) to create a mask for the outliers close to 0.
Threshold the grayscale image with a high low threshold value (e.g., 250) to create a mask for the outliers close to 255.
Combine both mask to create the outlier mask
Use the outlier mask to adaptive-filter the original input image substituting the median values where necessary.
Let's see the code:
# Imports:
import cv2
import numpy as np
# image path
path = "D://opencvImages//noisyNumbers//"
fileName = "noisy01.png"
# Reading an image in default mode:
inputImage = cv2.imread(path + fileName)
# Apply median filter:
filteredImage = cv2.medianBlur(inputImage, ksize=11)
# Convert input image to grayscale:
grayscaleImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2GRAY)
The median filtering with a kernel size of 11 looks like this:
The outliers are practically gone. Now, let's put this aside for the moment and compute a pair of binary masks for both outliers:
# Get low mask:
_, lowMask = cv2.threshold(grayscaleImage, 5, 255, cv2.THRESH_BINARY_INV)
# Get high mask:
_, highMask = cv2.threshold(grayscaleImage, 250, 255, cv2.THRESH_BINARY)
# Create outliers mask:
outliersMask = cv2.add(lowMask, highMask)
The outliers mask is this:
Now, you really don't provide your original data. You provide an image most likely plotted using matplotlib. That's a problem, because the image you posted is processed and compressed. This results in some sharp edges around the outliers on the original image. One straightforward solution is to dilate the outliers mask a little bit to cover this compression artifacts:
# Set kernel (structuring element) size:
kernelSize = 3
# Set operation iterations:
opIterations = 1
# Get the structuring element:
maxKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernelSize, kernelSize))
# Apply dilation:
outliersMask = cv2.dilate(outliersMask, maxKernel)
The outliers mask now looks like this:
Ok, let's adaptive-filter the original input using the median blurred image and the outliers mask. Just make sure to reshape all the numpy arrays to their proper size for broadcasting:
# Re-shape the binary mask to a 3-channeled image:
augmentedBinary = cv2.merge([outliersMask, outliersMask, outliersMask])
# Apply the adaptive filter:
cleanedImage = np.where(augmentedBinary == (255, 255, 255), filteredImage, inputImage)
# Show the result
cv2.imshow("Adaptive Filtering", cleanedImage)
cv2.waitKey(0)
For the first image, this is the result:
More results:
I have a contour that is represented in a numpy array in such a way that the boundary points have 1 and the rest are 0. An example image is shown below. How could I smooth this contour ?
I am trying the get a contour that is smoother than the one in the image right now
You can use active_contour to match a spline with a set number of points to your contour. If you lower the number of points you get a smoother contour.
You can use morphological transformations such as erosion and dilation. OpenCV has a nice tutorial here: https://docs.opencv.org/master/d9/d61/tutorial_py_morphological_ops.html
As #Mad Physicist says you can use morphological techniques I recommend using Opening (Dilation and erosion) followed by a dilation as follows:
import cv2
import numpy as np
img = cv2.imread('img.png',0)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(11,11))
opening = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel,iterations= 5)
opening = cv2.dilate(opening,kernel,iterations = 1)
img_window = np.hstack((img,opening))
cv2.imshow("image",img_window)
cv2.waitKey(0)
cv2.destroyAllWindows()
results:
I want to eliminate gray lines in 16 bit image you can see.
Final goal is remove line in object image(second image) with background image(first image).
I thought it need FFT, but i don't know how FFT applied. There will be other ways, too.
please help me.
One simple way using Python/OpenCV is to use morphology close multiple times with a small vertical rectangular kernel.
Input:
import cv2
import numpy as np
img = cv2.imread('lines.png')
# do morphology multiple times to remove horizontal lines
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1,5))
result = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel, iterations = 9)
# write result to disk
cv2.imwrite("lines_removed.png", result)
# display it
cv2.imshow("result", result)
cv2.waitKey(0)
However, it will modify the image everywhere slightly
I have this python code that applies a series thresholding to an image of an eye so that it would it would be able to detect the pupil. I wrote this code using python 2.7 in windows 10. It actually worked great since I was able to get my desired output.
Here is the code that I wrote in windows 10:
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('C:\Users\User\Documents\module4\input\left.jpg',0)
image = cv2.medianBlur(img,5)
#Apply Adaptive Threshold with Laplacian
th = cv2.adaptiveThreshold(image,255,cv2.ADAPTIVE_THRESH_MEAN_C,
cv2.THRESH_BINARY,11,2)
laplacian = cv2.Laplacian(th,cv2.CV_64F)
cv2.imwrite('C:\Users\User\Documents\module4\output\output1.jpg', laplacian)
#Apply Inverse Binary Threshold
binthresh = cv2.imread('C:\Users\User\Documents\module4\output\output1.jpg',0)
ret,thresh2 = cv2.threshold(laplacian,127,255,cv2.THRESH_BINARY_INV)
cv2.imwrite('C:\Users\User\Documents\module4\output\output2.jpg', thresh2)
#Apply First Otsu's Threshold
otsuthresh1 = cv2.imread('C:\Users\User\Documents\module4\output\output2.jpg',0)
blur = cv2.GaussianBlur(otsuthresh1,(5,5),0)
ret3,th3 = cv2.threshold(blur,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
cv2.imwrite('C:\Users\User\Documents\module4\output\output3.jpg', th3)
#Apply Gaussian Blur
gaussblur = cv2.imread('C:\Users\User\Documents\module4\output\output3.jpg',0)
blur2 = cv2.GaussianBlur(gaussblur,(5,5),0)
cv2.imwrite('C:\Users\User\Documents\module4\output\output4.jpg', blur2)
#Apply Second Otsu's Threshold
otsuthresh2 = cv2.imread('C:\Users\User\Documents\module4\output\output4.jpg',0)
blur3 = cv2.GaussianBlur(otsuthresh2,(5,5),0)
ret4,th4 = cv2.threshold(blur3,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
#Apply Circular Hough Transform
circles = cv2.HoughCircles(th4,cv2.HOUGH_GRADIENT,1,20,param1=50,param2=30,minRadius=0,maxRadius=0)
circles = np.uint16(np.around(circles))
for i in circles[0,:]:
# draw the outer circle
cv2.circle(th4,(i[0],i[1]),i[2],(100,100,0),2)
# draw the center of the circle
cv2.circle(th4,(i[0],i[1]),2,(0,50,100),3)
cv2.imshow('combined', th4)
cv2.imwrite('C:\Users\User\Documents\module4\output\output5.jpg', th4)
cv2.waitKey(0)
cv2.destroyAllWindows()
Here is a screenshot of all the outputs of the code (including the original input image):
I tried running this same code in my raspberry pi, I just changed the file path of the input image as well as where to store the output images.
Here is the code that I ran in my Raspberry Pi:
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('/home/pi/IPD/images/image1.jpg',0)
image = cv2.medianBlur(img,5)
#Apply Adaptive Threshold with Laplacian
th = cv2.adaptiveThreshold(image,255,cv2.ADAPTIVE_THRESH_MEAN_C,
cv2.THRESH_BINARY,11,2)
laplacian = cv2.Laplacian(th,cv2.CV_64F)
#Apply Inverse Binary Threshold
binthresh = cv2.imread('/home/pi/IPD/temp/output1.jpg',0)
ret,thresh2 = cv2.threshold(binthresh,127,255,cv2.THRESH_BINARY_INV)
cv2.imwrite('/home/pi/IPD/temp/output2.jpg', thresh2)
#Apply First Otsu's Threshold
otsuthresh1 = cv2.imread('/home/pi/IPD/temp/output2.jpg',0)
blur = cv2.GaussianBlur(otsuthresh1,(5,5),0)
ret3,th3 = cv2.threshold(blur,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
cv2.imwrite('/home/pi/IPD/temp/output3.jpg', th3)
#Apply Gaussian Blur
gaussblur = cv2.imread('/home/pi/IPD/temp/output3.jpg',0)
blur2 = cv2.GaussianBlur(gaussblur,(5,5),0)
cv2.imwrite('/home/pi/IPD/temp/output4.jpg', blur2)
#Apply Second Otsu's Threshold
otsuthresh2 = cv2.imread('C/home/pi/IPD/temp/output4.jpg',0)
blur3 = cv2.GaussianBlur(otsuthresh2,(5,5),0)
ret4,th4 = cv2.threshold(blur3,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
#Apply Circular Hough Transform
circles = cv2.HoughCircles(th4,cv2.HOUGH_GRADIENT,1,20,param1=50,param2=30,minRadius=0,maxRadius=0)
circles = np.uint16(np.around(circles))
for i in circles[0,:]:
# draw the outer circle
cv2.circle(th4,(i[0],i[1]),i[2],(100,100,0),2)
# draw the center of the circle
cv2.circle(th4,(i[0],i[1]),2,(0,50,100),3)
cv2.imshow('combined', th4)
cv2.imwrite('/home/pi/IPD/images/final.jpg', th4)
cv2.waitKey(0)
cv2.destroyAllWindows()
However I get the following error:
Traceback (most recent call last):
File "/home/pi/IPD/mod4.py", line 18, in
ret,thresh2 = cv2.threshold(binthresh,127,255,cv2.THRESH_BINARY_INV)
error: /build/opencv-ISmtkH/opencv-2.4.9.1+dfsg/modules/core/src/matrix.cpp:269: error: (-215) m.dims >= 2 in function Mat
Actually, I've also encountered this error when I first wrote the code in windows 10 but I solved it by writing the newly thresholded image and just loading it again (as you can see in my code. I know it's an inefficient way) so that I can apply a new threshold to it. I've tried searching for possible explanations why this might be and I figured it has something to do with how many channels the signal I'm inputting is (I think). However, I'm still new to using opencv and image processing in general and I really don't understand the concept really that well (even though I've already researched it).
If you guys can help me and point me in the right direction, I would be really grateful. And also if you guys can suggest how I can avoid storing the newly thresholded image and loading it again (which is really an inefficient way of going about this) without causing any errors, I would really, really appreciate it.
I spotted 2 errors, I hope those are the only 2.
1) In your first code you have:
laplacian = cv2.Laplacian(th,cv2.CV_64F)
cv2.imwrite('C:\Users\User\Documents\module4\output\output1.jpg', laplacian)
#Apply Inverse Binary Threshold
binthresh = cv2.imread('C:\Users\User\Documents\module4\output\output1.jpg',0)
ret,thresh2 = cv2.threshold(laplacian,127,255,cv2.THRESH_BINARY_INV)
here you are doing the laplacian operator and saving it in a CV_64F image (doubles), but threshold ONLY takes CV_8U or CV_32F. Here you have two options, one is to change this 64F to 32F or to use the function normalize and convert it to 8U image. Something like:
cv2.normalize(laplacian, output1, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)
2) In the second code you are missing:
cv2.imwrite('C:\Users\User\Documents\module4\output\output1.jpg', laplacian)
So, you are not saving such an image, thus you are not loading it either... no image, an error jumps out.
General suggestions, always use imshow to see what is going on until what point. Use relative paths for the saving and loading of the temp images, this way you only change the input path.
I'm currently using morphology transformations on binary images with OpenCV 2.4
I just noticed that using the built-in functions of OpenCV, all my pixels' positions are shifted right and down by one (i.e. the pixel previously located at (i,j) is now located at (i+1, j+1))
import cv2
import numpy as np
from skimage.morphology import opening
image = cv2.imread('input.png', 0)
kernel = np.ones((16,16), np.uint8)
opening_opencv = cv2.morphologyEx(image, cv2.MORPH_OPEN, kernel)
opening_skimage = opening(image, kernel)
cv2.imwrite('opening_opencv.png', opening_opencv)
cv2.imwrite('opening_skimage.png', opening_skimage)
Input :
Output :
As I didn't understand why, I just tied the same operation using skimage, and it doesn't make this "gap" during the morphological transformation.
Ouput :
Any idea about this issue ?
Thanks !
It is the way you commented, but the exact inverse :)
Structuring elements with even size lead to shifts, there is no middle pixel. With an odd size, you get a middle pixel and (n-1)/2 pixels at each size.
Other way of saying it is that a SE with odd sizes is symmetric and even size is asymmetric.