I'm currently working on a computer vision project, and got most of my algorithm working. However I'm currently doing background subtraction manually on every image. This is because the most common background subtraction algorithms that I can find make use of thresholding, and my project should deal with backgrounds both brighter and darker than the object I want to extract.
This is the way I am subtracting the background currently (using python and the scikit stack):
val = filters.threshold_otsu(image)
return image > val
Of course, this only works with backgrounds darker than the subject.
I had the idea of finding whether or not the background is bright, and then depending on that change the sign of the inequality, but could not find a way to do that.
Is there a background subtraction algorithm which is able to handle both bright and dark backgrounds, or is there another way to solve this problem?
There are no fixed method of solving your problem generally. Foreground and background can be defined differently according to situations.
That being said, it is not impossible to use some heuristic method to make the algorithm work on your dataset. It will be helpful if you can share some of the images to give us a better understanding of your definition of foreground and background.
Here are some of heuristic method that might help:
Run Ostu thresholding with both THRESH_BINARY and THRESH_BINARY_INV. Then assuming your foreground is always centered, choose the result where the a large portion of the center region is white.
If the foreground is always larger than backgorund or vice versa, calculate the area of white region instead.
There are several automatic thresholding techiques available. One of them is Otsu.
http://www.labbookpages.co.uk/software/imgProc/otsuThreshold.html
It is implemented in opencv (https://docs.opencv.org/trunk/d7/d4d/tutorial_py_thresholding.html)
import cv2
img = cv2.imread('noisy2.png',0)
ret2,th2 = cv2.threshold(img,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
Related
I'm looking for ideas to help improve my current approach for real-time object detection using computer vision (specifically the opencv library). My goal is to accurately detect a golf-ball through image processing in a large variety of environments/lighting conditions. My detection process works quite well probably 80% of the time, but I'm hitting edge cases that cause failures that I can't ignore. The edge case I'm focusing on right now is extreme shadows being cast by the golf-ball. Here is a pair of example images. The coloured image is my source, and the black and white image is my post-processed result.
There are a few important variables to consider with my application
Source coming from a video feed, and being processed in real-time
It can be windy, so camera shake can be an issue
Camera isn't guaranteed to be incredible quality, so need to account for extra noise/not incredible resolution
I won't go into full details in the processing I'm doing to detect moving objects (Kalman Filter, Background Subtraction, ...) as in this specific example I'm failing to detect a stationary object (ie. ball has gone to rest).
Grab initial frame before any balls are in-frame as my base frame (this will be used for background subtraction)
convert image to greyscale
apply a median blur to eliminate noise, which can otherwise be pretty extreme due to a combination of camera shake, poor camera quality
apply an adaptive threshold on the image. I'm using ADAPTIVE_THRESH_GUASSIAN_C and have been tuning the block size and C constant values as best I can
apply background subtraction (I'm using the built-in CNT Subtractor)
Apply a small dilation kernel to the entire image to try and increase the size of the contours that are left after the above processing, as they can sometimes become quite small after the blur filter for example
use opencv's "findContours" with RETR_TREE, and CHAIN_APPROX_SIMPLE parameters
walk the contour hierarchy, looking for "filled in" contours. The idea being that the golf-balls should mostly be completely filled in, compared to other objects which will have an outline, and I can use the hierarchy to determine which contours are filled in or not (ie do they have child contours)
for each filled in contour, do an enclosing circle. Compare area of enclosing circle to contour area, filter by an acceptable difference to determine how circular the object is
another pass, filtering by min/max area size since I can assume the camera will always be at a similar height to "hone-in" on the object
As you can see from the images above, this approach runs into problems when the ball itself has a lot of contrast due to shadows. To me it looks like the adaptive threshold pass is filtering out the darker part of the ball (due to shadow) which creates a non-circular shape. Perhaps I need to dial in the adaptive threshhold pass to allow for a bit more contrast since we can assume shadows are always on the dark side and a ground shadow should be darker than the shaded part of the ball? I'd also like to completely eliminate the leftover outline of the ground-shadow if possible. My guess is that the edges of the shadow being slightly lighter is the reason they don't get filtered out by my adaptive threshold pass. Open to any and all suggestions :-)
Working on object detection in Python with opencv.
I have two pictures
The reference picture with no object in it.
Picture with object.
The result of the images is:
The problem is, the pattern of the reference image is now on my objects. I want to remove this pattern and I don't know how to do it. For further image processing I need the the correct outline of the objects.
Maybe you know how to fix it, or have better ideas to exctract the object.
I would be glad for your help.
Edit: 4. A black object:
As #Mark Setchell commented, the difference of the two images shows which pixels contain the object, you shouldn't try to use it as the output. Instead, find the pixels with a significant difference, and then read those pixels directly from the input image.
Here, I'm using Otsu thresholding to find what "significant difference" is. There are many other ways to do this. I then use the inverse of the mask to blank out pixels in the input image.
import PyDIP as dip
bg = dip.ImageReadTIFF('background.tif')
bg = bg.TensorElement(1) # The image has 3 channels, let's use just the green one
fg = dip.ImageReadTIFF('object.tif')
fg = fg.TensorElement(1)
mask = dip.Abs(bg - fg) # Difference between the two images
mask, t = dip.Threshold(mask, 'otsu') # Find significant differences only
mask = dip.Closing(mask, 7) # Smooth the outline a bit
fg[~mask] = 0 # Blank out pixels not in the mask
I'm using PyDIP above, not OpenCV, because I don't have OpenCV installed. You can easily do the same with OpenCV.
An alternative to smoothing the binary mask as I did there, is to smooth the mask image before thresholding, for example with dip.Gauss(mask,[2]), a Gaussian smoothing.
Edit: The black object.
What happens with this image, is that its illumination has changed significantly, or you have some automatic exposure settings in your camera. Make sure you have turned all of that off so that every image is exposed exactly the same, and that you use the raw images directly off of the camera for this, not images that have gone through some automatic enhancement procedure or even JPEG compression if you can avoid it.
I computed the median of the background image divided by the object image (fg in the code above, but for this new image), which came up to 1.073. That means that the background image is 7% brighter than the object image. I then multiplied fg by this value before computing the absolute difference:
mask = dip.Abs(fg * dip.Median(bg/fg)[0][0] - bg)
This helped a bit, but it showed that the changes in contrast are not consistent across the image.
Next, you can change the threshold selection method. Otsu assumes a bimodal histogram, and works well if you have a significant number of pixels in each group (foreground and background). Here we'll have fewer pixels belonging to the object, because only some of the object pixels have a different color from the background. The 'triangle' method is suitable in this case:
mask, t = dip.Threshold(mask, 'triangle')
This will lead to a mask that contains only some of the object pixels. You'll have to add some additional knowledge about your object (i.e. it is a rotated square) to find the full object. There are also some isolated background pixels that are being picked up by the threshold, those are easy to eliminate using a bit of blurring before the threshold or a small opening after.
Getting the exact outline of the object in this case will be impossible with your current setup. I would suggest you improve your setup by either:
making the background more uniform in illumination,
using color (so that there are fewer possible objects that match the background color so exactly as in this case),
using infrared imaging (maybe the background could have different properties from all the objects to be detected in infrared?),
using back-illumination (this is the best way if your aim is to measure the objects).
How to remove the shadows of the seeds? Also I would like to know if there is a way to change the color of all the seeds to red colour?
It seems rather easy to detect the seeds since your background is homogeneous. You can start by some simple image processing (contrast enhancement, thresholding, contour detection) to detect the seeds and then you can plot red blobs (with the same area as the detected regions) on the original image. As for the shadows, you can check this question (How to remove the shadow in image by using openCV?).
I think you can solve with this paper and it will make you interesting.
The algorithm described there works quite well and this will be a good example for you in using opencv.
And you can find the source code here
Regards.
I am trying to enhance old hand drawn maps which were digitized by scanning and this process has caused some blacked out areas in the image making the text inside them very hard to read.
I tried adaptive histogram equalization and couple of other histogram based approach using MATLAB but nothing gives me the desired result. I could probably lighten the darker shades of grey and make it look a bit better using adaptive histogram equalization but it doesn't really help with the text.
Specifically, I tried adapthisteq() with different variations which is a function available in MATLAB.
Something like this:
A = adapthisteq(I,'NumTiles',X,'clipLimit',0.01,'Distribution','uniform');
... and also tried to change the pixel values directly by having a look at image, something like this :
I(10 > I & I > 0) = 0;
I(30 > I & I > 10) = 10;
I(255 > I & I > 30) = 255;
Can I enhance the image and get an end result which has only black and white where the lines and text (basically all the information) turns into black (0) and the shades of grey and whiter regions turn into white (255 or 1)?
Is this even possible? If not, how close can I even get to it or what is the best solution to get as close as possible to the desired result. Any help is appreciated.
Here's what the original image looks like:
Here's what the result looks like after I tried out my solution using adaptive histogram equalization:
Sounds like a classic case of using adaptive thresholding. Adaptive thresholding in a general sense works by taking a look at local image pixel neighbourhoods, compute the mean intensity and seeing if a certain percentage of pixels exceed this mean intensity. If it does, we set the output to white and if not, we set this to black.
One classic approach is to use the Bradley-Roth algorithm.
If you'd like to see an explanation of the algorithm, you can take a look at a previous answer that I wrote up about it:
Bradley Adaptive Thresholding -- Confused (questions)
However, if you want the gist of it, an integral image of the grayscale version of the image is taken first. The integral image is important because it allows you to calculate the sum of pixels within a window in O(1) complexity. However, the calculation of the integral image is usually O(n^2), but you only have to do that once. With the integral image, you scan neighbourhoods of pixels of size s x s and you check to see if the average intensity is less than t% of the actual average within this s x s window then this is pixel classified as the background. If it's larger, then it's classified as being part of the foreground. This is adaptive because the thresholding is done using local pixel neighbourhoods rather than using a global threshold.
On this post: Extract a page from a uniform background in an image, there is MATLAB code I wrote that is an implementation of the Bradley-Roth algorithm, so you're more than welcome to use it.
However, for your image, the parameters I used to get some OK results was s = 12 and t = 25.
After running the algorithm, I get this image:
Be advised that it isn't perfect... but you can start to see some text that you didn't see before. Specifically at the bottom, I see Lemont Library - Built 1948.... and we couldn't see that before in the original image.
Play around with the code and the parameters, read up on the algorithm, and just try things out yourself.
Hope this helps!
I'm trying to use Python to detect how many objects are on a white surface. An example image is found at the end of this post.
I'm wondering how I should do this, mainly because the background is white and most of the time it gets detected as foreground.
What I have now in Python based on this tutorial (http://pythonvision.org/basic-tutorial) uses several libraries and detects the white as the object so count is 1, the tools get detected as background and thus are ignored:
dna = mahotas.imread('dna.jpeg')
dna = dna.squeeze()
dna = pymorph.to_gray(dna)
print dna.shape
print dna.dtype
print dna.max()
print dna.min()
dnaf = ndimage.gaussian_filter(dna, 8)
T = mahotas.thresholding.otsu(dnaf)
labeled, nr_objects = ndimage.label(dnaf > T)
print nr_objects
pylab.imshow(labeled)
pylab.jet()
pylab.show()
Are there any options for getting the white part as background and the tools as foreground?
Thanks in advance!
Example image:
The segmented image where red is foreground and blue background (the few tools merging is not a problem):
If the shadow is not a problem
You can label the images in mahotas (http://mahotas.readthedocs.org/en/latest/labeled.html) given this binary image. You can also use skimage.morphology (which uses ndlabel as was mentioned in comments). http://scikit-image.org/docs/dev/auto_examples/plot_label.html
These are examples of connect-component algorithms and are standard in any general image processing package. ImageJ also makes this quite simple.
If the shadow is a problem
Otsu thresholding returns a single value: a pixel brightness, and all you're doing is keeping all pixels that are dimmer than this threshold. This method is getting tripped up by your shadows, so you need to try another segmentation algorithm, preferably one that does local segmentation (IE it segments small regions of the image individually.)
Adaptive or local methods don't have this problem and would be really well-suited to your image's shadows:
http://scikit-image.org/docs/dev/auto_examples/plot_threshold_adaptive.html#example-plot-threshold-adaptive-py
In mahotas there should be other segmentation methods but I'm only knowledgeable about scikit-image. If you want a serious overview on segmentation, check out this paper: https://peerj.com/preprints/671/
Full disclosure, it's my paper.