Working on object detection in Python with opencv.
I have two pictures
The reference picture with no object in it.
Picture with object.
The result of the images is:
The problem is, the pattern of the reference image is now on my objects. I want to remove this pattern and I don't know how to do it. For further image processing I need the the correct outline of the objects.
Maybe you know how to fix it, or have better ideas to exctract the object.
I would be glad for your help.
Edit: 4. A black object:
As #Mark Setchell commented, the difference of the two images shows which pixels contain the object, you shouldn't try to use it as the output. Instead, find the pixels with a significant difference, and then read those pixels directly from the input image.
Here, I'm using Otsu thresholding to find what "significant difference" is. There are many other ways to do this. I then use the inverse of the mask to blank out pixels in the input image.
import PyDIP as dip
bg = dip.ImageReadTIFF('background.tif')
bg = bg.TensorElement(1) # The image has 3 channels, let's use just the green one
fg = dip.ImageReadTIFF('object.tif')
fg = fg.TensorElement(1)
mask = dip.Abs(bg - fg) # Difference between the two images
mask, t = dip.Threshold(mask, 'otsu') # Find significant differences only
mask = dip.Closing(mask, 7) # Smooth the outline a bit
fg[~mask] = 0 # Blank out pixels not in the mask
I'm using PyDIP above, not OpenCV, because I don't have OpenCV installed. You can easily do the same with OpenCV.
An alternative to smoothing the binary mask as I did there, is to smooth the mask image before thresholding, for example with dip.Gauss(mask,[2]), a Gaussian smoothing.
Edit: The black object.
What happens with this image, is that its illumination has changed significantly, or you have some automatic exposure settings in your camera. Make sure you have turned all of that off so that every image is exposed exactly the same, and that you use the raw images directly off of the camera for this, not images that have gone through some automatic enhancement procedure or even JPEG compression if you can avoid it.
I computed the median of the background image divided by the object image (fg in the code above, but for this new image), which came up to 1.073. That means that the background image is 7% brighter than the object image. I then multiplied fg by this value before computing the absolute difference:
mask = dip.Abs(fg * dip.Median(bg/fg)[0][0] - bg)
This helped a bit, but it showed that the changes in contrast are not consistent across the image.
Next, you can change the threshold selection method. Otsu assumes a bimodal histogram, and works well if you have a significant number of pixels in each group (foreground and background). Here we'll have fewer pixels belonging to the object, because only some of the object pixels have a different color from the background. The 'triangle' method is suitable in this case:
mask, t = dip.Threshold(mask, 'triangle')
This will lead to a mask that contains only some of the object pixels. You'll have to add some additional knowledge about your object (i.e. it is a rotated square) to find the full object. There are also some isolated background pixels that are being picked up by the threshold, those are easy to eliminate using a bit of blurring before the threshold or a small opening after.
Getting the exact outline of the object in this case will be impossible with your current setup. I would suggest you improve your setup by either:
making the background more uniform in illumination,
using color (so that there are fewer possible objects that match the background color so exactly as in this case),
using infrared imaging (maybe the background could have different properties from all the objects to be detected in infrared?),
using back-illumination (this is the best way if your aim is to measure the objects).
Related
I know how to Gaussian blur with Pillow, but can't track how to mask it by intensity of the radius value with a mask.
I am using MiDaS package to produce depth maps form 2D images. What I want to do is be able to blur the original image by the depth mask as a pseudo depth of field.
Here is a visual demonstration of the result I'm after with CV2 or Pillow (I don't understand which can do what I'm after.)
Note: I'm sorry if this is considered junk, I've sat on this question for a month. I tried scouring the net for something like this, and all I found was Poor Man's Portrait Mode which I could not get to work, and also would be reproducing depth maps when I already have them from my script and used for the 3D image creation.
Edit:
I did come up with this, using composite Not sure why I didn't take note of it before. Though I have to say, the results aren't too great. I think I really do need to emulate some sort of shape blur like bokeh.
sharpen = 3
boxBlur = 5
oimg = Image.open('2.png').convert('RGB')
width, height = oimg.size
mimg = Image.open('2_depth.png').resize((width, height)).convert('L')
bimg = oimg.filter(ImageFilter.BoxBlur(int(boxBlur)))
bimg = bimg.filter(ImageFilter.BLUR)
for i in range(sharpen):
bimg = bimg.filter(ImageFilter.SHARPEN)
rimg = Image.composite(oimg, bimg, mimg)
Basically get your image, and mask, ensure the mask matches the image (I had a issue where images didn't match, but were the same size, just saved different from 2 saved the same way)
Blur your image to a new variable, however you like, Gaussian, etc. Gaussian was too soft for me. Add whatever extra filtering you want
Composite the results together, using depth map as a mask for composite.
Note: If someone knows how to achieve a different sort of blur that mimics bokeh, I'd like to know, and have adjusted the question title. I read about a discBlur but couldn't find anything for PIL/CV2.
I’ve got only a brute-force solution with iteration over pixels: Variable blur intensity.
My code is working but not as efficiently as I want.
You can try. Open your image as input and put your depth map in the variable blur_map.
I hope you're all doing well!
I'm new to Image Manipulation, and so I want to apologize right here for my simple question. I'm currently working on a problem that involves classifying an object called jet into two known categories. This object is made of sub-objects. My idea is to use this sub-objects to transform each jet in a pixel image, and then applying convolutional neural networks to find the patterns.
Here is an example of the pixel images:
jet's constituents pixel distribution
To standardize all the images, I want to find the two most intense pixels and make sure the axis connecting them is in the vertical direction, as well as make sure that the most intense pixel is at the top. It also would be good to impose that one of the sides (left or right) of the image contains the majority of the intensity and to normalize the intensity of the whole image to 1.
My question is: as I'm new to this kind of processing, I don't know if there is a library in Python that can handle these operations. Are you aware of any?
PS: the picture was taken from here:https://arxiv.org/abs/1407.5675
You can look into OpenCV library for Python:
https://docs.opencv.org/master/d6/d00/tutorial_py_root.html.
It supports a lot of image processing functions.
In your case, it probably would be easier to convert the image into a more suitable color space in which one axis stands for color intensity (e.g HSI, HSL, HSV) and trying to find indices of the maximum values along this axis (this should return the pixels with the highest intensity in the image).
Generally, in Python, we use PIL library for basic manipulations with images and OpenCV for advances ones.
But, if understand your task correctly, you can just think of an image as a multidimensional array and use numpy to manipulate it.
For example, if your image is stored in a variable of type numpy.array called img, you can find maximum value along the desired axis just by writing:
img.max(axis=0)
To normalize image you can use:
img /= img.max()
To find which image part is brighter, you can split an img array into desired parts and calculate their mean:
left = img[:, :int(img.shape[1]/2), :]
right = img[:, int(img.shape[1]/2):, :]
left_mean = left.mean()
right_mean = right.mean()
I am trying to use OpenCV to measure size of filament ( that plastic material used for 3D printing)
What I am trying to do is measuring filament size ( that plastic material used for 3D printing ). The idea is that I use led panel to illuminate filament, then take image with camera, preprocess the image, apply edge detections and calculate it's size. Most filaments are fine made of one colour which is easy to preprocess and get fine results.
The problem comes with transparent filament. I am not able to get useful results. I would like to ask for a little help, or if someone could push me the right directions. I have already tried cropping the image to heigh that is a bit higher than filament, and width just a few pixels and calculating size using number of pixels in those images, but this did not work very well. So now I am here and trying to do it with edge detections
works well for filaments of single colour
not working for transparent filament
Code below is working just fine for common filaments, the problem is when I try to use it for transparent filament. I have tried adjusting tresholds for Canny function. I have tried different colour-spaces. But I am not able to get the results.
Images that may help to understand:
https://imgur.com/gallery/CIv7fxY
image = cv.imread("../images/img_fil_2.PNG") # load image
gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY) # convert image to grayscale
edges = cv.Canny(gray, 100, 200) # detect edges of image
You can use the assumption that the images are taken under the same conditions.
Your main problem is that the reflections in the transparent filament are detected as edges. But, since the image is relatively simple, without any other edges, you can simply take the upper and the lower edge, and measure the distance between them.
A simple way of doing this is to take 2 vertical lines (e.g. image sides), find the edges that intersect the line (basically traverse a column in the image and find edge pixels), and connect the highest and the lowest points to form the edges of the filament. This also removes the curvature in the filament, which I assume is not needed for your application.
You might want to use 3 or 4 vertical lines, for robustness.
I've been attempting to overlay two images in python to match coordinates, the top left and bottom right corners have the same coordinates and their aspects are almost identical bar a few pixels. Although they are different resolutions.
Using PIL I have been able to overlay the images, though after overlaying them the image output is square but the resolution is that of the background image, the foreground image is also re-sized incorrectly (As far as I can see). I must be doing something wrong.
import Image
from PIL import Image
#load images
background = Image.open('ndvi.png')
foreground = Image.open('out.png')
#resizing
foreground.thumbnail((643,597),Image.ANTIALIAS)
#overlay
background.paste(foreground, (0, 0), foreground)
#save
background.save("overlay.png")
#display
background.show()
When dropping the images into something horrible like powerpoint the image aspects are almost identical. I've included an example image, the image on the left is my by hand overlay and the image on the right is the output from python. The background at some point in the code is squashed vertically, also affecting the overlay. I'd like to be able to do this in python and make it correctly look like the left hand image.
A solution upfront.
Background image
width/height/ratio: 300 / 375 / 0.800
Foreground image
width/height/ratio: 400 / 464 / 0.862
Overlay
from PIL import Image
imbg = Image.open("bg.png")
imfg = Image.open("fg.png")
imbg_width, imbg_height = imbg.size
imfg_resized = imfg.resize((imbg_width, imbg_height), Image.LANCZOS)
imbg.paste(imfg_resized, None, imfg_resized)
imbg.save("overlay.png")
Discussion
The most important information you have given in your question were:
the aspect ratios of your foreground and background images are not equal, but similar
the top left and bottom right corners of both images need to be aligned in the end.
The conclusion from these points is: the aspect ratio of one of the images has to change. This can be achieved with the resize() method (not with thumbnail(), as explained below). To summarize, the goal simply is:
Resize the image with larger dimensions (foreground image) to the exact dimensions of the smaller background image. That is, do not necessarily maintain the aspect ratio of the foreground image.
That is what the code above is doing.
Two comments on your approach:
First of all, I recommend using the newest release of Pillow (Pillow is the continuation project of PIL, it is API-compatible). In the 2.7 release they have largely improved the image re-scaling quality. The documentation can be found at http://pillow.readthedocs.org/en/latest/reference.
Then, you obviously need to take control of how the aspect ratio of both images evolves throughout your program. thumbnail(), for instance, does not alter the aspect ratio of the image, even if your size tuple does not have the same aspect ratio as the original image. Quote from the thumbnail() docs:
This method modifies the image to contain a thumbnail version of
itself, no larger than the given size. This method calculates an
appropriate thumbnail size to preserve the aspect of the image
So, I am not sure where you were going exactly with your (643,597) tuple and if you are possibly relying on the thumbnail to have this exact size afterwards.
I'm trying to use Python to detect how many objects are on a white surface. An example image is found at the end of this post.
I'm wondering how I should do this, mainly because the background is white and most of the time it gets detected as foreground.
What I have now in Python based on this tutorial (http://pythonvision.org/basic-tutorial) uses several libraries and detects the white as the object so count is 1, the tools get detected as background and thus are ignored:
dna = mahotas.imread('dna.jpeg')
dna = dna.squeeze()
dna = pymorph.to_gray(dna)
print dna.shape
print dna.dtype
print dna.max()
print dna.min()
dnaf = ndimage.gaussian_filter(dna, 8)
T = mahotas.thresholding.otsu(dnaf)
labeled, nr_objects = ndimage.label(dnaf > T)
print nr_objects
pylab.imshow(labeled)
pylab.jet()
pylab.show()
Are there any options for getting the white part as background and the tools as foreground?
Thanks in advance!
Example image:
The segmented image where red is foreground and blue background (the few tools merging is not a problem):
If the shadow is not a problem
You can label the images in mahotas (http://mahotas.readthedocs.org/en/latest/labeled.html) given this binary image. You can also use skimage.morphology (which uses ndlabel as was mentioned in comments). http://scikit-image.org/docs/dev/auto_examples/plot_label.html
These are examples of connect-component algorithms and are standard in any general image processing package. ImageJ also makes this quite simple.
If the shadow is a problem
Otsu thresholding returns a single value: a pixel brightness, and all you're doing is keeping all pixels that are dimmer than this threshold. This method is getting tripped up by your shadows, so you need to try another segmentation algorithm, preferably one that does local segmentation (IE it segments small regions of the image individually.)
Adaptive or local methods don't have this problem and would be really well-suited to your image's shadows:
http://scikit-image.org/docs/dev/auto_examples/plot_threshold_adaptive.html#example-plot-threshold-adaptive-py
In mahotas there should be other segmentation methods but I'm only knowledgeable about scikit-image. If you want a serious overview on segmentation, check out this paper: https://peerj.com/preprints/671/
Full disclosure, it's my paper.