I am working on a python program to detect if there is something missing in a picture as compared to another picture. Basically, it should take pictures of a place, save it, and after that whenever we run the program, it should take pictures from the camera of the same place and compare them to the picture we saved earlier to detect if an object is missing in the picture.
Requirement:
For example, we take a reference photo of the floor with two apples and one orange laying on the ground, then save that picture for future reference.
After that, we take one apple from the floor and now there is only one apple and one orange on the floor. When we run the program, it should take a photo using a camera and compare that photo to the one we took earlier. It should detect that a certain area of the picture (where the missing apple was placed) is different in the new picture and alert the user (print a message)...This is the whole idea.
Here is what I achieved so far:
I used a tool called ImageChops from the Pillow library. The problem with it is that it detects even very small changes that can occur due to noise. But I want it to detect major changes like an object missing or added to the picture. I would also like to control it with a threshold. I searched for doing this over the internet but couldn't find anything useful. Here is the code that I used
from PIL import Image, ImageChops
import cv2
import os
import sys
img1 = Image.open('1.jpg')
img2 = Image.open('2.jpg')
diff = ImageChops.difference(img1, img2)
if diff.getbbox():
print("Total {} differences found".format(diff.getbbox()))
diff.show()
else:
print('No difference found')
I would really appreciate it if someone can point me in the right direction. Thanks
The general problem you're describing is called "feature matching". OpenCV has lots of tooling for this exact purpose: a simple approach is this one https://docs.opencv.org/4.x/dc/dc3/tutorial_py_matcher.html, which compares ORB features (basically, generalized corners) between two images. These kinds of features are more robust to lighting and noise than just comparing pixel values directly.
Related
I have a collection of images taken with different lens (same distorsion) and I want to see if theres a difference in the colour.
I think the best way to compare is simple average, for which I have:
import cv2
import numpy
myimg = cv2.imread('image.jpg')
avg_color_per_row = numpy.average(myimg, axis=0)
avg_color = numpy.average(avg_color_per_row, axis=0)
print(avg_color)
From: How to find the average colour of an image in Python with OpenCV?
I am struggling to find out if I should be using squared values of rgb and how i would do this in python. Any help highly appreciated & any advice on other ways to compare colour of identical images. I am new here so pls advise on any protocol ive missed out on too. Thank you.
That is a very broad question. And it depends on what you call "difference of color".
Are your images supposed to be strictly (to the pixel) aligned? Your comparison by rows seems to indicate so. But the rest of the text makes it very unrealistic (in fact, even in industrial computer vision, for example to control defects on parts, in very controlled environment, strictly positioned parts and cameras, this is unrealistic).
Are differences of luminosity considered difference of color?
Are the images contents supposed to be identical?
...
Very generally speaking, if what you are trying to assess is color impact of a lense, on an otherwise very similar image (same lighting, same object), I would just compute histograms of colors and compare them.
Forgive me but I'm new in OpenCV.
I would like to delete the common background in 3 images, where there is a landscape and a man.
I tried some subtraction codes but I can't solve the problem.
I would like output each image only with the man and without landscape
Are there in OpenCV Algorithms what do this do? (then without any manual operation so no markers or other)
I tried this python code CV - Extract differences between two images
but not works because in my case i don't have an image with only background (without man).
I thinks that good solution should to Compare all the images and save those "points" that are the same at least in an image.
In this way I can extrapolate a background (which we call "Result.jpg") and finally analyze each image and cut those portions that are also present in "Result.jpg".
You say it's a good idea? Do you have other simplest ideas?
Without semantic segmentation, you can't do that.
Because all you can compute is where two images differ, and this does not give you the silhouette of the person, but an overlapping of two silhouettes. You'll never know the exact outline.
I have tried 3 algorithms:
Compare by Compare_ssim.
Difference detection by PIL (ImageChops.difference).
Images subtraction.
The first algorithm:
(score, diff) = compare_ssim(img1, img2, full=True)
diff = (diff * 255).astype("uint8")
The second algorithm:
from PIL import Image ,ImageChops
img1=Image.open("canny1.jpg")
img2=Image.open("canny2.jpg")
diff=ImageChops.difference(img1,img2)
if diff.getbbox():
diff.show()
The third algorithm:
image3= cv2.subtract(image1,image2)
The problem is these algorithms are so sensitive. If the images have different noise, they consider that the two images are totally different. Any ideas to fix that?
These pictures are different in many ways (deformation, lighting, colors, shape) and simple image processing just cannot handle all of this.
I would recommend a higher level method that tries to extract the geometry and color of those tubes, in the form of a simple geometric graph. Then compare the graphs rather than the images.
I acknowledge that this is easier said than done, and will only work with this particular kind of scene.
It is very difficult to help since we don't really know which parameters you can change, like can you keep your camera fixed? Will it always be just about tubes? What about tubes colors?
Nevertheless, I think what you are looking for is a framework for image registration and I propose you to use SimpleElastix. It is mainly used for medical images so you might have to get familiar with the library SimpleITK. What's interesting is that you have a lot of parameters to control the registration. I think that you will have to look into the documentation to find out how to control a specific image frequency, the one that create the waves and deform the images. Hereafter I did not configured it to have enough local distortion, you'll have to find the best trade-off, but I think it should be flexible enough.
Anyway, you can get such result with the following code, I don't know if it helps, I hope so:
import cv2
import numpy as np
import matplotlib.pyplot as plt
import SimpleITK as sitk
fixedImage = sitk.ReadImage('1.jpg', sitk.sitkFloat32)
movingImage = sitk.ReadImage('2.jpg', sitk.sitkFloat32)
elastixImageFilter = sitk.ElastixImageFilter()
affine_registration_parameters = sitk.GetDefaultParameterMap('affine')
affine_registration_parameters["NumberOfResolutions"] = ['6']
affine_registration_parameters["WriteResultImage"] = ['false']
affine_registration_parameters["MaximumNumberOfSamplingAttempts"] = ['4']
parameterMapVector = sitk.VectorOfParameterMap()
parameterMapVector.append(affine_registration_parameters)
parameterMapVector.append(sitk.GetDefaultParameterMap("bspline"))
elastixImageFilter.SetFixedImage(fixedImage)
elastixImageFilter.SetMovingImage(movingImage)
elastixImageFilter.SetParameterMap(parameterMapVector)
elastixImageFilter.Execute()
registeredImage = elastixImageFilter.GetResultImage()
transformParameterMap = elastixImageFilter.GetTransformParameterMap()
resultImage = sitk.Subtract(registeredImage, fixedImage)
resultImageNp = np.sqrt(sitk.GetArrayFromImage(resultImage) ** 2)
cv2.imwrite('gray_1.png', sitk.GetArrayFromImage(fixedImage))
cv2.imwrite('gray_2.png', sitk.GetArrayFromImage(movingImage))
cv2.imwrite('gray_2r.png', sitk.GetArrayFromImage(registeredImage))
cv2.imwrite('gray_diff.png', resultImageNp)
Your first image resized to 256x256:
Your second image:
Your second image registered with the first one:
Here is the difference between the first and second image which could show what's different:
This is one of the classical problems of image treatment - and one which does not have an answer which holds universally. The possible answers depend highly on what type of images you have, and what type of information you want to extract from them and the differences between them.
You can reduce noise by two means:
a) take several images of the same object, such that the object does not change. You can stack the images and noise is reduced by square-root of the number of images.
b) You can run a blur filter over the image. The more you blur, the more noise is averaged. Noise is here reduced by square-root of the number of pixels you average over. But so is detail in the images.
In both cases (a) and (b) you run the difference analysis after you applied either method.
Probably not applicable to you as you likely cannot get hold of either: it helps, if you can get hold of flatfields which give the inhomogeneity of illumination and pixel sensitivity of your camera and allow correcting the images prior to any treatment. Similar goes for darkfields which give an estimate of the influence of the read-out noise of the camera and allow correcting images for those.
There is somewhat another 3rd option, which is more high-level: run your object analysis first at a detailed-enough level. And compare the results.
I'm trying to use Python to detect how many objects are on a white surface. An example image is found at the end of this post.
I'm wondering how I should do this, mainly because the background is white and most of the time it gets detected as foreground.
What I have now in Python based on this tutorial (http://pythonvision.org/basic-tutorial) uses several libraries and detects the white as the object so count is 1, the tools get detected as background and thus are ignored:
dna = mahotas.imread('dna.jpeg')
dna = dna.squeeze()
dna = pymorph.to_gray(dna)
print dna.shape
print dna.dtype
print dna.max()
print dna.min()
dnaf = ndimage.gaussian_filter(dna, 8)
T = mahotas.thresholding.otsu(dnaf)
labeled, nr_objects = ndimage.label(dnaf > T)
print nr_objects
pylab.imshow(labeled)
pylab.jet()
pylab.show()
Are there any options for getting the white part as background and the tools as foreground?
Thanks in advance!
Example image:
The segmented image where red is foreground and blue background (the few tools merging is not a problem):
If the shadow is not a problem
You can label the images in mahotas (http://mahotas.readthedocs.org/en/latest/labeled.html) given this binary image. You can also use skimage.morphology (which uses ndlabel as was mentioned in comments). http://scikit-image.org/docs/dev/auto_examples/plot_label.html
These are examples of connect-component algorithms and are standard in any general image processing package. ImageJ also makes this quite simple.
If the shadow is a problem
Otsu thresholding returns a single value: a pixel brightness, and all you're doing is keeping all pixels that are dimmer than this threshold. This method is getting tripped up by your shadows, so you need to try another segmentation algorithm, preferably one that does local segmentation (IE it segments small regions of the image individually.)
Adaptive or local methods don't have this problem and would be really well-suited to your image's shadows:
http://scikit-image.org/docs/dev/auto_examples/plot_threshold_adaptive.html#example-plot-threshold-adaptive-py
In mahotas there should be other segmentation methods but I'm only knowledgeable about scikit-image. If you want a serious overview on segmentation, check out this paper: https://peerj.com/preprints/671/
Full disclosure, it's my paper.
I am trying to add two images of different sizes using bitwise operations in OpenCV using python. I want a particular point in Image1(an image of face of a person) to coincide with a particular point in Image2(image of a spectacle frame). The particular points are not the cornermost points of the images.I know the 2 mid points of the frame glasses and the pupil of the eyes. I want the frame mid points to coincide with the pupil points of the eyes in the face. The code which I am using adds the second image's leftmost corner point to the specific point of Image1 as in Line 10, whereas i want the mid point of left glass frame to be added.
The face image can be any random image and the spectacle image is as -
I am using the code:
import cv2
import numpy as np
img_frame = cv2.imread('image1.jpg',1)
img_in = cv2.imread('face.jpg',1)
new_image = np.zeros(img_frame.shape,dtype=np.uint8)
i,j,k = img_frame.shape
for ii in range (1,i):
for jj in range (1,j):
pixel = img_frame[ii,jj]
img_in[339+ii,468+jj] = pixel
cv2.imwrite('pc2_with_frame_7.jpg',img_in)
cv2.imshow('win',img_in)
cv2.waitKey(0)
cv2.destroyWindow('win')
Any kind of help would be appreciated.
Thank you.
Ok, it seems nobody else much can help so I will offer what I can...
What you are trying to do is called alpha-compositing. You can read about it here on Wikipedia and also here in the OpenCV documentation.
My tool of choice for this would be ImageMagick, which is free and has Perl, Python, C/C++ bindings as well as command-line tools. If I start with this photo (face.jpg):
and take your glasses.jpg file and convert it to a PNG with transparency, whcih looks like this:
I can run the following ImageMagick command at the Terminal
composite glasses.png face.jpg out.jpg
and I get this:
It seems that OpenCV has problems maybe with transparency, and a solution is presented here. If you want to try the masking method suggested by #ypnos in that post, I have made you the necessary input files and you can download them from my website at:
glasses.png with alpha channel
input-mask.png