Develop Reference

Compare two pictures to determine whether there is the same object inside

I am defining a problem: I have two pictures, e.g. two photos with a 1€ coin.
How can I compare the two images to get "yes they contain both a 1€ coin"? of course the test should return false if the second picture contains a 2€ coin.
I tried the openCV methods, but there is nothing so precise.
Also, a ML approach has to handle the issue of recognising two objects in two images without any other previous exposure to them.
EDIT I noted the question is a bit too vague: I am trying to redefine it here a bit.
Given two images, how do I write a boolean function are_the_same(img1, img2) returning True if both images contain the same object?
Here what I tried so far:
SIFT, you find keypoints in images and if a certain number of them matches you state they contain the same object.
CNN siamese network, you train your network to encode same object pictures to close points in the embedding space, and different object images to points that are far from each other in the embedding space.

It depends a lot on what types of images you have, but if it's clear top down images, you can use the goldish band/center to distinguish between them.
First a mask is made based on the goldish color. (You'll probably have to make the color range more specific - I had an easy image. I used this convenient script to determine the color range.) Next some noise is removed and then contours are detected. Contours that have no child- or parent-contour are the solid center of e €2 coin. Contours with a child but no parent are the band of a €1 coin. Contours with a parent but no child are the center of a €1 coin and are ignored.
€2 gets drawn red, €1 blue.
import cv2
import numpy as np
# load image
img = cv2.imread("E1E2.jpg")
# Convert to HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# define range wanted color in HSV
lower_val = np.array([0,25,0])
upper_val = np.array([179,255,255])
# Threshold the HSV image to get only goldish colors
mask = cv2.inRange(hsv, lower_val, upper_val)
# remove noise
kernel = np.ones((5,5))
mask_open = cv2.morphologyEx(mask,cv2.MORPH_OPEN,kernel)
mask_close = cv2.morphologyEx(mask_open,cv2.MORPH_CLOSE,kernel)
# find contours
contours, hier = cv2.findContours(mask_close,cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# loop through contours, check hierarchy, draw contours
for i, cnt in enumerate(contours):
(prev, nxt, child, parent) = hier[0][i]
if child == -1 and parent == -1 :
# €2
cv2.drawContours(img, [cnt],0,(0,0,255), 3)
if child != -1 and parent == -1 :
# €1
cv2.drawContours(img, [cnt],0,(255,0,0), 3)
# display image
cv2.imshow("Res", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Cut out a piece from image the piece is not rectangular (eg trapeze), and turn it into a rectangle that fits with the board

I will bring an example I have a picture of a swimming pool with some tracks I want to take only the three middle tracks Now what is the best way to cut the image in a trapeze shape then how to take this trapeze and try to fit it to the size of the window that will have a relatively similar ratio between the two sides (upper and lower)
image for the example

I modified this example
Result:
import numpy as np
import cv2
# load image
img = cv2.imread('pool.jpg')
# resize to easily fit on screen
img = cv2.resize(img,None,fx=0.5, fy=0.5, interpolation = cv2.INTER_CUBIC)
# determine cornerpoints of the region of interest
pts1 = np.float32([[400,30],[620,30],[50,700],[1000,700]])
# provide new coordinates of cornerpoints
pts2 = np.float32([[0,0],[300,0],[0,600],[300,600]])
# determine transformationmatrix
M = cv2.getPerspectiveTransform(pts1,pts2)
# apply transformationmatrix
dst = cv2.warpPerspective(img,M,(300,600))
# display image
cv2.imshow("img", dst)
cv2.waitKey(0)
cv2.destroyAllWindows()
Note the rezise function, you may wish to delete that line, but you will have to change the coordinates of the cornerpoints accordingly.
I used about the height and width of the base of the trapezoid for the new image (300,600).
You can tweak the cornerpoints and final image size as you see fit.

You can use imutils.four_point_transform function. You can read more about it here.
Basic usage is finding the document contours on a canny edge detected image (again, you can use imutils package that I linked), find the contours on that image and then apply four_point_transform on that contour.
EDIT: How to use canny edge detection and four_point_transform
For finding contours you can use openCV and imutils like this:
cnts = cv2.findContours(edged_image.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
Now, when you have the contours just iterate through and see which one is the biggest and has four points (4 vertices). Then just pass the image and the contour to the four_point_transform function like this:
image_2 = four_point_transform(image, biggest_contour)
That's it.

Finding the boundary points of a contour in opencv numpy

complete noob at open cv and numpy here. here is the image: here is my code:
import numpy as np
import cv2
im = cv2.imread('test.jpg')
imgray = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
imgray = cv2.medianBlur(imgray, ksize=7)
ret, thresh = cv2.threshold(imgray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)
_, contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
print ("number of countours detected before filtering %d -> "%len(contours))
new = np.zeros(imgray.shape)
new = cv2.drawContours(im,contours,len(contours)-1,(0,0,255),18)
cv2.namedWindow('Display',cv2.WINDOW_NORMAL)
cv2.imshow('Display',new)
cv2.waitKey()
mask = np.zeros(imgray.shape,np.uint8)
cv2.drawContours(mask,[contours[len(contours)-1]],0,255,-1)
pixelpoints = cv2.findNonZero(mask)
cv2.imwrite("masked_image.jpg",mask)
print(len(pixelpoints))
print("type of pixelpoints is %s" %type(pixelpoints))
the length of pixelpoints is nearly 2 million since it contains all the point covered by the contours. But i only require the bordering point of that contour. How do I do it? I have tried several methods from opencv documentation but always errors with tuples and sorting operations. please...help?
I only require the border points of the contour :(

Is this what you mean by border points of a contour?
The white lines you see are points that I have marked out in white against the blue drawn contours. There's a little spot at the bottom right because I think its most likely that your black background isn't really black and so when I did thresholding and a floodfill to get this,
there was a tiny white speck at the same spot. But if you play around with the parameters and do a more proper thresholding and floodfill it shouldn't be an issue.
In openCV's drawContours function, the cnts would contain lists of contours and each contour will contain an array of points. Each point is also of type numpy.ndarray. If you want to place all points of each contour in one place so it returns you a set of points of boundary points (like the white dots outline in the image above), you might want to append them all into a list. You can try this:
#rgb is brg instead
contoured=cv2.drawContours(black, cnts, -1, (255,0,0), 3)
#list of ALL points of ALL contours
all_pixels=[]
for i in range(0, len(cnts)):
for j in range(0,len(cnts[i])):
all_pixels.append(cnts[i][j])
When I tried to
print(len(all_pixels))
it returned me 2139 points.
Do this if you want to mark out the points for visualization purposes (e.g. like my white points):
#contouredC is a copy of the contoured image above
contouredC[x_val, y_val]=[255,255,255]
If you want less points, just use a step function when iterating through to draw the white points out. Something like this:
In python, for loops are slow so I think there's better ways of replacing the nested for loops with a np.where() function or something instead. Will update this if/when I figure it out. Also, this needs better thresholding and binarization techniques. Floodfill technique referenced from: Python 2.7: Area opening and closing binary image in Python not so accurate.
Hope it helps.

Python Fit ellipse to an image

I have a webcam feed using OpenCV, and I am trying to fit an ellipse in real time.
The code I am using at the moment works, but it fails to fit an ellipse to the image a lot of the time. What other methods of ellipse fitting to an image can I pursue?
Current code:
def find_ellipses(img): #img is grayscale image of what I want to fit
ret,thresh = cv2.threshold(img,127,255,0)
_,contours,hierarchy = cv2.findContours(thresh, 1, 2)
if len(contours) != 0:
for cont in contours:
if len(cont) < 5:
break
elps = cv2.fitEllipse(cont)
return elps #only returns one ellipse for now
return None
Where elps is of the form (x_centre,y_centre),(minor_axis,major_axis),angle
Here is an example of what I want to successfully fit an ellipse to. My current code fails with this image when I don't want it to.

Turns out I was wrong is just getting the first ellipse from the function. While I thought the first calculated ellipse was the most correct one, what I actually had to do was go through all the ellipses - and choose the most suitable one that bounded the object in the image.

I would define my contours outside of the function, as you don't need to keep re-defining them in this image.
def create_ellipse(thresh,cnt):
ellipse = cv2.fitEllipse(cnt)
thresh = cv2.ellipse(thresh,ellipse,(0,255,0),2)
return thresh
What this code is doing is im taking my thresh image stream and adding an ellipse on top of it. Later on in my code when I want to call it I use the line
thresh = create_ellipse(thresh,cnt)

Python OpenCV detect a white object from a binary image and crop it

My goal is detecting a piece of white paper from this binary image and then crop this white paper and make a new subset binary image just for this white paper.
Now my Python code with OpenCV can find this white paper. For the first step, I created a mask for finding this white paper:
As you guys can see, the small white noise and small piece have been removed. And then the problem become how I can crop this white paper object from this binary image for making a new subset binary image?
My current code is:
import cv2
import numpy as np
QR = cv2.imread('IMG_0352.TIF', 0)
mask = np.zeros(QR.shape,np.uint8)
contours, hierarchy = cv2.findContours(QR,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
if cv2.contourArea(cnt)>1000000:
cv2.drawContours(mask,[cnt],0,255,-1)
Looking for the cnt var, there are four elements, but they are nonsense to me.
I used code to fit a box:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
The box information doesn't seem right.
Thanks for any suggestions.
Follow up:
I have figured out this problem, which is very easy. The code is attached:
import cv2
import numpy as np
QR_orig = cv2.imread('CamR_IMG_0352.TIF', 0)
QR = cv2.imread('IMG_0352.TIF', 0) # read the QR code binary image as grayscale image to make sure only one layer
mask = np.zeros(QR.shape,np.uint8) # mask image the final image without small pieces
# using findContours func to find the none-zero pieces
contours, hierarchy = cv2.findContours(QR,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)
# draw the white paper and eliminate the small pieces (less than 1000000 px). This px count is the same as the QR code dectection
for cnt in contours:
if cv2.contourArea(cnt)>1000000:
cv2.drawContours(mask,[cnt],0,255,-1) # the [] around cnt and 3rd argument 0 mean only the particular contour is drawn
# Build a ROI to crop the QR
x,y,w,h = cv2.boundingRect(cnt)
roi=mask[y:y+h,x:x+w]
# crop the original QR based on the ROI
QR_crop = QR_orig[y:y+h,x:x+w]
# use cropped mask image (roi) to get rid of all small pieces
QR_final = QR_crop * (roi/255)
cv2.imwrite('QR_final.TIF', QR_final)

the contour object is an arbitrary vector (list) of points that enclose the object detected.
An easy brain dead way of accomplishing this is to walk through all the pixels after your thresholding and simply copy the white ones.
I believe findContours() alters the image ( side effect ) so check QR.
However, you need to (usually) get the biggest contour.
Example:
# Choose largest contour
best = 0
maxsize = 0
count = 0
for cnt in contours:
if cv2.contourArea(cnt) > maxsize :
maxsize = cv2.contourArea(cnt)
best = count
count = count + 1
x,y,w,h = cv2.boundingRect(cnt[best])
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)

I actually figured out the solution of this problem, which obviously is very simple!!
import cv2
import numpy as np
QR_orig = cv2.imread('CamR_IMG_0352.TIF', 0)
QR = cv2.imread('IMG_0352.TIF', 0) # read the QR code binary image as grayscale image to make sure only one layer
mask = np.zeros(QR.shape,np.uint8) # mask image the final image without small pieces
# using findContours func to find the none-zero pieces
contours, hierarchy = cv2.findContours(QR,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)
# draw the white paper and eliminate the small pieces (less than 1000000 px). This px count is the same as the QR code dectection
for cnt in contours:
if cv2.contourArea(cnt)>1000000:
cv2.drawContours(mask,[cnt],0,255,-1) # the [] around cnt and 3rd argument 0 mean only the particular contour is drawn
# Build a ROI to crop the QR
x,y,w,h = cv2.boundingRect(cnt)
roi=mask[y:y+h,x:x+w]
# crop the original QR based on the ROI
QR_crop = QR_orig[y:y+h,x:x+w]
# use cropped mask image (roi) to get rid of all small pieces
QR_final = QR_crop * (roi/255)
cv2.imwrite('QR_final.TIF', QR_final)