I have a hundreds of ID card images which some of them provided below:
(Disclaimer: I downloaded the images from the Web, all rights (if exist) belong to their respective authors)
As seen, they are all different in terms of brightness, perspective angle and distance, card orientation. I'm trying to extract only rectangular card area and save it as a new image. To achieve this, I came to know that I must convert an image to grayscale image and apply some thresholding methods. Then, cv2.findCountours() is applied to threshold image to get multiple vector points.
I have tried many methods and come to use cv2.adaptiveThreshold() as it is said that it finds a value for threshold (because, I can't manually set threshold values for each image). However, when I apply it to images, I am not getting what I want. For example:
My desired output should look like this:
It seems like it also includes affine transformations to make the card area (Obama case) proper but I am finding it difficult to understand. If that's possible I'd further extract and save the image separately.
Is there any other method or algorithm that can achieve what I want? It should consider different lighting conditions and card orientations. I am expecting one-fits-all solution given there will be only one rectangle ID card. Please, guide me through this with whatever you think will be of help.
Note that I can't use CNNs as object detector, it must be based on purely image-processing.
Thank you.
EDIT:
The code for the above results is pretty simple:
image = cv2.imread(args["image"])
gray_img = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh_img = cv2.adaptiveThreshold(gray_img,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY_INV,51,9)
cnts = cv2.findContours(thresh_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
area_treshold = 1000
for cnt in cnts:
if cv2.contourArea(cnt) > area_treshold:
x,y,w,h = cv2.boundingRect(cnt)
print(x,y,w,h)
cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 3)
resize = ResizeWithAspectRatio(image, width=640)
cv2.imshow("image", resize)
cv2.waitKey()
EDIT 2:
I provide the gradient magnitude images below:
Does it mean I must cover both low and high intensity values? Because the edges of the ID cards at the bottom are barely noticeable.
Related
as the title states, I'm trying to crop the largest circle out of an image. I'm using OpenCV in python. To be exact, it's a shooting target, which always has the same format, but the picture of it can be taken with any mobile device and in different lighting conditions (I will include some examples lower).
I'm completely new to image recognition, so I have been trying out many different ways of doing this, but couldn't figure out a universal solution, that would work on all of my target images.
Why I'm trying to do this:
My assignment is to calculate score of one or multiple shots on the given target image. I have tried color segmentation to find the shots, but since the shots can be on different backgrounds, this wouldn't work properly. So now I'm trying to see the difference between the empty shooting target image and the already shot on target image. Also, I need to be able to tell, which target it was shot on (there are two target types). So I'm trying to crop out only the target from image to get rid of the background interference and then continue with the shot identifications.
What I have tried so far:
1) Finding the largest circle with HoughCircles. My next step would be to somehow remove the outer part of that found circle. I have played with the configuration of HoughCircles method for quite some time, but always one of the example images wasn't highlighting the most outer circle correctly or wasn't highlighting any of the circles :/.
My final configuration looked something like this:
img = cv2.GaussianBlur(img, (3, 3), 0)
cv2.HoughCircles(img, cv2.HOUGH_GRADIENT, 2, 10000, param1=50, param2=100, minRadius=200, maxRadius=0)
It seemed like using HoughCircles wouldn't be the right way to do this, so I moved on to another possible solution I found on the internet.
2) Finding all the countours by filtering the 'black' color range in which the circles seem to be on the pictures and than finding the largest one. The problem with this solution seemed to be that sometimes the pictures had a shadow that destroyed the outer circle and therefore it seemed impossible to crop by it.
My code looked like this:
# black color boundaries [B, G, R]
lower = [0, 0, 0]
upper = [150, 150, 150]
# create NumPy arrays from the boundaries
lower = np.array(lower, dtype="uint8")
upper = np.array(upper, dtype="uint8")
# find the colors within the specified boundaries and apply the mask
mask = cv2.inRange(img, lower, upper)
output = cv2.bitwise_and(img, img, mask=mask)
ret, thresh = cv2.threshold(mask, 40, 255, 0)
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
if len(contours) != 0:
# draw in blue the contours that were founded
cv2.drawContours(output, contours, -1, 255, 3)
# find the biggest countour (c) by the area
c = max(contours, key=cv2.contourArea)
x, y, w, h = cv2.boundingRect(c)
After that, I would try to draw a circle by the largest found contour (c) and crop by it. But I have already seen that the drawn circles weren't complete (probably due to some shadow on the picture) and therefore this wouldn't work anyway.
After those failures, I have tried so many solutions from other questions on here, but none would work for my problem.
Example images:
Target example 1
Target example 2
Target to calc score 1
Target to calc score 2
To be completely honest with you, I'm really lost on how to go about this. I would appreciate any help, advice, anything.
There are two different types of target in your samples. You may want to process them separately or ask the user for the input, what kind of target it is. Basically, you want to know how large the black part of the target, does it cover 7-10 or 4-10.
Binarize your image. Build a histogram along X and Y -- you'll find the position of the black part of your target as (x_left, x_right, y_top, y_bottom). Once you know that, you can calculate the center ((top+bottom)/2, (left+right)/2). After that you can easily calculate the score for every pixel of the image, since you know the center, the black spot size and the number of different score areas within.
I am defining a problem: I have two pictures, e.g. two photos with a 1€ coin.
How can I compare the two images to get "yes they contain both a 1€ coin"? of course the test should return false if the second picture contains a 2€ coin.
I tried the openCV methods, but there is nothing so precise.
Also, a ML approach has to handle the issue of recognising two objects in two images without any other previous exposure to them.
EDIT I noted the question is a bit too vague: I am trying to redefine it here a bit.
Given two images, how do I write a boolean function are_the_same(img1, img2) returning True if both images contain the same object?
Here what I tried so far:
SIFT, you find keypoints in images and if a certain number of them matches you state they contain the same object.
CNN siamese network, you train your network to encode same object pictures to close points in the embedding space, and different object images to points that are far from each other in the embedding space.
It depends a lot on what types of images you have, but if it's clear top down images, you can use the goldish band/center to distinguish between them.
First a mask is made based on the goldish color. (You'll probably have to make the color range more specific - I had an easy image. I used this convenient script to determine the color range.) Next some noise is removed and then contours are detected. Contours that have no child- or parent-contour are the solid center of e €2 coin. Contours with a child but no parent are the band of a €1 coin. Contours with a parent but no child are the center of a €1 coin and are ignored.
€2 gets drawn red, €1 blue.
import cv2
import numpy as np
# load image
img = cv2.imread("E1E2.jpg")
# Convert to HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# define range wanted color in HSV
lower_val = np.array([0,25,0])
upper_val = np.array([179,255,255])
# Threshold the HSV image to get only goldish colors
mask = cv2.inRange(hsv, lower_val, upper_val)
# remove noise
kernel = np.ones((5,5))
mask_open = cv2.morphologyEx(mask,cv2.MORPH_OPEN,kernel)
mask_close = cv2.morphologyEx(mask_open,cv2.MORPH_CLOSE,kernel)
# find contours
contours, hier = cv2.findContours(mask_close,cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
# loop through contours, check hierarchy, draw contours
for i, cnt in enumerate(contours):
(prev, nxt, child, parent) = hier[0][i]
if child == -1 and parent == -1 :
# €2
cv2.drawContours(img, [cnt],0,(0,0,255), 3)
if child != -1 and parent == -1 :
# €1
cv2.drawContours(img, [cnt],0,(255,0,0), 3)
# display image
cv2.imshow("Res", img)
cv2.waitKey(0)
cv2.destroyAllWindows()
For a little experiment in Python I'm doing I want to find small scratches on fruits. The scratches are very small and hard to detect by human eye.
I'm using a high resolution camera for that experiment.
Here is the defect I want to detect:
Original Image:
This is my result with very few lines of code:
So I found the contours of my fruit. How can I proceed to finding the scratch? The RGB Value is similar to other parts of the fruit. So how can I differentiate between A scratch, and a part of the fruit?
My code:
# Imports
import numpy as np
import cv2
import time
# Read Image & Convert
img = cv2.imread('IMG_0441.jpg')
result = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# Filtering
lower = np.array([1,60,50])
upper = np.array([255,255,255])
result = cv2.inRange(result, lower, upper)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(9,9))
result = cv2.dilate(result,kernel)
# Contours
im2, contours, hierarchy = cv2.findContours(result.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
result = cv2.cvtColor(result, cv2.COLOR_GRAY2BGR)
if len(contours) != 0:
for (i, c) in enumerate(contours):
area = cv2.contourArea(c)
if area > 100000:
print(area)
cv2.drawContours(img, c, -1, (255,255,0), 12)
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),12)
# Stack results
result = np.vstack((result, img))
resultOrig = result.copy()
# Save image to file before resizing
cv2.imwrite(str(time.time())+'_0_result.jpg',resultOrig)
# Resize
max_dimension = float(max(result.shape))
scale = 900/max_dimension
result = cv2.resize(result, None, fx=scale, fy=scale)
# Show results
cv2.imshow('res',result)
cv2.waitKey(0)
cv2.destroyAllWindows()
I changed your image to HSL colour space.
I can't see the scratch in the L channel, so the greyscale approach suggested earlier is going to be difficult.
But the scratch is quite noticeable in the hue plane.
You could use an edge detector to find the blemish in the hue channel. Here I use a difference of gaussians detector (with sizes 20 and 4).
personal guess is to use some algorithm to detect the grayscale change. The grayscale variation around the scratch should be bigger than the variation in other area. Sobel and Scharr Derivatives could be an option. This is a link to python-openCV about image gradient. You can first crop out the fruit with coutour application
If you really want to use conventional computer vision techniques, you should start with edges that can be detected on the fruit. Some of the edges are caused by the bumps on the fruit, so you have to look at various features of the area around the edges to find the difference between scratches and bumps. After you look at about a hundred scratches, you should be able to come up with some rules.
But this process is going to be very tiring, and my guess is you will not have much luck. A better way to approach this problem is to train a deep neural network by manually annotating scratches on about 100 images, and letting the network find out by itself how to distinguish scratches from the rest of the fruit.
If you are a beginner to these stuff, search for PyImageSearch and LearnOpenCV. Both are very resourceful sites where you can learn quickly.
The image below shows an aerial photo of a house block (re-oriented with the longest side vertical), and the same image subjected to Adaptive Thresholding and Difference of Gaussians.
Images: Base; Adaptive Thresholding; Difference of Gaussians
The roof-print of the house is obvious (to the human eye) on the AdThresh image: it's a matter of connecting some obvious dots. In the sample image, finding the blue-bounded box below -
Image with desired rectangle marked in blue
I've had a crack at implementing HoughLinesP() and findContours(), but get nothing sensible (probably because there's some nuance that I'm missing). The python script-chunk that fails to find anything remotely like the blue box, is as follows:
import cv2
import numpy as np
from matplotlib import pyplot as plt
# read in full (RGBA) image - to get alpha layer to use as mask
img = cv2.imread('rotated_12.png', cv2.IMREAD_UNCHANGED)
grey = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Otsu's thresholding after Gaussian filtering
blur_base = cv2.GaussianBlur(grey,(9,9),0)
blur_diff = cv2.GaussianBlur(grey,(15,15),0)
_,thresh1 = cv2.threshold(grey,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
thresh = cv2.adaptiveThreshold(grey,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY,11,2)
DoG_01 = blur_base - blur_diff
edges_blur = cv2.Canny(blur_base,70,210)
# Find Contours
(ed, cnts,h) = cv2.findContours(grey, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
cnts = sorted(cnts, key = cv2.contourArea, reverse = True)[:4]
for c in cnts:
approx = cv2.approxPolyDP(c, 0.1*cv2.arcLength(c, True), True)
cv2.drawContours(grey, [approx], -1, (0, 255, 0), 1)
# Hough Lines
minLineLength = 30
maxLineGap = 5
lines = cv2.HoughLinesP(edges_blur,1,np.pi/180,20,minLineLength,maxLineGap)
print "lines found:", len(lines)
for line in lines:
cv2.line(grey,(line[0][0], line[0][1]),(line[0][2],line[0][3]),(255,0,0),2)
# plot all the images
images = [img, thresh, DoG_01]
titles = ['Base','AdThresh','DoG01']
for i in xrange(len(images)):
plt.subplot(1,len(images),i+1),plt.imshow(images[i],'gray')
plt.title(titles[i]), plt.xticks([]), plt.yticks([])
plt.savefig('a_edgedetect_12.png')
cv2.destroyAllWindows()
I am trying to set things up without excessive parameterisation. I'm wary of 'tailoring' an algorithm for just this one image since this process will be run on hundreds of thousands of images (with roofs/rooves of different colours which may be less distinguishable from background). That said, I would love to see a solution that 'hit' the blue-box target - that way I could at the very least work out what I've done wrong.
If anyone has a quick-and-dirty way to do this sort of thing, it would be awesome to get a Python code snippet to work with.
The 'base' image ->
Base Image
You should apply the following:
1. Contrast Limited Adaptive Histogram Equalization-CLAHE and convert to gray-scale.
2. Gaussian Blur & Morphological transforms (dialation, erosion, etc) as mentioned by #bad_keypoints. This will help you get rid of the background noise. This is the most tricky step as the results will depend on the order in which you apply (first Gaussian Blur and then Morphological transforms or vice versa) and the window sizes you choose for this purpose.
3. Apply Adaptive thresholding
4. Apply Canny's Edge detection
5. Find contour having four corner points
As said earlier you need to tweak with input parameters of these functions and also need to validate these parameters with other images. As it might be possible that it will work for this case but not for other cases. Based on trial and error you need to fix the parameter values.
I'm trying to build a simple image analyzing tool that will find items that fit in a color range and then finds the centers of said objects.
As an example, after masking, I'm analyzing an image like this:
What I'm doing so far code-wise is rather simple:
import cv2
import numpy
bound = 30
inc = numpy.array([225,225,225])
lower = inc - bound
upper = inc + bound
img = cv2.imread("input.tiff")
cv2.imshow("Original", img)
mask = cv2.inRange(img, lower, upper)
cv2.imshow("Range", mask)
contours = cv2.findContours(mask, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_TC89_L1)
print contours
This, however, gives me a countless number of contours. I'm somewhat at a loss while reading the corresponding manpage. Can I use moments to reasonably analyze the contours? Are contours even the right tool for this?
I found this question, that vaguely covers finding the center of one object, but how would I modify this approach when there are multiple items?
How do I find the centers of the objects in the image? For example, in the above sample image I'm looking to find three points (the centers of the rectangle and the two circles).
Try print len(contours). That will give you around the expected answer. The output you see is the full representation of the contours which could be thousands of points.
Try this code:
import cv2
import numpy
img = cv2.imread('inp.png', 0)
_, contours, _ = cv2.findContours(img.copy(), cv2.RETR_CCOMP, cv2.CHAIN_APPROX_TC89_L1)
print len(contours)
centres = []
for i in range(len(contours)):
moments = cv2.moments(contours[i])
centres.append((int(moments['m10']/moments['m00']), int(moments['m01']/moments['m00'])))
cv2.circle(img, centres[-1], 3, (0, 0, 0), -1)
print centres
cv2.imshow('image', img)
cv2.imwrite('output.png',img)
cv2.waitKey(0)
This gives me 4 centres:
[(474, 411), (96, 345), (58, 214), (396, 145)]
The obvious thing to do here is to also check for the area of the contours and if it is too small as a percentage of the image, don't count it as a real contour, it will just be noise. Just add something like this to the top of the for loop:
if cv2.contourArea(contours[i]) < 100:
continue
For the return values from findContours, I'm not sure what the first value is for as it is not present in the C++ version of OpenCV (which is what I use). The second value is obviously just the contours (an array of arrays) and the third value is a hierarchy holding information on the nesting of contours, which can be very handy.
You can use the opencv minEnclosingCircle() function on your contours to get the center of each object.
Check out this example which is in c++ but you can adapt the logic Example