cv2.matchTemplate finds wrong template in image - python

I'm trying to create a program that knows what number is on an image with the following function:
def img_in_img(big_picture, small_picture, tamper):
big_picture = str(big_picture)
small_picture = str(small_picture)
if os.path.isfile(big_picture) == False or os.path.isfile(small_picture) == False:
return "Image does not exist"
img = cv2.imread(big_picture,0)
templace_loc = small_picture
template = cv2.imread(templace_loc,0)
w, h = template.shape[::-1]
method = cv2.TM_CCOEFF
tamper = int(tamper)
res = cv2.matchTemplate(img,template,method)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)
top_left = max_loc
bottom_right = (top_left[0] + w, top_left[1] + h)
height, width, channels = cv2.imread(templace_loc).shape
if int(top_left[0]) < int(height + tamper) and int(top_left[0]) > int(height - tamper) and int(top_left[1]) < int(width + tamper) and int(top_left[1]) > int(width - tamper):
return True
else:
return False
But then when I check if 7.png is in img.png with the code
nur = "7"
if img_in_img("2020-01-14-17-36-08.537043/verification_image2.png", "verifynr/" + "old_orange" + "/" + nur + ".png", 25):
print(Style.BRIGHT + Fore.GREEN + "Color: " + "old_orange" + ", Num: " + nur + Fore.RESET)
else:
print(Style.BRIGHT + Fore.RED + "Color: " + "old_orange" + ", Num: " + nur + Fore.RESET)
it gives me in RED: Color: old_orange, Num: 7
but then if I check if 6.png is in img.png by changing nur from 7 to 6 it gives me in Green: Color: old_orange, Num: 6, but that's the wrong image.
I've also tried the following code:
img_rgb = cv2.imread("2020-01-14-17-36-08.537043/verification_image2.png")
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
template = cv2.imread('verifynr/old_orange/7.png',0)
w, h = template.shape[::-1]
res = cv2.matchTemplate(img_gray,template,cv2.TM_SQDIFF)
threshold = 0.8
loc = np.where( res >= threshold)
pt = list(zip(*loc[::-1]))
if len(pt) >= 1:
print("True")
which prints True, but it does that for every number png I saved.
How do I make my program recognize the 7.png in the img.png without recognizing every single number png?
img.png:
6.png:
7.png:

Template matching not for object detection or pattern recognition, it's function is to find the most similar patch position. For detection use detectors (haar, dnn based ones, ...), for recognition use classifiers (descriptor based or nn based).
'

Here is an example of using a template mask to do template matching at one resolution level using Python/OpenCV. The black region of the mask tells the template matching to ignore those regions when matching, so that when the template has a background color different from that in the larger image, it does not degrade the match score. Only the color regions of the template corresponding to the white in the mask contribute to the matching score. Note that masked template matching in OpenCV only works for methods TM_SQDIFF and TM_CCORR_NORMED.
Input:
Template:
Template Mask:
import cv2
import numpy as np
# read image
img = cv2.imread('logo.png')
# read template
tmplt = cv2.imread('hat.png')
hh, ww, cc = tmplt.shape
# read template mask as grayscale
tmplt_mask = cv2.imread('hat_mask.png', cv2.COLOR_BGR2GRAY)
# do template matching
corrimg = cv2.matchTemplate(img,tmplt,cv2.TM_CCORR_NORMED, mask=tmplt_mask)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrimg)
max_val_ncc = '{:.3f}'.format(max_val)
print("correlation match score: " + max_val_ncc)
xx = max_loc[0]
yy = max_loc[1]
print('xmatch =',xx,'ymatch =',yy)
# draw red bounding box to define match location
result = img.copy()
pt1 = (xx,yy)
pt2 = (xx+ww, yy+hh)
cv2.rectangle(result, pt1, pt2, (0,0,255), 1)
cv2.imshow('image', img)
cv2.imshow('template', tmplt)
cv2.imshow('template_mask', tmplt_mask)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()
# save results
cv2.imwrite('logo_hat_match.png', result)
Result:
Textual Information Listed:
correlation match score: 1.000
xmatch = 417 ymatch = 44
(Apologies for borrowing and modifying the ImageMagick logo. I already had that template matching example)

These are some few potential causes:
Scale and rotation difference: If the template image and the image you are searching for the template in have different scales or rotations, cv2.matchTemplate() may not find a match. You can try using techniques such as scaling or rotating the template image to match the image you are searching in.
Lighting and contrast differences: If the lighting or contrast of the template image and the image you are searching in are different, it can affect the accuracy of the match. You can try adjusting the brightness and contrast of the images to make them more similar.
Template size: If the template is too small or too big compared to the area you are searching in, it can affect the accuracy of the match. You should try using a template size that is similar to the expected size of the object in the image.
Similarity threshold: The matchTemplate() method has a threshold value that you can set to control the level of similarity required for a match. If the threshold value is set too high, it may not find any matches even if there are similar templates in the image. If it's set too low, it may return false positives.
Similarity Measurement Algorithm: matchTemplate() uses a specific algorithm for measuring the similarity between the template and the search image. Depending on the image and the template, one algorithm might work better than another.

Related

Why is this code of Object detection using Template Matching does not run successfully

The output that I get is just the reference image and no bounding box is seen in the output.
I have tried this code from this website: https://www.sicara.fr/blog-technique/object-detection-template-matching
Here's the reference image
Reference Image
Here are the templates:
1:
templates 2:
templates 3:
As compared to the website, using the code the output should look like this:
Expected Output:
I am expecting to have this output as discussed in the website, however, when I tried to run this code, nothing seems to be detected. Here is the code that I copied:
import cv2
import numpy as np
DEFAULT_TEMPLATE_MATCHING_THRESHOLD = 0.9
class Template:
"""
A class defining a template
"""
def __init__(self, image_path, label, color, matching_threshold=DEFAULT_TEMPLATE_MATCHING_THRESHOLD):
"""
Args:
image_path (str): path of the template image path
label (str): the label corresponding to the template
color (List[int]): the color associated with the label (to plot detections)
matching_threshold (float): the minimum similarity score to consider an object is detected by template
matching
"""
self.image_path = image_path
self.label = label
self.color = color
self.template = cv2.imread(image_path)
self.template_height, self.template_width = self.template.shape[:2]
self.matching_threshold = matching_threshold
image = cv2.imread("reference.jpg")
templates = [
Template(image_path="Component1.jpg", label="1", color=(0, 0, 255), matching_threshold=0.99),
Template(image_path="Component2.jpg", label="2", color=(0, 255, 0,) , matching_threshold=0.91),
Template(image_path="Component3.jpg", label="3", color=(0, 191, 255), matching_threshold=0.99),
detections = []
for template in templates:
template_matching = cv2.matchTemplate(template.template, image, cv2.TM_CCORR_NORMED)
match_locations = np.where(template_matching >= template.matching_threshold)
for (x, y) in zip(match_locations[1], match_locations[0]):
match = {
"TOP_LEFT_X": x,
"TOP_LEFT_Y": y,
"BOTTOM_RIGHT_X": x + template.template_width,
"BOTTOM_RIGHT_Y": y + template.template_height,
"MATCH_VALUE": template_matching[y, x],
"LABEL": template.label,
"COLOR": template.color
}
detections.append(match)
def compute_iou(boxA, boxB):
xA = max(boxA["TOP_LEFT_X"], boxB["TOP_LEFT_X"])
yA = max(boxA["TOP_LEFT_Y"], boxB["TOP_LEFT_Y"])
xB = min(boxA["BOTTOM_RIGHT_X"], boxB["BOTTOM_RIGHT_X"])
yB = min(boxA["BOTTOM_RIGHT_Y"], boxB["BOTTOM_RIGHT_Y"])
interArea = max(0, xB - xA + 1) * max(0, yB - yA + 1)
boxAArea = (boxA["BOTTOM_RIGHT_X"] - boxA["TOP_LEFT_X"] + 1) * (boxA["BOTTOM_RIGHT_Y"] - boxA["TOP_LEFT_Y"] + 1)
boxBArea = (boxB["BOTTOM_RIGHT_X"] - boxB["TOP_LEFT_X"] + 1) * (boxB["BOTTOM_RIGHT_Y"] - boxB["TOP_LEFT_Y"] + 1)
iou = interArea / float(boxAArea + boxBArea - interArea)
return iou
def non_max_suppression(objects, non_max_suppression_threshold=0.5, score_key="MATCH_VALUE"):
"""
Filter objects overlapping with IoU over threshold by keeping only the one with maximum score.
Args:
objects (List[dict]): a list of objects dictionaries, with:
{score_key} (float): the object score
{top_left_x} (float): the top-left x-axis coordinate of the object bounding box
{top_left_y} (float): the top-left y-axis coordinate of the object bounding box
{bottom_right_x} (float): the bottom-right x-axis coordinate of the object bounding box
{bottom_right_y} (float): the bottom-right y-axis coordinate of the object bounding box
non_max_suppression_threshold (float): the minimum IoU value used to filter overlapping boxes when
conducting non-max suppression.
score_key (str): score key in objects dicts
Returns:
List[dict]: the filtered list of dictionaries.
"""
sorted_objects = sorted(objects, key=lambda obj: obj[score_key], reverse=True)
filtered_objects = []
for object_ in sorted_objects:
overlap_found = False
for filtered_object in filtered_objects:
iou = compute_iou(object_, filtered_object)
if iou > non_max_suppression_threshold:
overlap_found = True
break
if not overlap_found:
filtered_objects.append(object_)
return filtered_objects
NMS_THRESHOLD = 0.2
detections = non_max_suppression(detections, non_max_suppression_threshold=NMS_THRESHOLD)
image_with_detections = image.copy()
for detection in detections:
cv2.rectangle(
image_with_detections,
(detection["TOP_LEFT_X"], detection["TOP_LEFT_Y"]),
(detection["BOTTOM_RIGHT_X"], detection["BOTTOM_RIGHT_Y"]),
detection["COLOR"],
2,
)
cv2.putText(
image_with_detections,
f"{detection['LABEL']} - {detection['MATCH_VALUE']}",
(detection["TOP_LEFT_X"] + 2, detection["TOP_LEFT_Y"] + 20),
cv2.FONT_HERSHEY_SIMPLEX, 0.5,
detection["COLOR"], 1,
cv2.LINE_AA,
)
# NMS_THRESHOLD = 0.2
# detection = non_max_suppression(detections, non_max_suppression_threshold=NMS_THRESHOLD)
print("Image written to file-system: ", status)
cv2.imshow("res", image_with_detections)
cv2.waitKey(0)
this is how his final output looks like:
5
Here's my attempt in detecting the larger components, the code was able to detect them and here is the result:
Result
Here are the resize templates and the original components that I wanted to detect but unfortunately can't:
1st 2nd 3rd
Here is a method of finding multiple matches in template matching in Python/OpenCV using your reference and smallest template. I have remove all the white padding you had around your template. My method simply draws a black rectangle over the correlation image where it matches and then repeats looking for the next best match in the modified correlation image.
I have used cv2.TM_CCORR_NORMED and a match threshold of 0.90. You have 4 of these templates showing in your reference image, so I set my search number to 4 and spacing of 10 for the non-maximum suppression by masking. You have other small items of the same shape and size, but the text on them is different. So you will need different templates for each.
Reference:
Template:
import cv2
import numpy as np
# read image
img = cv2.imread('circuit_board.jpg')
# read template
tmplt = cv2.imread('circuit_item.png')
hh, ww, cc = tmplt.shape
# set arguments
match_thresh = 0.90 # stopping threshold for match value
num_matches = 4 # stopping threshold for number of matches
match_radius = 10 # approx radius of match peaks
match_radius2 = match_radius//2
# get correlation surface from template matching
corrimg = cv2.matchTemplate(img,tmplt,cv2.TM_CCORR_NORMED)
hc, wc = corrimg.shape
# get locations of all peaks higher than match_thresh for up to num_matches
imgcopy = img.copy()
corrcopy = corrimg.copy()
for i in range(0, num_matches):
# get max value and location of max
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrcopy)
x1 = max_loc[0]
y1 = max_loc[1]
x2 = x1 + ww
y2 = y1 + hh
loc = str(x1) + "," + str(y1)
if max_val > match_thresh:
print("match number:", i+1, "match value:", max_val, "match x,y:", loc)
# draw draw white bounding box to define match location
cv2.rectangle(imgcopy, (x1,y1), (x2,y2), (255,255,255), 1)
# insert black rectangle over copy of corr image so not find that match again
corrcopy[y1-match_radius2:y1+match_radius2, x1-match_radius2:x1+match_radius2] = 0
i = i + 1
else:
break
# save results
# power of 4 exaggeration of correlation image to emphasize peaks
cv2.imwrite('circuit_board_multi_template_corr.png', (255*cv2.pow(corrimg,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('circuit_board_multi_template_corr_masked.png', (255*cv2.pow(corrcopy,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('circuit_board_multi_template_match.png', imgcopy)
# show results
# power of 4 exaggeration of correlation image to emphasize peaks
cv2.imshow('image', img)
cv2.imshow('template', tmplt)
cv2.imshow('corr', cv2.pow(corrimg,4))
cv2.imshow('corr masked', cv2.pow(corrcopy,4))
cv2.imshow('result', imgcopy)
cv2.waitKey(0)
cv2.destroyAllWindows()
Original Correlation Image:
Modified Correlation Image after 4 matches:
Matches Marked on Input as White Rectangles:
Match Locations:
match number: 1 match value: 0.9982172250747681 match x,y: 128,68
match number: 2 match value: 0.9762057065963745 match x,y: 128,90
match number: 3 match value: 0.9755787253379822 match x,y: 128,48
match number: 4 match value: 0.963689923286438 match x,y: 127,107

Replacing specific Characters in an Image using OpenCV

I have the next Image:
I want to find all the occurrences of an specific character, let's say "a", and then replacing it with another character, let's say "e"
This can be done in Python3 using OpenCV and Numpy, to do that I'm using the next two images in order to do template matching:
my code is the next:
import cv2 as cv
import numpy as np
img = cv.imread('source_image.jpg') # Importing Source Image
imgray = cv.cvtColor(img, cv.COLOR_BGR2GRAY) # Converting Source Image to Gray Scale
template_a = cv.imread('template_a.jpg') # Reading Template "a" Character
template_a_gray = cv.cvtColor(template_a, cv.COLOR_BGR2GRAY) # Converting Template "a" Character to Gray Scale
template_e = cv.imread('template_e.jpg') # Reading Template "e" Character
template_e_gray = cv.cvtColor(template_e, cv.COLOR_BGR2GRAY) # Converting Template "e" Character to Gray Scale
# Template Matching
w, h = template_a_gray.shape[::-1] # Obtaining Width and Height of Template "a" Character
res = cv.matchTemplate(imgray, template_a_gray, cv.TM_CCOEFF_NORMED) # Template Matching
threshold = 0.6
loc = np.where( res >= threshold)
for pt in zip(*loc[::-1]):
cv.rectangle(imgray, pt, (pt[0] + w, pt[1] + h), color=(255,255,255), thickness=cv.FILLED) # Removing Matched Template "a" Characters
y_offset = pt[1]
x_offset = pt[0]
x_end = x_offset + template_e_gray.shape[1]
y_end = y_offset + template_e_gray.shape[0]
imgray[y_offset:y_end,x_offset:x_end] = template_e_gray # Pasting Template "e" Characters
filename = 'savedImage.jpg'
cv.imwrite(filename, imgray) # Saving Image
cv.waitKey(0)
cv.destroyAllWindows()
When executed the result is:
The problem is that when zoom in, the pasted image isn't exactly the same as the template character for e
Blobs appear above and in the left of the original "e" template character. Why is this happening? How do I get rid of them?
Edit: in Response of Christoph Rackwitz's Comment:
Edit: Code fixed upon Christoph Rackwitz's Suggestions
This is because template_e_gray is pasted multiple times. Blobs are the edges of another e pasted just before. Print pt and see where e is pasted.
There are several ways to deal with this problem. The easiest way is to set the threshold to 0.8.
threshold = 0.8
If this is sufficient, use this. But may not be sufficient for some applications because the threshold needs to be tuned per image.
Another way is to fill white with one pixel larger.
imgray[y_offset - 1:y_end + 1, x_offset - 1:x_end + 1] = 255
imgray[y_offset:y_end, x_offset:x_end] = template_e_gray # Pasting Template "e" Characters
This is easy too, but inefficient, misaligned, and may not work with large images.
My recommandation is to use non maximum suppression (NMS). NMS is a technique commonly used in object detection that delete all overlapping rectangles except the one with the highest score.
Borrowing the implementation from here, this is the complete code.
import cv2 as cv
import numpy as np
# https://github.com/rbgirshick/fast-rcnn/blob/master/lib/utils/nms.py
def nms(dets, thresh):
x1 = dets[:, 0]
y1 = dets[:, 1]
x2 = dets[:, 2]
y2 = dets[:, 3]
scores = dets[:, 4]
areas = (x2 - x1 + 1) * (y2 - y1 + 1)
order = scores.argsort()[::-1]
keep = []
while order.size > 0:
i = order[0]
keep.append(i)
xx1 = np.maximum(x1[i], x1[order[1:]])
yy1 = np.maximum(y1[i], y1[order[1:]])
xx2 = np.minimum(x2[i], x2[order[1:]])
yy2 = np.minimum(y2[i], y2[order[1:]])
w = np.maximum(0.0, xx2 - xx1 + 1)
h = np.maximum(0.0, yy2 - yy1 + 1)
inter = w * h
ovr = inter / (areas[i] + areas[order[1:]] - inter)
inds = np.where(ovr <= thresh)[0]
order = order[inds + 1]
return keep
img = cv.imread('source_image.jpg') # Importing Source Image
imgray = cv.cvtColor(img, cv.COLOR_BGR2GRAY) # Converting Source Image to Gray Scale
template_a = cv.imread('temp/template_a.jpg') # Reading Template "a" Character
template_a_gray = cv.cvtColor(template_a, cv.COLOR_BGR2GRAY) # Converting Template "a" Character to Gray Scale
template_e = cv.imread('temp/template_e.jpg') # Reading Template "e" Character
template_e_gray = cv.cvtColor(template_e, cv.COLOR_BGR2GRAY) # Converting Template "e" Character to Gray Scale
# Template Matching
w, h = template_a_gray.shape[::-1] # Obtaining Width and Height of Template "a" Character
res = cv.matchTemplate(imgray, template_a_gray, cv.TM_CCOEFF_NORMED) # Template Matching
threshold = 0.6
loc = np.where(res >= threshold)
top, left = loc
score = res[loc]
iou_threshold = 0.5
matched_boxes = np.array([left, top, left + w, top + h, score]).T
valid_indices = nms(matched_boxes, iou_threshold)
boxes = matched_boxes[valid_indices]
boxes = boxes[..., :-1].astype(int)
for pt in boxes:
print(pt)
cv.rectangle(imgray, pt[:2], pt[2:], color=(255, 255, 255),
thickness=cv.FILLED) # Removing Matched Template "a" Characters
y_offset = pt[1]
x_offset = pt[0]
x_end = x_offset + template_e_gray.shape[1]
y_end = y_offset + template_e_gray.shape[0]
imgray[y_offset:y_end, x_offset:x_end] = template_e_gray # Pasting Template "e" Characters
filename = 'temp/savedImage.jpg'
cv.imwrite(filename, imgray) # Saving Image
cv.waitKey(0)
cv.destroyAllWindows()
This code works perfectly as expected, but you will see the following results. See the area circled in red.
This is a trace of a, not e.
This is caused by the difference in size between a in the image and a in the template, and is unrelated to the original issue. I mentioned this because I thought you would suspect that the same issue remains.

Are there ways to do some crops in an image based on a given pattern?

I have an image of pots of the same size, the user must crop an area (let's say he crops the first pot (top left corner)) of ​​the image and depending on the pattern designed or cropped by the user, I must automatically perform other cropping and save their coordinates. Is there another technique to do this without template matching or do you think I can improve my code to do it with template matching only?
So far I have tried with template matching and saved the coordinates of each corresponding matched, but as you can see in the attached image, the result is not quite satisfying because I don't match all the pots and for some area I draw several rectangles for just one pot (the more I lower the threshold). Any help is higly appreciated.
Here is my code
# import necessary dependies
import cv2
import numpy as np
# Read the source image
img_rgb = cv2.imread('image.jpg')
# Convert the source image to gray
img_gray = cv2.cvtColor(img_rgb, cv.COLOR_BGR2GRAY)
# Read the pattern image designed by the user
template = cv2.imread('mattern.png',0)
# Get the shape of the pattern image
w, h = template.shape[::-1]
# Apply cv2 template matching functon
res = cv2.matchTemplate(img_gray,template,cv.TM_CCOEFF_NORMED)
# List of coordinates of the matched templates
list_coordinates = []
labeled_coordinates = []
# Threshold applied to the template matching
threshold = 0.55
# Apply the threshold to cv2 the template matching
loc = np.where( res >= threshold)
# Directory to save the matched templates (pattern)
s_dir = "s_img/"
# Counter to add in the name of the saved image
i = 1
for pt in zip(*loc[::-1]):
# Draw a rectangle to area that satifies the condition
cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 1)
top_left = pt
# Crop every matched templates (pattern)
crop_img = img_rgb[top_left[1]:top_left[1] + h, top_left[0]:top_left[0]+w]
bottom_right = (top_left[0] + w, top_left[1] + h)
# Save the crop patterns for future use
cv2.imwrite(s_dir+"crop_1_"+str(i)+".png", crop_img)
# Label the coordinates for future use
labeled_coordinates = ["crop_1_"+str(i), top_left[0], top_left[1], bottom_right[0], bottom_right[1]]
# Add the coordinates in a list
list_coordinates.append(labeled_coordinates)
i += 1
cv2.imshow('template',template)
cv2.imshow('mathced',img_rgb)
cv2.waitKey(0)
cv2.destroyAllWindows()

Does template matching using Gaussian filter (pyramid) give poor results?

I am attempting to perform template matching using pyramid representation, but getting bad results. I am unsure if the bad results are due to a problem with my algorithm/code or because that is just the result.
The code follows, but I have basically taken an image, converted it to grayscale, cropped the image to get a template, scaled the template, implemented the sliding window method for matching each of the pyramid images (scaled image) to the query image using the metrics Zero-mean Correlation, SSD, and NCC, and determined the localization error
# Load the images
image1 = cv2.imread("image1.jpg")
# Convert the images to grayscale
gray_image1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)
# Crop the image
y_1 = 100
x_1 = 100
w_1 = 15
h_1 = 15
template_image1 = gray_image1[y_1:y_1 + h_1, x_1:x_1 + w_1].copy()
# Scale the template image
template_image1_5 = cv2.resize(template_image1, None, fx=0.5, fy=0.5, interpolation=cv2.INTER_CUBIC)
template_image1_1 = cv2.resize(template_image1, None, fx=1.0, fy=1.0, interpolation=cv2.INTER_CUBIC)
template_image1_2 = cv2.resize(template_image1, None, fx=2.0, fy=2.0, interpolation=cv2.INTER_CUBIC)
# Store the scaled template and their associated scaling factors in a dictionary
scaled_templates_image1 = {'0.5': template_image1_5, '1.0': template_image1_1, '2.0': template_image1_2}
# Put the three different methods (Zero-mean correlation, SSD, and NCC) in a list
methods = ['cv2.TM_CCORR', 'cv2.TM_SQDIFF', 'cv2.TM_CCORR_NORMED']
for meth in methods:
method = eval(meth)
# Apply template matching
for scale, scaled_template_image1 in scaled_templates_image1.items():
gray_image1_copy = gray_image1.copy()
result = cv2.matchTemplate(gray_image1_copy, scaled_template_image1, method)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
# If the method is TM_SQDIFF, then take the minimum
if method in [cv2.TM_SQDIFF]:
top_left = min_loc
else:
top_left = max_loc
bottom_right = (top_left[0] + w_3, top_left[1] + h_3)
cv2.rectangle(gray_image1_copy, top_left, bottom_right, 255, 2)
print(meth, top_left, bottom_right)
euclidean_distance = np.linalg.norm(np.subtract((x_1, y_1), top_left))
print("The localization error for the " + scale + " scale using " + meth + " method is: " +
np.str(euclidean_distance))
plt.subplot(121), plt.imshow(result, cmap='gray')
plt.title('Matching Result'), plt.xticks([]), plt.yticks([])
plt.subplot(122), plt.imshow(gray_image1_copy, cmap='gray')
plt.title('Detected Point'), plt.xticks([]), plt.yticks([])
plt.suptitle("The " + scale + " scale using " + meth + " method")
plt.show()
The localization errors are everywhere. Without scaling, the localization errors for SSD and NCC are 0.0, while the localization error for Zero-mean correlation is above 100 (expected). When applying scaling, the localization errors are almost all above 100. Am I going about this wrong or should I just expect such results? Maybe I'm not understanding the concept and-or problem correctly. Any help is appreciated.

Copy a part of an image in opencv and python

I'm trying to split an image into several sub-images with opencv by identifying templates of the original image and then copy the regions where I matched those templates. I'm a TOTAL newbie to opencv! I've identified the sub-images using:
result = cv2.matchTemplate(img, template, cv2.TM_CCORR_NORMED)
After some cleanup I get a list of tuples called points in which I iterate to show the rectangles. tw and th is the template width and height respectively.
for pt in points:
re = cv2.rectangle(img, pt, (pt[0] + tw, pt[1] + th), 0, 2)
print('%s, %s' % (str(pt[0]), str(pt[1])))
count+=1
What I would like to accomplish is to save the octagons (https://dl.dropbox.com/u/239592/region01.png) into separated files.
How can I do this? I've read something about contours but I'm not sure how to use it. Ideally I would like to contour the octagon.
Thanks a lot for your help!
If template matching is working for you, stick to it. For instance, I considered the following template:
Then, we can pre-process the input in order to make it a binary one and discard small components. After this step, the template matching is performed. Then it is a matter of filtering the matches by means of discarding close ones (I've used a dummy method for that, so if there are too many matches you could see it taking some time). After we decide which points are far apart (and thus identify different hexagons), we can do minor adjusts to them in the following manner:
Sort by y-coordinate;
If two adjacent items start at a y-coordinate that is too close, then set them both to the same y-coord.
Now you can sort this point list in an appropriate order such that the crops are done in raster order. The cropping part is easily achieved using slicing provided by numpy.
import sys
import cv2
import numpy
outbasename = 'hexagon_%02d.png'
img = cv2.imread(sys.argv[1])
template = cv2.cvtColor(cv2.imread(sys.argv[2]), cv2.COLOR_BGR2GRAY)
theight, twidth = template.shape[:2]
# Binarize the input based on the saturation and value.
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
saturation = hsv[:,:,1]
value = hsv[:,:,2]
value[saturation > 35] = 255
value = cv2.threshold(value, 0, 255, cv2.THRESH_OTSU)[1]
# Pad the image.
value = cv2.copyMakeBorder(255 - value, 3, 3, 3, 3, cv2.BORDER_CONSTANT, value=0)
# Discard small components.
img_clean = numpy.zeros(value.shape, dtype=numpy.uint8)
contours, _ = cv2.findContours(value, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
for i, c in enumerate(contours):
area = cv2.contourArea(c)
if area > 500:
cv2.drawContours(img_clean, contours, i, 255, 2)
def closest_pt(a, pt):
if not len(a):
return (float('inf'), float('inf'))
d = a - pt
return a[numpy.argmin((d * d).sum(1))]
match = cv2.matchTemplate(img_clean, template, cv2.TM_CCORR_NORMED)
# Filter matches.
threshold = 0.8
dist_threshold = twidth / 1.5
loc = numpy.where(match > threshold)
ptlist = numpy.zeros((len(loc[0]), 2), dtype=int)
count = 0
print "%d matches" % len(loc[0])
for pt in zip(*loc[::-1]):
cpt = closest_pt(ptlist[:count], pt)
dist = ((cpt[0] - pt[0]) ** 2 + (cpt[1] - pt[1]) ** 2) ** 0.5
if dist > dist_threshold:
ptlist[count] = pt
count += 1
# Adjust points (could do for the x coords too).
ptlist = ptlist[:count]
view = ptlist.ravel().view([('x', int), ('y', int)])
view.sort(order=['y', 'x'])
for i in xrange(1, ptlist.shape[0]):
prev, curr = ptlist[i - 1], ptlist[i]
if abs(curr[1] - prev[1]) < 5:
y = min(curr[1], prev[1])
curr[1], prev[1] = y, y
# Crop in raster order.
view.sort(order=['y', 'x'])
for i, pt in enumerate(ptlist, start=1):
cv2.imwrite(outbasename % i,
img[pt[1]-2:pt[1]+theight-2, pt[0]-2:pt[0]+twidth-2])
print 'Wrote %s' % (outbasename % i)
If you want only the contours of the hexagons, then crop on img_clean instead of img (but then it is pointless to sort the hexagons in raster order).
Here is a representation of the different regions that would be cut for your two examples without modifying the code above:
I am sorry, I didn't understand from your question on how do you relate matchTemplate and Contours.
Anyway, below is a small technique using contours. It is on the assumption that your other images are also like the one you provided. I am not sure if it works with your other images. But I think it would help to get a startup. Try this yourself and make necessary adjustments and modifications.
What I did :
1 - I needed the edge of octagons . So Thresholded Image using Otsu and apply dilation and erosion (or use any method you like that works well for all your images, beware of the edges in left edge of image).
2 - Then found contours (More about contours : http://goo.gl/r0ID0
3 - For each contours, find its convex hull, find its area(A) & perimeter(P)
4 - For a perfect octagon, P*P/A = 13.25 approximately. I used it here and cut it and saved it.
5 - You can see cropping it also removes some edges of octagon. If you want it, adjust the cropping dimension.
Code :
import cv2
import numpy as np
img = cv2.imread('region01.png')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
thresh = cv2.dilate(thresh,None,iterations = 2)
thresh = cv2.erode(thresh,None)
contours,hierarchy = cv2.findContours(thresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)
number = 0
for cnt in contours:
hull = cv2.convexHull(cnt)
area = cv2.contourArea(hull)
P = cv2.arcLength(hull,True)
if ((area != 0) and (13<= P**2/area <= 14)):
#cv2.drawContours(img,[hull],0,255,3)
x,y,w,h = cv2.boundingRect(hull)
number = number + 1
roi = img[y:y+h,x:x+w]
cv2.imshow(str(number),roi)
cv2.imwrite("1"+str(number)+".jpg",roi)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Those 6 octagons will be stored as separate files.
Hope it helps !!!

Categories

Resources