Let's say that we have the following black and white image (image.png):
We load the image using OpenCV2 with the following code:
import cv2
img = cv2.imread('image.png')
How can we detect the corners of all white rectangles in an image? We can assume that all rectangles are parallel to the corners of the image.
Result in this case should be in the following form (res[0] is left rectangle and res[1] is right rectangle):
res = [
[
(20, 30), # upper left
(40, 30), # upper right
(20, 80), # bottom left
(40, 80) # bottom right
],
[
(100, 20), # upper left
(140, 20), # upper right
(100, 70), # bottom left
(140, 70) # bottom right
]
]
There are a few possibilities:
the "Harris Corner Detector" is good at finding corners - see here
you can use OpenCV's findContours()
you could use "Hit-or-Miss" morphology to look for corners
you could convolve the image with a kernel and look for specific outputs
So, looking at the last option, if we slide a 2x2 pixel kernel over the image and multiply each of the elements underneath the kernel by 1 and add them all together, and then find all the pixels where that total comes to 255, that will be a 2x2 square where exactly one pixel is white - and that is a corner:
import cv2
import numpy as np
# Load image as greyscale
im = cv2.imread('tZHHE.png',cv2.IMREAD_GRAYSCALE)
# Make a 2x2 kernel of ones
kernel = np.ones((2,2), dtype=np.uint8)
# Convolve the image with the kernel
res = cv2.filter2D(im.astype(np.int16), -1, kernel)
# Get coordinates of points where the sum of the 2x2 window comes to 255
corners = np.where(res==255)
Sample Output
(array([101, 101, 118, 118, 166, 166, 174, 174]),
array([274, 307, 117, 134, 274, 307, 117, 134]))
Looking at the "Hit-or-Miss" morphology method, I will do it with ImageMagick straight in the Terminal, but you can equally do it with other Python libraries:
magick tZHHE.png -alpha off -morphology HMT Corners result.png
As always, I am indebted to Anthony Thyssen for his ImageMagick examples as linked above. We are looking for these specific shapes with the "Hit-or-Miss" morphology:
Keywords: Python, OpenCV, image processing, convolution, corner detect, corner detector, corner detection, ImageMagick Hit-or-Miss morphology.
This works for any number of rectangles:
import cv2 as cv
import pprint as pprint
img = cv.imread("tZHHE.png") # read image
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY) # make grayscale image
cv.imshow("Our initial image",img) # show our original image
corners = cv.goodFeaturesToTrack(gray,2000,0.01,5) # find our corners, 2000 is the number of corners we can detect, 5 is the distance between corners
xylist = [] #put all of our xy coords in here
for corn in corners: # extract our corners and put them in xylist
x,y = corn[0]
xylist.append((x,y))
x = int(x)
y = int(y)
cv.rectangle(img, (x-2,y-2), (x+2,y+2), (100,100,0),-1) # now mark where our corners are on our original image
res = [[] for i in range(int(len(xylist)/4))] # generate i nested lists for our rectangles
for index, item in enumerate(xylist): # format coordinates as you want them
res[index % int(len(xylist)/4)].append(item)
print("\n"+"found ",int(len(xylist)/4) ,"rectangles\n") # how many rectangles did we have?
print(res)
cv.imshow("Corners?", img) # show our new image with rectangle corners marked
Try findContours()
I suggest you to try findContours() with its companion boundingRect().
Here is how you can make it work.
Load the image in grayscale, then pass it to the function findContours().
img = cv2.imread('tZHHe.png', cv2.IMREAD_GRAYSCALE)
contours, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
Getting the bounding box from contours, it returns x, y coordinates of the top left corner and w, h the width and height of the box:
[cv2.boundingRect(contour) for contour in contours]
#=> [(117, 118, 17, 56), (274, 101, 33, 65)]
View in action
Try this maybe in a Jupyter Notebook to see a kind of animation:
def corners_from_bounding_rect(bounding_rect):
x, y, w, h = bounding_rect
points = {'top_left': (x, y), 'top_right':(x+w, y), 'bottom-left': (x, y+h), 'bottom-rigth':(x+w, y+h)}
return points
Retrieve the points from contours using the method defined:
corner_groups = [corners_from_bounding_rect(cv2.boundingRect(cnt)) for cnt in contours]
# [{'top_left': (117, 118),
# 'top_right': (134, 118),
# 'bottom-left': (117, 174),
# 'bottom-rigth': (134, 174)},
# {'top_left': (274, 101),
# 'top_right': (307, 101),
# 'bottom-left': (274, 166),
# 'bottom-rigth': (307, 166)}]
Then plot the sequence:
pinned_img = img.copy()
for n, corners in enumerate(corner_groups):
for name, point in corners.items():
cv2.circle(pinned_img, point, 10, 255)
plt.title(f'{n}-{name}')
plt.imshow(pinned_img)
plt.show()
The first image from the squence:
Try this code
import cv2
img=cv2.imread('tZHHE.png') # Read my Image
imgContours=img.copy() # Copy my Image for Contours
imgCanny=cv2.Canny(imgContours,10,50) # Image to Edges
contours,hierarchy =cv2.findContours(imgCanny,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE)
rectangles=[]
for cont in reversed(contours):
area=cv2.contourArea(cont)
x, y, w, h = cv2.boundingRect(cont)
rectangles.append([[x,y],[x+w,y],[x+w,y+h],[x,y+h]])
cv2.imshow("con", imgContours)
cv2.waitKey(0)
print(rectangles)
cv2.destroyAllWindows()
left to right rectangles
Output:
[[[273, 100], [307, 100], [307, 166], [273, 166]], [[116, 117], [134, 117], [134, 174], [116, 174]]]
Related
I would like to photograph a LEGO building instruction from different perspectives and determine the perspective of the entire sheet based on the LEGO logo depicted on it.
As a first approach, I detected the square of the logo using a mask in the color space in order to then recognize the edges and use the edges to display the entire image from the correct perspective. (The goal later is to be able to read the ID reliably, regardless of the perspective from which the picture is taken.)
https://i.stack.imgur.com/U5ZkE.png
filterLegoLogo(image):
img = image.copy()
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
lower1 = np.array([0, 100, 20])
upper1 = np.array([10, 255, 255])
lower2 = np.array([170, 200, 20])
upper2 = np.array([179, 255, 255])
lower_mask = cv2.inRange(hsv, lower1, upper1)
upper_mask = cv2.inRange(hsv, lower2, upper2)
full_mask = lower_mask + upper_mask;
return full_mask
Output:
https://i.stack.imgur.com/SL2Op.png
What I want to achieve:
https://i.stack.imgur.com/1DmDj.png
Later I want to transform the recognized area and read out the ID.
I use this to transform the image:
image = cv2.imread('images/lego.png')
w, h, _ = np.shape(image)
pts1 = np.float32([[170, 337], [714, 379], [199, 600], [841, 621]])
# Size of transformed image
pts2 = np.float32([[200, 200], [500, 200], [200, 500], [500, 500]])
for val in pts1:
cv2.circle(image, (int(val[0]), int(val[1])), 5, (0, 255, 0), -1)
M = cv2.getPerspectiveTransform(pts1, pts2)
dst = cv2.warpPerspective(image, M, (h, w))
plt.imshow(dst)
plt.title('Transformed Image')
plt.show()
And as a result I get this:
https://i.stack.imgur.com/4473V.png
The points for pts1 are entered manually. I would like to automate edge detection.
Does anyone have an idea?
Thank you in advance!
Here is one way to do that in Python/OpenCV.
Read the input
Threshold using inRange()
Apply morphology to clean it up
Get the convex hull
Get the 4 corner points using approxPolyDP()
Specify the corresponding output corner points
Warp the image
Save the results
Input:
import cv2
import numpy as np
# read image
img = cv2.imread('lego.png')
hh, ww = img.shape[:2]
# threshold on red color
lower=(90,100,170)
upper=(130,150,220)
thresh = cv2.inRange(img, lower, upper)
# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3))
morph = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (7,7))
morph = cv2.morphologyEx(morph, cv2.MORPH_CLOSE, kernel)
# get convex hull
points = np.column_stack(np.where(morph.transpose() > 0))
hull = cv2.convexHull(points)
peri = cv2.arcLength(hull, True)
hullimg = img.copy()
hullimg = cv2.polylines(hullimg, [hull], True, (0,255,0), 1)
# get 4 corners
poly = cv2.approxPolyDP(hull, 0.01 * peri, True)
plist = poly.tolist()
print(plist)
# plist ordered cw from bottom right
cornerimg = img.copy()
cv2.polylines(cornerimg, [np.int32(poly)], True, (0,0,255), 1, cv2.LINE_AA)
# specify control point pairs
# order pts cw from bottom right
inpts = np.float32(poly)
outpts = np.float32([[500, 500], [200, 500], [200, 200], [500, 200]])
# warp image
M = cv2.getPerspectiveTransform(inpts, outpts)
warpimg = cv2.warpPerspective(img, M, (ww,hh))
# save results
cv2.imwrite("lego_thresh.png", thresh)
cv2.imwrite("lego_morph.png", morph)
cv2.imwrite("lego_convexhull.png", hullimg)
cv2.imwrite("lego_corners.png", cornerimg)
cv2.imwrite("lego_warped.png", warpimg)
cv2.imshow('thresh',thresh)
cv2.imshow('morph',morph)
cv2.imshow('hullimg',hullimg)
cv2.imshow('cornerimg',cornerimg)
cv2.imshow('warpimg',warpimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
Threshold Image:
Morphology Cleaned Image:
Convex Hull Image:
4 Corner Image:
Warped Result:
The Mask image gives a good starting point. Extract the contours and find a way to filter out the interesting one (base on size, length, straightness or whatever criterion that works). Then find the point farthest from the center, and the point farthest from the first point (these define the longest diagonal). Next find the two points farthest form the diagonal, on both sides. Now you have the four interesting corners.
is it possible to get the coordinates of the incomplete circle? i am using opencv and python. so i can find the most of the circles.
But i have no clue how can i detect the incomplete cirlce in the picture.
I am looking for a simple way to solve it.
import sys
import cv2 as cv
import numpy as np
## [load]
default_file = 'captcha2.png'
# Loads an image
src = cv.imread(cv.samples.findFile(default_file), cv.IMREAD_COLOR)
## [convert_to_gray]
# Convert it to gray
gray = cv.cvtColor(src, cv.COLOR_BGR2GRAY)
## [convert_to_gray]
## [reduce_noise]
# Reduce the noise to avoid false circle detection
gray = cv.medianBlur(gray, 3)
## [reduce_noise]
## [houghcircles]
#rows = gray.shape[0]
circles = cv.HoughCircles(gray, cv.HOUGH_GRADIENT, 1, 5,
param1=1, param2=35,
minRadius=1, maxRadius=30)
## [houghcircles]
## [draw]
if circles is not None:
circles = np.uint16(np.around(circles))
for i in circles[0, :]:
center = (i[0], i[1])
# circle center
cv.circle(src, center, 1, (0, 100, 100), 3)
# circle outline
radius = i[2]
cv.circle(src, center, radius, (255, 0, 255), 3)
## [draw]
## [display]
cv.imshow("detected circles", src)
cv.waitKey(0)
## [display]
Hi - there is an other Picture. I want the x and y cords of the incomplete circle, light blue on the lower left.
Here the original Pic:
You need to remove the colorful background of your image and display only circles.
One approach is:
Get the binary mask of the input image
Apply Hough Circle to detect the circles
Binary mask:
Using the binary mask, we will detect the circles:
Code:
# Load the libraries
import cv2
import numpy as np
# Load the image
img = cv2.imread("r5lcN.png")
# Copy the input image
out = img.copy()
# Convert to the HSV color space
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# Get binary mask
msk = cv2.inRange(hsv, np.array([0, 0, 130]), np.array([179, 255, 255]))
# Detect circles in the image
crc = cv2.HoughCircles(msk, cv2.HOUGH_GRADIENT, 1, 10, param1=50, param2=25, minRadius=0, maxRadius=0)
# Ensure circles were found
if crc is not None:
# Convert the coordinates and radius of the circles to integers
crc = np.round(crc[0, :]).astype("int")
# For each (x, y) coordinates and radius of the circles
for (x, y, r) in crc:
# Draw the circle
cv2.circle(out, (x, y), r, (0, 255, 0), 4)
# Print coordinates
print("x:{}, y:{}".format(x, y))
# Display
cv2.imshow("out", np.hstack([img, out]))
cv2.waitKey(0)
Output:
x:178, y:60
x:128, y:22
x:248, y:20
x:378, y:52
x:280, y:60
x:294, y:46
x:250, y:44
x:150, y:62
Explanation
We have three chance for finding the thresholding:
Simple Threshold result:
Adaptive Threshold
Binary mask
As we can see the third option gave us a suitable result. Of course, you could get the desired result with other options, but it might take a long time for finding the suitable parameters. Then we applied Hough circles, played with parameter values, and got the desired result.
Update
For the second uploaded image, you can detect the semi-circle by reducing the first and second parameters of the Hough circle.
crc = cv2.HoughCircles(msk, cv2.HOUGH_GRADIENT, 1, 10, param1=10, param2=15, minRadius=0, maxRadius=0)
Replacing the above line in the main code will result in:
Console result
x:238, y:38
x:56, y:30
x:44, y:62
x:208, y:26
I have a depth image (in blue) and of it, I generate an outline image, where I want to take some points automatically (highlighted in red).
I already have the points (x,y) of the outline, but I would like to pick up the points marked in red automatically.
For this task, I am using python 3.6 (OpenCV ...) and I have a 435i realsense camera that gives me the depth image.
Would it be possible to get an approximate value of these coordinates automatically?
Image: https://i.stack.imgur.com/qiTvh.png
a small example of the code:
# function tha captures the edges
def get_edges(frame):
image = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
image = cv2.GaussianBlur(image, (5, 5), 0)
_, image = cv2.threshold(image, 100, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
image = cv2.bitwise_not(image)
element = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
image = cv2.erode(image, element)
image = cv2.dilate(image, element)
image = cv2.Canny(image, 100, 200, L2gradient=True)
return image
# ... read the frames, process the image ...
image_process = get_edges(depth_frame)
# taking the contour indices
indices = np.where(image_process == 255)
# get coordiniates
coordinates = zip(indices[1], indices[0])
My variable "coordinates" has all the points (x,y) of the outline, but I would like to automatically set the points in red highlighted in the image.
I managed to solve using Pose Estimation
Final image
Where I have the return of the points and coordinates:
points = {'A': (67, 121), 'B': (66, 21), 'C': (165, 38), 'D': (133, 143)}
I have a picture( Basic X-Y plot image, where the plot line is in blue color and x,y axis are in black color), where in that I need to detect the edges based on the color. I came across below code, where its detecting all the lines by using canny edge detection and hough algorithm. But I need to detect only the blue color line in this image. What can i do in order to detect that?
Below is the code that I used.
import numpy as np
import cv2
img = cv2.imread('xyplot.png')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
cv2.imshow('grayimage',gray)
edges = cv2.Canny(gray,50,150,apertureSize = 3)
cv2.imshow('edgesimage',edges)
print img.shape[1]
print img.shape
minLineLength=img.shape[1]-300
lines = cv2.HoughLinesP(image=edges,rho=0.02,theta=np.pi/500,
threshold=10,lines=np.array([]),
minLineLength=minLineLength, maxLineGap=100)
a,b,c = lines.shape
for i in range(a):
cv2.line(img, (lines[i][0][0], lines[i][0][1]), (lines[i][0][2],
lines[i][0] [3]), (0, 0, 255), 3, cv2.LINE_AA)
cv2.imshow('edges', edges)
cv2.imshow('result', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Apply a (blue) color mask on your image before converting to gray scale. You can do so by defining the lower and upper boundary of all 3 channels and performing BITWISE_AND on the original image. You will have to play around with the values of the channel ranges to make sure only the pixels you want get captured.
lower = np.array([200, 20, 20], dtype = "uint8")
upper = np.array([255, 100, 100], dtype = "uint8")
mask = cv2.inRange(img, lower, upper)
img = cv2.bitwise_and(img, img, mask = mask)
Note: The three channels in the list are B, G and R respectively.
I'm doing finger detection i.e. the fingertips using VGA camera. I have used HSV and image thresholding and I am able to detect the fingertips.
The Problem: I can now find the centroid of the white spot in a black and white image if there is only one white spot (if I place just one finger); but if I place multiple fingers more white spots will come in the final image. So I want to find each centroid separately.
I want to find all centroids of each of the white spots i.e. if I place more than one finger in front of the camera. Look at the code below
thresholded_img = cv.CreateImage(cv.GetSize(hsv_img), 8, 1)
cv.InRangeS(hsv_img, (0,0,200), (0,0,255), thresholded_img)
moments = cv.Moments(cv.GetMat(thresholded_img,1), 0)
area = cv.GetCentralMoment(moments, 0, 0)
x = cv.GetSpatialMoment(moments, 1, 0)/area
y = cv.GetSpatialMoment(moments, 0, 1)/area
posY=y
posX=x
Here thresholded_img is a black and white image where the fingertips alone is represented as white and all other in black.
In this code if the thresholded_img contains a single white spot then I can get the x and y coordinates of the centroid of that dot correctly!
But how to find centroid of each white dots in this image?
But if there is multiple white dots in the thresholded image then it's finding wrong centroid!
How can I change the above code to find (x,y coordinates) separate centroid of each white dots in a single frame (image)?
Refer this image please.
http://www.csksoft.net/data/pic/laserkbd/036.jpg
I tested the following code with the image you uploaded.
I got the following textual output:
cv2 version: 2.4.4
centroids: [(580, 547), (437, 546), (276, 545), (115, 545), (495, 425), (334, 424), (174, 424), (24, 423), (581, 304), (437, 303), (277, 303), (117, 302), (495, 182), (334, 181), (174, 181), (25, 181), (581, 60), (438, 59), (277, 59), (117, 59)]
and this image:
#!/usr/bin/env python
import cv2
img = cv2.imread('HFOUG.jpg',cv2.CV_LOAD_IMAGE_GRAYSCALE)
_,img = cv2.threshold(img,0,255,cv2.THRESH_OTSU)
h, w = img.shape[:2]
contours0, hierarchy = cv2.findContours( img.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
moments = [cv2.moments(cnt) for cnt in contours0]
# Nota Bene: I rounded the centroids to integer.
centroids = [( int(round(m['m10']/m['m00'])),int(round(m['m01']/m['m00'])) ) for m in moments]
print 'cv2 version:', cv2.__version__
print 'centroids:', centroids
for c in centroids:
# I draw a black little empty circle in the centroid position
cv2.circle(img,c,5,(0,0,0))
cv2.imshow('image', img)
0xFF & cv2.waitKey()
cv2.destroyAllWindows()
See also this answer https://stackoverflow.com/a/9059648/15485 to the question Python OpenCV - Find black areas in a binary image