Problem:
I want to detect lines in a given image using OpenCV in Python. Although there are multiple obvious vertical lines, neither normal HoughLines nor probabilistic HoughLines does find them. As I spent plenty of time playing around with the Parameters, I guess I am doing something fundamental wrong here. I am Aware of the fact, that hough-lines is usually applied on edges, e.g. after using canny. Due to canny´s non-maximum supression, canny does not give good results here.
Image, where detecting the vertical lines Fails :
Why:
Given this (image of a water meter) :
I want to detect the rectangle around each digit. To detect the rectangles, I used sobel filters in x and y direction and calculated Magnitude and angle/Phase of the Gradient. As I assume the image to be rotated correctly in this step, I extract vertical and horizontal edges as shown in the image. My hope was to make use of houghLines to find the bounding boxes. Finding the horizontal lines works perfectly, as seen in the
Debug plot containing further insights on the Problem, where as I does not work on the vertical components (second row) :
Detecting the rectangles around each digit would help me to
locate the Region of Interest
cut out the region inside the rectangle, in other words the digit. Several other approches to detect the digits directly by using contours, all had the problem of the outer rectangles interfering with the digit.
Update: the Code for detecting the vertical lines:
#img is initialized with the binarized, vertical component image, as shown above
minLength = 30
maxGap = 7
angle_res = np.pi / 180
rad_res = 2
threshold_val = 100
linesP = cv2.HoughLinesP(img, rad_res, angle_res, threshold_val, minLineLength=minLength, maxLineGap=maxGap)
cdst = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
cdstP = np.copy(cdst)
if linesP is None:
print("Error when finding lines (probabilistic hough transformation). No lines detected")
else:
# Copy edges to the images that will display the results in BGR
for i in range(0, len(linesP)):
l = linesP[i][0]
cv2.line(cdstP, (l[0], l[1]), (l[2], l[3]), (255,0,0), 3, cv2.LINE_AA)
plt.imshow(cdstP); plt.show()
First apply canny edge with proper settings of threshold. Then apply probabilistic hough line transform. After applying hough transform filter the lines with slope. You want to filter the box so you need to filter horizontal and vertical lines. After filtering lines apply morphological dilation and erosion operation back to back to resultant image to get neat box around each digit. While applying hough transform select parameters minimum line length, maximum line length and maximum line gap appropriately.
You can use trackbar function while selecting appropriate parameters. The sample code is given below for selection of threshold for canny edge.
import cv2
import numpy as np
cv2.namedWindow('Result')
img = cv2.imread('qkEuE.png')
v1 = 0
v2 = 0
def doEdges():
edges = cv2.Canny(img,v1,v2)
edges = cv2.cvtColor(edges,cv2.COLOR_GRAY2BGR)
res = np.concatenate((img,edges),axis = 0)
cv2.imshow('Result',res)
def setVal1(val):
global v1
v1 = val
doEdges()
def setVal2(val):
global v2
v2 = val
doEdges()
cv2.createTrackbar('Val1','Result',0,500,setVal1)
cv2.createTrackbar('Val2','Result',0,500,setVal2)
cv2.imshow('Result',img)
cv2.waitKey(0)
cv2.destroyAllWindows
Hope it helps you.
Related
I am working on a problem where i need to find the path the robot can take without hitting any crop rows.Raw Image
My initial approach was to convert this into birds eye view and then use canny and skeletonize techniques.Then I applied Hough transform to come up with the crop rows.This works well when the rows are straight but if i rotate the image by 45 degrees I couldn't find any rows with Hough transform.So I decided to use another approach.
First I only selected the green region and applied morphological filters to remove small branches which come out
img = cv2.imread('''')
min_green2 = np.array([45, 50, 50])
max_green2 = np.array([75, 250, 250])
image_blur = cv2.GaussianBlur(img, (5, 5), 0)
image_blur_hsv = cv2.cvtColor(image_blur, cv2.COLOR_BGR2HSV)
image_green = cv2.inRange(image_blur_hsv, min_red2, max_red2)
se1 = cv2.getStructuringElement(cv2.MORPH_RECT, (7,7))
se2 = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
mask = cv2.morphologyEx(image_green, cv2.MORPH_OPEN, se1)
I ended up with thisFinal_output
Now I want to detect the path the robot can take which is the black region.So only the first row is my region of interest and I tried different methods to draw a line in center of the row but couldn't find any help in opencv.I did manage to get a work around by splitting the image into two vertically and used cv2.fitline function to get a line joining one side of row and did the same with other side of row and finally I plotted the center line.But this is not an ideal approach and I feel like there might be some opencv functions to do it in much better way.Can some one help me with this or guide me in right way.
This is the final output I am looking for
Final expected result with green color showing the center of path
So, here is my approach using numpy and scipy, which yielded this result:
.
Without doing any bluring or morphological operations, use the Canny edge detector:
edges = cv2.Canny(image, 100, 200, None, 3, cv2.DIST_L2)
Notice that most of the edges surround the track your robot wants to follow. Since each edge is a collection of white pixels, we could calculate a column's total intensity:
normalized = cv2.normalize(edges, None, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX)
column_intensity = normalized.sum(axis=0)
Plotting the results, we get
If we were to find the minimum of the graph, then we would find the x direction, where most of the edges are avoided. But first, let's smooth the function so as to avoid some noise.
# smooth function through moving average
window_size = 30
window = np.ones((window_size,)) / window_size
smoothed = np.convolve(column_intensity, window, mode="valid")
Since there are a lot of local minima, our additional constraint is that the x-direction the robot should take is the closest to the center of the image.
# find indices of local minima and select the one closest to the center
indices = scipy.signal.argrelmin(smoothed)[0]
distances = np.abs(indices - int(width / 2))
x = indices[np.argmin(distances)]
Now that we have the x-direction, we need to determine a y coordinate so as to estimate the angle the robot should rotate (tan(angle)=y/x). There are as many choices as there are rows in the image, which means the y coordinate needs to be manually set. If we choose a y closer to the robot, the angle will be more volatile as the robot advances. Conversely, if we choose a y that is far from the robot, then it will be less volatile but less accurate as well. That is up to you; the final image was created with a y = 400.
I hope this fits your needs :)
I have multiple images of bees in my dataset. I am also given the midpoint, orientation and major and minor axis length of each bee. Now, I want a mask that surrounds the bee. I was thinking of extracting around 15-20 points around the bees, that would represent 1 bee. How do I go about doing that?
Can I somehow use OpenCV to get the points around the ellipse?
EDIT
After using cv2.ellipse, I have an image like this. I want around 15-20 corner points for each bee.
gray = cv2.cvtColor(image_with_blue_blobs, cv2.COLOR_BGR2GRAY)
_, contours, _ = cv2.findContours(gray,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
epsilon = 0.01*cv2.arcLength(contours[0],True)
green_dots = cv2.approxPolyDP(contours[0],epsilon,True)
Problem - green_dots variable just has the corners of the image.
You can use OpenCV to draw an ellipse.
cv2.ellipse(img,(256,256),(100,50),0,0,360,255,-1)
You could calculate the points, or you can just find them using OpenCV:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
contours, _ = cv2.findContours(gray,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
epsilon = 0.01*cv2.arcLength(contours[0],True)
green_dots = cv2.approxPolyDP(contours[0],epsilon,True)
I have some hundreds of images (scanned documents), most of them are skewed. I wanted to de-skew them using Python.
Here is the code I used:
import numpy as np
import cv2
from skimage.transform import radon
filename = 'path_to_filename'
# Load file, converting to grayscale
img = cv2.imread(filename)
I = cv2.cvtColor(img, COLOR_BGR2GRAY)
h, w = I.shape
# If the resolution is high, resize the image to reduce processing time.
if (w > 640):
I = cv2.resize(I, (640, int((h / w) * 640)))
I = I - np.mean(I) # Demean; make the brightness extend above and below zero
# Do the radon transform
sinogram = radon(I)
# Find the RMS value of each row and find "busiest" rotation,
# where the transform is lined up perfectly with the alternating dark
# text and white lines
r = np.array([np.sqrt(np.mean(np.abs(line) ** 2)) for line in sinogram.transpose()])
rotation = np.argmax(r)
print('Rotation: {:.2f} degrees'.format(90 - rotation))
# Rotate and save with the original resolution
M = cv2.getRotationMatrix2D((w/2,h/2),90 - rotation,1)
dst = cv2.warpAffine(img,M,(w,h))
cv2.imwrite('rotated.jpg', dst)
This code works well with most of the documents, except with some angles: (180 and 0) and (90 and 270) are often detected as the same angle (i.e it does not make difference between (180 and 0) and (90 and 270)). So I get a lot of upside-down documents.
Here is an example:
The resulted image that I get is the same as the input image.
Is there any suggestion to detect if an image is upside down using Opencv and Python?
PS: I tried to check the orientation using EXIF data, but it didn't lead to any solution.
EDIT:
It is possible to detect the orientation using Tesseract (pytesseract for Python), but it is only possible when the image contains a lot of characters.
For anyone who may need this:
import cv2
import pytesseract
print(pytesseract.image_to_osd(cv2.imread(file_name)))
If the document contains enough characters, it is possible for Tesseract to detect the orientation. However, when the image has few lines, the orientation angle suggested by Tesseract is usually wrong. So this can not be a 100% solution.
Python3/OpenCV4 script to align scanned documents.
Rotate the document and sum the rows. When the document has 0 and 180 degrees of rotation, there will be a lot of black pixels in the image:
Use a score keeping method. Score each image for it's likeness to a zebra pattern. The image with the best score has the correct rotation. The image you linked to was off by 0.5 degrees. I omitted some functions for readability, the full code can be found here.
# Rotate the image around in a circle
angle = 0
while angle <= 360:
# Rotate the source image
img = rotate(src, angle)
# Crop the center 1/3rd of the image (roi is filled with text)
h,w = img.shape
buffer = min(h, w) - int(min(h,w)/1.15)
roi = img[int(h/2-buffer):int(h/2+buffer), int(w/2-buffer):int(w/2+buffer)]
# Create background to draw transform on
bg = np.zeros((buffer*2, buffer*2), np.uint8)
# Compute the sums of the rows
row_sums = sum_rows(roi)
# High score --> Zebra stripes
score = np.count_nonzero(row_sums)
scores.append(score)
# Image has best rotation
if score <= min(scores):
# Save the rotatied image
print('found optimal rotation')
best_rotation = img.copy()
k = display_data(roi, row_sums, buffer)
if k == 27: break
# Increment angle and try again
angle += .75
cv2.destroyAllWindows()
How to tell if the document is upside down? Fill in the area from the top of the document to the first non-black pixel in the image. Measure the area in yellow. The image that has the smallest area will be the one that is right-side-up:
# Find the area from the top of page to top of image
_, bg = area_to_top_of_text(best_rotation.copy())
right_side_up = sum(sum(bg))
# Flip image and try again
best_rotation_flipped = rotate(best_rotation, 180)
_, bg = area_to_top_of_text(best_rotation_flipped.copy())
upside_down = sum(sum(bg))
# Check which area is larger
if right_side_up < upside_down: aligned_image = best_rotation
else: aligned_image = best_rotation_flipped
# Save aligned image
cv2.imwrite('/home/stephen/Desktop/best_rotation.png', 255-aligned_image)
cv2.destroyAllWindows()
Assuming you did run the angle-correction already on the image, you can try the following to find out if it is flipped:
Project the corrected image to the y-axis, so that you get a 'peak' for each line. Important: There are actually almost always two sub-peaks!
Smooth this projection by convolving with a gaussian in order to get rid of fine structure, noise, etc.
For each peak, check if the stronger sub-peak is on top or at the bottom.
Calculate the fraction of peaks that have sub-peaks on the bottom side. This is your scalar value that gives you the confidence that the image is oriented correctly.
The peak finding in step 3 is done by finding sections with above average values. The sub-peaks are then found via argmax.
Here's a figure to illustrate the approach; A few lines of you example image
Blue: Original projection
Orange: smoothed projection
Horizontal line: average of the smoothed projection for the whole image.
here's some code that does this:
import cv2
import numpy as np
# load image, convert to grayscale, threshold it at 127 and invert.
page = cv2.imread('Page.jpg')
page = cv2.cvtColor(page, cv2.COLOR_BGR2GRAY)
page = cv2.threshold(page, 127, 255, cv2.THRESH_BINARY_INV)[1]
# project the page to the side and smooth it with a gaussian
projection = np.sum(page, 1)
gaussian_filter = np.exp(-(np.arange(-3, 3, 0.1)**2))
gaussian_filter /= np.sum(gaussian_filter)
smooth = np.convolve(projection, gaussian_filter)
# find the pixel values where we expect lines to start and end
mask = smooth > np.average(smooth)
edges = np.convolve(mask, [1, -1])
line_starts = np.where(edges == 1)[0]
line_endings = np.where(edges == -1)[0]
# count lines with peaks on the lower side
lower_peaks = 0
for start, end in zip(line_starts, line_endings):
line = smooth[start:end]
if np.argmax(line) < len(line)/2:
lower_peaks += 1
print(lower_peaks / len(line_starts))
this prints 0.125 for the given image, so this is not oriented correctly and must be flipped.
Note that this approach might break badly if there are images or anything not organized in lines in the image (maybe math or pictures). Another problem would be too few lines, resulting in bad statistics.
Also different fonts might result in different distributions. You can try this on a few images and see if the approach works. I don't have enough data.
You can use the Alyn module. To install it:
pip install alyn
Then to use it to deskew images(Taken from the homepage):
from alyn import Deskew
d = Deskew(
input_file='path_to_file',
display_image='preview the image on screen',
output_file='path_for_deskewed image',
r_angle='offest_angle_in_degrees_to_control_orientation')`
d.run()
Note that Alyn is only for deskewing text.
I'm trying to use OpenCV to segment a bent rod from it's background then find the bends in it and calculate the angle between each bend.
The first part luckily is trivial with a enough contrast between the foreground and background.
A bit of erosion/dilation takes care of reflections/highlights when segmenting.
The second part is where I'm not sure how to approach it.
I can easily retrieve a contour (top and bottom are very similar so either would do),
but I can't seem to figure out is how to get split the contour into the straight parts and the bend rods to calculate the angles.
So far I've tried simplyfying the contours, but either I get too many or too few points and it feels difficult to pin point the right
settings to keep the straight parts straight and the bent parts simplified.
Here is my input image(bend.png)
And here's what I've tried so far:
#!/usr/bin/env python
import numpy as np
import cv2
threshold = 229
# erosion/dilation kernel
kernel = np.ones((5,5),np.uint8)
# contour simplification
epsilon = 0
# slider callbacks
def onThreshold(x):
global threshold
print "threshold = ",x
threshold = x
def onEpsilon(x):
global epsilon
epsilon = x * 0.01
print "epsilon = ",epsilon
# make a window to add sliders/preview to
cv2.namedWindow('processed')
#make some sliders
cv2.createTrackbar('threshold','processed',60,255,onThreshold)
cv2.createTrackbar('epsilon','processed',1,1000,onEpsilon)
# load image
img = cv2.imread('bend.png',0)
# continuously process for quick feedback
while 1:
# exit on ESC key
k = cv2.waitKey(1) & 0xFF
if k == 27:
break
# Threshold
ret,processed = cv2.threshold(img,threshold,255,0)
# Invert
processed = (255-processed)
# Dilate
processed = cv2.dilate(processed,kernel)
processed = cv2.erode(processed,kernel)
# Canny
processed = cv2.Canny(processed,100,200)
contours, hierarchy = cv2.findContours(processed,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
if len(contours) > 0:
approx = cv2.approxPolyDP(contours[0],epsilon,True)
# print len(approx)
cv2.drawContours(processed, [approx], -1, (255,255,255), 3)
demo = img.copy()
cv2.drawContours(demo, [approx], -1, (192,0,0), 3)
# show result
cv2.imshow('processed ',processed)
cv2.imshow('demo ',demo)
# exit
cv2.destroyAllWindows()
Here's what I've got so far, but I'm not convinced this is the best approach:
I've tried to figure this out visually and what I've aimed for is something along these lines:
Because the end goal is to calculate the angle between bent parts something like this feels simpler:
My assumption that fitting lines and compute the angles between pairs of intersecting lines could work:
I did a quick test using the HoughLines OpenCV Python tutorial, but regardless of the parameters passed I didn't get great results:
#!/usr/bin/env python
import numpy as np
import cv2
threshold = 229
minLineLength = 30
maxLineGap = 10
houghThresh = 15
# erosion/dilation kernel
kernel = np.ones((5,5),np.uint8)
# slider callbacks
def onMinLineLength(x):
global minLineLength
minLineLength = x
print "minLineLength = ",x
def onMaxLineGap(x):
global maxLineGap
maxLineGap = x
print "maxLineGap = ",x
def onHoughThresh(x):
global houghThresh
houghThresh = x
print "houghThresh = ",x
# make a window to add sliders/preview to
cv2.namedWindow('processed')
#make some sliders
cv2.createTrackbar('minLineLength','processed',1,50,onMinLineLength)
cv2.createTrackbar('maxLineGap','processed',5,30,onMaxLineGap)
cv2.createTrackbar('houghThresh','processed',15,50,onHoughThresh)
# load image
img = cv2.imread('bend.png',0)
# continuously process for quick feedback
while 1:
# exit on ESC key
k = cv2.waitKey(1) & 0xFF
if k == 27:
break
# Threshold
ret,processed = cv2.threshold(img,threshold,255,0)
# Invert
processed = (255-processed)
# Dilate
processed = cv2.dilate(processed,kernel)
processed = cv2.erode(processed,kernel)
# Canny
processed = cv2.Canny(processed,100,200)
lineBottom = np.zeros(img.shape,np.uint8)
contours, hierarchy = cv2.findContours(processed,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
if len(contours) > 0:
cv2.drawContours(lineBottom, contours, 0, (255,255,255), 1)
# HoughLinesP
houghResult = img.copy()
lines = cv2.HoughLinesP(lineBottom,1,np.pi/180,houghThresh,minLineLength,maxLineGap)
try:
for x in range(0, len(lines)):
for x1,y1,x2,y2 in lines[x]:
cv2.line(houghResult,(x1,y1),(x2,y2),(0,255,0),2)
except Exception as e:
print e
# show result
cv2.imshow('lineBottom',lineBottom)
cv2.imshow('houghResult ',houghResult)
# exit
cv2.destroyAllWindows()
Is this a feasible approach ? If so, what's the correct way of doing line fitting in OpenCV Python ?
Otherwise, that's the best way to tackle this problem ?
Update Following Miki's advise I've tried OpenCV 3's LSD and got nicer results than with HoughLinesP but it looks like there's still some tweaking needed, although it doesn't look other than cv2.createLineSegmentDetector there aren't many options to play with:
It may be convenient to use curvature to find line segments. Here example of splitting contour by minimal curvature points, it may be better to use maximal curvature points in your case. B You can split your curve to parts, then each part approximate with line segment using RANSAC method.
I know this is old but I found this after having a similar problem
The method I used (after finding binary image) was along the lines of:
Find ends (points with fewest neighbors)
skeletonize (optional)
Starting at one end find a few closest points using skimage cdist
Perform a linear regression with these points and find all points in the image within a few pixels error of the line of best fit. I used query_ball_point
This gives additional points within the same straight line. Order them by distance from the last fiducial point. Some of these might be projections of the line onto distant parts of the object and should be deleted.
Repeat steps 4 and 5 until no more points are added.
Once no more points are added to the line you find the start of the next valid line by looking at R-squared for the fit. The line should have very high R squared eg. > 0.95 (depending on the image - I was getting > 0.99). Keep changing starting point until high R squared is achieved.
This gives a bunch of line segments from where it should be easy to find the angles between them. One potential problem occurs when the segment is vertical (or horizontal) and the slope becomes infinite. When this occurred I just flipped the axes around. You can also get around this by defining end points of a line and finding all points within a threshold distance from that line rather than doing the regression.
This involves a lot more coding than using the other methods suggested but execution time is fast and it gives much greater control over what is happening.
Once you have the contour, you can analyze it using a method like the one proposed in this paper: https://link.springer.com/article/10.1007/s10032-011-0175-3
Basically, the contour is tracked calculating the curvature at each point.
Then you can use a curvature threshold to segment the contour into straight and curved sections.
I am looking to detect lines connecting corners over a hand drawn image like this. I am using Harris Corner Detection to find the corners of the image. Next I am connecting all of the corners with lines and iterating though the points to see if they match the pixels from original image and setting a threshold for each line pixel cover to say what is acceptable to say it is a correct line connecting corners.Image of connected lines. It works... but it is very slow. Is there a better way to do this or different method I should use? (Hough lines will not work because of the possibility of curved lines, I only want the lines connecting corners.
for i in c_corners: #corners thru harris and coorected with subpix
x1,y1 = i.ravel()
for k in c_corners:
x2,y2 = k.ravel()
if x1 != x2 and y1 != y2: #ignore vertical lines
linePoints = line_points(x1,y1, x2,y2) # function to get line pnts
totalLinePoints = len(linePoints)
coverPoints = 0
########## This is where I think the slow down is happening and could be optimized
for m in originalImage: #image is dialated to help detection
for n in linePoints:
match = np.all(m == n)
if match == True:
coverPoints += 1
print("Line Cover = ", (coverPoints/totalLinePoints))
if (coverPoints/totalLinePoints) > .65:
good_lines.append([x1,y1,x2,y2])
Any help at all is appreciated, thank you!
My original approach was to create a blank image and draw each line on it, and then use cv2.bitwise_and() with the binary (dilated) image to count how many pixels were in agreement, and if they met a threshold, then draw those lines over the original image. However setting a threshold for the number of pixels penalizes small lines. A better indicator would be the ratio of the number of correct matches to incorrect matches (I realize now that's what you were actually doing). Furthermore this is a little more robust towards dilation and the line thickness you choose to draw your lines.
However the general method you're using is not very robust to issues in the drawing where, like this one, synthetic lines may be able to fit easily to lines they don't belong to, because many drawn curves may hit a line segment. You can see this issue in the output of my code:
I simply hardcoded some corner estimates and went from there. Note the use of itertools to help create all possible pairs of points to define line segments.
import cv2
import numpy as np
import itertools
img = cv2.imread('drawing.png')
bin_inv = cv2.bitwise_not(img) # flip image colors
bin_inv = cv2.cvtColor(bin_inv, cv2.COLOR_BGR2GRAY) # make one channel
bin_inv = cv2.dilate(bin_inv, np.ones((5,5)))
corners = ((517, 170),
(438, 316),
(574, 315),
(444, 436),
(586, 436))
lines = itertools.combinations(corners,2) # create all possible lines
line_img = np.ones_like(img)*255 # white image to draw line markings on
for line in lines: # loop through each line
bin_line = np.zeros_like(bin_inv) # create a matrix to draw the line in
start, end = line # grab endpoints
cv2.line(bin_line, start, end, color=255, thickness=5) # draw line
conj = (bin_inv/255 + bin_line/255) # create agreement image
n_agree = np.sum(conj==2)
n_wrong = np.sum(conj==1)
if n_agree/n_wrong > .05: # high agreements vs disagreements
cv2.line(line_img, start, end, color=[0,200,0], thickness=5) # draw onto original img
# combine the identified lines with the image
marked_img = cv2.addWeighted(img, .5, line_img, .5, 1)
cv2.imwrite('marked.png', marked_img)
I tried a lot of different settings (playing with thickness, dilation, different ratios, etc) and couldn't get that spurious longer line from appearing. It fits the original black pixels super well though, so I'm not sure how you would be able to get rid of it if you used this method. It's got the curve from the top-right line going for it, as well as the middle line it crosses, and the curve at the bottom right which trends that direction for a bit. Regardless, this only takes two seconds to run, so at least it's faster than your current code.