So I'm working on this piece of code to extract data from some graphs in images. These images are all scanned from a book. Since we're talking about 100+ images here, I would like to automate the process of course. My first step was to make sure that all images are aligned. Because the pages of the book were scanned by hand, the scans are all slightly shifted or rotated in regards to each other. Luckily there are some dotted lines on the images, which can be used as a reference point to align them. Afterwards I can then divide the image into smaller subimages, by slicing the image on these dotted lines. In that way, all subimages will be equal for all scanned images.
So, first step of course is to detect these dotted lines. My strategy can be described in 4 steps:
turn the dotted lines into solid lines, using Morphological Transformation
detect all edges, using Canny Edge Detection
identify the lines, using HoughLines
draw these lines on a mask for further usage
Now there are several problems which may occur. Sometimes HoughLines will detect a wrong line (such as the fold of the next page in the book), but this could potentially be fixed by cropping the image a little on the right side (better solutions are always welcome). The second (and biggest) problem is that HoughLines sometimes tends to identify a single line as multiple lines. I think this has something to do with Canny Edge Detection being too rough or vague about the edges, so that HoughLines actually sees multiple lines. Is there a way I could "smooth" the output from Canny so that HoughLines detects each line exactly once?
In the case of this specific image, the vertical dotted lines in the middle didn't get identified, whereas the fold of the next page in the book did. Furthermore the vertical dotted lines got identified as multiple lines. (left source image, middle edges detected, right lines detected)
# load image
img_large = cv2.imread("image.png")
# resize for ease of use
img_ori = cv2.resize(img_large, None, fx=0.2, fy=0.2, interpolation=cv2.INTER_CUBIC)
# create grayscale
img = cv2.cvtColor(img_ori, cv2.COLOR_BGR2GRAY)
# create mask for image size
mask = np.zeros((img.shape[:2]), dtype=np.uint8)
# do a morphologic close to merge dotted line
kernel = np.ones((8, 8))
res = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel)
# detect edges for houghlines
edges = cv2.Canny(res, 50, 50)
# detect lines
lines = cv2.HoughLines(edges, 1, np.pi/180, 200)
# draw detected lines
for line in lines:
rho, theta = line[0]
a = np.cos(theta)
b = np.sin(theta)
x0 = a*rho
y0 = b*rho
x1 = int(x0 + 1000*(-b))
y1 = int(y0 + 1000*a)
x2 = int(x0 - 1000*(-b))
y2 = int(y0 - 1000*a)
cv2.line(mask, (x1, y1), (x2, y2), 255, 2)
cv2.line(img, (x1, y1), (x2, y2), 127, 2)
In your script, the pixel-bins and the rotation bins are too fine for the threshold you've set:
lines = cv2.HoughLines(edges, 1, np.pi/180, 200)
So you can tune the threshold parameter (200) to get only one line, or tune the rho (1) and theta (np.pi/180) parameters, or tune all these. You can select a set of image that contain only one horizontal or vertical line from your images. Then do grid search to find the parameters that detect only one line in your set of test images.
Related
I am trying to learn techniques on image feature detection.
I have managed to detect horizontal line(unbroken/continuous), however I am having trouble detecting all the dotted/broken lines in an image.
Here is my test image, as you can see there are dotted lines and some text/boxes etc.
So far I have used the following code which detected only one dotted line.
import cv2
import numpy as np
img=cv2.imread('test.jpg')
img=functions.image_resize(img,1000,1000) #function from a script to resize image to fit my screen
imgGray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
imgEdges=cv2.Canny(imgGray,100,250)
imgLines= cv2.HoughLinesP(imgEdges,2,np.pi/100,60, minLineLength = 10, maxLineGap = 100)
for x1,y1,x2,y2 in imgLines[0]:
cv2.line(img,(x1,y1),(x2,y2),(0,255,0),2)
cv2.imshow('Final Image with dotted Lines detected',img)
My output image is below. As you can see I only managed to detect the last dotted line. I have played around with the parameters rho,theta,min/max line but no luck.
Any advice is greatly appreciated :)
This solution:
import cv2
import numpy as np
img=cv2.imread('test.jpg')
kernel1 = np.ones((3,5),np.uint8)
kernel2 = np.ones((9,9),np.uint8)
imgGray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
imgBW=cv2.threshold(imgGray, 230, 255, cv2.THRESH_BINARY_INV)[1]
img1=cv2.erode(imgBW, kernel1, iterations=1)
img2=cv2.dilate(img1, kernel2, iterations=3)
img3 = cv2.bitwise_and(imgBW,img2)
img3= cv2.bitwise_not(img3)
img4 = cv2.bitwise_and(imgBW,imgBW,mask=img3)
imgLines= cv2.HoughLinesP(img4,15,np.pi/180,10, minLineLength = 440, maxLineGap = 15)
for i in range(len(imgLines)):
for x1,y1,x2,y2 in imgLines[i]:
cv2.line(img,(x1,y1),(x2,y2),(0,255,0),2)
cv2.imshow('Final Image with dotted Lines detected', img)
If you have an idea about the dot size, you can use black-hat transform to filter out the dotted lines. Black-hat is the difference between the closing of the image and the image. Then you can try hough line transform.
So, try
Convert bgr-to-gray
Apply black-hat using morphologyEx: this will leave only the black dots in the resulting image.
Invert the result and try hough line transform.
Here, you will have to experiment with the kernel size to filter only the dots. If that proves to be not very robust, another approach would be to use a blob detector. Invert the image and apply opencv blob detector or find contours. Filter the blobs/contours by area. Letters and other structures will have a larger area than the dots, so you can remove any structures that are larger than the dots. Then apply the hough line transform.
because you choose just one line to draw.
You change function draw line to
for i, line in enumerate(imgLines):
for x1, y1, x2, y2 in line:
cv2.line(img, (x1,y1), (x2,y2), (0,255,0), 2)
print(i, x1, y1, x2, y2)
I'm trying to measure water level in a glass channel using OpenCV and Python. I've decided to use HaughLines in a selected ROI and find the midpoints of the said lines so I can calculate the difference between the ones that I want and multiply it with a set reference size that I'll get later on. So far the part where I find the lines look like this:
import cv2
import numpy as np
def midpoint(ptA, ptB, ptC, ptD):
return ((ptA + ptC) * 0.5, (ptB + ptD) * 0.5)
img = cv2.imread("b2924.JPG")
img = cv2.resize(img, None, fx=3/10, fy=3/10)
r = cv2.selectROI("main", img, False, False)
cropped = img[r[1]:(r[1]+r[3]), r[0]:(r[0]+r[2])]
cv2.destroyWindow("main")
imgray = cv2.cvtColor(cropped, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(imgray, 35, 75)
lines = cv2.HoughLinesP(edges, 1, np.pi/180, 75, maxLineGap=1000)
midPoint = []
for line in lines:
x1, y1, x2, y2 = line[0]
cv2.line(cropped, (x1, y1), (x2, y2), (0, 0, 255), 1)
mP = midpoint(x1, y1, x2, y2)
midPoint.append(mP)
midPoint.sort(key = lambda x: x[1])
img[r[1]:(r[1]+r[3]), r[0]:(r[0]+r[2])] = cropped
print(lines)
print(midPoint)
cv2.imshow("img", img)
cv2.waitKey()
cv2.destroyAllWindows()
Depending on the image and the ROI I select I find inconsistent results. Image examples and where I select the ROIs:
Note that base of the channel starts where the duct tape reaches. It looks like I can almost never find that exact line because how noisy it is at the base. Right now these threshold values with no morphology seem to give the better results. I tried to use sobel derivative aswell instead of canny but got worse results.
Is it even possible to get exact measurements in this enviroment? Is it a matter of coding or changing the way I take the pictures or both? In the future I will possibly need to map the water profile during heavy turbulance, should I simply move away from OpenCV for that, since the noise is too much? Any help is appreciated.
I would not invest in any image processing with that setup.
If you insist on image processing (if you are only interested in the level at a few positions you might be better off using conventional level sensors)
Add LED panels or any other kind of homogeneous background illumination to the back of the basin. Add dye to the water to get some contrast.
Get rid of the window reflections. Clean the glass.
Alternatively make the background dark and add something to the water that makes it stray light or fluorescent.
You could also add stuff that floats on the surface and is either retroreflective or self-illuminated. That way you would get a bright surface level indicator that is easily detected in an image.
I have an image like so:
I would like to automatically identify the dense white box area in the top left and then fill it and black out the rest of image. Producing something like this:
Essentially, I just want to return the co-ordinates of the densest cluster. I have tried ad-hoc methods such as erosion, dilation and binary closing but they do not quite suite my needs. I'm not sure if I could use k-means here? Looking for an efficient method, any help is appreciated.
You could erode the image a little bit more, to remove more of the noise, and then find the contours and filter them by area. Here is what I would use (not tested):
kernel = np.ones((2, 2), np.uint8)
img = cv2.erode(img, kernel, iterations = 2)
#Finding contours of white square:
_, conts, hierarchy = cv2.findContours(img, cv2.RETR_EXTERNAL , cv2.CHAIN_APPROX_SIMPLE)
for cnt in conts:
area = cv2.contourArea(cnt)
#filter more noise
if area > 200: # optimize this number
x1, y1, w, h = cv2.boundingRect(cnt)
x2 = x1 + w # (x1, y1) = top-left vertex
y2 = y1 + h # (x2, y2) = bottom-right vertex
rect = cv2.rectangle(img, (x1, y1), (x2, y2), (255,0,0), 2)
One right approach here would be to apply a large square averaging filter. If you know approximately the size of the box you're looking for, then match that size with the filter. After applying this filter, the largest pixel value in the image will be at the middle of the densest region. Let's call this point p.
Next, apply segmentation and connected component labeling to your original image. From your example image, it seems that the box you're looking for is connected. You might want to apply some morphological operations to make sure it's connected. You can also paint a reasonably-sizes blob centered at point p, it'll connect lots of small regions that together form a dense area.
Next, remove all connected components except the one containing point p. You can do this by finding the label at pixel p, and comparing all pixels in the labeled image for equality with that label.
This should leave you a connected, compact region. You can find the bounding box of this region, and paint it on your image, if you really want to enforce that the found area be a box.
I am trying to develop a program that can detect lanes on the road. I have experimented with both Hough Line Transform and Probabilistic Hough Line Transform. However none of these are getting the results that I want.
Original Image:
Hough Line Transform
Probabilistic Hough Line Transform
It seems that for Hough Line Transform, I can at least detect the entire lane, but unfortunately, the line just goes on infinitely (until they move off the picture), to the point where the lines intersect with each other, which is not a good graphical lane detection marker.
I also tried Probalistic Hough Line Transform, and the green line used for lane detection does not go off to infinitely like the other one, but it fails to mark and detect the entire lane.
I am trying to replicate results here (by writing it in Python)
http://www.transistor.io/revisiting-lane-detection-using-opencv.html
What can I do to fix this problem?
Code:
import numpy as np
import cv2
from matplotlib import pyplot as plt
from PIL import Image
import imutils
def invert_img(img):
img = (255-img)
return img
def canny(imgray):
imgray = cv2.GaussianBlur(imgray, (5,5), 200)
canny_low = 5
canny_high = 150
thresh = cv2.Canny(imgray,canny_low,canny_high)
return thresh
def filtering(imgray):
thresh = canny(imgray)
minLineLength = 1
maxLineGap = 1
lines = cv2.HoughLines(thresh,1,np.pi/180,0)
#lines = cv2.HoughLinesP(thresh,2,np.pi/180,100,minLineLength,maxLineGap)
print lines.shape
# Code for HoughLinesP
'''
for i in range(0,lines.shape[0]):
for x1,y1,x2,y2 in lines[i]:
cv2.line(img,(x1,y1),(x2,y2),(0,255,0),2)
'''
# Code for HoughLines
for i in range(0,5):
for rho,theta in lines[i]:
a = np.cos(theta)
b = np.sin(theta)
x0 = a*rho
y0 = b*rho
x1 = int(x0 + 1000*(-b))
y1 = int(y0 + 1000*(a))
x2 = int(x0 - 1000*(-b))
y2 = int(y0 - 1000*(a))
cv2.line(img,(x1,y1),(x2,y2),(0,0,255),2)
return thresh
img = cv2.imread('images/road_0.bmp')
imgray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img = imutils.resize(img, height = 500)
imgray = imutils.resize(imgray, height = 500)
thresh = filtering(imgray)
cv2.imshow('original', img)
cv2.imshow('result', thresh)
cv2.waitKey(0)
Cool topic! First of all, why did you add the Gaussian blur? Your source article doesn't mention that at all. If I remove that, I instantly get extra crazy lines, which I can tone down with the canny_low and canny_high. About the best I could find was low=100 and high=180.
Second, you did quite a good job translating the article to Python. However, I think you left out a crucial detail. The author writes:
// Canny algorithm
Mat contours;
Canny(image,contours,50,350);
Mat contoursInv;
threshold(contours,contoursInv,128,255,THRESH_BINARY_INV);
You implement the Canny function (cv2.canny()), but you don't call the threshold function. According to documentation I found, this function "applies a fixed-level threshold to each array element." I experimented with your code and came up with the following.
#thresh = canny(imgray) # original
edges = canny(imgray) # docs refer to return value as "edges"
retval, dst = cv2.threshold(edges, 128, 255,cv2.THRESH_BINARY_INV)
Two values are returned - retval isn't particularly important for us right now. dst is the destination 2D array of image data after thresholding. You would then update your call to cv2.HoughLines and cv2.HoughLinesP replacing "thresh" with "dst." When I did this I got a lot more interesting behavior, though I was not able to find the correct tuning values to make the lines work well.
So, hopefully that gives you some pointers. Try my tips, and also read the article once or twice more to double check that you have the same program flow as the author. This seems like a fun project, have fun!
See, I have been trying to detect books in a bookshelf:
I used Contours for bounding boxes. However, I just want to capture the actual book objects. If I lessen the threshold from Canny, it won't detect the book edges themselves but it detects the book titles or some images from the spine.
I used houghlines and it worked well for detecting the book edge. How can I apply bounding boxes but with houghlines instead of contours?
code I used for Contour finding:
edges = cv2.Canny(blur,thresh,thresh*2)
drawing = np.zeros(img.shape,np.uint8)
contours,hierarchy = cv2.findContours(edges,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
x,y,w,h = cv2.boundingRect(cnt)
cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
rect = cv2.minAreaRect(cnt)
box = cv2.cv.BoxPoints(rect)
box = np.int0(box)
where:
img = cv2.imread('books3.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(5,5),0)
For the houghlines:
lines = cv2.HoughLines(edges,1,np.pi/180,120)
for rho,theta in lines[0]:
a = np.cos(theta)
b = np.sin(theta)
x0 = a*rho
y0 = b*rho
x1 = int(x0 + 1000*(-b))
y1 = int(y0 + 1000*(a))
x2 = int(x0 - 1000*(-b))
y2 = int(y0 - 1000*(a))
where:
im = cv2.imread('books2.jpg')
gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray,100,300,apertureSize = 3)
Thank you so much in advance.
I am actually working on something similar myself. trying to segment the books from one another in a bookshelf. I wish to ask what is the progress on your side so far?
I have yet to tried contours method. However, what I did try was to pre-process the image, canny the image before using HoughLines. The image belows shows a rough result.
I admit I have to perfectly segment out the books either. As you can see in the image, there are more lines then I actually wanted due to the nature of the book spine. I am looking into preprocessing methods that can help me to rid such problem.
I noticed you mentioned that "If I lessen the threshold from Canny, it won't detect the book edges themselves but it detects the book titles or some images from the spine." Maybe for HoughLine Parameters, you can adjust the theta? for instance to 90 degrees so that the book titles, etc will not be detected.
You can also try HoughLineP which is basically Probabilistic Hough Line Transform. More details about that can be found in:
http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/hough_lines/hough_lines.html
Hope my methods give some ideas.I also hope to hear updates from you in regards to your contour method. Hope we can share tips and work together as we have a common goal (: Hope to hear from you soon.