1. The problem
Given the images of a house roof, I am trying to find the contours of the roofs. I have labelled data available (as polygon vertices) which I interpolate and create the truth image which is shown below
I use canny, hough-lines, LBP features to train an ML model the results look decent. the model output is shown in the middle, and overlay on test image is shown on right.
2. What I need.
The final output should really be a set of polygons and I need to find the points on which these polygons should be drawn (see the highlighted points in image below). So the output can be set of n line segments. where each line segment is 2 points [(x1,y1),(x2,y2)]
3. What are my thoughts/ideas;
a. Erosion,Dilation,Opening,closing,skeletonize operations
While these operations make the lines in the above image much neater, they don’t help me find the polygon vertices I am looking for.
I'd like to fit (a number of) lines to the white pixels in the image (something like hough lines).
The intersections of these lines would give me the vertices for the polygons I am looking for.
I am wondering if there is a more standard/better way of accomplishing the above.
I think HoughLinesP will help you in your goal. It will find line segments and output them in a vector [x1,y1,x2,y2] where (x,y) pairs represent the start and endpoints of line segments.
Each vertex should be near the end of 2 or more line segments. You go through each of the endpoints and count how many times they appear. When you've processed all the points you can eliminate any that have less than 2 occurances. Of course you will need some small threshold for determining a point is unique because the gaps in the lines some psuedocode: dist(point1, point2) < some_delta_threshold
I'm not sure how you would find the polygons at this point, but hopefully this offers some assistance
Related
I made a model that predicts electrical symbols and junctions:
image of model inference.
Given the xywh coordinates of each junctions' bounding box in a form of a dataframe: image of the dataframe, how would I make an output that stores the location of all the wires in a .txt file in a form of: (xstart,ystart), (xend,yend).
I'm stuck at writing a way to check if there is a valid line (wire) between any two given junctions.
data = df.loc[df['name'] == 'junction']
# iterates through all of the junctions
for index, row in data.iterrows():
for index2, row2 in data.iterrows():
check_if_wire_is_valid()
My attempt was to erase all electrical symbols (make everything in bounding boxes white except for junctions) from the inference image and run cv.HoughLinesP to find wires. How can I write a function that checks if the cv.HoughLinesP output lies between two junctions?
Note that the minimum distance that lies between two junctions should be greater than 1px because if I have a parallel circuit like such: top left and bottom right junction would "detect" more than 1px of line between them and misinterpret that as a valid line.
EDIT: minAreaRect on contours . I've drawn this circuit with no elements for simplification and testing. This is the resulting minAreaRect found for the given contours. I can't seem to find a way to properly validate lines from this.
My initial solution was to compare any two junctions and if they are relatively close on the x-axis, then I would say that those two junctions form a vertical wire, and if other two junctions were close on the y-axis I would conclude they form a horizontal wire. junction distance to axis.
Now, this would create a problem if I had a diagonal line. I'm trying find a solution that is consistent and applicable to every circuit. I believe I'm onto something with HoughLinesP method or contours but that's as far as my knowledge can assist me.
The main goal is to create an LTSpice readable circuit for simulating purposes. Should I change my method of finding valid lines? If so, what is your take on the problem?
This should be doable using findContours(). A wire is always a (roughly) straigt line, right ?
Paint the classified boxes white, as you said
threshold() to get a binary image with the wires (and other symbols and letters) in white, everything else black.
run findContours() on that to extract objects.
Get the bounding boxes (minAreaRect) for all contours
discard all contours with a too wide side ratio, those are letter or symbols, keep only those slim enough to be a wire
Now you got all wires as objects, similiar to the junction list. Now, for how to merge those two... Some options that come to mind:
Grow the boxes by a certain amount, and check if they overlap.
Interpolate a line from the wire boxes and check if they cross any intersection box close by.
Or the other way around: draw a line between intersections and check how much of it goes through a certain wire box.
This is a pure math problem, and i don't know what you performance requirements are. So i'll leave it at that.
I want to separate a noiseless 1-bit (black and white) image with white circles based on the concave part of the outline.
Please refer to the picture below.
This is the white object to separate:
The target result is:
Here is my implementation with the watershed algorithm:
The above result is not what I want.
If the size of the separated objects is similar, my algorithm is fine, but if the size difference is large, a problem occurs as shown in the picture above.
I would like to implement an opencv algorithm that can segment a region like the second picture.
However, the input photo is not necessarily a perfect circle.
It can be oval like the picture below:
Or it can be squished:
However, I would like to separate it based on the concave part of the outline anyway.
I think it can be implemented by using the distanceTransform function well, but I'm not sure how to approach it.
Please let me know which way to refer.
Thank you.
Here is an algorithm which should give you a good start.
Compute all contours.
For each contour compute the convexity defects. If there is no defect the contour is an isolated circle and you can segment it out.
After you handled all the isolated circles, you can work out the remaining contours by counting the convexity defects: the number of circles N for each contour is the number of convexity defects divided by 2.
Use a clustering algorithm (https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html should do well given the shapes you have) and cluster the "white" points using N as the number of clusters to be found.
If you want to find the minimal openings, you can use a medial axis based approach.
Pseudo code:
compute contours of bitmap
compute medial-axis of bitmap
for each point on medial-axis:
get minimal distance d from medial axis algorithm
for each local minimum of distance d:
get two points on bitmap contours with minimal distance that are at least d apart from each other
use these points for deviding line
If you need a working implementation in python, please let me know. I would use skimage lib. For other languages you might have to implement medial-axis on your own. But that shouldn't be a big deal.
I'd like to find the corners of the following box
However, as you can see I'm detecting a lot of corners I don't want to find. I'm completly stuck on this one. No matter what I try, I always seem to find corners in the dots on the box. I used the function goodFeaturesToTrack() but I also tried cornerHarris()
The most important thing to me is to find the coordinates of the corner pixels so I can draw a wire frame.
Kind regards, Schweini
Edit:
To draw the wire frame onto the image, following process can be thinkable.
When extracting outline of the black box region, the outline consists with 6 straight line segments.
Therefore, you'll able to find at least 6 corners of the box as the intersection of each two adjacent line segments.
Additionally, it looks like that, outline of 6 surfaces of the box will be able to coarsely estimated from each adjacent line segment pairs. (with assuming parallelogram)
This means estimating the positions of the remaining two corners (to draw the wire frame).
Furthermore, if you want, comparing the estimation result with your examination (corner detecition) result will be able to refine the coordinates of one corner.
I'm trying to detect the key intersection points of lines on a sports court to allow me to use homography to map a player's position in the camera frame to the corresponding position on a 2D model of the court. The camera position and environment details such as lighting will vary.
My current approach has been to use Canny edge detection and Hough Transform to detect the lines then to use kmeans to group them into horizontal/vertical groups. Once I obtain the key points (court floor corners and intersection points of service lines) I know I can use findHomography to obtain the perspective transform for the court. My problems are:
Even if I get the intersection points for all the lines, given the camera position (and therefore frame rotation) is not fixed how can I know which lines/intersection points are which? Is there a technique to take the intersections and try and fit them to a model court? I have the exact dimensions of the court floor which is the area I care about.
There's lots of "noise" created when using HoughLines and HoughLinesP - see image below. What would be the best way to merge these lines? Or if fitting to a model is that not a problem?
Is this the best modern approach or would attempting something like training a CV model to do segmentation be a better approach?
Sample output from Input/HoughLines/HoughLinesP (L to R):
I have an Image (or several hundreds of them) that need to be analyzed. The goal is to find all black spots close to each other.
For example all black spots with a Horizontal distance of 160 pixel and vertical 40 pixel.
For now I just look at each Pixel and if there is a black pixel I call a recursive Method to find its neighbours (i can post the code too if you want to)
It works, but its very slow. At the moment the script runs about 3-4 minutes depending on image size.
Is there some easy/fast way to accomplish this (best would be a scikit-image method to help out here) I'm using Python.
edit: I tried to use scikit.measure.find_contours, now i have an array with arrays containing the contours of the black spots. Now I only need to find the contours in the neighbourhood of these contours.
When you get the coordinates of the different black spots, rather than computing all distances between all pairs of black pixels, you can use a cKDTree (in scipy.spatial, http://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.cKDTree.html#scipy.spatial.cKDTree). The exact method of cKDTree to use depends on your exact criterion (you can for example use cKDTree.query_ball_tree to know whether there exists a pair of points belonging to two different labels, with a maximal distance that you give).
KDTrees are a great method to reduce the complexity of problems based on neighboring points. If you want to use KDTrees, you'll need to rescale the coordinates so that you can use one of the classical norms to compute the distance between points.
Disclaimer: I'm not proficient with the scikit image library at all, but I've tackled similar problems using MATLAB so I've searched for the equivalent methods in scikit, and I hope my findings below help you.
First you can use skimage.measure.label which returns label_image, i.e. an image where all connected regions are labelled with the same number. I believe you should call this function with background=255 because from your description it seems that the background in your images is the while region (hence the value 255).
This is essentially an image where the background pixels are assigned the value 0 and the pixels that make up each (connected) spot are assigned the value of an integer label, so all the pixels of one spot will be labelled with the value 1, the pixels of another spot will be labelled with the value 2, and so on. Below I'll refer to "spots" and "labelled regions" interchangeably.
You can then call skimage.measure.regionprops, that takes as input the label_image obtained in the previous step. This function returns a list of RegionProperties (one for each labelled region), which is a summary of properties of a labelled region.
Depending on your definition of
The goal is to find all black spots close to each other.
there are different fields of the RegionProperties that you can use to help solve your problem:
bbox gives you the set of coordinates of the bounding box that contains that labelled region,
centroid gives you the coordinates of the centroid pixel of that labelled region,
local_centroid gives you the centroid relative to the bounding box bbox
(Note there are also area and bbox_area properties which you can use to decide whether to throw away very small spots that you might not be interested in, thus reducing computation time when it comes to comparing proximity of each pair of spots)
If you're looking for a coarse comparison, then comparing the centroid or local_centroid between each pair of labelled regions might be enough.
Otherwise you can use the bbox coordinates to measure the exact distance between the outer bounds of any two regions.
If you want to base the decision on the precise distance between the pixel(s) of each pair of regions that are closest to each other, then you'll likely have to use the coords property.
If your input image is binary, you could separate your regions of interest as follows:
"grow" all the regions by the expected distance (actually half of it, as you grow from "both sides of the gap") with binary_dilation, where the structure is a kernel (e.g. rectangular: http://scikit-image.org/docs/dev/api/skimage.morphology.html#skimage.morphology.rectangle) of, let's say, 20x80pixels;
use the resulting mask as an input to skimage.measure.label to assign different values for different regions' pixels;
multiply your input image by the mask created above to zero dilated pixels.
Here are the results of proposed method on your image and kernel = rectange(5,5):
Dilated binary image (output of step 1):
Labeled version of the above (output of step 2):
Multiplication results (output of step 3):