I'd like to find the corners of the following box
However, as you can see I'm detecting a lot of corners I don't want to find. I'm completly stuck on this one. No matter what I try, I always seem to find corners in the dots on the box. I used the function goodFeaturesToTrack() but I also tried cornerHarris()
The most important thing to me is to find the coordinates of the corner pixels so I can draw a wire frame.
Kind regards, Schweini
Edit:
To draw the wire frame onto the image, following process can be thinkable.
When extracting outline of the black box region, the outline consists with 6 straight line segments.
Therefore, you'll able to find at least 6 corners of the box as the intersection of each two adjacent line segments.
Additionally, it looks like that, outline of 6 surfaces of the box will be able to coarsely estimated from each adjacent line segment pairs. (with assuming parallelogram)
This means estimating the positions of the remaining two corners (to draw the wire frame).
Furthermore, if you want, comparing the estimation result with your examination (corner detecition) result will be able to refine the coordinates of one corner.
Related
I made a model that predicts electrical symbols and junctions:
image of model inference.
Given the xywh coordinates of each junctions' bounding box in a form of a dataframe: image of the dataframe, how would I make an output that stores the location of all the wires in a .txt file in a form of: (xstart,ystart), (xend,yend).
I'm stuck at writing a way to check if there is a valid line (wire) between any two given junctions.
data = df.loc[df['name'] == 'junction']
# iterates through all of the junctions
for index, row in data.iterrows():
for index2, row2 in data.iterrows():
check_if_wire_is_valid()
My attempt was to erase all electrical symbols (make everything in bounding boxes white except for junctions) from the inference image and run cv.HoughLinesP to find wires. How can I write a function that checks if the cv.HoughLinesP output lies between two junctions?
Note that the minimum distance that lies between two junctions should be greater than 1px because if I have a parallel circuit like such: top left and bottom right junction would "detect" more than 1px of line between them and misinterpret that as a valid line.
EDIT: minAreaRect on contours . I've drawn this circuit with no elements for simplification and testing. This is the resulting minAreaRect found for the given contours. I can't seem to find a way to properly validate lines from this.
My initial solution was to compare any two junctions and if they are relatively close on the x-axis, then I would say that those two junctions form a vertical wire, and if other two junctions were close on the y-axis I would conclude they form a horizontal wire. junction distance to axis.
Now, this would create a problem if I had a diagonal line. I'm trying find a solution that is consistent and applicable to every circuit. I believe I'm onto something with HoughLinesP method or contours but that's as far as my knowledge can assist me.
The main goal is to create an LTSpice readable circuit for simulating purposes. Should I change my method of finding valid lines? If so, what is your take on the problem?
This should be doable using findContours(). A wire is always a (roughly) straigt line, right ?
Paint the classified boxes white, as you said
threshold() to get a binary image with the wires (and other symbols and letters) in white, everything else black.
run findContours() on that to extract objects.
Get the bounding boxes (minAreaRect) for all contours
discard all contours with a too wide side ratio, those are letter or symbols, keep only those slim enough to be a wire
Now you got all wires as objects, similiar to the junction list. Now, for how to merge those two... Some options that come to mind:
Grow the boxes by a certain amount, and check if they overlap.
Interpolate a line from the wire boxes and check if they cross any intersection box close by.
Or the other way around: draw a line between intersections and check how much of it goes through a certain wire box.
This is a pure math problem, and i don't know what you performance requirements are. So i'll leave it at that.
If I have an image with a black background on which various colored rectangles are drawn, how can I find them?
The rectangles are empty but with colored edges (each rectangle has different colors), each rectangle intersects at least one other (the sides of different rectangles do not overlap long but cross and the corners of different rectangles do not overlap), moreover they are aligned with the axis (so they cannot be oblique).
I would like to find how many these rectangles are, which is the smallest of all and what is the order of these rectangles (i.e. see which was drawn first up to which which last) identifying them by color. So I would like to find the length of the edges and the vertices (to see which one is smaller). Without import libraries.
So far I have reasoned like this:
for line in range (len (image)):
for column in range (len (line)):
Then I thought about "saving" the colors I encounter in a dictionary, but I don't know how to go on because as the rectangles intersect I'm afraid of working hours on a wrong code.
Can you give me a hand?
This is an example:
https://pasteboard.co/JBHMImK.png
Example image used in program
I am trying to find the coordinates of a polygon in an image
(just like flood fill algorithm we are given a coordinate and we need to search the surrounding pixels for the boundary, if boundary is found we need to append its coordinate to the list if not we need to keep searching other pixels.)and if all the pixels are traversed the program should stop returning the list of pixels.
usually color of boundary is black and image is a gray scale image of maps of building.
It seems that flood-fill will be good enough to completely fill a room, despite the extra annotations. After filling, extract the outer outline. Now you can detect the straight portions of the outline by checking the angle formed by three successive points. I would keep a spacing between them to avoid local inaccuracies.
You will find a sequence of line segments, possibly interrupted at corners. Optionally use line fitting to maximize accuracy, and recompute the corners by intersecting the segments. Also consider joining aligned segments that are interrupted by short excursions.
If the rooms are not well closed, flood filling can leak and you are a little stuck. Consider filling with a larger brush, though this can cause other problems.
I'm using OpenCV to find boxes within a depth grayscale image.
First, I cut the depth image from the top of the box as shown below:
Next, I cut the depth image from the bottom of the box as shown below:
After some filtering, closing and doing some basic operations, I get an image of the top cut merged with where the box bottoms are along with bounding boxes. Please see the image below:
The goal now is to attempt to stretch the inner rectangles until they touch the corners of the top edges. The issue is that the boxes could be rotated which makes filling much harder than just looping horizontally/vertically and filling in gaps.
An approach would be getting a side of the inner rectangle, keep moving it in the direction of stretching until it touches some pixels of value 255. However, this takes a lot of time and isn't efficient.
Another approach is eliminating all pixels that aren't along the direction of stretching the segment, finding the contours of the remaining objects, and attempting the same method.
A third approach and the one that makes most sense to me:
Draw the other 2 segments connected to the side of the rectangle but as lines so to fill the entire image.
Anything outside the 2 lines is ignored
Find get a list of coordinates of the contours of the remaining objects in the direction of stretching/filling
Find the closest pixel to the side; this is where we'll stretch the box to
This is still rather inefficient.
Any help, guidance, or insights would be much appreciated. Thank you!
Basically, I'm working on a robot arm that will play checkers.
There is a camera attached above the board supplying pictures (or even videomaterial but I guess that is just a series of images and since checkers is not really a fast paced game I can just take a picture every few seconds and go from there)
I need to find a way to translate the visual board into a e.g a 2d array to feed into the A.I to compute the robots moves.
I have a line detection working which draws lines on along the edges of the squares (and also returns edges in canny as a prior step). Moreover I detect green and red (the squares of my board are green and red) and return these both as a mask each.
I also have a sphere detection in place to detect the position of the pieces and some black and white color detection returning a mask each with the black or white detected areas.
My question is how I can now combine these things I have and as a result get some type of array out of which I can deduct information over in which squares my pieces are ?
Like how would i build the 2d array (or connect any 8x8) array to the image of the board with the lines and/or the masks of the red/green tiles ? I guess I have to do some type of calibration ?
And secondly is there a way to somehow overlay the masks so that I then know which pieces are in which squares ?
Well, first of all remember that chess always starts with the same pieces on the same positions e.g. black knight starts at 8-B which can be [1][7] in your 2D array. If I were you I would start with a 2D array with the begin positions of all the chess pieces.
As to knowing which pieces are where: you do not need to recognize the pieces themselves. What I would do if I were you is detect the empty spots on the chessboard which is actually quite easy in comparison to really recognizing the different chess pieces.
Once your detection system detects that one of the previously empty spots is now no longer empty you know that a chess piece was moved there. Since you can also detect a new open spot(the spot where the chess piece came from) you also know the exact chess piece which was moved. If you keep track of this list during the whole game you can always know which pieces are moved and which pieces are where.
Edit:
As noted in the comments my answer was based on chess instead of checkers. The idea is however still the same but instead of chess pieces you can now put men and kings in the 2D array.
Based on either the edge detector or the red/green square detector, calculate the center coordinates of each square on the game board. For example, average the x-coordinate of the left and right edge of a square to get the x-coordinate of the square's center. Similarly, average the y-coordinate of the top and bottom edge to get the y-coordinate of the center.
It might also be possible to find the top, left, bottom and right edge of the board and then interpolate to find the centers of all the squares. The sides of each square are probably more than a hundred pixels in length, so the calculations don't need to be that accurate.
To determine where the pieces are, iterate of a list of the center coordinates and look at the color of the pixel. If it is red or green, the square is empty. If it is black or white, the square has a corresponding piece in it. Use the information to fill an array with the information for the AI.
If the images are noisy, it might be necessary to average several pixels near the center or to average the center pixel over several frames.
It would work best if the camera is above the center of the board. If it is off to the side, the edges wouldn't be parallel/orthogonal in the picture, which might complicate the math for finding the centers.