Basically, I'm working on a robot arm that will play checkers.
There is a camera attached above the board supplying pictures (or even videomaterial but I guess that is just a series of images and since checkers is not really a fast paced game I can just take a picture every few seconds and go from there)
I need to find a way to translate the visual board into a e.g a 2d array to feed into the A.I to compute the robots moves.
I have a line detection working which draws lines on along the edges of the squares (and also returns edges in canny as a prior step). Moreover I detect green and red (the squares of my board are green and red) and return these both as a mask each.
I also have a sphere detection in place to detect the position of the pieces and some black and white color detection returning a mask each with the black or white detected areas.
My question is how I can now combine these things I have and as a result get some type of array out of which I can deduct information over in which squares my pieces are ?
Like how would i build the 2d array (or connect any 8x8) array to the image of the board with the lines and/or the masks of the red/green tiles ? I guess I have to do some type of calibration ?
And secondly is there a way to somehow overlay the masks so that I then know which pieces are in which squares ?
Well, first of all remember that chess always starts with the same pieces on the same positions e.g. black knight starts at 8-B which can be [1][7] in your 2D array. If I were you I would start with a 2D array with the begin positions of all the chess pieces.
As to knowing which pieces are where: you do not need to recognize the pieces themselves. What I would do if I were you is detect the empty spots on the chessboard which is actually quite easy in comparison to really recognizing the different chess pieces.
Once your detection system detects that one of the previously empty spots is now no longer empty you know that a chess piece was moved there. Since you can also detect a new open spot(the spot where the chess piece came from) you also know the exact chess piece which was moved. If you keep track of this list during the whole game you can always know which pieces are moved and which pieces are where.
Edit:
As noted in the comments my answer was based on chess instead of checkers. The idea is however still the same but instead of chess pieces you can now put men and kings in the 2D array.
Based on either the edge detector or the red/green square detector, calculate the center coordinates of each square on the game board. For example, average the x-coordinate of the left and right edge of a square to get the x-coordinate of the square's center. Similarly, average the y-coordinate of the top and bottom edge to get the y-coordinate of the center.
It might also be possible to find the top, left, bottom and right edge of the board and then interpolate to find the centers of all the squares. The sides of each square are probably more than a hundred pixels in length, so the calculations don't need to be that accurate.
To determine where the pieces are, iterate of a list of the center coordinates and look at the color of the pixel. If it is red or green, the square is empty. If it is black or white, the square has a corresponding piece in it. Use the information to fill an array with the information for the AI.
If the images are noisy, it might be necessary to average several pixels near the center or to average the center pixel over several frames.
It would work best if the camera is above the center of the board. If it is off to the side, the edges wouldn't be parallel/orthogonal in the picture, which might complicate the math for finding the centers.
Related
I'd like to find the corners of the following box
However, as you can see I'm detecting a lot of corners I don't want to find. I'm completly stuck on this one. No matter what I try, I always seem to find corners in the dots on the box. I used the function goodFeaturesToTrack() but I also tried cornerHarris()
The most important thing to me is to find the coordinates of the corner pixels so I can draw a wire frame.
Kind regards, Schweini
Edit:
To draw the wire frame onto the image, following process can be thinkable.
When extracting outline of the black box region, the outline consists with 6 straight line segments.
Therefore, you'll able to find at least 6 corners of the box as the intersection of each two adjacent line segments.
Additionally, it looks like that, outline of 6 surfaces of the box will be able to coarsely estimated from each adjacent line segment pairs. (with assuming parallelogram)
This means estimating the positions of the remaining two corners (to draw the wire frame).
Furthermore, if you want, comparing the estimation result with your examination (corner detecition) result will be able to refine the coordinates of one corner.
I have the following problem. If you take a look at the image below, you see a microscopic image of a circular strand of DNA, called a teardrop. An initial skeletonization (morphology.skeletonize) yielded the trace in red. The trace is an nd-array of shape (N, 2), where N is the number of points in my trace. In orange, the strand is traced after I applied some sort of improvement algorithm, which I designed myself. Now, as you can see, around the bright region, the trace twists a little bit and doesn't resemble the real shape. To work around that, I want to do the following:
Define a radius around the blue dot which is fixed (i.e. 5 nanometres)
Remove all points of the trace that lie within that radius (my problem)
After those points are removed, connect the blue dot with the first point that lies outside the circle
In the end, I don't know how I can tell Python that for a given array of coordinates, I want to delete all those that lie within an area of my image. That area is, as explained, a circle around the blue dot.
Example image used in program
I am trying to find the coordinates of a polygon in an image
(just like flood fill algorithm we are given a coordinate and we need to search the surrounding pixels for the boundary, if boundary is found we need to append its coordinate to the list if not we need to keep searching other pixels.)and if all the pixels are traversed the program should stop returning the list of pixels.
usually color of boundary is black and image is a gray scale image of maps of building.
It seems that flood-fill will be good enough to completely fill a room, despite the extra annotations. After filling, extract the outer outline. Now you can detect the straight portions of the outline by checking the angle formed by three successive points. I would keep a spacing between them to avoid local inaccuracies.
You will find a sequence of line segments, possibly interrupted at corners. Optionally use line fitting to maximize accuracy, and recompute the corners by intersecting the segments. Also consider joining aligned segments that are interrupted by short excursions.
If the rooms are not well closed, flood filling can leak and you are a little stuck. Consider filling with a larger brush, though this can cause other problems.
I need to find the coordinates of the largest empty rectangle in a PNG image. The rectangle should consist of light colors (if that is too difficult, white pixels only are fine) and should be axis-oriented.
I am new to computer vision and I found out about OpenCV, I am currently using the python interface to it and started tackling this problem with the SimpleBlobDetector interface, but it gives me only the center of the Blob with a certain radius.
Can anyone point me in the right direction for this?
EDIT: I need to do this with a regular colored PNG image, not a binary matrix
You can use a contour extractor, with the given point list you can check the size of the rectangle by checking the sizes of the lists, assuming that all the rectangles are parallel to the cardinal axis. If not you need to compute the distance of a pixel and the next for all the pixels in the contour list by using the x and y coordinates on each.
Using Python, OpenCV, and live webcam input, I can't figure out how to set a point based on an x y coordinate and track where it moves.
Below is a simple example to track a yellow object.
https://github.com/abidrahmank/OpenCV-Python/blob/master/Other_Examples/track_yellow_draw_line.py
Here is the method to track yellow color:
1) Extract the first frame of video
2) Convert frame into HSV color space. Take H plane and threshold it for yellow color so that you get binary image with yellow object as white (also called blob) and remaining as black.
3) Now you find centre point of blob. You can use moments or contours(especially if you have more than one blob. In the example above, very simple logic is used. Just find leftmost,rightmost,topmost and bottommost points on blob and draw a rectangle around it). And store this values.
4) Extract next frame and follow all above steps to get new position. Join these two position and draw a line.
Over.
There are a few blogs that explain the basics. Check out this one: Object tracking in OpenCV and Python 2.6.
Edit: I don't think you can track arbitrary points. To be able to make a correspondence between one point in two images, you need to know something unique about the point to track. This is often done with interest points, which are "unique enough" to be compared across images. Other methods are based making the point easy to detect using a projection scheme.