Develop Reference

How to find the rotation matrix with known points

The corner points of a rectangle are given in 2D coordinates. I also know the real distance of the points and I have the camera matrix.
Now I want to find the rotation vector with respect to the camera, but without using the cv2.calibrateCamera() method with the chessboard corners.

Finding minimum bounding box for a set of points

Im trying to find the minimum bounding box for a set of points. I used the library https://bitbucket.org/william_rusnack/minimumboundingbox/src/master/
I want to enclose these coordinates in the minimum bounding box :-
#INPUT COORDINATES
('297.153338872911', '373.3796368193535')
('222.75', '278.7499999999999')
('222.75', '278.7499999999999')
('42.25', '251.75')
('179.25', '390.5')
('298.403338872911', '373.3796368193535')
('0.6400000000000537', '240.0000000000201')
('75.26400000000632', '390.9120000000328')
# OUTPUT BOUNDING BOX COORDINATES , PLOTTED
[(-22.685403520024096, 373.8408037637353),
(0.6400000000000432, 240.0),
(288.84381342985046, 428.133250381587),
(312.16921694987457, 294.2924466178517)]
But as you can see from the output itself
the bounding box is coming out as tilted/inclined . That's all right, but what I want is for the bounding box to be straight to the horizontal & vertical plane of my desktop. How can I achieve this without losing the bounding box functionality (i.e. any point popping out of the bounding box while making it straight) ?
I tried this code also, but it rotates the rectangle and finds the best possible bounding box which mostly would be inclined rectangle. But I strictly want a straight rectangle.

For sake of notation, let's assume that your points are a list of pairs (2-tuples), each pair being the x,y coordinates of a point.
point_list = [(10,12), (11,14), (15,20), ...]
Your orthogonal bounding box has simply x and y limits: the min and max of each coordinate:
x_min = min(point[0] for point in point_list)
x_max = max(point[0] for point in point_list)
y_min = min(point[1] for point in point_list)
y_max = max(point[1] for point in point_list)
Your bounding box is the four edges defined by those min/max values.

How would you render a 3D arrow (cone + cylinder) onto a 2D image in Python without OpenGL?

Assuming I've been given image I and a 3D vector V_3d = [V_x, V_y, V_z]. To make things easier, assume they're both in the same coordinate system. That is, the image plane normal is Z axis. The Y axis points downwards and X points to the right.
The closest example:
Their solution is complicated and requires OpenGL.

How do I fit rectangles to an image in python and obtain their coordinates

I'm looking for a way to split a number of images into proper rectangles. These rectangles are ideally shaped such that each of them take on the largest possible size without containing a lot of white.
So let's say that we have the following image
I would like to get an output such as this:
Note the overlapping rectangles, the hole and the non axis aligned rectangle, all of these are likely scenario's I have to deal with.
I'm aiming to get the coordinates describing the corner pieces of the rectangles so something like
[[(73,13),(269,13),(269,47)(73,47)],
[(73,13),(73,210),(109,210),(109,13)]
...]
In order to do this I have already looked at the cv2.findContours but I couldn't get it to work with overlapping rectangles (though I could use the hierarchy model to deal with holes as that causes the contours to be merged into one.
Note that although not shown holes can be nested.

A algorithm that works roughly as follow should be able to give you the result you seek.
Get all the corner points in the image.
Randomly select 3 points to create a rectangle
Count the ratio of yellow pixels within the rectangle, accept if the ratio satisfy a threshold.
Repeat 2 to 4 until :
a) every single combination of point is complete or
b) all yellow pixel are accounted for or
c) after n number of iteration
The difficult part of this algorithm lies in step 2, creating rectangle from 3 points.
If all the rectangles were right angle, you can simply find the minimum x and y to correspond for topLeft corner and maximum x and y to correspond for bottomRight corner of your new rectangle.
But since you have off axis rectangle, you will need to check if the two vector created from the 3 points have a 90 degree angle between them before generating the rectangle.

How to get the camera rotation? (Aruco Library)

I've been trying to understand the output of the aruco_test.cpp program that is included when you download the Aruco Library.
The output has this format:
22=(236.87,86.4296) (422.581,78.3856) (418.21,228.032) (261.347,228.529) Txyz=0.00813142 -0.0148134 0.140595 Rxyz=-2.14032 0.0777095 0.138929
22 is the unique identifier of the marker, the next four pairs of numbers are the four corners of the marker. My problem here is the two vectors Tvec and Rvec.
I've been reading on the Internet that tvec is the translation vector from my camera's center to my object (the marker in this case) and that rvec is the rotation of the object with respect to my camera.
I've got a few questions regarding this:
How can I know the axis of my camera? I mean, is there a way to know where the x, y and z are facing?
How can I get the rotation of the camera from the rotation of the object wrt the camera?
Can someone explain me the meaning of the vectors better so I can really understand it? I think my main problem here is that I don't really know what those numbers mean for real.
EDIT: I've been doing some testing to check how the rotation works and I don't really understand the results:
Moving the camera, marker fixed on the floor:
Initial position: camera looking at the marker - 'z' axis of the marker looking to the camera, 'y' is going upwards and 'x' goes to the right: Rxyz=2.40804 -0.0823451 0.23141
Moving the camera on the 'x' axis of the marker (tilt the camera up): Rxyz=-1.97658 -0.0506794 -0.020052
Moving the camera on the 'y' axis of the marker (incline the camera to the right): Rxyz=2.74544 -0.118551 -0.973627
Turn the camera 90 degrees (to the right): Rxyz=1.80194 -1.86528 0.746029
Moving the marker instead of the camera, leaving the camera fixed looking to the marker:
Using the same initial position as in the previous case.
Moving the marker on its 'x' axis: Rxyz=2.23619 -0.0361307 -0.0843008
Moving the marker on its 'y' axis: Rxyz=-2.9065 -0.0291299 -1.13356
Moving the marker on its 'z' axis (90º turn to the right): Rxyz=1.78398 1.74161 -0.690203
I've been assuming that each number of the vector was the rotation on a respective axis but I think I'm assuming wrong as this values don't make so much sense if that was the case.

How can I know the axis of my camera? I mean, is there a way to know
where the x, y and z are facing?
This is defined in the OpenCV library. x-axis increases from left to right of the image, y-axis increases from top to bottom of the image, and z axis increases towards the front of the camera. Below image explains this axis selection.
How can I get the rotation of the camera from the rotation of the
object wrt the camera?
rvec is the rotation of the marker with respect to the camera frame. You can convert rvec to a 3x3 rotation matrix using the built-in Rodrigues function. If the marker is aligned with camera frame, this rotation matrix should read 3x3 identity matrix.
If you get the inverse of this matrix (this is a rotation matrix, so the inverse is the transpose of the matrix), that is the rotation of the camera with respect to the marker.
Can someone explain me the meaning of the vectors better so I can really understand it?
I think my main problem here is that I don't really know what those
numbers mean for real.
tvec is the distance from the origin of the camera frame to the center of the detected marker (this is F_c - P line on the figure. rvec is as described in the above answer.