Transform 3D points to points in 2D image - python

Is there a way how I could transform 3D points to 2D points corresponding to the positions of a 2D image?
Is there a function in Python/OpenCV that has following properties:
input: 3D points, camera position in 3D, direction of the camera, maybe additional features of the camera
output: corresponding 2D points on the 2D image of the camera?


Have you seen cv2.projectPoints method?
Python: cv2.projectPoints(objectPoints, rvec, tvec, cameraMatrix, distCoeffs[, imagePoints[, jacobian[, aspectRatio]]]) → imagePoints, jacobian
The function computes projections of 3D points to the image plane given intrinsic and extrinsic camera parameters.
more details here

Related

How to find the rotation matrix with known points

The corner points of a rectangle are given in 2D coordinates. I also know the real distance of the points and I have the camera matrix.
Now I want to find the rotation vector with respect to the camera, but without using the cv2.calibrateCamera() method with the chessboard corners.

Projecting a Texture Mask onto an existing 3D Mesh given the camera extrinsics

Given an image mask, I want to project the pixels onto a mesh in respect to the position and orientation of the camera and convert these pixels into a pointcloud. I have the intrinsic and extrinsic parameters of the camera in respect to the world, and the location of the mesh in world coordinates. I know the mapping from world coordinates to camera image is as follow:
imgpoint = Intrinsic * Extrinsic * worldpoint
So when I want to the opposite i do the inverse of the intrinsic and extrinsic matrices:
worldpoint= Intrinsic^(-1) * Extrinsic^(-1) * imgpoint
However, the idea that I had was to obtain two points from one pixel, with different depth values, to obtain a line and then look for the closest intersection for the mesh I want with the line, but I do not know how to properly generate a point away from the original camera plane. How can I find this extra point and/or am I complicating this problem?

The top equation below shows how to project a point (x,y,z) onto a pixel (u,v);
The extrinsic parameters are the 3x3 rotation matrix R and translation t.
The intrinsic parameters are the focal distances f_x, f_y and
principal point (c_x, c_y). The value alpha is the perspective foreshortening term that is divided out.
The bottom equation reverses the process by describing how to project
a ray from the camera position through through the pixel (u,v) out into the scene as the parameter alpha varies from 0 to infinity.
Now we have converted the problem into a ray casting problem.
Find the intersection of the ray with your mesh which is a
standard computer graphics problem.

Camera calibration and point projection from 3D to 2D image

I am trying to project a 3D point from a pointcloud to a 2D panorama image. I have manually picked 8 points on the 2D image and the corresponding 3D pointcloud points and used the OpenCV library to find the camera matrix, translation, and rotation vectors that are then used in projection method.
#initial camera matrix
camera_matrix = cv2.initCameraMatrix2D([objectPoints_3D],[imagePoints_2D], image_size_2D)
#calibrate cameras
intrinsics_error, intrinsic_matrix, intrinsics_distortion_coeffs, rotation_vectors, translation_vectors = cv2.calibrateCamera([objectPoints_3D], [imagePoints_2D], image_size_2D, camera_matrix, None, flags=cv2.CALIB_USE_INTRINSIC_GUESS) #camera_matrix and intrinsic_matrix are the same
#project a 3D point in the pointcloud onto a 2D image
projected_point_2D = cv2.projectPoints(selected_point_3D, rotation_vectors[0], translation_vectors[0], intrinsic_matrix, intrinsics_distortion_coeffs)
The projected_point_2D seems reasonable for the same image that I used when calibrating, but fails in another set of 2D and 3D images. Am I doing this wrong?
Images contain buildings and trees

How to extract the surface mesh from binary mask of 3D images?

I have a binary mask of 3D medical images in nii format which is reconstructed from 2D slices and I need to extract surface mesh from the mask. AFAK, a surface mesh is constructed from the surface points of 3D surface, but I do not have the point sets (vertices).
Do you know any function or software that I can extract the mesh from the binary mask?
Thanks

Getting the projection matrix which can transform 3d points to 2d points in an image as done by cv2.projectPoints function in opencv

The opencv function cv2.projectPoints() takes 3d points, rotation vector, translation vector, camera matrix and distortion coefficients and aspect ratio and generate the 2d projection of the 3d point in the image plane. Is there some way to get the full projection matrix which does the same to any 3d point so that i can do the projection without using the projectPoints fucntion

Categories

Resources