I want to fit a camera-view to a mesh from a fixed point. See attached image.
Example
So I need to adjust the camera rotation, focal length and frame width/height.
What is the best way to do this with python?
What you're asking is relatively complex in terms of operation. You're adjusting multiple camera properties to frame an object.
I recommend you decompose the problem into parts and ignore focal length all together. Simply transform a camera so it frames the object. You can then add a supplementary step to modify the width and height of the camera to tightly frame it.
The gist of what you want to do is as follows:
get bounding box of object
get camera aspect ratio
get active viewport representation
get camera matrix based on object bounding box and corresponding camera aspect ratio mapped from active viewport
apply matrix to camera
This will be much easier if you're familiar with the OpenMaya API. The OpenMayaUI.M3dView and the OpenMaya.MFnCamera classes should get you started.
https://help.autodesk.com/view/MAYAUL/2019/ENU/?guid=__py_ref_class_open_maya_u_i_1_1_m3d_view_html
https://help.autodesk.com/view/MAYAUL/2019/ENU/?guid=__py_ref_class_open_maya_1_1_m_fn_camera_html
If you're unfamiliar with the API, then scour the mel scripts and check how the FrameSelectedWithoutChildren Runtime Command (F key in the viewport) shortcut works, and use that to automate the process.
Related
I am working on a pet project. I want to develop a system where I can measure a 3D object's height, width, and length. But, the fact is that I wish the measurement to be done automatically from the 3D point cloud without any intervention from me or needing the help of any reference object.
To be more precise, the object is in a point-cloud format encapsulated in a 3D bounding box.
You can analyze geomdl, rhino3dm and verb.py libraries. You can use bounding box to get object and analyze height, width and surface area of the 3D object.
I have a dataset of meshes, which I want to use to generate partial view data as point clouds. In other words, simulating the way an RGB-D sensor would work.
My solution is very naive, so feel free to disregard it and suggest another method.
It consists of taking a rendered RGB and a rendered D image from an o3d visualization, as such:
vis.add_geometry(tr_mesh)
... # set some params to have a certain angle
vis.capture_screen_image('somepath.png')
vis.capture_depth_image('someotherpath.png')
These are saved as PNG files. Then I combine them into an o3d RGBDImage:
# Load the RGB image
rgb = o3d.io.read_image(rgb_path)
# Load the depth image
depth = o3d.io.read_image(depth_path)
# Convert the RGB and depth images into pointcloud
rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(color=o3d.geometry.Image(rgb),
depth=o3d.geometry.Image(depth),
convert_rgb_to_intensity=False)
And convert this to a PointCloud
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(image=rgbd,
intrinsic=o3d.camera.PinholeCameraIntrinsic(o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
This has many limitations:
the depth is quantized to 256 values. These are literally in 0...255 so they don't match the scale of the image height and width. Not to mention, it completely loses the original scale of the mesh object itself.
the camera params (focal length etc) are not recreated identically so the point cloud is deformed.
Again, this is a very naive solution, so completely different approaches are very welcome.
This is not a duplicate of Can I generate Point Cloud from mesh? as I want partial views. So think that question, but with back-face culling.
Getting back with the solution.
No image capture is needed. There is a function called vis.capture_depth_point_cloud()
So the partial view can be generated by simply running
vis.add_geometry(tr_mesh)
... # set some params to have a certain angle
vis.capture_depth_point_cloud("somefilename.pcd")
This also has a parameter called convert_to_world_coordinate which is very useful.
There doesn't seem to be a way to change the resolution of the sensor. Though up-(or down-)scaling the object, capturing the point cloud, then down-(or up-)scaling the point cloud should obtain the same effect.
I have a photo taken from a camera (whose focal length, principle point, and distortion coefficients I know). The photo has a 8cm x 8cm post-in on a table and the center of the post-it is the origin (0, 0) again in cm. I've also indicated the positive-y axis on the post-it.
From this information is it possible to compute the location of the camera and the vector in which the camera is looking in Python using OpenCV? If someone has a snippet of code that does that (assuming you know the coordinates of the post-it corners already) that would be amazing!
Use OpenCV's solvePnP specifying SOLVEPNP_IPPE_SQUARE in the flags. With only 4 points (and a postit) the solution will be quite sensitive to how accurately you mark their images, so ask yourself whether you really need the camera pose and location for your application, and how accurately. E.g., if you just want to make a flat CG "sticker" stay fixed on the table while the camera moves, all you need is estimating a homography, a much simpler task.
It does look like you have all the information required. The marker you use can be easily segmented. Shape analysis will provide corners. I did something similar to get basic eyesight tracking:
Here is a complete example.
Segmentation result for the example:
Please notice, accuracy really matters, so it might be useful to rely on several sets of points.
I have two questions relating to stereo calibration with opencv. I have many pairs of calibration images like these:
Across the set of calibration images the distance of the chessboard away from the camera varies, and it is also rotated in some shots.
From within this scene I would like to map pairs of image coordinates (x,y) and (x',y') onto object coordinates in a global frame: (X,Y,Z).
In order to calibrate the system I have detected pairs of image coordinates of all chessboard corners using cv2.DetectChessboardCorners(). From reading Hartley's Multiple View Geometry in Computer Vision I gather I should be able to calibrate this system up to a scale factor without actually specifying the object points of the chessboard corners. First question: Is this correct?
Investigating cv2's capabilities, the closest thing I've found is cv2.stereoCalibrate(objectpoints,imagepoints1,imagepoints2).
I have obtained imagepoints1 and imagepoints2 from cv2.findChessboardCorners. Apparently from the images shown I can approximately extract (X,Y,Z) relative to the frame on the calibration board (by design), which would allow me to apply cv2.stereoCalibrate(). However, I think this will introduce error, and it prevents me from using all of the rotated photos of the calibration board which I have. Second question: Can I calibrate without object points using opencv?
Thanks!
No. You must specify the object points. Note that they need not change across the image sequence, since you can interpret the change as due to camera motion relative to the target. Also, you can (should) assume that Z=0 for a planar target like yours. You may specify X,Y up to scale, and thus obtain after calibration translations up to scale.
No
Clarification: by "need not change across the image sequence" I mean that you can assume the target fixed in the world frame, and interpret the relative motion as due to the camera only. The world frame itself, absent a better prior, can be defined by the pose of the target in any one of the images (say, the first one). Obviously, I do not mean that the pose of the target relative to the camera does not change - in fact, it must change in order to obtain a calibration. If you do have a better prior, you should use if. For example, if the target moves on a turntable, you should solve directly for the parameters of the cylindrical motion, since there is less of them (one constant axis, one constant radius, plus one angle per image, rather than 6 parameters per image).
I have an image that represents the elevation of some area. But the drone that made it didn't necessarily go in a straight line(although image is always rectangular). I also have gps coordinates generated every 20cm of the way.
How can I "bend" this rectangular image (curve/mosaic) so that it represents the curved path that the drone actually went through? (in python)
I haven't managed to write any code as I have no idea what is the name of this "warping" of the image. Please find the attached image as a wanted end state, and normal horizontal letters as a start state.
There might be a better answer, but I guess you could use the remapping functions of openCV for that.
The process would look like that :
From your data, get your warping function. This will be a function that maps (x,y) pixel values from your input image I to (x,y) pixel values from your output image O
Compute the size needed in the output image to host your whole warped image, and create it
Create two maps, mapx and mapy, which will tell the pixel coordinates in I for every pixel in 0 (that's, in a sense, the inverse of your warping function)
Apply OpenCV remap function (which is better than simply applying your maps because it interpolates if the output image is larger than the input)
Depending on your warping function, it might be very simple, or close to impossible to apply this technique.
You can find an example with a super simple warping function here : https://docs.opencv.org/2.4/doc/tutorials/imgproc/imgtrans/remap/remap.html
More complex examples can be looked at in OpenCV doc and code when looking at distortion and rectification of camera images.