I have a UAV with a LiDAR onboard flying and scanning on 3D space. I have the UAV's GPS position with good precision, and I wanted to know how to build a 3D map using the LiDAR's pointCloud. Our goal is for the UAV to avoid obstacles in front of it, and it would be very helpful in visualizing the operation remotely.
I have ardupilot's GPS and orientation data through mavlink and publish it on ROS for my application, as well as the LiDAR's scan as a PointCloud2. Can I somehow set a GPS static origin and build a map around it, using something like octomap_server?
Any tips on what to look for would be greatly appreciated!
Thank you kindly.
The octomap_server will assume the origin is (0,0) and try to build a map around that. As your question implies, if you're using the lidar for object detection and avoidance, you should not be operating in the GPS frame. If you're trying to use the lidar and with octomap_server for a long term data reference, this work should be done in the map frame; here you shouldn't have any issues with setting default points. If you would like to just use it for a short term reference and continuous object avoidance, it should be done in the odom frame; again, default position shouldn't be a problem here.
Related
I intend to make a 3D model based on multi view stereo images ( basically 2D plane images of the same object from different angles and orientation) inside Blender from scratch.However, I am new to Blender.
I wanted to know if there are any tutorials of how to project a single pixel or point in the space of Blender's 3D environment using python. If not tutorial, any documentation. I am still learning about this whole 3D construction thing and pretty new to this, so I am not sure maybe these points are displayed using a 3 dimensional matrix/array ?
Basically I want to implement 3D construction based on a paper written by some researchers. Mostly every such project is in C++. I want to do it in Python in Blender, and if I am capable enough, make these libraries open source.
Suggest me any pre-requisite if you think that shall help me. I have just started my 3rd year of BSc Computer Science course, and very new to the world of Computer Graphics.
(My skillset is C, Java and Python.)
I would be very glad and appreciate any help.
Thank You
[Link to websitehttps://vision.in.tum.de/research/image-based_3d_reconstruction/multiviewreconstruction[][1]]
image2
Yes, it can very likely be done in Blender, and in Python at least for small geometries / low resolution.
A valid approach for the kind of scenarios you seem to want to play with is based on the idea of "space carving" or "silhouette projection". A good description in is an old paper by Kutulakos and Seitz, which was based in part on earlier work by Szelisky.
Given a good estimation of the silhouettes, these methods can correctly reconstruct all convex portions of the object's surface, and the subset of concavities that are resolved in the photo hull. The remaining concavities are "patched" over and need to be reconstructed using a different method (e.g. stereo, or structured light). For the surfaces that can be reconstructed, space carving is generally more robust than stereo (since it is insensitive to the color and surface texture of the object), and can work on surfaces where structured light struggles (e.g. surfaces with specularities, or very dark objects with low reflectance for a laser stripe)
The basic idea is to use the silhouettes of the projection of the object in cameras around it to "remove" mass from an initial volume (e.g. a box) encompassing the object, a bit like a sculptor carving a statue by removing material from a block of marble.
Computationally, you can do it representing the volume of space of interest using an octree, initialized with a minimal level of subdivision, and then progressively refined. The refinement consists of projecting the vertices of the octree leaves in the cameras, and identifying which leaves are completely outside or partially inside the silhouettes. The former are pruned, while the latter are split, and the process continues until no more leaves can be split or a maximul level of subdivision is reached. The hull of the octree is then extracted as a "watertight" mesh using standard methods.
Apart from the above paper, a way more detailed description can be found on an old patent by Geometrix - it sold a scanner based on the above ideas around year 2000. Here is what it looked like:
I am developing a robot based on StereoPI. I have successfully calibrated the cameras and obtained a fairly accurate depth map. However, I am unable to convert my depth map to point cloud so that I can obtain the actual distance of an object. I have been trying to use cv2.reprojectImageTo3D, but see no success. May I ask if there is a tutorial or guide which teaches how to convert disparity map to point cloud?
I am trying very hard to learn and find reliable sources but see on avail. So, Thank you very much in advance.
By calibrating your cameras you compute their interior orientation parameters (IOP - or intrinsic parameters). To compute the XYZ coordinates from the disparity you need also the exterior orientation parameters (EOP).
If you want your point cloud relative to the robot position, the EOP can be simplified, otherwise, you need to take into account the robot's position and rotation, which can be retrieved with a GNSS receiver and intertial measurement unit (IMU). Note that is very likely that such data need to be processed with a Kalman filter.
Then, assuming you got both (i) the IOP and EOP of your cameras, and (ii) the disparity map, you can generate the point cloud by intersection. There are several ways to accomplish this, I suggest using the collinearity equations.
I'd like to create synthetic training data for DL models for segmentation and classification in point clouds. The ground truth / real data comprise LiDAR point clouds. I scripted a simple mesh sampling model in python/open3d and I'm able to quickly transfer 3D scenes to point clouds (see fig 1), but I need to include certain characteristics of LiDAR sensors.
Blensor (https://www.blensor.org/) works the way I need it (fig 2), but I don't want to use blender atm. Also the results don't have a sufficient quality for my use case.
In the first step I'd just like to cut off the points, which are not reachable by a certain position of a LiDAR sensor, mainly to create the "shadows", which are important to make the training data more realistic. Do you have any suggestions for a simple and fast workaround? My point cloud is saved in a pandas dataframe including x,y,z and nx,ny,nz values.
Thx in advance,
reiti
If your 3D scene can be described in the form of distance functions (essentially consisting of a range of simple geometric shapes opposed to point cloud data) you may be good to go with an easily modified ray tracing algorithm that emulates a LiDAR sensor.
For each LiDAR "ray" (i.e. for every direction) you only need to save the first scene collision's xyz coordinates. This also gives you full freedom to match the original real world sensor properties (like angles and number of points).
How easy the calculation of the distance between scene and sensor-ray will be depends on the scene you have set up and how it is represented. Sorry for not being able to provide you with a ready-to-use implementation, but this might give you some direction.
I am working with 3D point clouds acquired from an object and I need to align them in a single global point cloud. I am having an hard time in understanding the difference between SLAM and registration. Especially since both of them can implement ICP
The point clouds have been acquired in spatial and temporal order and hav extended overlapping area; therefore I should could SLAM for aligning them.
Anyone can clarify this point to me?
Thanks!
anna
Based on your question it sounds like you are working with a depth sensor of some kind moving through an environment. You would like to create a consistent map or point cloud with this sensor. I would have commented this, but my reputation is too low at the moment.
Registration just refers to the alignment of two measurements through some transformation. For image registration this is typically when you find some transformation whether it be a simple translation or an affine warp between two images which makes them 'look' similar. Point cloud registration typically refers to finding a rotation and translation which aligns two point clouds.
SLAM, as you probably know, refers to simultaneous localization and mapping. The goal of SLAM is to find the sensors motion through a scene, and map the scene at the same time.
I think the reason you are having a hard time seeing the difference between the two is because, for your application, registration is a way of accomplishing a simple form of SLAM. The reason for this is because ICP essentially is finding the relative transformation of your depth sensor between two measurements. This is acts as odometry for your sensor.
However, registration is not necessarily going to give you a relative sensor pose in all applications. For example a KLT tracker is a form of simple image registration, but it does not give the relative transformation of two cameras directly.
I hope this clears it up.
I am using OpenCV and Python.
Let say I have this sequence video of the car. And I have tracked some 'interesting points' of the car with the cv2.goodFeaturesToTrack and cv2.calcOpticalFlowPyrLK. Now, given the traced points, I want to estimate a very rough shape (maybe a 3D box) of the car and its distance from the camera. It doesn't need to be that accurate.
On top of that, I want it to be keep updating in real time. The closest youtube video I can find that can give a view of what I am trying to achieve is this. I have found a new Structure from Motion module in OpenCV, but it is more on building a 3D model from a collection of points.
The question is, what is the best way of achieving this and what kind of library I can use (especially in order to construct the 3D space)?
And it is also OK if somehow I need to use C++ for this (although I am still not good in it yet).
Thanks.