I am working a gps-denied UAV localization project. I am reading this paper GPS-Denied UAV Localization using Pre-existing Satellite Imagery. In this paper, they try to align a UAV frame sequentially with a satellite map using an homography. The first homography is estimated using the GPS information stored in the first UAV frame. In the code, I don't see any information about this part. I am wondering if someone can explain this or point me to some reference that can help.
Related
I have a large model of a house, with every internal object like walls, tables, doors and tv's inside. The file is a 3D object, either a .obj or a .fbx file. I also have a pointcloud from a 180 degrees lidar scanner that has scanned from somewhere inside the house. I know where the scanner stood with a precision of about 3 meters, and I want to find out what my pointcloud corresponds to in my 3D model. In other words, I want to find the translation and rotation required to move my pointcloud to the correct position in the model.
I have tried to turn the 3D model into a pointcloud, and then use ICP (iterative closest point), but as the points I generate does not necessarily correspond with those from the scanner, I get quite weird results from time to time.
I have also looked at this one Match 3D point cloud to CAD model, but in my case I only have a scan of a small portion of the full model.
Does anyone have any advice on how to do this with python?
Given a 2D image of blade and its corresponding 3D Scan data in stl/ply/pcd format. Is it possible to map the 2D image onto the 3D scan data using python? Or is it possible that we extract the color information from the 2D Image and map the color texture onto the 3D scan data using python libraries?
I am working on a project where I want to localize the defect position on the blade, I have already implemented AI algorithm to locate the defect on the 2D image, but now I want to transfer this information to the 3D CAD Data.
Ps. I am new to handling 3D data, so any suggestion would be of great help.
if your 3d data means a point cloud data from a lidar device in one timestamp, you can calculate the transformation from your camera to your lidar and project your lidar to the image plane and get color. And you can reference code from here
if your 3d was collected from stereo vision software, like colmap. There is also a pose relationship between images and the 3d model. You can make the same approach as situation 1.
the worst condition is that your model doesn't have any relative pose between the images and the 3d model. The solution is to calculate an image-to-geometry result. image-to-geometry methods will guess the 3d pose of the images relative to the 3d model.
if you only want to know how to map a texture to a 3d model. There is an answer.
I am developing a vision system based on ArUco visual markers. The ArUco library used is cv2.aruco.DICT_7X7_50 and the marker I have used is 2 cm on each side.
I have made the calibration using a 3.6 cm board for each square, obtaining results that I consider satisfactory.
The problem appeared when I started to validate the results provided by the vision system. To perform the validation, a UR10e collaborative robot has been used, in which the ArUco marker has been strategically placed on the tool connected to it. In this way, the robot tool is positioned in a known spatial position and, using a Logitech C525 camera, the marker is located and its position and orientation with respect to the camera is calculated (actually the position of the tool is calculated, for this the reference system is transferred from the center of the aruco to the center of the tool using homogeneous transformation matrices).
The problem I have is that when the tool and its marker are placed just in front of the camera, i.e. without any rotation in X or Y, I seem to have an ambiguity problem because the vision system returns a rotation value in Y. In the same way, when I try to correct the error manually, this numerical error is transferred to the X rotation.
I have posted this problem on GitHub and on the OpenCV forum, but they didn't do much help. Attached are the links to each of the topics:
https://forum.opencv.org/t/ambiguity-problem-in-the-detection-of-the-aruco-marker-when-it-is-parallel-to-the-camera-plane/7783?u=danielbilbao12
https://github.com/opencv/opencv_contrib/issues/3190#issue-1157441872
If you need more detailed information, please do not hesitate to ask for it.
I have a UAV with a LiDAR onboard flying and scanning on 3D space. I have the UAV's GPS position with good precision, and I wanted to know how to build a 3D map using the LiDAR's pointCloud. Our goal is for the UAV to avoid obstacles in front of it, and it would be very helpful in visualizing the operation remotely.
I have ardupilot's GPS and orientation data through mavlink and publish it on ROS for my application, as well as the LiDAR's scan as a PointCloud2. Can I somehow set a GPS static origin and build a map around it, using something like octomap_server?
Any tips on what to look for would be greatly appreciated!
Thank you kindly.
The octomap_server will assume the origin is (0,0) and try to build a map around that. As your question implies, if you're using the lidar for object detection and avoidance, you should not be operating in the GPS frame. If you're trying to use the lidar and with octomap_server for a long term data reference, this work should be done in the map frame; here you shouldn't have any issues with setting default points. If you would like to just use it for a short term reference and continuous object avoidance, it should be done in the odom frame; again, default position shouldn't be a problem here.
I've got a processed image array from UAV and want to write it into a projected tiff. I am aware with the array to tiff writting process with python gdal, however not sure how to project it correctly.
I have got the central GPS, UAV height, pixel size of the image array, the array is northward. The orginal UAV image's metadata can not be recognized by gdal, so I have to extract them out and then rearrange them to project the array.
Many thanks!
This question is too vague. The process you need to look into is called "ortho-rectification". You should read about the process and then break it down into stages. Then, figure out the specific pieces you have and don't have.
Fundamentally, in order to create an ortho-image, you need a digital elevation model (DEM), intrinsic camera parameters, and extrinsic parameters (pose). You can find documentation on the algorithm online or in a standard Remote Sensing book.
Another option is if your camera provides Rational Polynomial Coefficients (RPCs), which I assume is no.
Generic Amazon Search of Remote Sensing Books
https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=remote+sensing