I'm a beginner in python and raspberry pi and for a school project we're using a raspberry pi 3 and a camera module (fixed) and the idea is when u move something in the camera frame the program gives an output of where this object is according to the camera (in 3D) and the distance this object has traveled, it would be something like ( x=2.98m, y=5.56m, z=3.87m, distance=0.677m ) and all of this using optical flow and python. Is it possible to make this and if not Is there something close or similar to this.
Any help appreciated.
The first thing you can do is camera calibration. If you have the intrinsics of the camera you can infer the direction of the vector in 3D from your camera to your object.
The problem is how to find the distance, or the length of this vector.
There are two solutions I can think of:
use two cameras - if you have two calibrated cameras (intrinsics and extrinsics) you can triangulate the two points of the object detected in the images and get the object position in 3D.
If you know the size of the object, you can infer its distance from the camera using the intrinsics.
hope I helped.
Related
I am searching to create a script for Stereo-Opticalflow using OPENCV. My problem is reconstrution of cartesian coordinates using two images. i have two virtual cameras place at an angle 90° between cameras. i used cv.stereoCalibrate to calibrate but i don't know how to put back together the cartesian coordinate system. Can someone help me please ?
Thank you for your time.
I realise there is a lot of information about calibrating cameras in a single set up and even in a stereo set up all these instances differed to me somewhat. My set up is as follows:
Within the calibration zone there will be a big chess board in which I will calibrate the camera with opencv findchessboard. The problem I have at the moment is that I have been using the calibration only for the pictures distortion, which is what I have mostly found only online. However I need to somehow find the respective positions of the cameras (x,y,z) and the angle of inclination if possible. I though surely this would be possible with some sort of calibration as to line the points up as both cameras will be looking that the same calibration configuration. Essentially my question boils down to:
Is there a opencv calibration which will help me with the relative distances and angles (x,y,z,θ)
Is it as trivial as calibrating the two cameras individually
If not opencv how would this be accomplished.
I am working on an application using an IFM 3D camera to identify parts prior to a robot pickup. Currently I am able to find the centroid of these objects using contours from a depth image and from there calculate the center point of these objects in pixel space.
My next task is to then transform the 2D centroid coordinates to a 3D point in 'real' space. I am able to train the robot such that it's coordinate frame is either at the center of the image or at the traditional (0,0) point of an image (top left).
The 3D camera I am using provides both an intrinsic and extrinsic matrix. I know I need to use some combination of these matrices to project my centroid into three space but the following questions remain:
My current understanding from googling is the intrinsic matrix is used to fix lens distortion (barrel and pinhole warping, etc.) whereas the extrinsic matrix is used to project points into the real world. Is this simplified assumption correct?
How can a camera supply a single extrinsic matrix? I know traditionally these matrices are found using the checkerboard corners method but are these not dependent on the height of the camera?
Is the solution as simple as taking the 3x4 extrinsic matrix and multiplying it by a 3x1 matrix [x, y, 1] and if so, will the returned values be relative to the camera center or the traditional (0,0) point of an image.
Thanks in advance for any insight! Also if it's any consolation I am doing everything in python and openCV.
No. I suggest you read the basics in Multiple View Geometry of Hartley and Zisserman, freely available in the web. Dependent on the camera model, the intrinsics contain different parameters. For the pinhole camera model, these are the focal length and the principal point.
The only reason why you maybe could directly transform your 2D centroid to 3D is that you use a 3D camera. Read the manual of the camera, it should be explained how the relation between 2D and 3D coordinates is given for your specific model.
If you have only image data, you can only compute a 3D point from at least two views.
No, of course not. Please don't be lazy and start reading the basics about camera projection instead of asking for others to explain the common basics that are written down everywhere in the web and literature.
I followed the opencv tutorials called 'Camera calibration and 3d reconstruction' (https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_calib3d/py_table_of_contents_calib3d/py_table_of_contents_calib3d.html) and I do not understand what is the correlation between camera calibration and creating a depth map. I can see that focal length is in the equation for disparity, but according to code there's no need to compute the camera matrix 'cause its not used anywhere.
Am I right or someone can point me my mistake?
All code is in the opencv link.
I'm using python.
I have two questions relating to stereo calibration with opencv. I have many pairs of calibration images like these:
Across the set of calibration images the distance of the chessboard away from the camera varies, and it is also rotated in some shots.
From within this scene I would like to map pairs of image coordinates (x,y) and (x',y') onto object coordinates in a global frame: (X,Y,Z).
In order to calibrate the system I have detected pairs of image coordinates of all chessboard corners using cv2.DetectChessboardCorners(). From reading Hartley's Multiple View Geometry in Computer Vision I gather I should be able to calibrate this system up to a scale factor without actually specifying the object points of the chessboard corners. First question: Is this correct?
Investigating cv2's capabilities, the closest thing I've found is cv2.stereoCalibrate(objectpoints,imagepoints1,imagepoints2).
I have obtained imagepoints1 and imagepoints2 from cv2.findChessboardCorners. Apparently from the images shown I can approximately extract (X,Y,Z) relative to the frame on the calibration board (by design), which would allow me to apply cv2.stereoCalibrate(). However, I think this will introduce error, and it prevents me from using all of the rotated photos of the calibration board which I have. Second question: Can I calibrate without object points using opencv?
Thanks!
No. You must specify the object points. Note that they need not change across the image sequence, since you can interpret the change as due to camera motion relative to the target. Also, you can (should) assume that Z=0 for a planar target like yours. You may specify X,Y up to scale, and thus obtain after calibration translations up to scale.
No
Clarification: by "need not change across the image sequence" I mean that you can assume the target fixed in the world frame, and interpret the relative motion as due to the camera only. The world frame itself, absent a better prior, can be defined by the pose of the target in any one of the images (say, the first one). Obviously, I do not mean that the pose of the target relative to the camera does not change - in fact, it must change in order to obtain a calibration. If you do have a better prior, you should use if. For example, if the target moves on a turntable, you should solve directly for the parameters of the cylindrical motion, since there is less of them (one constant axis, one constant radius, plus one angle per image, rather than 6 parameters per image).