I wanted to check if my understanding is correct. I want to calculate the depth of a crack in an object.
I have a video of the object with a crack on it captured from a smartphone. video is captured from different angles covering almost whole object. using segmentation models, I have changed the background to black keeping only object in the video.
Generate a depth map for each frame of the video using Pytorch depth estimation deep learning model.
Generate .ply file, and point cloud using depth map and frames.
calculate depth using a .ply file, but how?
Does this workflow makes sense and is it feasible? Correct me if I am wrong. Also kindly guide me in the right direction.
Removed the background of the object in the video so there should not be noise in the final output.
Related
I'm trying to automatically draw a mesh or grid over a face, similar to the image below, to use the result in a blog post that I'm writing. However, my knowledge of computer vision is not enough to recognize which model or algorithm is behind these types of cool visualizations.
Could someone help pointing me some link to reador or a starting point?
Using Python, OpenCV and dlib the closest thing I found is something called delauny triangulation but I'm not sure if that's exactly what I'm looking for seeing the results.
Putting it in a few words what I have so far is:
Detect all faces on image and calculate their landmarks using dlib.get_frontal_face_detector() and dlib.shape_predictor() methods from dlib.
Use the method cv2.Subdiv2D() from OpenCV to compute a 2D subdivision based on my landmarks. In particulary I'm getting the delauny subdivision using the getTriangleList() method from the resulting subdivision.
The complete code is available here.
However, the result is not so attractive perhaps because the division is using triangles instead of polygons and I want to check if I can improve it!
I am doing a 3D Scan and export the result as a wavefront object (.obj with the mesh, and .mtl and .jpeg for the texture). the link between the faces of the mesh and the texture are in the .obj.
I am trying to detect "circles" for example on the 3D scans which are on the texture.
there is no "3D variation on the mesh specific to the feature that I want to detect. the variation is due to the object on which is set the texture. so I have to detect "only based on the texture".
My problem is that I don't know how to start. the .jpeg is "puzzled".
I can do thresholding and " basic pre-processing" on the .jpeg but I don't know how to do the feature detection which need "geodesic distance information" in addition to the texture. at least I don't know an efficient way.
I could even detect some of the features directly on the jpeg. some of them are not split on different portion of the jpeg. don't know if it can help...
My first thought was to loop on every vertex (which already doesn't sound very optimized), try to reconstruct a local (approximately the size of the feature I want to detect) not puzzled "jpeg" and do the feature detection on this not puzzled jpeg.
I tried by "attributing" the average "color value" on each vertex but I lose too much of the texture information.
My question, is there any python libraries existing to manipulate this format ?
Is there any python libraries to do feature detection on texture on mesh with this kind of texture "puzzled" ?
if not what would be the best way to process ?
I guess the loop on all vertex and local jpeg reconstruction is not ideal...
This is my first post here, so hello everyone.
I am working on a project that involves writing a program in c++ or python that will detect obstacles and will be used for AR.Drone 2.0. However, I don't know which approach should I take.
Initially, I was adviced to use opencv and optocal flow. I've found some videos and papers about it and one way is that: divide every frame from AR.Drone's camera on 2 (left/right side) or 4 (additionally up and bottom) and calculate optical flow for each part. Then, fly in the direction where the optical flow is less.
However I have some doubts about it:
1)Which method of optical flow calculation should I use? I know that in opencv there are provided methods for calculating both dense or sparse optical flow. Which one should I choose in this application? Won't dense optical flow be too slow to meet real-time requirements?
2)I guess that in time when UAV moves left-right or up-down I'll get some "fake" vectors caused by the movement of a drone and not because of the looming obstacle. How to prevent this?
Another solution I was told about a method shown here (link for paper in description) and someone who implemented it github link however the author admitted that he "never get obstacle detection working properly on the drone".
Another option I was told about is attaching a realsense camera to a drone and extract an information about the obstacle somehow using it.
So, my question is - which path should I take? Or is there some other method to do this that will work for application I described and is relatively easy to implement?
Thanks in advance for every reply.
I'm not sure the scope of your project, whether or not this is academic or professional, but my recommendation would be to use object detection of a control image with the camera facing directly forward on the drone. if the object is detected, you can estimate it's distance from your drone based on it's size. Since it is a control image it should have a constant size and you should record how many pixels across that is at various distances from your camera. This way you can build up a model. Once you know how far away the object is you can determine if it is an obstacle or not.
Once the detection becomes large enough, determine if it is in the flight path. Then you move the drone such that the coordinates of the detection box are no longer in your flight path.
For the detection, you can either use Google's detection api which comes with a number of solid detectors/classifiers, or if you are looking to add a layer of depth to the project you can train your own. PyImageSearch is a great place to start. And if you are feeling extra scientific you can dive right into Tensorflow.
Best of luck!
Try the open source project https://github.com/generalized-intelligence/GAAS
It uses stereo camera and SLAM to detect obstacles.
I have an image captured by android camera. Is it possible to calculate depth of object in the image ? Image contains object and background only. Any suggestion, explanation or links that you think can help me will be appreciated.
OpenCV is the library you need.
I did some depth identification of water levels in pure white background a few days ago. Generally, if you want to identify the depth, you can convert the question to identify the edge of the changing colors. In this case, you can convert the colorful pictures to grey and identify the changing of while-black-grey interface. OpenCV is capable of doing the job at high speed.
Hope it helps. Let me know if you need further help.
Edits:
If you want to find the actual depths, you need to project the coordinate system of your pictures to the real world, or vice versa. To do it, you have to know a fix location as your reference and the relationship between pixels and real distances.
What I did is find the fixed location and set it as zero. Afterwards, I measured a length of an object in the picture, and also calculated the pixel amount of the object. Therefore I obtained the relationship between pixels and real distances.
Note that these procedures may involve errors in the identification. I did it very carefully and the error was acceptable in my case.
With only one image, accurate depth estimation is near impossible. However, there are various methods of estimating depth under certain assumptions or the availability of the camera calibration matrix. As mentioned by #WenlongLiu, OpenCV is a very good place to start with.
There are several packages and methods for segmentation in Python. However, if I know apriori that certain pixels (and no others) correspond to a particular object, how can I use that to segment other objects?
Which methods implemented in python would lend themselves to this approach?
Thanks.
You'll want to take a look at semi-automated image segmentation. Image segmentation in a semi-automated perspective means that you know before hand what class certain pixels belong to - either foreground or background. Given this a priori information, the goal is to minimize an energy function that best segments the rest of the pixels into foreground and background.
The best two methods that I know of are Graph Cuts and Random Walks. If you want to study the fundamentals of both of them, you should read the canonical papers by Boykov (Graph Cuts) and Grady (Random Walks) respectively:
Graph Cuts - Boykov: http://www.csd.uwo.ca/~yuri/Papers/ijcv06.pdf
Random Walks - Grady: http://webdocs.cs.ualberta.ca/~nray1/CMPUT615/MRF/grady2006random.pdf
For Graph Cuts, OpenCV uses the GrabCut algorithm, which is an extension of the original Graph Cuts algorithm: http://en.wikipedia.org/wiki/GrabCut. Essentially, you surround a box around the object you want segmented, and Gaussian Mixture Models are used to model the foreground and background and the object will be segmented from the background inside this box. Additionally, you can add foreground and background markers inside the box to further constrain the solution to ensure you get a good result.
Take a look at this official OpenCV tutorial for more details: http://docs.opencv.org/trunk/doc/py_tutorials/py_imgproc/py_grabcut/py_grabcut.html
For Random Walks, this is implemented in the scikit-image library and here's a great tutorial on how to get the segmentation up and running off of their official website: http://scikit-image.org/docs/dev/auto_examples/plot_random_walker_segmentation.html
Good luck!