I am trying to create a custom VR headset that displays a live feed from a remote camera, and in order for the view to be clear from the vr headset, I need to duplicate the image from the camera for both eyes and apply barrel distortion (see attached picture) to them to offset the distortion from the lenses. Duplicating the image should be simple, but I do not know how to apply the distortion.
Most of the solutions I've found online are built in to some sort of game engine or VR SDK, but I don't want to use a game engine since I'm only processing a raw camera feed.
I am planning on using OpenCV to do this and I'm hoping to get at least 30fps at 1080p (hardware is an NVIDIA Jetson Nano with a CSI camera). What would be the best way to go about doing this?
Related
I have a Canon VB-H45 and I know how to access the camera and take pictures with cv2, but I would like to be able to use rotation and zoom as well.
Is there a way to track down and nicely blur faces or part of face (like hair) for multiple 360 degree images via python opencv. ? I'am using Windows OS and python3.8
Two methods with opencv and python
Using a Gaussian blur to anonymize faces in images and video streams
Applying a “pixelated blur” effect to anonymize faces in images and video
The method is well explained here and you can access code.
Now, a more advance solution if you are using GPU, and you want to run the application on a live video stream.. its with nvidia DS and Deep Learning. The github here reports results on T4, i believe you should be able to run it on Jetson nano. Here is the link
Yes, there is. First you need to detect the face(s) using Haar-cascade, which will provide you the rectangle coordinates of the face location. Then you can use this answer to blur the desired portion of an image.
This is my first post here, so hello everyone.
I am working on a project that involves writing a program in c++ or python that will detect obstacles and will be used for AR.Drone 2.0. However, I don't know which approach should I take.
Initially, I was adviced to use opencv and optocal flow. I've found some videos and papers about it and one way is that: divide every frame from AR.Drone's camera on 2 (left/right side) or 4 (additionally up and bottom) and calculate optical flow for each part. Then, fly in the direction where the optical flow is less.
However I have some doubts about it:
1)Which method of optical flow calculation should I use? I know that in opencv there are provided methods for calculating both dense or sparse optical flow. Which one should I choose in this application? Won't dense optical flow be too slow to meet real-time requirements?
2)I guess that in time when UAV moves left-right or up-down I'll get some "fake" vectors caused by the movement of a drone and not because of the looming obstacle. How to prevent this?
Another solution I was told about a method shown here (link for paper in description) and someone who implemented it github link however the author admitted that he "never get obstacle detection working properly on the drone".
Another option I was told about is attaching a realsense camera to a drone and extract an information about the obstacle somehow using it.
So, my question is - which path should I take? Or is there some other method to do this that will work for application I described and is relatively easy to implement?
Thanks in advance for every reply.
I'm not sure the scope of your project, whether or not this is academic or professional, but my recommendation would be to use object detection of a control image with the camera facing directly forward on the drone. if the object is detected, you can estimate it's distance from your drone based on it's size. Since it is a control image it should have a constant size and you should record how many pixels across that is at various distances from your camera. This way you can build up a model. Once you know how far away the object is you can determine if it is an obstacle or not.
Once the detection becomes large enough, determine if it is in the flight path. Then you move the drone such that the coordinates of the detection box are no longer in your flight path.
For the detection, you can either use Google's detection api which comes with a number of solid detectors/classifiers, or if you are looking to add a layer of depth to the project you can train your own. PyImageSearch is a great place to start. And if you are feeling extra scientific you can dive right into Tensorflow.
Best of luck!
Try the open source project https://github.com/generalized-intelligence/GAAS
It uses stereo camera and SLAM to detect obstacles.
I am looking into image processing using an SJ4000 camera, linked up via USB to a Raspberry Pi (running Raspbian Jessie) for image processing with OpenCV in Python. I have achieved quite a bit using my webcam but now need to port it into the SJ4000's environment, however I am stuck at this hurdle.
The code I've used is identical to the answer to this question: rotated face detection.
On my laptop's webcam, I get a reasonably good framerate. When the SJ4000 is connected to my laptop via USB as well, I get a good framerate. However, on the Raspberry Pi, when I execute the same code, the image is just frozen for some reason. I then need to force quit the video viewer window which shows up as it's simply frozen.
EDIT 1: After closing the Spyder IDE and loading it up again a few times, and executing the same code, I can see a feed, but the framerate is very low (2-3 seconds per frame) and it will just freeze after some time.
EDIT 2: I've done further testing and find that when I include the face detection code, it takes a long time for the feed to be displayed as there is a TEN second delay. When I forward the feed live without any processing, it's very responsive.
How should I get around this? Is the only way getting a more powerful processor?
Thanks for any help!
Like others said, face detection is very computationally expensive using HOG/Haar descriptors. You won't be able to do real time face detection on the Raspberry Pi. On my Raspberry Pi 3, I can do human body detection on a 300x300 image at around 5 fps.
What I recommend is: Do motion detection. When motion is detected, start face detection.
Further optimization can be done by running face detection in its own thread, and have motion detection feed a FIFO of frames to be analyzed by face detector if motion is detected in a frame. That way, your face detector can operate asynchronously, and not hold up the main thread capturing the video frames, and doing motion detection.
I want to build a webcam based 3D scanner, since I'm going to use a lot of webcams I doing tests before.
I have orderer 3 exact camera that I will drive in python to take snapshot at the same time.
Obviously the bus is going to be saturated when there will be 50 of them.
What I want to know is if the camera are able to hold the picture until they are transfered to the computer.
To simulate this behavior I'd like to slow down the USB bus and make a snapshot with 3 camera,
I'm under windows 7 pro, is this possible?
Thanks.
PS : couldn't I saturate the USB BUS by pluggin some USB external harddrive and doing some file transfert?
What I want to know is if the camera are able to hold the picture until they are transfered to the computer.
That depends on the camera model, but since you mention in your post you are using "webcams", then the answer is almost certainly no. You could slow down the requests you make to the camera to take a picture though.
This sequence of events is possible:
wait
request camera takes picture
camera returns picture as normal
wait
This sequence of events is not possible (with webcams at least)
wait
request camera takes picture
wait
camera returns picture at a significantly later time that you want
to have control over
wait
If you need the functionality displayed in the last sequence I provide (a controllable time between capture and readout of the picture) you will need to upgrade to a better camera, such as a machine vision camera. These cameras usually cost considerably more than webcams and are unlikely to interface over USB (though you might find some that do).
You might be able to find some other solution to your problem (for instance what happens if you request 50 photos from 50 cameras nd saturate the USB bus? Do the webcams you have buffer the data well enough so that it achieves your ultimate goal, or does this affect the quality of the picture?)