I am trying to rapidly select and process different frames from a video using OpenCV Python. To select a frame, I have used the 'CAP_PROP_POS_FRAMES' (or cap.set(2, frame_no)). However when using this I noticed a delay of about 200 ms to decode the selected frame. My script will be jumping in between frames a lot (not necessarily chronological) which means this will cause a big delay between each iteration.
I suspected OpenCV is buffering the upcoming frames after I set the frame number. Therefore I tried pre-decoding of the video by basically putting the entire video as a list so it can be accessed from RAM. This worked fantastic except bigger videos completely eat up my memory.
I was hoping someone knows a way to either set the frame number without this 200ms delay or to decode the video without using all of my memory space. Any suggestions are also welcome!
I don't know how to avoid that 200ms delay, but I have a suggestion on how you could decode the video first even if its size is greater than your RAM. You could use numpy's memmap:
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.memmap.html.
In practice you could have a function that initializes this memory-mapped matrix and then iterate over each frame of the video using VideoCapture and then store each frame in this matrix. After that you will be able to jump between each frame just by accessing this memory-mapped matrix.
Related
I am working with a system that uses an Allied Vision Camera with Vimba Python.
Currently, I grab frames synchronously inside a loop, convert them into numpy arrays and append those to a list.
for _ in range(10):
frame = cam.get_frame()
img = np.ndarray(buffer=frame._buffer, dtype=np.uint16, shape=(frame._frame.height, frame._frame.width))
vTmpImg.append(img)
I need to optimize this process because it takes a significant amount of time. It would be ideal that the camera started streaming, taking frames and putting them in a queue or something and the I could retrieve them when I needed them. I figured that a good way to handle it is taking the frames asynchronously.
I've read the examples that Vimba has on asynchronous_grab, but it is still not clear to me how can I grab the frames that the camera is taking.
Does anyone know how to approach it?
Thanks in advance.
What is unclear about the asynchronous grab? The code or the concept?
Maybe asynchronous_grab_opencv.py is easier to modify. It transforms the frame into an OpenCV frame that can then be modified/saved etc in the Handler class. Basically, switch out the imshow command line for whatever you want to do with your frames.
I'm struggling with a real-time application I'm currently writing. I capture a webcam stream and apply multiple image processing algorithms to each individual frame, e.g. to get the emotion of a person in the frame and to detect objects in it.
Unfortunately, the algorithms have different runtimes and since some are based on neural networks, those in particular are slow.
My goal is, that I want to show a video stream without lags. I don't care if an image processing algorithm grabs only every n-th frame or shows the results with a delay.
To get rid of the lags, I put the image processing in different threads but I wonder if there is a more sophisticated way to synchronize my analysis on the video stream's frames - or maybe even a library that helps building pipelines for real time data analytics?
Every hint is welcome!
I am currently extracting frames from a video so I can add some bounding boxes in each frame. Then, I want to put all the frames together and make a new video, using openCV. The problem is that every time I want to do that, I have to extract thousand of frames first. Is there a way to do it without having to extract the frames?
Thanks
That assumption isn't correct. You can use OpenCV's VideoCapture to load a video, get a frame, do some processing on it and save it using a VideoWriter object - one frame at a time. There is no need to load all frames into memory and then store them all at once.
Code:
clip = ImageSequenceClip(new_frames, fps=fps1)
clip.write_videofile("out.mp4", fps=fps1)
TL;DR:
This code produces a black screen video.
where fps1 is from the original video I stitch on
I am trying to stitch a video using frames from many videos.
I created an array containing all the images in their respective place and then passed frame by frame on each video and assigned the correct frame in the array. When I acted that way the result was ok, but the process was slow so I saved each frame to a file and loaded it within the stitching process. Python throw an exception that the array is to big and I chunked the video into parts and saved each chunk. The result came out as a black screen, even thought when I debugged I could show each frame on the ImageSequenceClip correctly. I tried reinstalling moviepy. I use windows 10 and I converted all frames to png type.
Well #BajMile was indeed right offering to use opencv.
What took me a while to realize is that I have to use only functions of opencv, also for the images I was opening and resizing.
I wrote some image analysis in OpenCV's Python API. Images are acquired in real-time from a webcam with the read() function of a cv2.VideoCapture object within a while True loop.
Processing a frame takes about 100ms. My camera would be capable of providing 30 fps. But if I even try to set its FPS to 15 my slow processing will lead to increasing lag. The processing happens on frames that get older and older in relation to "now". I can only run in real-time if I set the FPS to 5 which is a bit low. I assume incoming frames are buffered and once my loop returns to the start, the next frame is read from that buffer instead of straight from the camera.
I read elsewhere that running the frame grabbing and the processing in seperate threads would be the solution but I never used threading. Maybe I could get the most recent frame from the buffer instead?
I am using Python 3. I would prefer a OpenCV 3 answer if that is relevant, will accept a OpenCV 2 solution happily too.