Develop Reference

Python CV2 reads out-of-date frames from video stream

I am using Python 3.9 and Open-CV (cv2) to read frames from a video stream and save them as JPGs.
My program seems to run OK. It captures the video stream fine, obtains frames, and saves them as JPGs.
However, the frames it is obtaining from the stream are out-of-date - sometimes by several minutes. The clock in the video stream is running accurately, but the clock displays in the JPGs are all identical (to the second - but one or more minutes prior to the datetime in the program's "print()" output (and the saved JPG file time), and moving objects that were in view at the time they were saved are missing completely.
Strangely:
The JPG images are not identical in size. They grow by 10K - 20K as the sequence progresses. Even though they look identical to the eye, they show significant difference when compared using CV2 - but no difference if compared using PIL (which is about 10 - 15 times slower for image comparisons).
The camera can be configured to send a snapshot by email when it detects motion. These snapshots are up-to-date, and show moving objects that were in frame at the time (but no clock display). Enabling or disabling this facility has no effect on the out-of-date issue with JPGs extracted from the video stream. And, sadly, the snapshots are only about 60K, and too low resolution for our purposes (which is an AI application that needs images to be 600K or more).
The camera itself is ONVIF - and things like PTZ work nicely from Python code. Synology Surveillance Station works really well with it in every aspect. This model has reasonably good specs - zoom and good LPR anti-glare functionality. It is made in China - but I don't want to be 'a poor workman who blames his tools'.
Can anyone spot something in the program code that may be causing this?
Has anyone encountered this issue, and can suggest a work-around or different library / methodology?
(And if it is indeed an issue with this brand / model of camera, you are welcome to put in a plug for a mid-range LPR camera that works well for you in an application like this.)
Here is the current program code:
import datetime
from time import sleep
import cv2
goCapturedStream = None
# gcCameraLogin, gcCameraURL, & gcPhotoFolder are defined in the program, but omitted for simplicity / obfuscation.
def CaptureVideoStream():
global goCapturedStream
print(f"CaptureVideoStream({datetime.datetime.now()}): Capturing video stream...")
goCapturedStream = cv2.VideoCapture(f"rtsp://{gcCameraLogin}#{gcCameraURL}:554/stream0")
if not goCapturedStream.isOpened(): print(f"Error: Video Capture Stream was not opened.")
return
def TakePhotoFromVideoStream(pcPhotoName):
llResult = False ; laFrame = None
llResult, laFrame = goCapturedStream.read()
print(f"TakePhotoFromVideoStream({datetime.datetime.now()}): Result is {llResult}, Frame data type is {type(laFrame)}, Frame length is {len(laFrame)}")
if not ".jpg" in pcPhotoName.lower(): pcPhotoName += ".jpg"
lcFullPathName = f"{gcPhotoFolder}/{pcPhotoName}"
cv2.imwrite(lcFullPathName, laFrame)
def ReleaseVideoStream():
global goCapturedStream
goCapturedStream.release()
goCapturedStream = None
# Main Program: Obtain sequence of JPG images from captured video stream
CaptureVideoStream()
for N in range(1,7):
TakePhotoFromVideoStream(f"Test{N}.jpg")
sleep(2) # 2 seconds
ReleaseVideoStream()

Dan Masek's suggestions were very valuable.
The program (now enhanced significantly) saves up-to-date images correctly, when triggered by the camera's inbuilt motion detection (running in a separate thread and communicating through global variables).
The key tricks were:
A much faster loop reading the frames (and discarding most of them). I reduced the sleep to 0.1 (and even further to 0.01), and saved relatively few frames to JPG files only when required
Slowing down the frame rate on the camera (from 25 to 10 fps - even tried 5 at one point). This meant that the camera didn't get ahead of the software and send unpredictable frames.

Project problem, looking for advice about image processing

I'm a senior in high school and this year I have to do a project for my electronic class, I was hoping to get some advice from people with some experience.
My idea is kind of complicated and has a lot of different sensors but not too crazy, the problem begins with possible image processing. I have a camera who need to check for flashing light and send the video to a screen without the frames of the flashing (like just skipping the frame, so the video is always a frame in delay but the person won't notice it).
The fashing light is supposed to be like in a party or in a video game you get a warning on. The idea is to notice the extreme changing of lighting and to not show it on the screen.
My teacher is afraid that doing image processing might be too complicated and video processing as well... I don't have any knowledge in it, and I have a little background in Python and other languages, do you think it is possible? Can anyone give me an advice or a good video/tutorial to learn from?
Thank you in advance:)

your probleme if quite diificult, cause it envolved unknown environnement in a dynamic time range.
if you admit as an axiom that your camera has for exemple a frame rate of 20 FPS, the chances that your difference between Frame f' and next frame f+1 are quite low.
UNLESS you have a huge color change du to ligth flash,
So you can process with an image similarity such as ssim or psim
https://www.pyimagesearch.com/2017/06/19/image-difference-with-opencv-and-python/
if your image is over a certain treshold that you have to define ( can use also a kalmann filter to dynamically reajust the difference treshold)
so it will probably mean that your flash light is on.

Although it's a visual coding program (per se), Bonsai is a great open source software for doing what's in your description; as well, Bonsai supports applications that require combinations of different hardware (e.g. microcontrollers, cameras) and software components (e.g. Python).
To provide a similar application as an example, I have setup a workflow where Bonsai captures images sent from a Basler camera, it processes the input video frame-by-frame, and when it detects, within the cropped frame (that I cropped around an red LED), a threshold change in pixel intensity (i.e. the red LED turns ON or OFF), it sends an output signal (i.e. 5 volts) to an Arduino microcontroller while saving the image frame as a png file as well as a avi video file along with a vector of True/False (corresponding to the ON or OFF red LED frames) and corresponding timestamps that are saved as csv files, etc. Although this isn't identical to what you've described, I'm sure you can setup a similar Bonsai workflow to accomplish your goal.
Citation: https://www.frontiersin.org/articles/10.3389/fninf.2015.00007/full
Edit: I'm very familiar with Bonsai so if you need help with setting up a Bonsai workflow I'd be happy to help; I don't think there is direct message on StackOverFlow, but given that StackOverFlow doesn't list Bonsai as a programming language (because it's a visual programming language; or because it's not well known enough to include on StackOverFlow) feel free to reach out if you have any questions regarding Bonsai specifically (again, it's also an open source software).

Fast-Forward issue with saved video from PiCam

I'm working on a code, which reads incoming videos from Raspberry Pi, performs face detection on the frames, places frames around the faces, and then write backs the frames into an MP4 file with the same FPS. I use OpenCV to open and read from the PiCam.
When I looked into the saved video, it looks like it's moving too fast. I let my code to run for around 2 minutes, but my video has a length of 30 second. When I disable all post-processings (face detection), I can observe stable speed on the output video.
I can understand that Raspberry Pi has a small processor for heavy computations, but cannot understand why the video length is shorter? Is it possible that my face detection pipeline running much slower than the camera FPS, so the camera buffer should drop frames that are not going to be grabbed by the pipeline in a timely-fashion?
Any help here is highly appreciated!

Optimizing webcam picture rate python/cv2

I got a BRIO 4k Logitech Webcam for a research project deducing car velocity through pictures taken of traffic. It is working fine, I am currently trying to optimize the speed of taking pics with the highest possible resolution. I am using Logitech software to disable autofocus & set it to as far as possible (focus not changeable through cv2 as far as I know).
I put my current 3 program files here:
https://1drv.ms/f/s!AvdFsPpMX-gYrA9zo-r_W340B7b8
take_photos.py is the script you need to run, just put all three in a folder and create a subfolder "pics" and it should work.
func_cam.py includes functions that the main program is calling, params_cam.py some parameters like resolution (set to max throuch the 5k/5k input).
The output will be the time taken to yield each image. Here are some typical outputs I get:
1.657
1.587
1.033
0.927
0.719
0.573
0.689
0.508
1.097
1.516
1.409
1.767
So you see quite a difference (no change in position, light condition, no moving objects in the imaging area) in times.
How can I imporve image speeds, it is still to slow for my purposes (without reducing resolution)
The second challenge I am currently facing is the big differences of image rates within a run despite almost no changes in image area or setup. Maybe there is an idea on how to stabilize this (time differences above)?
I am looking forward to your help with these challenges I am facing! Thanks for this and all the best!

Can you slow down your USB bus?

I want to build a webcam based 3D scanner, since I'm going to use a lot of webcams I doing tests before.
I have orderer 3 exact camera that I will drive in python to take snapshot at the same time.
Obviously the bus is going to be saturated when there will be 50 of them.
What I want to know is if the camera are able to hold the picture until they are transfered to the computer.
To simulate this behavior I'd like to slow down the USB bus and make a snapshot with 3 camera,
I'm under windows 7 pro, is this possible?
Thanks.
PS : couldn't I saturate the USB BUS by pluggin some USB external harddrive and doing some file transfert?

What I want to know is if the camera are able to hold the picture until they are transfered to the computer.
That depends on the camera model, but since you mention in your post you are using "webcams", then the answer is almost certainly no. You could slow down the requests you make to the camera to take a picture though.
This sequence of events is possible:
wait
request camera takes picture
camera returns picture as normal
wait
This sequence of events is not possible (with webcams at least)
wait
request camera takes picture
wait
camera returns picture at a significantly later time that you want
to have control over
wait
If you need the functionality displayed in the last sequence I provide (a controllable time between capture and readout of the picture) you will need to upgrade to a better camera, such as a machine vision camera. These cameras usually cost considerably more than webcams and are unlikely to interface over USB (though you might find some that do).
You might be able to find some other solution to your problem (for instance what happens if you request 50 photos from 50 cameras nd saturate the USB bus? Do the webcams you have buffer the data well enough so that it achieves your ultimate goal, or does this affect the quality of the picture?)