What I'm trying to do.
Currently I am using PIL too grab a screen shot of my desktop and display the image in a constant loop. This produces a video like effect of screenshots. My intent is analyze each image taken with opencv and display the manipulated results.
What I'm currently doing.
At the moment I have a loop with a 1 millisecond delay taking a screenshot and then displaying it through opencv. Unfortunately grabbing and analyzing the image takes a while and produces the effect I am looking for, but at a lowered frame rate as it takes longer than a millisecond to grab the screenshot and display it.
What I'm looking for.
I'm hoping to asynchronously run this process so that concurrently I can get a smoother frame rate. Unfortunately when I attempt to use Process or threadingon my function I get a warning saying that the action must be taken from the main thread.
How may I be able to make the bellow code async/threaded?
def getImage(bbox):
while True:
#Grab pil image
pil_image = ImageGrab.grab(bbox=(bbox[0], bbox[1], bbox[2], bbox[3])).convert('RGB')
#conver the image to something opencv can use
image = cv2.cvtColor(numpy.array(pil_image), cv2.COLOR_RGB2BGR)
#show the image
cv2.imshow("image", image)
#wait a millisecond so that the image displays
cv2.waitKey(1)
Error:
*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'
Related
I am using Python 3.9 and Open-CV (cv2) to read frames from a video stream and save them as JPGs.
My program seems to run OK. It captures the video stream fine, obtains frames, and saves them as JPGs.
However, the frames it is obtaining from the stream are out-of-date - sometimes by several minutes. The clock in the video stream is running accurately, but the clock displays in the JPGs are all identical (to the second - but one or more minutes prior to the datetime in the program's "print()" output (and the saved JPG file time), and moving objects that were in view at the time they were saved are missing completely.
Strangely:
The JPG images are not identical in size. They grow by 10K - 20K as the sequence progresses. Even though they look identical to the eye, they show significant difference when compared using CV2 - but no difference if compared using PIL (which is about 10 - 15 times slower for image comparisons).
The camera can be configured to send a snapshot by email when it detects motion. These snapshots are up-to-date, and show moving objects that were in frame at the time (but no clock display). Enabling or disabling this facility has no effect on the out-of-date issue with JPGs extracted from the video stream. And, sadly, the snapshots are only about 60K, and too low resolution for our purposes (which is an AI application that needs images to be 600K or more).
The camera itself is ONVIF - and things like PTZ work nicely from Python code. Synology Surveillance Station works really well with it in every aspect. This model has reasonably good specs - zoom and good LPR anti-glare functionality. It is made in China - but I don't want to be 'a poor workman who blames his tools'.
Can anyone spot something in the program code that may be causing this?
Has anyone encountered this issue, and can suggest a work-around or different library / methodology?
(And if it is indeed an issue with this brand / model of camera, you are welcome to put in a plug for a mid-range LPR camera that works well for you in an application like this.)
Here is the current program code:
import datetime
from time import sleep
import cv2
goCapturedStream = None
# gcCameraLogin, gcCameraURL, & gcPhotoFolder are defined in the program, but omitted for simplicity / obfuscation.
def CaptureVideoStream():
global goCapturedStream
print(f"CaptureVideoStream({datetime.datetime.now()}): Capturing video stream...")
goCapturedStream = cv2.VideoCapture(f"rtsp://{gcCameraLogin}#{gcCameraURL}:554/stream0")
if not goCapturedStream.isOpened(): print(f"Error: Video Capture Stream was not opened.")
return
def TakePhotoFromVideoStream(pcPhotoName):
llResult = False ; laFrame = None
llResult, laFrame = goCapturedStream.read()
print(f"TakePhotoFromVideoStream({datetime.datetime.now()}): Result is {llResult}, Frame data type is {type(laFrame)}, Frame length is {len(laFrame)}")
if not ".jpg" in pcPhotoName.lower(): pcPhotoName += ".jpg"
lcFullPathName = f"{gcPhotoFolder}/{pcPhotoName}"
cv2.imwrite(lcFullPathName, laFrame)
def ReleaseVideoStream():
global goCapturedStream
goCapturedStream.release()
goCapturedStream = None
# Main Program: Obtain sequence of JPG images from captured video stream
CaptureVideoStream()
for N in range(1,7):
TakePhotoFromVideoStream(f"Test{N}.jpg")
sleep(2) # 2 seconds
ReleaseVideoStream()
Dan Masek's suggestions were very valuable.
The program (now enhanced significantly) saves up-to-date images correctly, when triggered by the camera's inbuilt motion detection (running in a separate thread and communicating through global variables).
The key tricks were:
A much faster loop reading the frames (and discarding most of them). I reduced the sleep to 0.1 (and even further to 0.01), and saved relatively few frames to JPG files only when required
Slowing down the frame rate on the camera (from 25 to 10 fps - even tried 5 at one point). This meant that the camera didn't get ahead of the software and send unpredictable frames.
I am trying to display 9 videos onto their own canvases using Tkinter. The problem I am running into is that the window is very laggy and unresponsive. This means nothing else can be done in the application whilst the videos are playing.
I have tried to use a second thread to perform steps 1-3 (as shown below), but this does not seem to help. I am struggling to see how I can make this process more efficient and any help would be much appreciated.
My process for doing this is as follows:
Read the video using cv2
import cv2
cap = cv2.VideoCapture(video_path)
Read a frame from the video
cap.set(1, frame_num)
ret, frame = cap.read()
Resize and convert to PhotoImage
frame = imutils.resize(frame, height=video_height, width=video_width)
photo = ImageTk.PhotoImage(image=Image.fromarray(frame))
Draw the image to canvas
im = self.create_image(0, 0, image=photo, anchor=NW)
This is done for all 9 videos, every 200ms.
The only solution for this is to buy a computer with more CPU cores / GPU / faster CPU, i'm afraid ! If the videos are displayed on a small canvas, you should resize them before playing them. That is, have a already resized video and play that instead of the original one. Reading frame by frame, then resizing and displaying on canvas the frame is very CPU expensive, already for one video, and you do it with 9 ! Thats why the GUI becomes laggy : Your CPU uses all its resources for resizing and playing, and has almost nothing left for responding to user events. A GPU would help there : it takes the part of resizing, which is the most CPU expensive.
Code:
clip = ImageSequenceClip(new_frames, fps=fps1)
clip.write_videofile("out.mp4", fps=fps1)
TL;DR:
This code produces a black screen video.
where fps1 is from the original video I stitch on
I am trying to stitch a video using frames from many videos.
I created an array containing all the images in their respective place and then passed frame by frame on each video and assigned the correct frame in the array. When I acted that way the result was ok, but the process was slow so I saved each frame to a file and loaded it within the stitching process. Python throw an exception that the array is to big and I chunked the video into parts and saved each chunk. The result came out as a black screen, even thought when I debugged I could show each frame on the ImageSequenceClip correctly. I tried reinstalling moviepy. I use windows 10 and I converted all frames to png type.
Well #BajMile was indeed right offering to use opencv.
What took me a while to realize is that I have to use only functions of opencv, also for the images I was opening and resizing.
I am trying to change the Frame rate i.e., FPS of an existing video using openCV library in python. Below is the code that I am trying to execute. Even after setting the FPS property using cv2.CAP_PROP_FPS the video is not playing faster in the cv2.imshow() method. Even After setting the FPS property the getter returns the older FPS value. So how do I set the FPS value higher and make the video play faster?
Used version:
python = 3.7.4 and
opencv-python - 4.1.0.25
import cv2
video = cv2.VideoCapture("yourVideoPath.mp4");
video.set(cv2.CAP_PROP_FPS, int(60))
if __name__ == '__main__':
print("Frame rate : {0}".format(video.get(cv2.CAP_PROP_FPS)))
while video.isOpened():
ret1, frame2 = video.read()
cv2.imshow("Changed", frame2)
if cv2.waitKey(10) & 0xFF == ord('q'): # press q to quit
break
video.release()
cv2.destroyAllWindows()
If you're only trying to play the video in the displayed window, the limiting factor is not the fps of the video but the time spent waiting with the code waitKey(10) which makes the program wait for 10ms between each frame.
The read() method of the VideoCapture class simply returns the next frame with no concept of waiting or frame rate. The only thing preventing this code running as fast as it can is the waitKey(10) section, which is thus the main factor determining speed. To change the frame rate as seen through the imshow() method, you'd need to edit the time spent waiting. This is likely the dominant factor, but not the only one as the reading of a frame does take time.
If you're actually trying to change the playback rate of an existing file and have that saved to that file, I am unsure if OpenCV actually supports this, and I imagine it would be dependent on what back end you're using - OpenCV implements the VideoCapture class using different 3rd party backends.. As per the documentation of VideoCapture.set() I'd investigate the return value of video.set(cv2.CAP_PROP_FPS, int(60)) as the documentation suggests it will return true if this has changed something.
As an alternative you could investigate using something like FFMPEG which supports this relatively easily. If you want to stick with OpenCV I know from personal experience you can do this with the VideoWriter class. In this method you would read in the video frame by frame using the VideoCapture class, and then save it at the desired frame rate with VideoWriter. I suspect FFMPEG will likely meet your needs however!
I am trying to use OpenCV to load a video file or access video stream from webcam but I'm unable to access all frames.
The video file is captured with the FPS of 60 but in my code I am only able to access a few frames per second. I tried using the threaded version of OpenCV, imutils. It works better but I still cannot access full frames.
In my code below, I use the threaded version video reader to load video file and resize it to smaller size to reduce the processing power required.
After the frame is grabbed successfully, I will so some image processing work (in the future). But now even with this boilerplate, I can at most read only a few (10+) frames and the results are stuttering. Is there a way to resolve this?
import cv2
from imutils import resize
from imutils.video import VideoStream
vs = VideoStream(src="./video.MOV").start()
while True:
frame = vs.read()
if frame is not None:
frame = resize(frame, 800)
# some heavy analytics work
cv2.imshow("Frame", frame)
if cv2.waitKey(1) & 0xFF == ord("q"):
break
cv2.destroyAllWindows()
vs.stop()
Experiment
I ran an experiment to calculate number of frames loaded and average time taken for each imshow function on both my iMac and an Ubuntu machine rocking Intel Core i5-7400 # 3.00Ghz with a 1080p monitor.
The video (h264) has a duration of 1:03 min and of size 185.7MB.
iMac can only load a total of 414 frames while the Ubuntu machine can load a total of 2826 frames.
Average time take for imshow function for both machine is 0.0003s
Basically, you just load the video and display, so there isn't any reason that you get low fps like that except that you are using a Macbook with Retina screen. Maybe it's the reason causes slow video display because the Retina screen has a lot more pixels, and it might take time for the compositing engine to render your image on the screen. I suggest to use an external screen.