I am using python 3.5 with opencv.
I want to use 2 threads:
Thread 1: Save the video to a file
Thread 2: Display the video to the user
To view/capture the video from webcam i am using snippets of code from the following website: opencv video docs
I can capture and save the video using the following code:
# Define the codec and create VideoWriter object
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))
while(True):
ret, frame = cap.read()
if ret==True:
frame = cv2.flip(frame,0)
# write the flipped frame
out.write(frame)
else:
break
out.release()
cv2.destroyAllWindows()
I can view the video using the following code:
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cv2.destroyAllWindows()
Each of these pieces of code are in their own functions called capture and display. I then call them in separate threads with pythons threading library as follows:
cap = cv2.VideoCapture(0)
Thread(target=capture).start()
Thread(target=display).start()
cap.release()
I get an error I assume is related to both threads wanting to access the video buffer at the same time.
I understand this can be done without threads but there are other things I would like to do further than can only be done in separate threads.
How can I access the cap video capture from both threads?
My flask/django experience is increadibly limited, so I am not sure how to do it for that exactly, but I will answer the question posted directly.
First you need to create a thread-safe object to avoid calling at the same time the read function in different threads.
import cv2
import threading
class VideoCamera(object):
# filename can be 0 to access the webcam
def __init__(self, filename):
self.lock = threading.Lock()
self.openVideo(filename)
def openVideo(self, filename):
self.lock.acquire()
self.videoCap = cv2.VideoCapture(filename)
self.lock.release()
With this, you should be able to create an object with a lock and to open safely a video (in case that you want to open another video with the same object).
Now you have 2 options, either you create a thread that updates the frame and stores the current one internally or update do in a thread safe manner the get next frame function. I will do the second one here to show you:
def getNextFrame(self):
self.lock.acquire()
img = None
# if no video opened return None
if self.videoCap.isOpened():
ret, img = self.videoCap.read()
self.lock.release()
return img
This way you should be able to access the video cap with 2 frames... however, the frames will be different every time the function is called.
I hope this helps you.
Related
I need a main while() loop, which updates the screenshot frame all the time, but when I get to a part of the code because the sync needs to be very precise, what I need is to create 2 threads or subprocess (I think using subprocess is better in this case).
One that keeps updating the frames and the other thread or subprocess that makes a delay of 3 seconds, only then to start working with the last frame that was updated (because of this delay it is so important to wait for the frames to be updated).
This is my code:
import multiprocessing
import time
import cv2
import numpy as np
#library for Optical Character Recognition (OCR)
import pytesseract #pip install pytesseract
pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract'
# Create a VideoCapture object
cap = cv2.VideoCapture(1)
# Check if camera opened successfully
if (cap.isOpened() == False):
print("Unable to read camera feed")
.
# We convert the resolutions from float to integer.
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
while(True):
ret, frame = cap.read()
if ret == True:
#HERE SHOULD BE THE FORK IN 2 INDEPENDENT PROCESSES
text = pytesseract.image_to_string(frame) #OCR in subprocess_2
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break #Close main loop
This is the flowchart of how the program should work. Showing how the subprocess_1 repeats updating the value of the variable frame until subprocess_2 finishes executing (in this particular case, in principle I plan to try 3 seconds delay).
I thought of using a separate function but I'm really having trouble implementing it. I would also like to know if it is possible to implement all frame updates in a single loop while() .
def handle_frame_requests(conn1):
try:
while True:
request = conn1.recv()
conn1.send(frame) # The frame must be pickle-able
except EOFError:
pass
def capture_cam(conn1):
global frame
frame = None
Thread(target=handle_frame_requests, args=(conn1,), daemon=True).start()
cap = cv2.VideoCapture(1) #the same webcam
if (cap.isOpened() == False):
print("Unable to read camera!")
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))
while(True):
ret, frame = cap.read() #here load the frame variable
if ret == True:
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'): break
else:
break
But regardless of how many nested while loops I use, the problem is that I can't get one process to keep updating the webcam, while another process keeps a timer, so that when the timer indicates it, both processes will exit and return to the webcam. main line, where the main while loop can continue to receive data from the webcam via the first loop while(True):
I am trying to create automatic attendance system with opencv2 in which i need to get rtsp stream from IP camera, find faces from it and recognize face.
I created different threads from frame catching and drawing because face recognition function needs some time to recognize face.
But just creating 2 threads, one for frame reading and other for drawing uses around 70% CPU.
and creating pytorch_facenet model increase usage 80-90% CPU.
does anyone know how to reduce CPU usage ?
my program:
import cv2
import threading
from facenet_pytorch import InceptionResnetV1
cap = cv2.VideoCapture("rtsp://test:Test12345#125.0.0.1")
resnet = InceptionResnetV1(pretrained='vggface2').eval()
ret, frame = cap.read()
exit = False
def th1():
global ret, frame, exit
while True:
ret, frame = cap.read()
if exit:
break
def th2():
global ret, frame, exit
while True:
cv2.imshow('frame', frame)
cv2.waitKey(1)
if cv2.getWindowProperty('frame',cv2.WND_PROP_VISIBLE) < 1:
exit = True
break
t1 = threading.Thread(target=th1)
t1.start()
t2 = threading.Thread(target=th2)
t2.start()
Update:
I used time.sleep(0.2) in my all threads except frame reading.
and it worked, my cpu usage is 30% now.
Two issues.
th2 runs in an almost-tight-loop. It won't consume a whole core of CPU because waitKey(1) sleeps for some time.
No synchronization at all between threads, but you need it. You need a threading.Event to notify the consumer thread of a fresh frame. The consumer thread must wait until a fresh frame is available, because it's pointless to display the same old frame again and again. You can be lazy and use waitKey(30) instead. For the displaying thread, that's good enough.
VideoCapture. You don't do any error checking at all! You must check:
cap = cv2.VideoCapture("rtsp://test:Test12345#125.0.0.1")
assert cap.isOpened()
...
and
while True:
ret, frame = cap.read()
if not ret:
break
...
This code works.
This first loop (thread) will be trying to read frames as fast as it can.
The frame can be updated 100 times per second or more, but it's too fast. Try to add time.sleep(0.03).
And in the second loop, you can change the waitKey() param to 30.
import time
def th1():
global ret, frame, exit
while True:
ret, frame = cap.read()
time.sleep(0.03)
if exit:
break
def th2():
global ret, frame, exit
while True:
cv2.imshow('frame', frame)
cv2.waitKey(30)
if cv2.getWindowProperty('frame',cv2.WND_PROP_VISIBLE) < 1:
exit = True
break
I'm writing a program to control the video playback speed as custom rate.
Is there is anyway to achieve that?
What code should be added to control the playback speed?
import cv2
cap = cv2.VideoCapture('video.mp4')
while(cap.isOpened()):
ret, frame = cap.read()
cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
In the docs it is stated:
Note
This function should be followed by waitKey function which displays
the image for specified milliseconds. Otherwise, it won’t display the
image. For example, waitKey(0) will display the window infinitely
until any keypress (it is suitable for image display). waitKey(25)
will display a frame for 25 ms, after which display will be
automatically closed. (If you put it in a loop to read videos, it will
display the video frame-by-frame)
In cv2.waitKey(X) function X means the number of milliseconds for an image to be displayed on the screen. In your case it is set to 1, so theoretically you are able to achieve 1000 fps (frames per seconds). But frame decoding takes time in VideoCapture object and limits your framerate. To change the playback speed you need to declare variable and use it as a parameter in waitKey function.
import cv2
cap = cv2.VideoCapture('video.mp4')
frameTime = 10 # time of each frame in ms, you can add logic to change this value.
while(cap.isOpened()):
ret, frame = cap.read()
cv2.imshow('frame',frame)
if cv2.waitKey(frameTime) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Alternatively, as frame decoding is the most time consuming task you can move it to the second thread and use a queue of decoded frames. See this link for details.
The third approach is to separate grabbing and decoding process and simply decode every nth frame. That will result in displaying only a subset of frames from the source video but from the user perspective the video will be played faster.
import cv2
cap = cv2.VideoCapture('video.mp4')
i=0 #frame counter
frameTime = 1 # time of each frame in ms, you can add logic to change this value.
while(cap.isOpened()):
ret = cap.grab() #grab frame
i=i+1 #increment counter
if i % 3 == 0: # display only one third of the frames, you can change this parameter according to your needs
ret, frame = cap.retrieve() #decode frame
cv2.imshow('frame',frame)
if cv2.waitKey(frameTime) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
You can use ffmpeg to speed up (or slow down) your video like fast forwarding by using "presentation time stamps".
For speed up an example would be:
ffmpeg -i YOUR_INPUT_MOVIE.mp4 -vf "setpts=0.20*PTS" YOUR_OUTPUT_MOVIE.mp4
which will speed up your movie by 5x.
For slow down an example would be:
ffmpeg -i YOUR_INPUT_MOVIE.mp4 -vf "setpts=5*PTS" YOUR_OUTPUT_MOVIE.mp4
which will slow down your movie by 5x.
Note: this method will drop frames.
Hi everyone and thanks for the help.
I've got this function to save a video from frames taken by my webcam.
import cv2
import multiprocessing
import threading
def rec():
# Define the codec and create VideoWriter object
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi',fourcc, 20.0, (640,480))
while(cap.isOpened()):
ret, frame = cap.read()
if ret==True:
out.write(frame)
else:
break
# Release everything if job is finished
cap.release()
out.release()
cv2.destroyAllWindows()
If i use it with threads, and so with this code, it works well:
s1 = threading.Thread(target=rec)
s1.start()
But if i want to start another process, using the following code, when i open the video it contains only black frames with some noise.
s1 = multiprocessing.Process(target=rec)
s1.start()
I searched all around but couldn't find any solution.
Also, i'm using Python 3.6
where is cap defined ? Try defining that in the function that you give to multiprocessing. If it is defined in the parent and is passed from the parent to the child, it is being pickled and that probably makes it unusable.
I solved the problem.
I was calling cap = cv2.VideoCapture(0) in my main and also in one of my imported modules, and that conflicted. I solved by calling it once.
I am using cv2.VideoCapture to read the frames of an RTSP video link in a python script. The .read() function is in a while loop which runs once every second, However, I do not get the most current frame from the stream. I get older frames and in this way my lag builds up. Is there anyway that I can get the most current frame and not older frames which have piped into the VideoCapture object?
I also faced the same problem. Seems that once the VideoCapture object is initialized it keeps storing the frames in some buffer of sort and returns a frame from that for every read operation. What I did is I initialized the VideoCapture object every time I wanted to read a frame and then released the stream. Following code captures 10 images at an interval of 10 seconds and stores them. Same can be done using while(True) in a loop.
for x in range(0,10):
cap = cv2.VideoCapture(0)
ret, frame = cap.read()
cv2.imwrite('test'+str(x)+'.png',frame)
cap.release()
time.sleep(10)
I've encountered the same problem and found a git repository of Azure samples for their computer vision service.
The relevant part is the Camera Capture module, specifically the Video Stream class.
You can see they've implemented a Queue that is being updated to keep only the latest frame:
def update(self):
try:
while True:
if self.stopped:
return
if not self.Q.full():
(grabbed, frame) = self.stream.read()
# if the `grabbed` boolean is `False`, then we have
# reached the end of the video file
if not grabbed:
self.stop()
return
self.Q.put(frame)
# Clean the queue to keep only the latest frame
while self.Q.qsize() > 1:
self.Q.get()
I'm working with a friend in a hack doing the same. We don't want to use all the frames. So far we found that very same thing: grab() (or read) tries to get you all the frames, and I guess with rtp: it will maintain a buffer and drop if you're not responsive enough.
Instead of read you can also use grab() and receive(). First one ask for the frame. Receives reads it into memory. So if you call grab several times it will effectively skip those.
We got away with doing this:
#show some initial image
while True:
cv2.grab()
if cv2.waitKey(10):
im = cv2.receive()
# process
cv2.imshow...
Not production code but...
Inside the 'while' you can use:
while True:
cap = cv2.VideoCapture()
urlDir = 'rtsp://ip:port/h264_ulaw.sdp'
cap.open(urlDir)
# get the current frame
_,frame = cap.read()
cap.release() #releasing camera
image = frame
Using the following was causing a lot of issues for me. The frames being passed to the function were not sequention.
cap = cv2.VideoCapture(0)
while True:
ret, frame = cap.read()
function_that_uses_frame(frame)
time.sleep(0.5)
The following also didn't work for me as suggested by other comments. I was STILL getting issues with taking the most recent frame.
cap = cv2.VideoCapture(0)
while True:
ret = capture.grab()
ret, frame = videocapture.retrieve()
function_that_uses_frame(frame)
time.sleep(0.5)
Finally, this worked but it's bloody filthy. I only need to grab a few frames per second, so it will do for the time being. For context, I was using the camera to generate some data for an ML model and my labels compared to what was being captured was out of sync.
while True:
ret = capture.grab()
ret, frame = videocapture.retrieve()
ret = capture.grab()
ret, frame = videocapture.retrieve()
function_that_uses_frame(frame)
time.sleep(0.5)
I made an adaptive system as the ones the others on here posted here still resulted in somewhat inaccurate frame representation and have completely variable results depending on the hardware.
from time import time
#...
cap = cv2.VideoCapture(url)
cap_fps = cap.get(cv2.CAP_PROP_FPS)
time_start = time()
time_end = time_start
while True:
time_difference = int((((end_time-start_time))*cap_fps)+1) #Note that the 1 might be changed to fit script bandwidth
for i in range(0, time_difference):
a = cap.grab()
_, frame = cap.read()
time_start = time()
#Put your code here
variable = function(frame)
#...
time_end = time()
This way the skipped frames adapt to the amount of frames missed in the video stream - allowing for a much smoother transition and a relatively real-time frame representation.