Develop Reference

python-onvif-zeep library - how to grab a frame?

I'm using an ONVIF IP security camera with the python-onvif-zeep library https://github.com/FalkTannhaeuser/python-onvif-zeep. Can anybody please provide a short, simple, and concise example that shows how to grab a frame?
The readme and the examples https://github.com/FalkTannhaeuser/python-onvif-zeep/tree/zeep/examples show how to set camera properties and some other things, but I can't find a working example that shows how to grab a frame.
For example, this is the simplest possible OpenCV program that grabs frames from the 1st attached webcam and shows them:
import cv2
def main():
vidCap = cv2.VideoCapture(0)
while True:
frameGrabSuccessful, openCvImage = vidCap.read()
cv2.imshow('openCvImage', openCvImage)
keyPress = cv2.waitKey(1)
if keyPress == 27 or keyPress == ord('q'):
break
# end if
# end while
# end function
if __name__ == '__main__':
main()
Can somebody please provide an equivalent example for the python-onvif-zeep library? i.e.:
from onvif import ONVIFCamera
def main():
camera = ONVIFCamera('192.168.1.100', 80, 'username', 'pass')
# what goes here ??
while True:
# what goes here ??
# end while
# end function
if __name__ == '__main__':
main()

Upon further investigation it seems python-onvif-zeep cannot be used for grabbing frames. It's necessary to use RTSP for that, see Access IP Camera in Python OpenCV

Real time OCR in python

The problem
Im trying to capture my desktop with OpenCV and have Tesseract OCR find text and set it as a variable, for example, if I was going to play a game and have the capturing frame over a resource amount, I want it to print that and use it. A perfect example of this is a video by Micheal Reeves
where whenever he loses health in a game it shows it and sends it to his Bluetooth enabled airsoft gun to shoot him. So far I have this:
# imports
from PIL import ImageGrab
from PIL import Image
import numpy as np
import pytesseract
import argparse
import cv2
import os
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter("output.avi", fourcc, 5.0, (1366, 768))
while(True):
x = 760
y = 968
ox = 50
oy = 22
# screen capture
img = ImageGrab.grab(bbox=(x, y, x + ox, y + oy))
img_np = np.array(img)
frame = cv2.cvtColor(img_np, cv2.COLOR_BGR2RGB)
cv2.imshow("Screen", frame)
out.write(frame)
if cv2.waitKey(1) == 0:
break
out.release()
cv2.destroyAllWindows()
it captures real-time and displays it in a window but I have no clue how to make it recognise the text every frame and output it.
any help?

It's fairly simple to grab the screen and pass it to tesseract for OCRing.
The PIL (pillow) library can grab the frames easily on MacOS and Windows. However, this feature has only recently been added for Linux, so the code below works around it not existing. (I'm on Ubuntu 19.10 and my Pillow does not support it).
Essentially the user starts the program with screen-region rectangle co-ordinates. The main loop continually grabs this area of the screen, feeding it to Tesseract. If Tesseract finds any non-whitespace text in that image, it is written to stdout.
Note that this is not a proper Real Time system. There is no guarantee of timeliness, each frame takes as long as it takes. Your machine might get 60 FPS or it might get 6. This will also be greatly influenced by the size of the rectangle your ask it to monitor.
#! /usr/bin/env python3
import sys
import pytesseract
from PIL import Image
# Import ImageGrab if possible, might fail on Linux
try:
from PIL import ImageGrab
use_grab = True
except Exception as ex:
# Some older versions of pillow don't support ImageGrab on Linux
# In which case we will use XLib
if ( sys.platform == 'linux' ):
from Xlib import display, X
use_grab = False
else:
raise ex
def screenGrab( rect ):
""" Given a rectangle, return a PIL Image of that part of the screen.
Handles a Linux installation with and older Pillow by falling-back
to using XLib """
global use_grab
x, y, width, height = rect
if ( use_grab ):
image = PIL.ImageGrab.grab( bbox=[ x, y, x+width, y+height ] )
else:
# ImageGrab can be missing under Linux
dsp = display.Display()
root = dsp.screen().root
raw_image = root.get_image( x, y, width, height, X.ZPixmap, 0xffffffff )
image = Image.frombuffer( "RGB", ( width, height ), raw_image.data, "raw", "BGRX", 0, 1 )
# DEBUG image.save( '/tmp/screen_grab.png', 'PNG' )
return image
### Do some rudimentary command line argument handling
### So the user can speicify the area of the screen to watch
if ( __name__ == "__main__" ):
EXE = sys.argv[0]
del( sys.argv[0] )
# EDIT: catch zero-args
if ( len( sys.argv ) != 4 or sys.argv[0] in ( '--help', '-h', '-?', '/?' ) ): # some minor help
sys.stderr.write( EXE + ": monitors section of screen for text\n" )
sys.stderr.write( EXE + ": Give x, y, width, height as arguments\n" )
sys.exit( 1 )
# TODO - add error checking
x = int( sys.argv[0] )
y = int( sys.argv[1] )
width = int( sys.argv[2] )
height = int( sys.argv[3] )
# Area of screen to monitor
screen_rect = [ x, y, width, height ]
print( EXE + ": watching " + str( screen_rect ) )
### Loop forever, monitoring the user-specified rectangle of the screen
while ( True ):
image = screenGrab( screen_rect ) # Grab the area of the screen
text = pytesseract.image_to_string( image ) # OCR the image
# IF the OCR found anything, write it to stdout.
text = text.strip()
if ( len( text ) > 0 ):
print( text )
This answer was cobbled together from various other answers on SO.
If you use this answer for anything regularly, it would be worth adding a rate-limiter to save some CPU. It could probably sleep for half a second every loop.

Tesseract is a single-use command-line application using files for input and output, meaning every OCR call creates a new process and initializes a new Tesseract engine, which includes reading multi-megabyte data files from disk. Its suitability as a real-time OCR engine will depend on the exact use case—more pixels requires more time—and which parameters are provided to tune the OCR engine. Some experimentation may ultimately be required to tune the engine to the exact scenario, but also expect the time required to OCR for a frame may exceed the frame time and a reduction in the frequency of OCR execution may be required, i.e. performing OCR at 10-20 FPS rather than 60+ FPS the game may be running at.
In my experience, a reasonably complex document in a 2200x1700px image can take anywhere from 0.5s to 2s using the english fast model with 4 cores (the default) on an aging CPU, however this "complex document" represents the worst-case scenario and makes no assumptions on the structure of the text being recognized. For many scenarios, such as extracting data from a game screen, assumptions can be made to implement a few optimizations and speed up OCR:
Reduce the size of the input image. When extracting specific information from the screen, crop the grabbed screen image as much as possible to only that information. If you're trying to extract a value like health, crop the image around just the health value.
Use the "fast" trained models to improve speed at the cost of accuracy. You can use the -l option to specify different models and the --testdata-dir option to specify the directory containing your model files. You can download multiple models and rename the files to "eng_fast.traineddata", "eng_best.traineddata", etc.
Use the --psm parameter to prevent page segmentation not required for your scenario. --psm 7 may be the best option for singular pieces of information, but play around with different values and find which works best.
Restrict the allowed character set if you know which characters will be used, such as if you're only looking for numerics, by changing the whitelist configuration value: -c tessedit_char_whitelist='1234567890'.
pytesseract is the best way to get started with implementing Tesseract, and the library can handle image input directly (although it saves the image to a file before passing to Tesseract) and pass the resulting text back using image_to_string(...).
import pytesseract
# Capture frame...
# If the frame requires cropping:
frame = frame[y:y + h, x:x + w]
# Perform OCR
text = pytesseract.image_to_string(frame, lang="eng_fast" config="--psm 7")
# Process the result
health = int(text)

Alright, I was having the same issue as you so I did some research into it and I'm sure that I found the solution! First, you will need these libraries:
cv2
pytesseract
Pillow(PIL)
numpy
Installation:
To install cv2, simply use this in a command line/command prompt: pip install opencv-python
Installing pytesseract is a little bit harder as you also need to pre-install Tesseract which is the program that actually does the ocr reading. First, follow this tutorial on how to install Tesseract. After that, in a command line/command prompt just use the command: pip install pytesseract
If you don't install this right you will get an error using the ocr
To install Pillow use the following command in a command-line/command prompt: python -m pip install --upgrade Pillow or python3 -m pip install --upgrade Pillow. The one that uses python works for me
To install NumPy, use the following command in a command-line/command prompt: pip install numpy. Thought it's usually already installed in most python libraries.
Code:
This code was made by me and as of right now it works how I want it to and similar to the effect that Michal had. It will take the top left of your screen, take a recorded image of it and show a window display of the image it's currently using OCR to read. Then in the console, it is printing out the text that it read on the screen.
# OCR Screen Scanner
# By Dornu Inene
# Libraries that you show have all installed
import cv2
import numpy as np
import pytesseract
# We only need the ImageGrab class from PIL
from PIL import ImageGrab
# Run forever unless you press Esc
while True:
# This instance will generate an image from
# the point of (115, 143) and (569, 283) in format of (x, y)
cap = ImageGrab.grab(bbox=(115, 143, 569, 283))
# For us to use cv2.imshow we need to convert the image into a numpy array
cap_arr = np.array(cap)
# This isn't really needed for getting the text from a window but
# It will show the image that it is reading it from
# cv2.imshow() shows a window display and it is using the image that we got
# use array as input to image
cv2.imshow("", cap_arr)
# Read the image that was grabbed from ImageGrab.grab using pytesseract.image_to_string
# This is the main thing that will collect the text information from that specific area of the window
text = pytesseract.image_to_string(cap)
# This just removes spaces from the beginning and ends of text
# and makes the the it reads more clean
text = text.strip()
# If any text was translated from the image, print it
if len(text) > 0:
print(text)
# This line will break the while loop when you press Esc
if cv2.waitKey(1) == 27:
break
# This will make sure all windows created from cv2 is destroyed
cv2.destroyAllWindows()
I hope this helped you with what you were looking for, it sure did help me!

I want to receive RTSP stream in OpenCV from web URL [duplicate]

I have recently set up a Raspberry Pi camera and am streaming the frames over RTSP. While it may not be completely necessary, here is the command I am using the broadcast the video:
raspivid -o - -t 0 -w 1280 -h 800 |cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/output.h264}' :demux=h264
This streams the video perfectly.
What I would now like to do is parse this stream with Python and read each frame individually. I would like to do some motion detection for surveillance purposes.
I am completely lost on where to start on this task. Can anyone point me to a good tutorial? If this is not achievable via Python, what tools/languages can I use to accomplish this?

Using the same method listed by "depu" worked perfectly for me.
I just replaced "video file" with "RTSP URL" of actual camera.
Example below worked on AXIS IP Camera.
(This was not working for a while in previous versions of OpenCV)
Works on OpenCV 3.4.1 Windows 10)
import cv2
cap = cv2.VideoCapture("rtsp://root:pass#192.168.0.91:554/axis-media/media.amp")
while(cap.isOpened()):
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()

Bit of a hacky solution, but you can use the VLC python bindings (you can install it with pip install python-vlc) and play the stream:
import vlc
player=vlc.MediaPlayer('rtsp://:8554/output.h264')
player.play()
Then take a snapshot every second or so:
while 1:
time.sleep(1)
player.video_take_snapshot(0, '.snapshot.tmp.png', 0, 0)
And then you can use SimpleCV or something for processing (just load the image file '.snapshot.tmp.png' into your processing library).

use opencv
video=cv2.VideoCapture("rtsp url")
and then you can capture framse. read openCV documentation visit: https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_gui/py_video_display/py_video_display.html

Depending on the stream type, you can probably take a look at this project for some ideas.
https://code.google.com/p/python-mjpeg-over-rtsp-client/
If you want to be mega-pro, you could use something like http://opencv.org/ (Python modules available I believe) for handling the motion detection.

Here is yet one more option.
It's much more complicated than the other answers.
But this way, with just one connection to the camera, you could "fork" the same stream simultaneously to several multiprocesses, to the screen, recast it into multicast, write it to disk, etc.
Of course, just in the case you would need something like that (otherwise you'd prefer the earlier answers)
Let's create two independent python programs:
Server program (rtsp connection, decoding) server.py
Client program (reads frames from shared memory) client.py
Server must be started before the client, i.e.
python3 server.py
And then in another terminal:
python3 client.py
Here is the code:
(1) server.py
import time
from valkka.core import *
# YUV => RGB interpolation to the small size is done each 1000 milliseconds and passed on to the shmem ringbuffer
image_interval=1000
# define rgb image dimensions
width =1920//4
height =1080//4
# posix shared memory: identification tag and size of the ring buffer
shmem_name ="cam_example"
shmem_buffers =10
shmem_filter =RGBShmemFrameFilter(shmem_name, shmem_buffers, width, height)
sws_filter =SwScaleFrameFilter("sws_filter", width, height, shmem_filter)
interval_filter =TimeIntervalFrameFilter("interval_filter", image_interval, sws_filter)
avthread =AVThread("avthread",interval_filter)
av_in_filter =avthread.getFrameFilter()
livethread =LiveThread("livethread")
ctx =LiveConnectionContext(LiveConnectionType_rtsp, "rtsp://user:password#192.168.x.x", 1, av_in_filter)
avthread.startCall()
livethread.startCall()
avthread.decodingOnCall()
livethread.registerStreamCall(ctx)
livethread.playStreamCall(ctx)
# all those threads are written in cpp and they are running in the
# background. Sleep for 20 seconds - or do something else while
# the cpp threads are running and streaming video
time.sleep(20)
# stop threads
livethread.stopCall()
avthread.stopCall()
print("bye")
(2) client.py
import cv2
from valkka.api2 import ShmemRGBClient
width =1920//4
height =1080//4
# This identifies posix shared memory - must be same as in the server side
shmem_name ="cam_example"
# Size of the shmem ringbuffer - must be same as in the server side
shmem_buffers =10
client=ShmemRGBClient(
name =shmem_name,
n_ringbuffer =shmem_buffers,
width =width,
height =height,
mstimeout =1000, # client timeouts if nothing has been received in 1000 milliseconds
verbose =False
)
while True:
index, isize = client.pull()
if (index==None):
print("timeout")
else:
data =client.shmem_list[index][0:isize]
img =data.reshape((height,width,3))
img =cv2.GaussianBlur(img, (21, 21), 0)
cv2.imshow("valkka_opencv_demo",img)
cv2.waitKey(1)
If you got interested, check out some more in https://elsampsa.github.io/valkka-examples/

Hi reading frames from video can be achieved using python and OpenCV . Below is the sample code. Works fine with python and opencv2 version.
import cv2
import os
#Below code will capture the video frames and will sve it a folder (in current working directory)
dirname = 'myfolder'
#video path
cap = cv2.VideoCapture("your rtsp url")
count = 0
while(cap.isOpened()):
ret, frame = cap.read()
if not ret:
break
else:
cv2.imshow('frame', frame)
#The received "frame" will be saved. Or you can manipulate "frame" as per your needs.
name = "rec_frame"+str(count)+".jpg"
cv2.imwrite(os.path.join(dirname,name), frame)
count += 1
if cv2.waitKey(20) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()

Use in this
cv2.VideoCapture("rtsp://username:password#IPAddress:PortNO(rest of the link after the IPAdress)").

Opencv imshow() freezes when updating

For my image processing algorithm I'm using python / OpenCV. The output of my algorithm shall be updated im the same window over and over again.
However sometimes the window freezes and doesn't update at all, but the algorithm is still running and updated the picture a multiple times in the meantime. The window turns dark gray on this Ubuntu machine.
Here is an excerpt of the involved code:
for i in range(0,1000):
img = loadNextImg()
procImg = processImg(img)
cv2.imshow("The result", procImg)
cv2.waitKey(1)
N.B.: processImg() takes about 1-2 s for its procedures. The line cv2.imshow(procImg) creates the window in first instance (i.e. there is no preceding invocation)

My suggestion is to use Matplotlib pyplot for displaying the image. I do it the following way.
import matplotlib.pyplot as plt
# load image using cv2....and do processing.
plt.imshow(cv2.cvtColor(image, cv2.BGR2RGB))
# as opencv loads in BGR format by default, we want to show it in RGB.
plt.show()
I know it does not solve the problem of cv2.imshow, but it solves our problem.

Increasing the wait time solves this issue. However in my opinion this is unnecessary time spent on sleeping (20 ms / frame), even though it's not much.
Changing
cv2.waitKey(1)
to
cv2.waitKey(20)
prevents the window from freezing in my case. The duration of this required waiting time may vary on different machines.

Just add cv2.destroyAllWindows() just after cv2.waitKey()

I have the very same issue and I noticed that the fps the window is updated is getting slower and slower until it freezes completely.
Increasing the waitKey(x) to something higher just extends the duration where the images are updated but when the time that cv2.imshow() needs to calculate exceeds the time from wait(Key) it just stops updating.
(Skip this complainment:)
I think the cv2.imshow() with waitKey() combination is a complete design error, why isn't imshow() just blocking until the UI is updated? That would make life so much easier without having to call waitKey() everytime...
P.S.: There is a possibility to start an own thread for opencv windows inside opencv:
import cv2
img = cv2.imread("image.jpg")
cv2.startWindowThread()
cv2.namedWindow("preview")
cv2.imshow("preview", img)
source: cv2.imshow command doesn't work properly in opencv-python
Well this doesn't work for me because I always get this errors when I run it:
(python3:1177): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
Attempt to unlock mutex that was not locked
Aborted
Maybe you could try it and report if it is working for you?
Edit:
Okay I solved the problem for me by creating a separate script imshow.py:
import cv2
import os.path
while True:
if os.path.exists("image.pgm"):
image = cv2.imread("image.pgm")
if not image is None and len(image) > 0:
cv2.imshow("Frame", image)
cv2.waitKey(20)
And I am writing the image out in my other program with: cv2.imwrite("image.pgm", image)
And I am calling the script like this:
import subprocess
subprocess.Popen(["python3", "imshow.py"])
Although this is creating some dirty reads sometimes it is sufficient enough for me, a better solution would be to use pipes or queues between the two processes.

So what I think is going on here is that the window,(an element of the highGUI) which is still active after the first call to imshow, is waiting for some sort of response from your waitKey function, but is becoming inactive since the program is stuck calculating in either the processImg of loadNextImg functions. If you don't care about a slight waste of efficiency (i.e. you're not running on an embedded system where every operation counts), you should just destroy the window after waitKey, and recreate before imshow. Since the window no longer exists during the time you are processing and loading images, the highGUI wont get stuck waiting for a call from waitKey, and it won't become unresponsive.

If your window is going grey then it might be take more processing power. So try to resize image into smaller size image and execute. Sometimes times it freezes while running in ipython notebooks due to pressing any key while performing operation. I had personally executed your problem but I didn't get grey screen while doing it. I did executing directly using terminal. Code and steps are shown below.
import argparse
import cv2
import numpy as np
# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image", required=True, help="Path to the image")
args = vars(ap.parse_args())
# load the image, grab its dimensions, and show it
image = cv2.imread(args["image"])
(h, w) = image.shape[:2]
cv2.imshow("Original", image)
cv2.waitKey(0)
for i in range(0,1000):
image = cv2.imread(args["image"])
cv2.imshow("The result",image);
cv2.waitKey(0)
Run it in terminal:
source activate env_name
python Filename.py --image Imagename.png
This will get to your result in one window only(updating each time) without freezing and if you want seperate image in every new window then add .format(i) as given below. But Remember to run in terminal only not in jupyter notebooks.
You can check using terminal commands in this video link
https://www.youtube.com/watch?v=8O-FW4Wm10s
for i in range(0,1000):
image = cv2.imread(args["image"])
cv2.imshow("The result{}".format(i),image);
cv2.waitKey(0)
This may help to get you 1000 images separately.

try:
import cv2
except:
print("You need to install Opencv \n Run this command \n pip install python-opencv")
exit()
print('Press q to quit frame')
def viewer(name,frame):
while True:
cv2.imshow(name,frame)
if cv2.waitKey(10) & 0xff ==ord('q'):
break
return
cv2.destroyWindow(name)
Save this program and from now onwards, import this and use the function viewer to display any frame/image and your display windows will not hang or crash.

Add the following two lines of code after cv2.imshow() function,
cv2.waitKey()
cv2.destroyAllWindows()

You can use while loop to take burst images without freezing. Here is an example for taking 10 images. You can also try to increase waitkey number and sleep time in while loop. This work for me.
key = cv2.waitKey(1)
webcam = cv2.VideoCapture(0)
sleep(1)
while True:
try:
check, frame = webcam.read()
cv2.imshow("Capturing", frame)
key = cv2.waitKey(1)
img_counter = 0
if key & 0xFF == ord('s'): #press s to take images
while img_counter < 10:
check, frame = webcam.read()
cv2.imshow("Capturing", frame)
key = cv2.waitKey(1)
path = 'F:/Projects/' #folder path to save burst images
img_name = "burst_{}.png".format(img_counter)
cv2.imwrite(os.path.join(path, img_name), img=frame)
print("Processing image...")
img_ = cv2.imread(img_name, cv2.IMREAD_ANYCOLOR) #save as RGB color format
print("{} written!".format(img_name))
img_counter += 1
sleep(0.2)
webcam.release()
cv2.destroyAllWindows()
break
elif key == ord('q'): #press q to quit without taking images
webcam.release()
cv2.destroyAllWindows()
break
except(KeyboardInterrupt):
print("Turning off camera.")
webcam.release()
print("Camera off.")
print("Program ended.")
cv2.destroyAllWindows()
break

This is an old thread but in case someone else encounters it, normally the issue happen when you update opencv/opencv-contrib versions with pip and still some of their dependencies are unmet (for example you might have numpy already installed so it wont reinstall it and this causes it to crash in the back).
Simply do
pip install opencv-python opencv-contrib-python --no-cache --force-reinstall
Version 4.5.2.52 is working fine on ubuntu 20.04 and 18.04 with python > 3.8

The correct manner to install OpenCV in RaspberryPi to use it with Python

So far I have followed this tutorial to install OpenCV in RaspberryPi for Python. It says simply to run:
sudo apt-get install libopencv-dev python-opencv
But I've looked around internet and I've found there are more libraries to install than that, as in these two similar tutorials:
1) http://denis.doublebuffer.net/lablog/2012/08/10/setting-everything-up-for-opencv-raspberry-pi/
2) http://eduardofv.com/read_post/185-Installing-OpenCV-on-the-Raspberry-Pi
Doubts about my incorrect installation of OpenCV are based on the delay in the visualisation of the sequence with this following simple code. It just gets the frames from the camera (RPi camera module) and shows them:
import cv2
cap = cv2.VideoCapture(0)
while True:
b,frame = cap.read()
cv2.imshow("frame", frame)
cv2.waitKey(1)
As I said above the visualization is flowing but delayed.

If you're able to import it, then I would guess that you've installed it correctly. If you're in doubt, you can install it from the source, although it takes a while on the Raspi.
Try this code:
import cv2
import cv2.cv as cv
import numpy
class test():
def __init__(self):
cv.NamedWindow("w1", cv.CV_WINDOW_NORMAL)
self.capture = cv.CreateCameraCapture(-1)
self.vid()
def vid(self):
while True:
self.frame = cv.QueryFrame(self.capture)
aframe = numpy.asarray(self.frame[:,:])
cv2.imshow("w1", aframe)
c = cv.WaitKey(5)
if c == 110:
exit()
p = test()
Also, see the answer to this question (assuming you're using the Raspicam)