Create openCV VideoCapture from interface name instead of camera numbers - python

The normal way to create a videocapture is this:
cam = cv2.VideoCapture(n)
where n corresponds to the number of /dev/video0, dev/video1
But because I'm building a robot that uses multiple cameras for different things, I needed to make sure that it was being assigned to the correct camera, I created udev rules that created devices with symbolic links to the correct port whenever a specific camera was plugged in.
They appear to be working because when I look in the /dev directory I can see the link:
/dev/front_cam -> video1
However I'm not sure how to actually use this now.
I thought I could just open it from the filename, as if it was a video, but cam = cv2.VideoCapture('/dev/front_cam') doesn't work.
Neither does cv2.VideoCapture('/dev/video1')
It doesn't throw an error, it does return a VideoCapture object, just not one that's opened (cam.isOpened() returns False).


Each of my video4linux devices creates 2 device nodes. For example, /dev/video0 and /dev/video1 are both related to my internal webcam. When I plug in a second USB webcam, /dev/video2 and /dev/video3 both appear. However, I can only use the lower-numbered device of each pair for video capture (i.e. /dev/video0 and /dev/video2).
I watched my device arrival with udevadm monitor, and then inspected each of the camera devices with udevadm info --path=$PATH_FROM_UDEVADM_MONITOR --attribute-walk. The devices which work for video capture have ATTR{index}=="0".
Maybe instead of trying to open /dev/video1, you just need to open /dev/video0:
cam = cv2.CaptureVideo("/dev/video0")


Instead of the suggested solution I found a shorter one, that feels a little bit hacky.
I just look at where the symbolic link points, find the integer in it, and then use that.
import subprocess
cmd = "readlink -f /dev/CAMC"
process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
# output of form /dev/videoX
out = process.communicate()[0]
# parse for ints
nums = [int(x) for x in out if x.isdigit()]
cap = cv2.VideoCapture(nums[0])


If you know the model of the camera, you can look it up in /dev/v4l/by-id/.... We are using an HDMI-USB video converter, and we connect to it like this:
#! /usr/bin/env python
import os
import re
import cv2
DEFAULT_CAMERA_NAME = '/dev/v4l/by-id/usb-AVerMedia_Technologies__Inc._Live_Gamer_Portable_2_Plus_5500114600612-video-index0'
device_num = 0
if os.path.exists(DEFAULT_CAMERA_NAME):
device_path = os.path.realpath(DEFAULT_CAMERA_NAME)
device_re = re.compile("\/dev\/video(\d+)")
info = device_re.match(device_path)
if info:
device_num = int(info.group(1))
print("Using default video capture device on /dev/video" + str(device_num))
cap = cv2.VideoCapture(device_num)
This follows the device name symlink to the /dev/video name, then parses that for the device number.


import re
import subprocess
import cv2
import os
device_re = re.compile("Bus\s+(?P<bus>\d+)\s+Device\s+(?P<device>\d+).+ID\s(?P<id>\w+:\w+)\s(?P<tag>.+)$", re.I)
df = subprocess.check_output("lsusb", shell=True)
for i in df.split('\n'):
if i:
info = device_re.match(i)
if info:
dinfo = info.groupdict()
if "Logitech, Inc. Webcam C270" in dinfo['tag']:
print "Camera found."
bus = dinfo['bus']
device = dinfo['device']
break
device_index = None
for file in os.listdir("/sys/class/video4linux"):
real_file = os.path.realpath("/sys/class/video4linux/" + file)
print real_file
print "/" + str(bus[-1]) + "-" + str(device[-1]) + "/"
if "/" + str(bus[-1]) + "-" + str(device[-1]) + "/" in real_file:
device_index = real_file[-1]
print "Hurray, device index is " + str(device_index)
camera = cv2.VideoCapture(int(device_index))
while True:
(grabbed, frame) = camera.read() # Grab the first frame
cv2.imshow("Camera", frame)
key = cv2.waitKey(1) & 0xFF
First search for desired string in USB devices list. Get BUS and DEVICE number.
Find symbolic link under video4linux directory. Extract device index from realpath and pass it to VideoCapture method.


A possibility that was not explored in the other answers is to use the "name" file in the /sys/class/video4linux/video*/ directories.
Example :
def get_camera(camera_name):
cam_num = None
for file in os.listdir("/sys/class/video4linux"):
real_file = os.path.realpath("/sys/class/video4linux/" + file + "/name")
with open(real_file, "rt") as name_file:
name = name_file.read().rstrip()
if camera_name in name:
cam_num = int(re.search("\d+$", file).group(0))
found = "FOUND!"
else:
found = " "
print("{} {} -> {}".format(found, file, name))
return cam_num
Which gives :
get_camera('HUE')
FOUND! video1 -> HUE HD Pro Camera: HUE HD Pro C
video0 -> HP HD Camera: HP HD Camera

Related

Python : how to have preview of DSLR camera

I came here as I am lost.
I have a code which basically provides a preview on screen before a picture is taken. The code runs on 2 camera (it doesn't make sense I know).
Pi camera : for preview
DSLR camera : for taking the pictures
The scripts works as expected. The issue is that the PIcamera zoom is not aligned with the DSLR.
I know it is possible to get the DSLR preview on screen which would be better. However I don't know how to do it. As I am new to python, I read as much as I could. I read solution with VLC but I didn't understand the specificities. I read the gphoto2 help but got lost in the technical parts. Hence I'd highly appreciate if someone can help me out or point me to the correct direction.
The DSLR camera is a Canon EOS D60 which should be compatible for screen preview.
I attached below the code
from picamera import PiCamera
import time
from time import sleep
import logging
import sys
import gphoto2 as gp
import locale
photonr = 1
countdown = 5
FolderRaw='/home/pi/'
def TakePicture():
global rafale
try:
rafale +=1
except Exception as e:
rafale =1
camera1 = PiCamera()
locale.setlocale(locale.LC_ALL, '')
logging.basicConfig(
format='%(levelname)s: %(name)s: %(message)s', level=logging.WARNING)
callback_obj = gp.check_result(gp.use_python_logging())
camera2 = gp.Camera()
camera2.init()
i = 1
while i < photonr+1:
preview = camera1.start_preview()
preview.fullscreen = True
picture_time = time.strftime("%Y-%m-%d")
global image
image = picture_time+ '_' + str(rafale) + '_' + str(i)
if i == 1 :
countdown2 = countdown +1
else :
countdown2 = 3
for counter in range(countdown2, 0, -1):
camera1.annotate_text = str(counter)
camera1.annotate_text_size = 120
sleep(1)
camera1.annotate_text = ''
print('Capturing image')
file_path = camera2.capture(gp.GP_CAPTURE_IMAGE)
print('Camera file path: {0}/{1}'.format(file_path.folder, file_path.name))
target = FolderRaw+image +'.jpg'
print('Copying image to', target)
camera_file = camera2.file_get(
file_path.folder, file_path.name, gp.GP_FILE_TYPE_NORMAL)
camera_file.save(target)
#camera.capture(FolderRaw+image +'.jpg')
camera1.stop_preview()
print ("Raw picture taken and saved on " + image )
i += 1
camera1.close()
camera2.exit()
TakePicture()
Thank you beforehand for any tips/direction/help

Python opencv open webcam faster

I have a product need to open multiple cameras(about 20) with each camera capturing one single image. However, the time to initiate each camera takes about 3-4 seconds, and having all these time spent in sequence is relatively long.
So the question is: Whether there is a way to open a usb camera faster, and whether there is a way to do it in a concurrent way.
Appreciate any advice.
Thanks to everyone! I have code attached in below.
def take_picture(camera_id):
cap = cv2.VideoCapture(camera_id, cv2.CAP_DSHOW)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
res = None
name = None
while(cap.isOpened()):
ret0, frame0 = cap.read()
if frame0.any():
real_location = 'station' + str(pos)
name = real_location + '-' + 'collection' + str(cursor) + '.png'
res = frame0
break
cap.release()
return [name, res]

Pytesseract really slow

so I'm trying to read out text from MS Teams and use that text to make inputs on the keyboard.
Right now, I work with the threading module to have one thread for the input and one thread for the image_to_string. Following is the function for the image_to_string.
def imToString():
global message
print("Image getting read")
pytesseract.pytesseract.tesseract_cmd ='C:\\Users\\gornicec\\AppData\\Local\\Programs\\Tesseract-OCR\\tesseract.exe'
while(True):
print("preIMGgrab")
cap = ImageGrab.grab(bbox=(177, 850, 283, 881))
grayCap = cv2.cvtColor(np.array(cap), cv2.COLOR_BGR2GRAY)
print("postIMGgrab")
t = time.perf_counter()
print("preMSG" + str(t))
message = pytesseract.image_to_string(
grayCap,
lang ='deu',config='--psm 6')
print(str(message) + "was read" + str(time.perf_counter() - t))
I don't know how but it takes about 8 seconds to read an image thats 1000 pixels big. I need this to be at highest 2 seconds. I'll add the whole code at the end. If there is any way to make it faster or to do it differently please tell me so.
WHOLE CODE:
import numpy as np
import time
import pytesseract
from win32gui import GetWindowText, GetForegroundWindow
import win32api
import cv2
import pyautogui
from PIL import ImageGrab
import threading
from ahk import AHK
import keyboard
message = ""
ahk = AHK(executable_path='C:\\Program Files\\AutoHotkey\\AutoHotkey.exe')
def Controls():
global message
while True:
booleanVal = True
if booleanVal:
#imToString()
print("message")
#print("rechts" in message.lower())
#print(f'LÄNGE: {len(message)}')
if "vorne" in message.lower():
# Control(message, 'w')
ahk.key_press('w')
#message = ""
if "hinten" in message.lower():
# Control(message, 's')
ahk.key_press('s')
#message = ""
if "links" in message.lower():
# Control(message, 'a')
ahk.key_press('a')
#message = ""
if "rechts" in message.lower():
# Control(message, 'd')
#print("HAHAHA")
ahk.key_press('d')
#message = ""
if "greif" in message.lower():
ahk.key_press('space')
#message = ""
time.sleep(0.5)
#IMGTOSTRING---
controls = threading.Thread(target=Controls)
controls.start()
grab = threading.Thread(target=imToString)
grab.start()

pytesseract is not suit for large amount of images or images that are already in memory, its write them to a file and then pass the file path to tesseract cli, if you want to improve the performance of you script try using library that works directly with tesseract api.
like this: https://pypi.org/project/tess-py-api/

Pepper Live 2-way Audio Streaming Error

I'm trying to establish a real-time audio communication between Pepper's tablet and my PC. I'm using Gstreamer to establish that. The audio from Pepper's mic to PC is working but there seems to be no audio from my PC to Pepper's tablet. What am I doing wrong?
PC side:
audio_pipeline = Gst.Pipeline('audio_pipeline')
audio_udpsrc = Gst.ElementFactory.make('udpsrc', None)
audio_udpsrc.set_property('port', args.audio)
audio_caps = Gst.caps_from_string('application/x-rtp,media=(string)audio, clock-rate=(int)44100, width=16, height=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96')
audio_filter = Gst.ElementFactory.make('capsfilter', None)
audio_filter.set_property('caps',audio_caps)
audio_depay = Gst.ElementFactory.make('rtpL16depay', None)
audio_convert = Gst.ElementFactory.make('audioconvert', None)
audio_sink = Gst.ElementFactory.make('alsasink', None)
audio_sink.set_property('sync',False)
audio_pipeline.add(audio_udpsrc,audio_filter,audio_depay,audio_convert,audio_sink)
audio_udpsrc.link(audio_filter)
audio_filter.link(audio_depay)
audio_depay.link(audio_convert)
audio_convert.link(audio_sink)
Robot side (Choregraphe):
audio_src = gst.element_factory_make('autoaudiosrc')
audio_convert = gst.element_factory_make('audioconvert')
audio_caps = gst.caps_from_string('audio/x-raw-int,channels=1,depth=16,width=16,rate=44100')
audio_filter = gst.element_factory_make('capsfilter')
audio_filter.set_property('caps',audio_caps)
# audio_enc = gst.element_factory_make('mad')
audio_pay = gst.element_factory_make('rtpL16pay')
audio_udp = gst.element_factory_make('udpsink')
audio_udp.set_property('host',user_ip)
audio_udp.set_property('port',int(user_audio_port))
self.audio_pipeline.add(audio_src,audio_convert,audio_filter,audio_pay,audio_udp)
gst.element_link_many(audio_src,audio_convert,audio_filter,audio_pay,audio_udp)
or
Robot's side (Python SDK):
GObject.threads_init()
Gst.init(None)
audio_pipeline = Gst.Pipeline('audio_pipeline')
audio_src = Gst.ElementFactory.make('autoaudiosrc')
audio_convert = Gst.ElementFactory.make('audioconvert')
audio_caps = Gst.ElementFactory.make('audio/x-raw-int,channels=2,depth=16,width=16,rate=44100')
audio_filter = Gst.ElementFactory.make('capsfilter')
audio_filter.set_property('caps',audio_caps)
audio_pay = Gst.ElementFactory.make('rtpL16pay')
audio_udp = Gst.ElementFactory.make('udpsink')
audio_udp.set_property('host',user_ip)
audio_udp.set_property('port',int(user_audio_port))
audio_pipeline.add(audio_src,audio_convert,audio_filter,audio_pay,audio_udp)
audio_src.link(audio_convert)
audio_convert.link(audio_filter)
audio_filter.link(audio_pay)
audio_pay.link(audio_udp)
audio_pipeline.set_state(Gst.State.PLAYING)
Computer's mic to Pepper:
audio_port = 80
s_audio_pipeline = Gst.Pipeline('s_audio_pipeline')
s_audio_src = Gst.ElementFactory.make('autoaudiosrc')
s_audio_convert = Gst.ElementFactory.make('audioconvert')
s_audio_caps = Gst.ElementFactory.make('audio/x-raw-int,channels=2,depth=16,width=16,rate=44100')
s_audio_filter = Gst.ElementFactory.make('capsfilter')
s_audio_filter.set_property('caps',audio_caps)
s_audio_pay = Gst.ElementFactory.make('rtpL16pay')
s_audio_udp = Gst.ElementFactory.make('udpsink')
s_audio_udp.set_property('host',ip)
s_audio_udp.set_property('port',int(audio_port))
s_audio_pipeline.add(s_audio_src,s_audio_convert,s_audio_filter,s_audio_pay,s_audio_udp)
s_audio_src.link(s_audio_convert)
s_audio_convert.link(s_audio_filter)
s_audio_filter.link(s_audio_pay)
s_audio_pay.link(s_audio_udp)
Pepper receiving:
audio = 80
r_audio_pipeline = Gst.Pipeline('r_audio_pipeline')
#defining audio pipeline attributes
r_audio_udpsrc = Gst.ElementFactory.make('udpsrc', None)
r_audio_udpsrc.set_property('port', audio)
r_audio_caps = Gst.caps_from_string('application/x-rtp,media=(string)audio, clock-rate=(int)44100, width=16, height=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)2, format=(string)S16LE, channel-positions=(int)1, payload=(int)96')
r_audio_filter = Gst.ElementFactory.make('capsfilter', None)
r_audio_filter.set_property('caps',r_audio_caps)
r_audio_depay = Gst.ElementFactory.make('rtpL16depay', None)
r_audio_convert = Gst.ElementFactory.make('audioconvert', None)
r_audio_sink = Gst.ElementFactory.make('alsasink', None)
r_audio_sink.set_property('sync',False)
#linking the various attributes
r_audio_pipeline.add(r_audio_udpsrc,r_audio_filter,r_audio_depay,r_audio_convert,r_audio_sink)
r_audio_udpsrc.link(r_audio_filter)
r_audio_filter.link(r_audio_depay)
r_audio_depay.link(r_audio_convert)
r_audio_convert.link(r_audio_sink)
r_audio_pipeline.set_state(Gst.State.PLAYING)
I think there might be a problem with the pepper's receiving port number... I tried different port numbers (including 9559) but nothing seemed to work. Is the source ID wrong?
Is it possible to run the 2-way stream in the same pipeline?
I took a look at other libraries like ffmpeg and PyAudio, but I couldn't any method for live streaming.

Make sure you run the Python script on the robot.
Also, did you run the GMainLoop ?
Choregraphe behaviors are run in NAOqi, and NAOqi runs a GMainLoop already in the background. Maybe this is what is missing in your stand-alone script.
Finally, you show no piece of code in your snippets that is meant to take the PC's audio to the network, nor from the network to Pepper's speakers.

Use decodebin with adder

I'm trying to create an audio stream that has a constant audio source (in this case, audiotestsrc) to which I can occasionally add sounds from files (of various formats, that's why I'm using decodebin) through the play_file() method. I use an adder for that purpose. However, for some reason, I cannot add the second sound correctly. Not only does the program play the sound incorrectly, it also completely stops the original audiotestsrc. Here's my code so far:
import gst; import gobject; gobject.threads_init()
pipe = gst.Pipeline()
adder = gst.element_factory_make("adder", "adder")
first_sink = adder.get_request_pad('sink%d')
pipe.add(adder)
test = gst.element_factory_make("audiotestsrc", "test")
test.set_property('freq', 100)
pipe.add(test)
testsrc = test.get_pad("src")
testsrc.link(first_sink)
output = gst.element_factory_make("alsasink", "output")
pipe.add(output)
adder.link(output)
pipe.set_state(gst.STATE_PLAYING)
raw_input('Press key to play sound')
def play_file(filename):
adder_sink = adder.get_request_pad('sink%d')
audiofile = gst.element_factory_make('filesrc', 'audiofile')
audiofile.set_property('location', filename)
decoder = gst.element_factory_make('decodebin', 'decoder')
def on_new_decoded_pad(element, pad, last):
pad.link(adder_sink)
decoder.connect('new-decoded-pad', on_new_decoded_pad)
pipe.add(audiofile)
pipe.add(decoder)
audiofile.link(decoder)
pipe.set_state(gst.STATE_PAUSED)
pipe.set_state(gst.STATE_PLAYING)
play_file('sample.wav')
while True:
pass

Thanks to moch on #gstreamer, I realized that all adder sources should have the same format. I modified the above script so as to have the caps "audio/x-raw-int, endianness=(int)1234, channels=(int)1, width=(int)16, depth=(int)16, signed=(boolean)true, rate=(int)11025" (example) go before every input in the adder.

Categories

Resources