Develop Reference

Replacing 3D submatrix in Numpy and OpenCV2

Replace sub part of matrix by another small matrix in numpy generally seems to work for my purposes but I'm running into something I can't reconcile. Consider the following code, that creates two 3D matrices the shape of OpenCV2 webcam input, (in my case (480, 640, 3)), one of all 1s (frame) and one of random floats (rgb_noise_mask), replaces a specified submatrix in frame with the same submatrix of rgb_noise_mask, and displays it to the screen. This code works as intended, displaying a block of RGB-based static on a field of white.
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
ret, frame = cap.read()
frame = np.ones(frame.shape)
rgb_noise_mask = np.random.random(size=frame.shape)
while True:
boxes = [[300,300,30,30]]
for box in boxes:
x, y, width, height = box
frame[y:y2, x:x2] = rgb_noise_mask[y:y2, x:x2]
cv2.imshow("frame", frame)
if cv2.waitKey(1) & 0xFF == ord("q"):
break
cap.release()
cv2.destroyAllWindows()
Now take off the training wheels and use the actual webcam input instead of faking it. That same box now appears as uniform black instead of the expected colors:
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
ret, frame = cap.read()
rgb_noise_mask = np.random.random(size=frame.shape)
while True:
ret, frame = cap.read()
boxes = [[300,300,30,30]]
for box in boxes:
x, y, width, height = box
frame[y:y2, x:x2] = rgb_noise_mask[y:y2, x:x2]
cv2.imshow("frame", frame)
if cv2.waitKey(1) & 0xFF == ord("q"):
break
cap.release()
cv2.destroyAllWindows()
Why is this, and how can I get around it? Further adding to my confusion is that if I replace frame[y:y2, x:x2] = rgb_noise_mask[y:y2, x:x2] with frame[y:y2, x:x2] = frame[y:y2, x:x2][::-1] in the second code sample it behaves as expected and displays the live output with that square mirrored.

The issue here is that ret, frame = cap.read() returns frame as a numpy array with dtype=np.uint8, while rgb_noise_mask is float between 0,1, so all 0 when converted to uint8.
A simple fix is to generate noise as integers with randint:
rgb_noise_mask = np.random.randint(0,256, size=frame.shape, dtype=np.uint8)

get frame from video with CV_CAP_PROP_POS_FRAMES in opencv python

I am trying to detect a (photography) flash in a video using OpenCV.
I detected the frame in which the flash occurs (average brightness above a threshold) and now I'd like to get the frame number.
I tried using CV_CAP_PROP_POS_FRAMES from the OpenCV docs without any success.
import numpy as np
import cv2
cap = cv2.VideoCapture('file.MOV')
while(cap.isOpened()):
ret, frame = cap.read()
BW = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)
average = np.average(v) #computes the average brightness
if average > 200: #flash is detected
cv2.imshow('frame',BW)
frameid = cap.get(CV_CAP_PROP_POS_FRAMES) # <--- this line does not work
print(frameid)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
Any tips ?

You can either:
Use cap.get(cv2.CAP_PROP_POS_FRAMES) (see here, also), or
increment a variable at each iteration: its current value is the current frame number

From opencv-doc:
When querying a property that is not supported by the backend used by the VideoCapture class, value 0 is returned
Probably it is not supported. In that case you have to count the frame number yourself.

In OpenCV I've got a mask of an area of a frame. How would I then insert another image into that location on the original frame?

I'm brand new to OpenCV and I can't seem to find a way to do this (It probably has to do with me not knowing any of the specific lingo).
I'm looping through the frames of a video and pulling out a mask from the video where it is green-screened using inRange. I'm looking for a way to then insert an image into that location on the original frame. The code i'm using to pull the frames/mask is below.
import numpy as np
import cv2
cap = cv2.VideoCapture('vid.mov')
image = cv2.imread('photo.jpg')
# green digitally added not much variance
lower = np.array([0, 250, 0])
upper = np.array([10, 260, 10])
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
cv2.imshow('frame', frame)
# get mask of green area
mask = cv2.inRange(frame, lower, upper)
cv2.imshow('mask', mask)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

Use Bitwise operations for masking and related binary operations. Please check below code to see how Bitwise operations are done.
Code
import numpy as np
import cv2
cap = cv2.VideoCapture('../video.mp4')
image = cv2.imread('photo.jpg')
# green digitally added not much variance
lower = np.array([0, 250, 0])
upper = np.array([10, 260, 10])
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
cv2.imshow('frame', frame)
# get mask of green area
mask = cv2.inRange(frame, lower, upper)
notMask = cv2.bitwise_not(mask)
imagePart=cv2.bitwise_and(image, image, mask = mask)
videoPart=cv2.bitwise_and(frame, frame, mask = notMask)
output = cv2.bitwise_or(imagePart, videoPart)
cv2.imshow('output', output)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
RGB bad color space
Since, you are doing color processing, I would suggest you to use appropriate color space. HSV would be a good start for you. For finding good range of HSV values, try this script.
Generating Video Output
You need a to create a video writer and once all image processing is done, write the processed frame to a new video. I am pretty sure, you cannot read and write to same video file.
Further see official docs here.
It has both examples of how to read and write video.

How do I display multiple webcam feeds side by side using OpenCV Beta 3.0.0 and Python?

I am working on a project that requires that I display 3 (and possibly more) webcam feeds side by side. To tackle this project, I am using OpenCV Beta 3.0.0 and Python 2.7.5 because I am slightly familiar with the language. Also, how do I display the video in color?
Here is my current code:
import cv2
import numpy as np
capture = cv2.VideoCapture(0)
capture1 = cv2.VideoCapture(1)
while True:
ret, frame = capture.read()
gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
cv2.imshow("frame",gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
capture.release()
cv2.destroyAllWindows()

import cv2
import numpy as np
capture = cv2.VideoCapture(0)
capture1 = cv2.VideoCapture(1)
while True:
_, frame1 = capture.read()
_, frame2 = capture1.read()
frame1 = cv2.cvtColor(frame1,cv2.COLOR_BGR2RGB)
frame2 = cv2.cvtColor(frame2,cv2.COLOR_BGR2RGB)
cv2.imshow("frame1",frame1)
cv2.imshow("frame2",frame2)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
capture1.release()
capture2.release()
cv2.destroyAllWindows()
To display color you simply don't convert to grayscale. To show two frames simultaneously just call imshow() twice. As for side by side, you can play with the frame positions if you really want. Also notice that the I converted the frames from BGR to RGB.

OpenCV - Trackbar slider keeps going to zero with video

I'm trying to use the slider to control my lower and upper bounds for HSV masking. I'm able to get the slider but can't get it to hold the position I set; it keeps going back to zero each time a new frame is pulled in.
import numpy as np
import cv2
def nothing(x):
pass
cap = cv2.VideoCapture(0)
while(True):
# Make a window for the video feed
cv2.namedWindow('frame',cv2.CV_WINDOW_AUTOSIZE)
# Capture frame-by-frame
ret, frame = cap.read()
# Make the trackbar used for HSV masking
cv2.createTrackbar('HSV','frame',0,255,nothing)
# Name the variable used for mask bounds
j = cv2.getTrackbarPos('HSV','image')
# Convert BGR to HSV
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
# define range of color in HSV
lower = np.array([j-10,100,100])
upper = np.array([j+10,255,255])
# Threshold the HSV image to get only selected color
mask = cv2.inRange(hsv, lower, upper)
# Bitwise-AND mask the original image
res = cv2.bitwise_and(frame,frame, mask= mask)
# Display the resulting frame
cv2.imshow('frame',res)
# Press q to quit
if cv2.waitKey(3) & 0xFF == ord('q'):
break
# When everything is done, release the capture
cap.release()
cv2.destroyAllWindows()

You are creating track-bar inside while loop, that's why you are getting new track-bar on each frame.
So change your code like,
# Make a window for the video feed
cv2.namedWindow('frame',cv2.CV_WINDOW_AUTOSIZE)
# Make the trackbar used for HSV masking
cv2.createTrackbar('HSV','frame',0,255,nothing)
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
........................
........................