Develop Reference

OpenCV : Optimized window size for cv.cornersubpix()

I am writing a code for calibrating cameras with Zhang's checkboard method and I am trying to find the best way to reduce the mean error while finding corners.
As I understand, having a window size as big as possible gives the best results but if it is too big (and more than one corner is found in the in the window) it completely false the results.
I would like to know if there is a better way than just trying all the combinations possible or asking a human to check that the corners have been correctly placed on all images.
Computational time is not a big issue here, it can take some time to get the best results
for the moment, I use the code found in the opencv tutorial (here giving me a window size of (23,23):
# termination criteria
criteria = (cv.TERM_CRITERIA_EPS + cv.TERM_CRITERIA_MAX_ITER, 30, 0.001)
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6*7,3), np.float32)
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)
# Arrays to store object points and image points from all the images.
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
images = glob.glob('*.jpg')
for fname in images:
img = cv.imread(fname)
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Find the chess board corners
ret, corners = cv.findChessboardCorners(gray, (7,6), None)
# If found, add object points, image points (after refining them)
if ret == True:
objpoints.append(objp)
corners2 = cv.cornerSubPix(gray,corners, (11,11), (-1,-1), criteria)
imgpoints.append(corners2)
# Draw and display the corners
cv.drawChessboardCorners(img, (7,6), corners2, ret)
cv.imshow('img', img)
cv.waitKey(500)
cv.destroyAllWindows()
My point is what do I have to change in this part, to have the best localization of the corner => essentially how to find the most appropriate winsize (here the (11,11))
corners2 = cv.cornerSubPix(gray,corners, (11,11), (-1,-1), criteria)
Thank you

Why is the performace of cv2.calibratecamera() decreasing drastically with more images?

I am using a camera calibration routine and I want to calibrate a camera with large set of images.
Code: (from here)
import numpy as np
import cv2
import glob
import argparse
# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
def calibrate():
height = 8
width = 10
""" Apply camera calibration operation for images in the given directory path. """
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(8,6,0)
objp = np.zeros((height*width, 3), np.float32)
objp[:, :2] = np.mgrid[0:width, 0:height].T.reshape(-1, 2)
# Arrays to store object points and image points from all the images.
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
# Get the images
images = glob.glob('thermal_final set/*.png')
# Iterate through the pairs and find chessboard corners. Add them to arrays
# If openCV can't find the corners in an image, we discard the image.
for fname in images:
img = cv2.imread(fname)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, corners = cv2.findChessboardCorners(gray, (width, height), None)
# If found, add object points, image points (after refining them)
if ret:
objpoints.append(objp)
corners2 = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
imgpoints.append(corners2)
# Draw and display the corners
# Show the image to see if pattern is found ! imshow function.
img = cv2.drawChessboardCorners(img, (width, height), corners2, ret)
e1 = cv2.getTickCount()
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)
e2 = cv2.getTickCount()
t = (e2 - e1) / cv2.getTickFrequency()
print(t)
return [ret, mtx, dist, rvecs, tvecs]
if __name__ == '__main__':
ret, mtx, dist, rvecs, tvecs = calibrate()
print("Calibration is finished. RMS: ", ret)
Now, the problem is that the time that cv2.calibratecamera() takes, based on number of points(derived from images) used.
Result with 40 images:
9.34462341234 seconds
Calibration is finished. RMS: 2.357820395255311
Result with 80 images:
66.378870749 seconds
Calibration is finished. RMS: 2.864052963156834
The time taken increases exponentially with increase in images.
Now, I have a really huge set of images (500).
I have tried calibrating camera with points from a single image and then calculating average of all the results I get, but they are different than what I get from this method.
Also, I am sure that my setup is using optimized OpenCV, check using:
print(cv2.useOptimized())
How do I make this process faster? Can I leverage threads here?
Edit: Updated the concept and language from "calibrating images" to "calibrating camera using images"

First, I strongly suspect the reason of your dramatic slowdown is memory related: you may be running out and starting to swap.
But the basic approach you seem to be following is incorrect. You don't calibrate images, you calibrate a camera, i.e. a lens + sensor combo.
Calibrating a camera means estimating the parameters of a mathematical model of that lens+sensor package. Therefore you only need use enough independent data points to achieve the desired level of accuracy in the parameter estimation.
A couple dozen well chosen images will be enough most of the time, provided you are following a well designed calibration procedure. Some time ago I wrote a few tips on how to do such a design, you can find them here.

Cut and save an object recognized by color

So i would like to make a program which can detect an object by color, position and sharpness.
Now I am there that I could detect the object by color and draw its contour and bounding box.
My problem is that i really dont know how to cut out the object from the picture and save it as picture file when the program recognise its contour or bounding box.
here's a picture of what my camera is seeing
input
output
I would like to cut out what is inside of the green colored boundig box as many times as fps in the video and as long as you can see it in the video. So if the video is 30 fps and the object is visible for 10 seconds it needs to take 300 pictures.
Here is the code:
i know it looks bad, im just trying to figure out what to use to make it work
import cv2 as cv
import numpy as np
import os
import uuid
cap = cv.VideoCapture(1)
font = cv.FONT_HERSHEY_COMPLEX
path = os.getcwd()
print(path)
def createFolder(directory):
try:
if not os.path.exists(directory):
os.makedirs(directory)
except OSError:
print('Error: Creating directory. ' + directory)
createFolder("./data")
# folderName = '%s' % (str(uuid.uuid4()))
while cap.isOpened():
_, frame = cap.read()
hsv = cv.cvtColor(frame, cv.COLOR_BGR2HSV)
# blue is the chosen one for now
lower_color = np.array([82, 33, 39])
upper_color = np.array([135, 206, 194])
mask = cv.inRange(hsv, lower_color, upper_color)
kernel = np.ones((5, 5), np.uint8)
mask = cv.erode(mask, kernel)
contours, hierarchy = cv.findContours(mask, cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)
# find contour
for contour in contours:
area = cv.contourArea(contour)
x, y, h, w = cv.boundingRect(contour)
if area > 100:
# bounding box
# cv.rectangle(frame, (x - 40, y - 30), (x + h * 3, y + w * 3), (0, 255, 0), 1)
# cutting and saving
ext_left = tuple(contour[contour[:, :, 0].argmin()][0] - 20)
ext_right = tuple(contour[contour[:, :, 0].argmax()][0] + 20)
ext_top = tuple(contour[contour[:, :, 1].argmin()][0] - 20)
ext_bot = tuple(contour[contour[:, :, 1].argmax()][0] + 20)
outfile = '%s.jpg' % (str(uuid.uuid4()))
cropped_image = frame[ext_top[1]:ext_bot[1], ext_left[0]:ext_right[0]]
# write images to a specified folder
cv.imwrite(os.path.join(path, "/data/", outfile), cropped_image)
# outputs
cv.imshow("Frame", frame)
cv.imshow("Mask", mask)
key = cv.waitKey(1)
if key == 27:
break
cap.release()
cv.destroyAllWindows()

Focusing on the question and ignoring the code style, I can say you are close to achieving your goal :)
For cropping the object, you can use the Mat copyTo method. Here is the official OpenCV documentation and here is an example from the OpenCV forums.
Now, for creating the mask from the contours, you can use the same drawCountours method you already use, but provide a negative value for the thickness parameters (for example, thickness=CV_FILLED). You can see a code snippet in this stackoverflow post and check details in the official documentation.
For saving the image to disk you can use imwrite.
So, in a nutshell, draw filled contours to a mask and use that mask to copy only the object pixels from the video frame to another mat that you can save the disk.
Instead of posting code, I will share this very similar question with an accepted answer that may have the code snippet you are looking for.

How to resize output images in python?

Hi i run this blurdetection code in python ( source : https://www.pyimagesearch.com/2015/09/07/blur-detection-with-opencv/ )
# import the necessary packages
from imutils import paths
import argparse
import cv2
def variance_of_laplacian(image):
# compute the Laplacian of the image and then return the focus
# measure, which is simply the variance of the Laplacian
return cv2.Laplacian(image, cv2.CV_64F).var()
# loop over the input images
for imagePath in paths.list_images("images/"):
# load the image, convert it to grayscale, and compute the
# focus measure of the image using the Variance of Laplacian
# method
image = cv2.imread(imagePath)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
fm = variance_of_laplacian(gray)
text = "Not Blurry"
# if the focus measure is less than the supplied threshold,
# then the image should be considered "blurry"
if fm < 100:
text = "Blurry"
# show the image
cv2.putText(image, "{}: {:.2f}".format(text, fm), (10, 30),
cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 0, 255), 3)
cv2.imshow("Image", image)
print("{}: {:.2f}".format(text, fm))
key = cv2.waitKey(0)
with this 2173 x 3161 input file
input image
and this is the output show
the output image
The image is zoom in and dont shown full.
In the source code, they use 450 x 600 px input image :
input in source code
and this is the output :
output in source code
I think the pixels of the image influences of the output. So, how can i get the output like the output in source code to all image?
do i have to resize the input image? How to? but if I do it I'm afraid it will affect the result of his blur

Excerpt from the DOCUMENTATION.
There is a special case where you can already create a window and load image to it later. In that case, you can specify whether window is resizable or not. It is done with the function cv2.namedWindow(). By default, the flag is cv2.WINDOW_AUTOSIZE. But if you specify flag to be cv2.WINDOW_NORMAL, you can resize window. It will be helpful when image is too large in dimension and adding track bar to windows.
I just used the code placed in the question but added line cv2.namedWindow("Image", cv2.WINDOW_NORMAL) as mentioned in the comments.
# import the necessary packages
from imutils import paths
import argparse
import cv2
def variance_of_laplacian(image):
# compute the Laplacian of the image and then return the focus
# measure, which is simply the variance of the Laplacian
return cv2.Laplacian(image, cv2.CV_64F).var()
# loop over the input images
for imagePath in paths.list_images("images/"):
# load the image, convert it to grayscale, and compute the
# focus measure of the image using the Variance of Laplacian
# method
image = cv2.imread(imagePath)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
fm = variance_of_laplacian(gray)
text = "Not Blurry"
# if the focus measure is less than the supplied threshold,
# then the image should be considered "blurry"
if fm < 100:
text = "Blurry"
# show the image
cv2.putText(image, "{}: {:.2f}".format(text, fm), (10, 30),
cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 0, 255), 3)
cv2.namedWindow("Image", cv2.WINDOW_NORMAL) #---- Added THIS line
cv2.imshow("Image", image)
print("{}: {:.2f}".format(text, fm))
key = cv2.waitKey(0)

In case you want to use the exact same resolution as the example you've given, you can just use the cv2.resize() https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#resize method or (in case you want to keep the ratio of the x/y coordinates) use the imutils class provided in https://www.pyimagesearch.com/2015/02/02/just-open-sourced-personal-imutils-package-series-opencv-convenience-functions/
You still have to decide if you want to do the resizing first. It shouldn't matter in which order you greyscale or resize.
Command you can add:
resized_image = cv2.resize(image, (450, 600))

Stereo Calibration Opencv Python and Disparity Map

I am interested in finding the disparity map of a scene. To start with, I did stereo calibration using the following code (I wrote it myself with a little help from Google, after failing to find any helpful tutorials for the same written in python for OpenCV 2.4.10).
I took images of a chessboard simultaneously on both cameras and saved them as left*.jpg and right*.jpg.
import numpy as np
import cv2
import glob
# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6*9,3), np.float32)
objp[:,:2] = np.mgrid[0:9,0:6].T.reshape(-1,2)
# Arrays to store object points and image points from all the images.
objpointsL = [] # 3d point in real world space
imgpointsL = [] # 2d points in image plane.
objpointsR = []
imgpointsR = []
images = glob.glob('left*.jpg')
for fname in images:
img = cv2.imread(fname)
grayL = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, cornersL = cv2.findChessboardCorners(grayL, (9,6),None)
# If found, add object points, image points (after refining them)
if ret == True:
objpointsL.append(objp)
cv2.cornerSubPix(grayL,cornersL,(11,11),(-1,-1),criteria)
imgpointsL.append(cornersL)
images = glob.glob('right*.jpg')
for fname in images:
img = cv2.imread(fname)
grayR = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, cornersR = cv2.findChessboardCorners(grayR, (9,6),None)
# If found, add object points, image points (after refining them)
if ret == True:
objpointsR.append(objp)
cv2.cornerSubPix(grayR,cornersR,(11,11),(-1,-1),criteria)
imgpointsR.append(cornersR)
retval,cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, R, T, E, F = cv2.stereoCalibrate(objpointsL, imgpointsL, imgpointsR, (320,240))
How do I rectify the images? What other steps should I do before going on to find the disparity map? I read somewhere that while calculating the disparity map, the features detected on both frames should lie on the same horizontal line. Please help me out here. Any help would be much appreciated.

you need cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2 and "newCameraMatrix" for cv2.undistort()
you can get "newCameraMatrix" using cv2.getOptimalNewCameraMatrix()
so in the remainder of your script paste this:
# Assuming you have left01.jpg and right01.jpg that you want to rectify
lFrame = cv2.imread('left01.jpg')
rFrame = cv2.imread('right01.jpg')
w, h = lFrame.shape[:2] # both frames should be of same shape
frames = [lFrame, rFrame]
# Params from camera calibration
camMats = [cameraMatrix1, cameraMatrix2]
distCoeffs = [distCoeffs1, distCoeffs2]
camSources = [0,1]
for src in camSources:
distCoeffs[src][0][4] = 0.0 # use only the first 2 values in distCoeffs
# The rectification process
newCams = [0,0]
roi = [0,0]
for src in camSources:
newCams[src], roi[src] = cv2.getOptimalNewCameraMatrix(cameraMatrix = camMats[src],
distCoeffs = distCoeffs[src],
imageSize = (w,h),
alpha = 0)
rectFrames = [0,0]
for src in camSources:
rectFrames[src] = cv2.undistort(frames[src],
camMats[src],
distCoeffs[src])
# See the results
view = np.hstack([frames[0], frames[1]])
rectView = np.hstack([rectFrames[0], rectFrames[1]])
cv2.imshow('view', view)
cv2.imshow('rectView', rectView)
# Wait indefinitely for any keypress
cv2.waitKey(0)
hope that gets you on your way to the next thing which might be calculating "disparity maps" ;)
Reference:
http://www.janeriksolem.net/2014/05/how-to-calibrate-camera-with-opencv-and.html

Try this code, I already have been able to solve the mistake:
retVal, cm1, dc1, cm2, dc2, r, t, e, f = cv2.stereoCalibrate(objpointsL, imgpointsL, imgpointsR, (320, 240), None, None,None,None)

firstly, use the opencv calibration application or matlab calibration toolbox to calculate your camera parameters. with the parameters, you can rectify your images.
after rectification, refer to the python sample in opencv's codebase (samples/python/stereo_match.py) to compute the disparity map.