Develop Reference

Python + OpenCV + Webcam Calibration: Doesn't detect full pattern

i am trying to undistort a photo of a laptop screen in order to learn how camera calibration and undistortion with opencv + python work.
On the basis of a camera matrix that i obtained from one photo with controlled content on the screen, i would like to undistort subsequent images.
Neither the camera, nor the display will move, so i think i need only one image for calibration (or is this assumption already completely wrong?)
My first image with controlled content is this 24x13 chessboard. The camera's distortion is nicely visible in the corners of the photo:
This is the script that i use for calibration and undistortion:
import numpy as np
import cv2
img = cv2.imread("img.png")
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
def chessboard_objectpoints(boxes_x, boxes_y):
objp = np.zeros(((boxes_x - 1) * (boxes_y - 1), 3), np.float32)
objp[:, :2] = np.mgrid[0:boxes_x-1, 0:boxes_y-1].T.reshape(-1, 2)
return objp
def chessboard_imagepoints(img_gray, boxes_x, boxes_y, out_img=None):
boxdim = (boxes_x - 1, boxes_y - 1)
ret, corners = cv2.findChessboardCorners(img_gray, boxdim, None)
assert ret
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
corners2 = cv2.cornerSubPix(img_gray, corners, (11, 11), (-1, -1), criteria)
if out_img is not None:
out_img = cv2.drawChessboardCorners(out_img, boxdim, corners2, ret)
return corners2
BOXES_X = 23 # should be 24
BOXES_Y = 12 # should be 13
obj_points = chessboard_objectpoints(BOXES_X, BOXES_Y)
img_points = chessboard_imagepoints(img_gray, BOXES_X, BOXES_Y, img)
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(
[obj_points], [img_points], img_gray.shape[::-1], None, None
)
h, w = img.shape[:2]
dimension = w, h
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(mtx, dist, dimension, 0)
dst = cv2.undistort(img, mtx, dist)
x, y, w, h = roi
dst = dst[y : y + h, x : x + w]
cv2.imshow("img", dst)
cv2.waitKey()
The resulting image of this script is the following, and it is indeed less distorted than the original:
Now i have 2 questions:
Why can cv2.findChessboardCorners only find a subset of the corners, see image and sourcecode? I expect it to be able to find a pattern of 23x12 corners, but that won't work.
with the smaller subset of the chessboard, there is a lot of remaining distortion, see image. It also doesn't look like the full pattern would really help, then. How do i undistort this kind of image completely?

Undistortion crops the image and distort edges

I’m trying to calibrate and undistort image from fish-eye camera.
My code is:
import cv2
import os
import numpy as np
CHECKERBOARD = (5,7)
subpix_criteria = (cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 0.1)
calibration_flags = cv2.fisheye.CALIB_RECOMPUTE_EXTRINSIC+cv2.fisheye.CALIB_CHECK_COND+cv2.fisheye.CALIB_FIX_SKEW
R = np.zeros((1, 1, 3), dtype=np.float64)
T = np.zeros((1, 1, 3), dtype=np.float64)
objp = np.zeros( (CHECKERBOARD[0]*CHECKERBOARD[1], 1, 3) , np.float64)
objp[:,0, :2] = np.mgrid[0:CHECKERBOARD[0], 0:CHECKERBOARD[1]].T.reshape(-1, 2)
_img_shape = None
objpoints = [] # 3d point in real world space
imgpoints = []
N_OK = len(objpoints)
images = os.listdir('./images/')
for fname in images:
img = cv2.imread(fname)
img = cv2.imread('./images/'+fname)
#print(fname + str(os.path.exists('./images/'+fname)))
ext = os.path.splitext(fname)[-1].lower()
if ext == ".jpg":
print(img.shape[:2])
if _img_shape == None:
_img_shape = img.shape[:2]
else:
assert _img_shape == img.shape[:2], "All images must share the same size."
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
ret, corners = cv2.findChessboardCorners(gray, CHECKERBOARD,cv2.CALIB_CB_ADAPTIVE_THRESH+cv2.CALIB_CB_FAST_CHECK+cv2.CALIB_CB_NORMALIZE_IMAGE)
if ret == True:
objpoints.append(objp)
cv2.cornerSubPix(gray,corners,(3,3),(-1,-1),subpix_criteria)
imgpoints.append(corners)
N_OK = len(objpoints)
K = np.zeros((3, 3))
D = np.zeros((4, 1))
rvecs = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(N_OK)]
tvecs = [np.zeros((1, 1, 3), dtype=np.float64) for i in range(N_OK)]
rms, K, D, rvecs, tvecs = \
cv2.fisheye.calibrate(
objpoints,
imgpoints,
gray.shape[::-1],
K,
D,
rvecs,
tvecs,
calibration_flags,
(cv2.TERM_CRITERIA_EPS+cv2.TERM_CRITERIA_MAX_ITER, 30, 1e-6)
)
DIM=_img_shape[::-1]
K=np.array(K.tolist())
D=np.array(D.tolist())
And function to undistort:
def undistort(img_path):
img = cv2.imread(img_path)
h,w = img.shape[:2]
nk = K.copy()
map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), K, DIM, cv2.CV_16SC2)
undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)
cv2.imwrite('calibresult1.png',undistorted_img)
It gives following image:
undistorted image
While original image is:
original image
The center seems to be undistorted, but corners are distorted and image itself is cropped.
I'm not sure that the calibration process is correct. If anyone has experience with it, I would be happy if you look at the code and probably find errors.

Fast answer: bad calibration. Your undistorted image was obtained with (very) wrong intrinsics and distortion coefficients.
Sorry I'm not debugging your code, but code can be fine and still not getting the right calibration. Not everything is code: you must choose useful chessboard poses and many images to improve calibration.
Recommendation: with fisheye calibration, let start trying to get only intrinsics (camera center and focal point), and avoid computing distortion coefficients.

Opencv camera calibration use for world to view port mapping

I am making an application in image processing where I would like to map the view port coordinates to world window co ordinates I am using python and has obtain transformation rotation matrices and distortion coefficients in opencv cpp documentation it is written that camera callibration can be use for this purpose but I can't able to find how any one can help?

for camera calibration, you have to print chess sheet first from this link : chess for print
then use these codes for calibration as follows:
1) Take different position of chess sheet in front of camera, this code open camera and detect chess sheet. After several detection of chess sheet, this calculates coefficients matrix to undistortion of camera image.
import numpy as np
import cv2
objp = np.zeros((6 * 7, 3), np.float32)
objp[ : , : 2] = np.mgrid[0 : 7, 0 : 6].T.reshape(-1, 2)
objpoints = []
imgpoints = []
criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
cam = cv2.VideoCapture(0)
(w, h) = (int(cam.get(4)), int(cam.get(3)))
while(True):
_ , frame = cam.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
ret, corners = cv2.findChessboardCorners(gray, (7, 6), None)
if ret == True:
objpoints.append(objp)
corners = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
imgpoints.append(corners)
cv2.drawChessboardCorners(frame, (7, 6), corners, ret)
cv2.imshow('Find Chessboard', frame)
cv2.waitKey(0)
cv2.imshow('Find Chessboard', frame)
print "Number of chess boards find:", len(imgpoints)
if cv2.waitKey(1) == 27:
break
ret, oldMtx, coef, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints,
gray.shape[: : -1], None, None)
newMtx, roi = cv2.getOptimalNewCameraMatrix(oldMtx, coef, (w, h), 1, (w, h))
print "Original Camera Matrix:\n", oldMtx
print "Optimal Camera Matrix:\n", newMtx
np.save("Original camera matrix", oldMtx)
np.save("Distortion coefficients", coef)
np.save("Optimal camera matrix", newMtx)
cam.release()
cv2.destroyAllWindows()
2) Then load matrices and apply undistort function to correct camera images with below sample code:
import numpy as np
import cv2
oldMtx = np.load("Original camera matrix.npy")
coef = np.load("Distortion coefficients.npy")
newMtx = np.load("Optimal camera matrix.npy")
cam = cv2.VideoCapture(0)
(w, h) = (int(cam.get(4)), int(cam.get(3)))
while(True):
_ , frame = cam.read()
undis = cv2.undistort(frame, oldMtx, coef, newMtx)
cv2.imshow("Original vs Undistortion", np.hstack([frame, undis]))
key = cv2.waitKey(1)
if key == 27:
break
cam.release()
cv2.destroyAllWindows()
these codes work for me. good luck

Assertion failed when use opencv in python

I am doing Camera Calibration in opencv in python and I followed the tutorials on this page. My code is completely copied from the page with tiny adjustment on the parameters.
Code:
import numpy as np
import cv2
import glob
# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6*7,3), np.float32)
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)
# Arrays to store object points and image points from all the images.
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
images = glob.glob('../easyimgs/*.jpg')
print('...loading')
for fname in images:
print(f'processing img:{fname}')
img = cv2.imread(fname)
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
print('grayed')
# Find the chess board corners
ret, corners = cv2.findChessboardCorners(gray, (8, 11),None)
# If found, add object points, image points (after refining them)
if ret == True:
print('chessboard detected')
objpoints.append(objp)
corners2 = cv2.cornerSubPix(gray,corners,(11,11),(-1,-1),criteria)
imgpoints.append(corners2)
# Draw and display the corners
img = cv2.drawChessboardCorners(img, (8,11), corners2,ret)
cv2.namedWindow('img',0)
cv2.resizeWindow('img', 500, 500)
cv2.imshow('img',img)
cv2.waitKey(500)
cv2.destroyAllWindows()
img2 = cv2.imread("../easyimgs/5.jpg")
print(f"type objpoints:{objpoints[0].shape}")
print(f"type imgpoints:{imgpoints[0].shape}")
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None)
h, w = img2.shape[:2]
newcameramtx, roi=cv2.getOptimalNewCameraMatrix(mtx,dist,(w,h),1,(w,h))
dst = cv2.undistort(img2, mtx, dist, None, newcameramtx)
# crop the image
x,y,w,h = roi
dst = dst[y:y+h, x:x+w]
cv2.namedWindow('result', 0)
cv2.resizeWindow('result', 400, 400)
cv2.imshow('result',dst)
cv2.destroyAllWindows()
but when I run it, an error shows up as:
Traceback (most recent call last):
File "*/undistortion.py", line 51, in <module>
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None)
cv2.error: OpenCV(3.4.2) C:\projects\opencv-python\opencv\modules\calib3d\src\calibration.cpp:3143: error: (-215:Assertion failed) ni == ni1 in function 'cv::collectCalibrationData'
Here is my image.
I have searched on the Internet that many people are also confronted with this problem. But most of the solution on blog is saying that this is caused by the type of the first and second parameters of calibrateCamera() which is objpoints and imgpoints. But those are all solution for opencv on c++.
Could any one tell me how to solve it in python?

The number of entries in objpoints and imgpoints must be the same. This assertion means they aren't. It looks like you're creating a set of 6*7=42 objpoints, which is intended for a chessboard of 6x7 intersections, but your actual chessboard has 8*11=88 intersections. So as you process images, your objpoints and imgpoints lists get different lengths. You need to modify your creation/initialization of objp to have 8*11=88 points, whose coordinates correspond to the real physical coordinates on your chessboard.
To do this, you'll need to really understand the code you are using. Putting in more debug statements will help you to trace through what is happening in your code.
And note that the Python API to OpenCV is just a wrapper around the C++ API, so any solution for using OpenCV with C++ are (usually) relevant in Python too.

Raspberry pi OpenCV error: (-215) ni == ni1 in function collectCalibrationData

I'm running python 3 on a raspberry pi 3 and have opencv installed. I took 10 images of a checkerboard, it detects all 10 images and displays them, but when it gets to the last line, it throws an error. Here's the images i used: https://imgur.com/gallery/IDfHH This is my code:
import numpy as np
import cv2
import glob
# termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6*7,3), np.float32)
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)
# Arrays to store object points and image points from all the images.
objpoints = [] # 3d point in real world space
imgpoints = [] # 2d points in image plane.
images = glob.glob('*.jpg')
for fname in images:
print('test')
img = cv2.imread(fname)
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
# Find the chess board corners
ret, corners = cv2.findChessboardCorners(gray, (6,9),None)
# If found, add object points, image points (after refining them)
if ret == True:
print('test2')
objpoints.append(objp)
corners2 = cv2.cornerSubPix(gray,corners,(11,11),(-1,-1),criteria)
imgpoints.append(corners2)
# Draw and display the corners
img = cv2.drawChessboardCorners(img, (6,9), corners2,ret)
cv2.imshow('img',img)
cv2.waitKey(500)
print('test3')
cv2.destroyAllWindows()
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None)

the example assumes that you have a 6x7 chessboard image, i think you have a 6x9.
you have to prepare the objp variable for a 6x9 calibration image, so the code has to be like this: objp = np.zeros((6*9,3), np.float32)
code:
objp = np.zeros((6*9,3), np.float32)

Thanks #Rui Sebastiao.
I was using 14 x 10 so I changed the following lines and at least no error :)
objp = np.zeros((14*10, 3), np.float32)
objp[:, :2] = np.mgrid[0:14, 0:10].T.reshape(-1, 2)