Single Camera Calibration

Besides OpenCV book, the code documentation section "Camera Calibration and 3d Reconstruction" also provide a brief overview of the elements involved in calibration.

OpenCV provides a set of functions to estimate camera intrinsic and distortion effects from a set of image-views taken from a variety of perspectives on a 3D/planar object. A set of built-in functions to support using NxM chessboard as a planar-object (Z=0) is also included.

Camera Intrinsic [ 3x3 matrix ]

• focal-lengths (fx, fy) in x, y directions, center-of-projection (cx, cy).

Distortion effects [ (5 to 8) x 1 vector ]

• Radial Distortion - a vector of 3 to 6 (k1,...k6) coefficients; The book only mentions up to k3.
• Tangential Distortion. a vector of 2 coefficients (p1, p2). The book suggests setting Tangential coefficient 0 zero for well-built camera. Value too close to 0 could introduce large noise values.

Camera Extrinsic [ 3x4 matrix per view ]

• 3x3 Rotational appended with 3x1 Translation. This transforms the objects coordinate-system to match that of projected image-plane. 
• It could be further simplified if the object is planar (2D, Z=0). 
• For computation efficiency the 3x3 rotational matrix is represented by a 3x1 vector. See Rodrigues Transform below.

The chessboard grid elements are by default represented as 1x1 unit. This is handy to describe the object-coordinates. Conversion that to real-life metrics (such as inches, millimeters) in post-processing is up to the user.

Essential functions

• calibrateCamera() - estimate the camera intrinsic, distortion coefficients and optionally the extrinsic for each input view. The RMS value of projection-error is returned for result evaluation.
• projectPoints() - could be used to calculate projection error to find other points of interest from a view that has been used to deduce camera parameters, together with perspective transform of this view.

ChessBoard Functions

• findChessBoardCorners()
• drawChessBoardCorners()
• ChessBoardGeneratorClass - from test and 'calibration_artificial sample'

Undistort functions - undistort the projected view with distortion parameters.

• undistort()
• undistortPoints()
• initUndistortRectifyMap()
• getOptimalNewCameraMatrix() - adjust the camera intrinsic such that would result in a zoomed view with fewer black regions. The new camera matrix will be passed to initUndistortRectifyMap().

Supporting functions

• Rodrigues(): perform Rodrigues Transform

Rodrigues Transform

It provides the two-way conversion between 3x3 rotational matrix and 3x1 rotational vector. My understanding is, the 3x3 rotational matrix is made up of a sequential rotation of the 3 axes. The direction of the corresponding 3x1 vector represents a new axis in the 3D space that is equivalent to the combined result of the 3 rotations. The amount of rotation is represented by magnitude of the 3x1 vector.

Chessboard Picture
OpenCV comes with a 9x7 Chessboard in the name of 'pattern.pdf' under OpenCV/docs directory.

Sample (calibration.cpp)

• Tested with 9x6 chessboard (left series), begin with zero-tangential distortion
• Discovered bug in saving extrinsic parameters to file-storage - the type of rvecs[i] and tvecs[i] does not match bigmat.
• Radial distortion example - see right edge of chessboard at left06.jpg before and after 'undistort'.
• Despite what the manual says, a thin black strip remains at the middle top from distortion-correction even by setting 'alpha' value to 0 to getOptimalNewCameraMatrix().
• Didn't have a chance to test the video capture method, try later.  
• calibrateCamera() returns the RMS of back-projection errors. The sample does it's own version in computeReprojectionErrors().
• Higher order Radial distortion coefficients support with flag CV_CALIB_RATIONAL_MODEL

Sample (Calibration_Artificial.cpp)

• The camera, chessboard, projected-chessboard-images are all created / defined in code, no actual chessboard/camera is used.
• The program define camera intrinsic and distortion coefficients for a "virtual camera". Uses chess-board-generator class (copied from 'test' module) to make up 20 chessboard pictures that could have been captured by this virtual camera. The perspective of the chessboards are constrained by the camera parameters.
• It detects the chess-corners from the generated pictures and call cameraCalibration() function with those detected points. We could see how well it could recover the original camera parameters and what the error rate is.
• All 'k' elements of a set of valid Distortion Coefficients should be used together. The estimated k1, k2 and k3 values of the previous calibration sample are non-zero (using the leftN.jpg series). Tried replacing the sample-defined distortion coefficients with only the k1 and k2 from the previous example yields projected chessboard images that does not look planar, especially at the 4 corners.
• ChessBoardGenerator::rendererResolutionMultiplier draws a finer resolution chessboard by first scaling up the square-vertexes locations with the multiplier. Connect the points with contours at the enlarged canvas before down-sampling the result to fit the window.
  Multiplier values - Avg Reprojection Error
  1: 0.0010
  4: 0.0005 (default)
  16: 0.0004