Gaussian Processes for Efficient Plane-Based Camera Calibration

Abstract

Camera calibration is a crucial pre-processing for 3D related computer vision applications. When non-expert calibration operators capture calibration images, they strongly require guidance what kind of images must be taken. In the literature, several types of supports have been proposed.

Focusing on the plane-based calibration method proposed by Zhang, this paper proposed to such non-expert calibration operators. The proposed method asks such operators to take calibration images with variety of position and orientation and the method takes a subset of good quality images. Thanks to Gaussian Process modeling, the proposed method can select a near optimum subset with some accuracy guarantee in the sense of submodularity. To enable this, we propose to use 4-point parameterized homography as a global image feature. We conduct a small experiment with both synthesized and a public dataset and validate the proposed method works.

Appendix A Plane-Based Camera Calibration

This section reminds the readers a plane-based camera calibration method proposed by Zhang [21].

1.1 Appendix A.1 Pin-Hole Camera Model

Here, we describe how a point in the world coordinate is observed as an image point. Let $\varvec{X}_{w} = [X_{w}, Y_{w}, Z_{w}]^\top \in \mathbb {R}^3$ denotes a point in the world coordinate. In the camera coordinate, the point is observed as $\varvec{X}_{c} = [X_{c}, Y_{c}, Z_{c}]^{\top }$. The transformation from the world coordinate to the camera coordinate is expressed as

$$\begin{aligned} \tilde{\varvec{X}}_c = \begin{bmatrix} \varvec{R} \mid \varvec{t} \end{bmatrix} \tilde{\varvec{X}}_w \end{aligned}$$

(23)

where $\tilde{X}$ denotes the homogeneous coordinate of X, $\varvec{R} \in \mathbb {R}^{3 \times 3}$ $3 \times 3$ rotation matrix, and $\varvec{t}$ translation vector respectively. The rotation matrix is a redundant representation of rotation vector $\varvec{r} \in \mathbb {R}^3$. The homogeneous coordinate of a point is described by its tilde as $\tilde{\varvec{x}} = [\varvec{x}^{\top }, 1]^{\top }$. Following a polynomial lens distortion model [3, 20], the point is observed on the canonical frame as

$$\begin{aligned}&\varvec{x}_{d} = \varvec{x}_{n} + \varvec{d}_{\mathrm{rad}} + \varvec{d}_{\mathrm{tan}}, \end{aligned}$$

(24)

$$\begin{aligned}&\varvec{x}_{n} = \begin{bmatrix} x_n \\ y_n \end{bmatrix} = \frac{1}{Z_c} \begin{bmatrix} X_c \\ Y_c \end{bmatrix} \end{aligned}$$

(25)

$$\begin{aligned}&\varvec{d}_{\mathrm{rad}} = \begin{bmatrix} (\delta _{1}r^{2} + \delta _{2}r^{4} + \delta _{5}r^{6})x_{n} \\ (\delta _{1}r^{2} + \delta _{2}r^{4} + \delta _{5}r^{6})y_{n} \end{bmatrix}, \end{aligned}$$

(26)

$$\begin{aligned}&\varvec{d}_{\mathrm{tan}} = \begin{bmatrix} 2 \delta _{3}x_{n}y_{n} + \delta _{4}(3x_{n}^{2} + y_{n}^{2}) \\ 2 \delta _{4}x_{n}y_{n} + \delta _{3}(x_{n}^{2} + 3y_{n}^{2}) \\ \end{bmatrix}, \end{aligned}$$

(27)

$$\begin{aligned}&r = \sqrt{x_{n}^{2} + y_{n}^{2}}, \end{aligned}$$

(28)

where $\varvec{\delta } = [\delta _{1},\ldots ,\delta _{5}]^{\top }$ denotes lens distortion parameters and $\varvec{d}_{\mathrm{rad}}$ and $\varvec{d}_{\mathrm{tan}}$ denote the radial distortion and tangential distortion vector respectively. The final pixel coordinate $\varvec{x}$ is described using a calibration matrix $\varvec{K} \in \mathbb {R}^{3 \times 3}$ as

$$\begin{aligned}&\varvec{K} = \begin{bmatrix} f_{x} &{} \theta &{} o_{x} \\ 0 &{} f_{y} &{} o_{y} \\ 0 &{} 0 &{} 1 \end{bmatrix}, \end{aligned}$$

(29)

$$\begin{aligned}&\tilde{\varvec{x}} = \varvec{K}\tilde{\varvec{x}}_{d}, \end{aligned}$$

(30)

where $[f_{x}, f_{y}]^{\top }$ denotes the focal length along x and y axes respectively, $\theta$ the skew parameter, and $[o_{x}, o_{y}]^{\top }$ the principle point.

1.2 Appendix A.2 Plane-Based Camera Calibration

Zhang proposed a user friendly plane-based camera calibration method [23].

We first prepare a plane calibration object that has $N (\ge 4)$ points $\{\varvec{X}_n = (X_n, Y_n)\}$² on its surface. For calibration, we take $M (\ge 3)$ images $\{\varvec{I}_m\}$ with different orientation. For each input image $\varvec{I}_m$, we first extract N feature points $\{\varvec{x}_{n,m}\}$ on the target and compute the homography $\varvec{H}_m$ describing a 2D-2D transformation from the reference target to the input image plane as

$$\begin{aligned} \varvec{x}_{n,m} = \varvec{H}_m \varvec{X}_n. \end{aligned}$$

(31)

The homography can be decomposed into

$$\begin{aligned} \varvec{H} = \varvec{K} \begin{bmatrix} \varvec{r}_1&\!\!\varvec{r}_2 \!\!&\varvec{t} \end{bmatrix}, \end{aligned}$$

(32)

where $\varvec{r}_1$ and $\varvec{r}_2$ denotes the first and second column vectors of rotation matrix $\varvec{R}$. From M homographies $\{ \varvec{H}_m \}$, we estimate the intrinsic parameter $\varvec{K}$ using orthogonality of vanishing points obtained from homographies [6]. Once the intrinsic parameters are estimated, we compute the extrinsic parameters $\{\varvec{r}_m, \varvec{t}_m\}$ given $\varvec{K}$ and $\varvec{H}_m$ by solving Perspective-n-Points problem. Lastly, we optimize all the parameters by non-linear least squares [8, 11].