Abstract
The Cayley framework here is meant to tackle the vision problems under the infinite Cayley transformation (ICT), its main advantage lies in its numerical stability. In this work, the stratified self-calibration under the Cayley framework is investigated. It is well known that the main difficulty of the stratified self-calibration in multiple view geometry is to upgrade a projective reconstruction to an affine one, in other words, to estimate the unknown 3-vector of the plane at infinity, called the normal vector. To our knowledge, without any prior knowledge about the scene or the camera motion, the only available constraint on a moving camera with constant intrinsic parameters is the well-known Modulus Constraint in the literature. Do other kinds of constraints exist? If yes, what they are? How could they be used? In this work, such questions will be systematically investigated under the Cayley framework. Our key contributions include: 1. The original projective expression of the ICT is simplified and a new projective expression is derived to make the upgrade easier from a projective reconstruction to a metric reconstruction. 2. The constraints on the normal vector are systematically investigated. For two views, two constraints on the normal vector are derived; one of them is the well-known modulus constraint, while the other is a new inequality constraint. There are only these two constraints for two views. For three views, besides the constraints for two views, two groups of new constraints are derived and each of them contains three constraints. In other words, there are 12 constraints in total for three views. 3. Based on our projective expression and these constraints, a stratified Cayley algorithm and a total Cayley algorithm are proposed for the metric reconstruction from images. It is experimentally shown that they both improve significantly the numerical stability of the classical algorithms. Compared with the global optimal algorithm under the infinite homography framework, the Cayley algorithms have comparable calibration accuracy, but substantially reduce the computational load.
Similar content being viewed by others
References
Beardsley, P., Torr, P., & Zisserman, A. (1996). 3D model acquisition from extended image sequences. In European Conference on Computer Vision ECCV’96 (pp. 683–695).
Cayley, A. (1846). Sur quelques propriétés des déterminants gauches. Journal für die Reine und Angewandte Mathematik (Crelle’s Journal), 32, 119–123.
Chandrakar, M., Agarwal, S., Kriegman, D., & Belongie, S. (2010). Globally optimal algorithms for stratified autocalibration. International Journal of Computer Vision, 90, 236–254.
Courant, R., & Hilbert, D. (1989). Methods of mathematical physics (1st English ed.). New York: Wiley.
Faugeras, D. (1992). What can be seen in three dimensions with an uncalibrated stereo rig? In European Conference on Computer Vision ECCV’92 (pp. 563–578).
Hartley, R. (1994). Euclidean reconstruction form uncalibration views. In J. L. Mundy, A. Zisserman, & D. Forsyth (Eds.), Applications of invariance in computer (pp. 237–256). Berlin: Springer.
Hartley, R. (1997). Self-calibration of stationary cameras. International Journal of Computer Vision, 22(1), 5–23.
Hartley, R., & Kahl, F. (2009). Global optimization through rotation space search. International Journal of Computer Vision, 82(1), 64–79.
Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.
Hartley, R., Gupta, R., & Chang, T. (1992). Stereo from uncalibrated cameras. In IEEE Conference on Computer Vision and, Pattern Recognition CVPR’92 (pp. 761–764).
Hartley, R., Hayman, E., de Agapito, L., & Reid, I. (1999). Camera calibration and the search for infinity. In International Conference on Computer Vision ICCV’99 (pp. 510–517).
Heyden, A., & Åström, K. (1996). Euclidean reconstruction from constant intrinsic parameters. In International Conference on, Pattern Recognition ICPR’96 (pp. 339–343).
Horaud, H., & Csurka, G. (1998). Self-calibration and euclidean reconstruction using motions of a stereo rig. In International Conference on Computer Vision ICCV’98 (pp. 96–103).
Morgan, M. (1987). Solving polynomial system using continuation for engineering and scientific problems. Englewood Cliffs, NJ: Prentice Hall.
Mohr, R., Boufama, B., & Brand, P. (1994). Accurate projective reconstruction. In J. L. Mundy, A. Zisserman, & D. Forsyth (Eds.), Applications of invariance in computer vision (pp. 297–316). Berlin: Springer.
Pollefeys, M. Koch, R., & Van Gool, L. (1998). Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In International Conference on Computer Vision ICCV’98 (pp. 90–95).
Pollefeys, M., & Van Gool, L. (1999). Stratified self-calibration with the modulus constraint. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 707–724.
Ponce, J., McHenry, K., Papadopulo, T., Teillaud, M., & Triggs, B. (2005). On the absolute quadratic complex and its application to autocalibration. In IEEE Conference on Computer Vision and, Pattern Recognition CVPR’05 (pp. 780–787).
Oliensis, J., & Hartley, R. (2007). Iterative extensions of the sturm/triggs algorithm: Convergence and non-convergence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2217–2233.
Schaffalitzky, F. (2000). Direct solution of modulus constraints. In Indian conference on computer vision, graphics and image processing (pp. 314–321).
Sherman, J., & Morrision, W. J. (1950). Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Annals of Mathematical Statistics, 21, 124–127.
Sturm, P., & Triggs, B. (1996). A factorization based algorithm for multi-image projective structure and motion. In European Conference on Computer Vision ECCV’96 (pp. 709–720).
Triggs, B. (1997). Auto-calibration and the absolute quadric. In IEEE Conference on Computer Vision and, Pattern Recognition CVPR’97 (pp. 609–614).
Torr, P. (1995). Motion segmentation and outlier detection. Ph.D. thesis, Department of Engineering Science, Oxford University Press
Wu, F., Wang, Z., & Hu, Z. (2009). Cayley transformation and numerical stability of calibration equation. International Journal of Computer Vision, 82, 156–184.
Acknowledgments
We wish to thank the anonymous reviewers for their inspiring comments and suggestions. Also, we gratefully acknowledge the support from the National Natural Science Foundation of China (60835003, 91120012).
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix A: Abbreviations and Notations
Abbreviations | |
---|---|
ICT | Infinite Cayley transformation |
ZEC | Zero-Eigenvalue constraint |
CEC | Conjugate-Eigenvalue constraint |
3-VRC | 3-View row constraint |
3-VCC | 3-View column constraint |
S-CA | Stratified Cayley algorithm |
S-CAref | Stratified Cayley algorithm with the refinement step |
T-CA | Total Cayley algorithm |
POLLref | Pollefeys’ algorithm with the refinement step |
ADQref | Absolute dual quadric based calibration with the refinement step |
CHANgoa | Global optimal algorithm of Chandrakar et al. |
Notations | |
---|---|
\(\text{ R}\) | Rotation matrix |
\([\mathbf{w}]_\times \) | Skew-symmetric matrix defined by a 3-vector \(\mathbf{w}\) |
\(\text{ K}\) | Intrinsic parameter matrix of camera |
\(\omega \) | Image of the absolute conic |
\(\omega ^*\) | Dual of \(\omega \) |
\(\text{ H}\) | Homography of a plane between two views |
\(\text{ H}_\infty \) | Infinite homography between two views |
\(\text{ C}_\infty \) | ICT between two views |
\(\text{ Q}_\infty ^*\) | Absolute dual quadric |
\(|\cdot |\) | Determinant of a matrix |
\(\text{ tr}(\cdot )\) | Trace of a matrix |
\(\text{ adj}(\cdot )\) | Adjoint of a matrix |
Appendix B: Cayley Representation of Rotation Matrix
Before introducing the Cayley representation, we first briefly review the Rodrigues representation of a rotation matrix. Let \(\mathbf{t} \in R^{3}\) and \({\hat{\mathbf{t}}}=\mathbf{t}/\left\Vert \mathbf{t} \right\Vert,\) then the rotation matrix \(\text{ R}\) with axis \({\hat{\mathbf{t}}}\) and angle \(\Vert \mathbf{t}\Vert \) can be expressed as (Hartley and Zisserman 2000)
The 3-vector \(\mathbf{t}\) in (53) is called the Rodrigues representation of rotation matrix \(\text{ R}.\) Note that in general case, the Rodrigues representation is a multiple-to-one representation, for example \(\mathbf{t}=(2n\pi +\alpha )\mathbf{v}\) represents the same rotation matrix, where \(\mathbf{v}\) is a unit vector and \(n=1, 2,\ldots .\) If let
then the Rodrigues representation is of one-to-one (Hartley and Kahl 2009), i.e. (54) defines a one-to-one mapping between \(S\) and the 3D rotation matrix group G(R). We always assume \(\mathbf{t}\in S\) to ensure the uniqueness of Rodrigues representation.
Let \(\text{ G}_\pi (\text{ R})=\{\text{ R}|\,\text{ angle(R)}= \pi \}\) be the set of rotation matrices with the angle \(\pi .\) Then, the Cayley transformation (3) is a one-to-one mapping between \(\text{ G}(\text{ R})\backslash \text{ G}_\pi (\text{ R})\) and the 3D linear space \(\text{ L}(\text{ W})\) defined by \(3\times 3\) symmetric matrices under matrix addition and scalar multiplication. And thus, the 3-vector \(\mathbf{w}\) is a representation of rotation matrix in \(\text{ G}(\text{ R})\backslash \text{ G}_\pi (\text{ R}),\) called the Cayley representation.
For \(\text{ R}\in \text{ G}_\pi (\text{ R}),\) its eigenvalues must be \(\{1,-1,-1\}.\) It is easy to verify \(\text{ R}=\text{2}\mathbf{vv}^{\mathrm{T}}-\text{ I}\) by the Rodrigues representation, where \(\mathbf{v}\) is the axis of \(\text{ R}.\) And the following equation holds:
This is because for an arbitrary \(\mathbf{w}\in \mathcal R ^{3},\) by a direct computation there is
Setting \(\mathbf{w}=s\mathbf{v}\) in the above equation,
Then, there must be
Hence
Therefore, the rotation matrix \(\text{ R}\) with angle \(\pi \) and axis \(\mathbf{v}\) is the limit of the Cayley transformation \(\varphi ([s\mathbf{v}]_\times )\) at \(s\rightarrow +\infty ,\) corresponding to the infinity on the direction \(\mathbf{v}.\)
By the above discussions, the Cayley transformation gives a one-to-one mapping between the 3D rotation matrix group \(\text{ G}(\text{ R})\) and the 3D projective space.
The following proposition shows the relationship of the Cayley representations of two rotation matrices and their product.
Proposition 12
Assume \(\text{ R}_1 ,\;\text{ R}_2 , \text{ R}_1 \text{ R}_2 \in \text{ G}(\text{ R})\backslash \text{ G}_\pi (\text{ R})\) and \(\mathbf{w}_1 , \mathbf{w}_2 \) and \(\mathbf{w}_3 \) are respectively the Cayley representation of \(\text{ R}_1 ,\;\text{ R}_2 \) and \(\text{ R}_1 \text{ R}_2 .\) Then
This equation can be verified by direct computation.
For the relationship between the Cayley and Rodrigues representations of rotation matrix, we have the following proposition.
Proposition 13
Assume w and t are respectively the Cayley representation and the Rodrigues representation of rotation matrix \(\text{ R} \, in \, \text{ G}(\text{ R})\backslash \text{ G}_\pi (\text{ R}), then\)
Proof
From \(e^{[\mathbf{t}]_\times }=\text{ R}=\dfrac{\text{ I}-[\mathbf{w }]_\times }{\text{ I}+[\mathbf{w }]_\times },\) we have
By \(e^{[\mathbf{t}]_\times }=e^{[\mathbf{t}/2]_\times }e^{[\mathbf{t}/2]_\times }\) and \((e^{[\mathbf{t}/2]_\times })^{-1}=e^{-[\mathbf{t}/2]_\times },\)
Then, by substituting them into (58)
From (53), we can obtain
By (61) and the power series \((1+x)^{-1}\!=\!\sum _{n=0}^\infty {(-1)^{n}x^{n}} (|x|<1),\) we have
Then, by substituting (60) and (62) into (59), we have
By the identity equation \([\mathbf{t}/2]_\times ^{2n+1} =(-1)^{n}\left\Vert {\mathbf{t}/2} \right\Vert^{2n}[\mathbf{t}/2]_\times ,\) (63) becomes
Hence, Proposition 13 holds. \(\square \)
By Proposition 13, we have \({\hat{\mathbf{{t}}}}=-{\hat{\mathbf{{w}}}}\) and \(\left\Vert \mathbf{t} \right\Vert=2\arctan (\left\Vert \mathbf{w} \right\Vert).\) Thus, the angle and axis of the rotation matrix corresponding to \(\mathbf{w}\) is respectively \(2\arctan (\left\Vert \mathbf{w} \right\Vert)\) and \(-{\hat{\mathbf{{w}}}}.\) Besides, by \(\left\Vert \mathbf{t} \right\Vert=2\arctan (\left\Vert \mathbf{w} \right\Vert)\) there should be \(\left\Vert \mathbf{w} \right\Vert\rightarrow +\infty \) when \(\left\Vert \mathbf{t} \right\Vert\rightarrow \pi ,\) this indicates once again that the rotation matrix \(\text{ R}\) with the angle \(\left\Vert \mathbf{t} \right\Vert=\pi \) and the axis \({\hat{\mathbf{{t}}}}\) is the limit of the Cayley transformation \(\phi ([s{\hat{\mathbf{{t}}}}]_\times )\) at \(s\rightarrow +\infty ,\) corresponding to the infinity on the direction \({\hat{\mathbf{{t}}}}.\)
Appendix C: Proof of Proposition 2
For \(\mathbf{w}_2 \in \text{ span}(\mathbf{w}_1 ),\) there must exist a scalar \(s\) such that \(\mathbf{w}_2 =s\mathbf{w}_1 ,\) then
Thus
So, Proposition 2(a) holds. The proof of Proposition 2(b) can be done by the following three steps:
-
(1)
If \(\mathbf{w}\in \text{ span}(\mathbf{w}_1 ,\;\mathbf{w}_2 ),\) i.e. there exist two scalars \(s_1 , s_2 \) such that \(\mathbf{w}\in s_1 \mathbf{w}_1 +s_2 \mathbf{w}_2 ,\) then
$$\begin{aligned} \text{ C}_\infty =\text{ K}[s_1 \mathbf{w}_1 +s_2 \mathbf{w}_2 ]_\times \text{ K}^{-1}=s_1 \text{ C}_{\infty 1} +s_2 \text{ C}_{\infty 2} . \end{aligned}$$Thus
$$\begin{aligned} \text{ C}_\infty \omega ^*+\omega ^*\text{ C}_\infty ^{\mathrm{T}}&= s_1(\text{ C}_{\infty 1} \omega ^*+\omega ^*\text{ C}_{\infty 1}^{\mathrm{T}} )\\&+s_2 (\text{ C}_{\infty 2} \omega ^*+\omega ^*\text{ C}_{\infty 2}^{\mathrm{T}} ). \end{aligned}$$Therefore, Proposition 2(b) holds in this case.
-
(2)
If \(\mathbf{w}\in \text{ span}(\mathbf{w}_1 \times \mathbf{w}_2 )\) where \(\mathbf{w}_1 \times \mathbf{w}_2 \) is the cross product of \(\mathbf{w}_1 \) and \(\mathbf{w}_2 ,\) then there exists a scalar s such that \(\mathbf{w}=s\mathbf{w}_1 \times \mathbf{w}_2 .\) From
$$\begin{aligned} \text{[C}_{\infty 1} ,\text{ C}_{\infty 2} ]&\triangleq \text{ C}_{\infty 1} \text{ C}_{\infty 2} -\text{ C}_{\infty 2} \text{ C}_{\infty 1}\\&= \text{ K([}\mathbf{w}_1 ]_\times \text{[}\mathbf{w}_2 ]_\times -\text{[}\mathbf{w}_2 ]_\times \text{[}\mathbf{w}_1 ]_\times )\text{ K}^{-1}\\&= \text{ K[}\mathbf{w}_1 \times \mathbf{w}_2 ]_\times \text{ K}^{-1}, \end{aligned}$$we have \(\text{ C}_\infty =s[\text{ C}_{\infty 1}, \text{ C}_{\infty \text{2}}]\) and
$$\begin{aligned}&\text{ C}_\infty \omega ^*+\omega ^*\text{ C}_\infty ^{\mathrm{T}} =s([\text{ C}_{\infty 1} ,\text{ C}_{\infty 2} ]\omega ^*\!+\!\omega ^ *[\text{ C}_{\infty 1} \text{,C}_{\infty 2} ]^{T})\\&\quad =s\left( \text{ C}_{\infty 2} (\text{ C}_{\infty 1} \omega ^ *+\omega ^*\text{ C}_{\infty 1}^{\mathrm{T}} )+(\text{ C}_{\infty 1} \omega ^*\right.\\&\qquad +\omega ^*\text{ C}_{\infty 1}^{\mathrm{T}}) \text{ C}_{\infty 2}^{\mathrm{T}} \\&\qquad \left. -\text{ C}_{\infty 1} (\text{ C}_{\infty 2} \omega ^*+\omega ^*\text{ C}_{\infty 2}^{\mathrm{T}} )-(\text{ C}_{\infty 2} \omega ^*\right.\\&\qquad \left.+\omega ^*\text{ C}_{\infty 2}^{\mathrm{T}} )\text{ C}_{\infty 1}^{\mathrm{T}} \right). \end{aligned}$$Therefore, Proposition 2(b) holds in this case.
-
(3)
For an arbitrary \(\mathbf{w}\in \mathcal R ^{3},\) there must be \(s_1 , \text{ s}_2 , \text{ s}_3\) such that
$$\begin{aligned} \mathbf{w}=s_1 \mathbf{w}_1 +s_2 \mathbf{w}_2 +s_3 (\mathbf{w}_1 \times \mathbf{w}_2 ). \end{aligned}$$Thus
$$\begin{aligned} \text{ C}_\infty =s_1 \text{ C}_{\infty 1} +s_2 \text{ C}_{\infty 2} +s_3 \text{[C}_{\infty 1} ,\text{ C}_{\infty 2} ]. \end{aligned}$$By the steps (1) and (2), Proposition 2(b) holds for the general case.
Appendix D: Proof of the Inequality (30)
For the normal vector, there must have
Similarly
Thus, the ICT’s expression (22) in Sect. 4 can be rewritten as
On the other hand
here \(\text{ R}_{ij} \) is the rotation matrix of the view pair \(\left\{ {i,j} \right\} \) and \(\phi (\text{ R}_{ij} )=\frac{\text{ I}-\text{ R}_{ij} }{\text{ I}+\text{ R}_{ij} }\) is the Cayley transformation of \(\text{ R}_{ij} .\) By substituting (65) into (64), we can obtain
Let \(\{1,\;e^{i\theta },e^{-i\theta }\}\) be eigenvalues of \(\text{ R}_{ij} \) and \(\{\mathbf{u}, \mathbf{v}, {\bar{\mathbf{v}}}\}\) the corresponding eigenvectors. Then, eigenvalues of \(\phi (\text{ R}_{ij} )\) are \(\{\phi (1), \phi (e^{i\theta }), \phi (e^{-i\theta })\}=\{0, -i\tan (\theta /2), i\tan (\theta /2)\}\) and the corresponding eigenvectors still are \(\{\mathbf{u}, \mathbf{v},{\bar{\mathbf{v}}}\}.\) And thus, from (66) we have
Since \(\frac{\delta _{ij} }{\mu _{ij} \mu _{ji} \left| {\text{ H}_i } \right|\left| {\text{ H}_j } \right|}\ne 0,\) by (67) we obtain again the ZEC
By substituting (69) into (68) and since \(\delta _{ij} = \frac{\mu _{ij} }{\mu _{ji} (\sigma _i \mu _{ji} |\text{ H}_{ij}|+\mu _{ij}^2 )},\)
Then
Or equivalently
Therefore, the inequality (30) holds.
Rights and permissions
About this article
Cite this article
Wu, F.C., Zhang, M. & Hu, Z.Y. Self-Calibration Under the Cayley Framework. Int J Comput Vis 103, 372–398 (2013). https://doi.org/10.1007/s11263-013-0610-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-013-0610-7