Multi-Camera Structure from Motion with Eye-to-Eye Calibration

  • Sandro Esquivel
  • Reinhard Koch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9358)


Imaging systems consisting of multiple conventional cameras are of increasing interest for computer vision applications such as Structure from Motion (SfM) due to their large combined field of view and high composite image resolution. In this work we present a SfM framework for multi-camera systems w/o overlapping camera views that integrates on-line extrinsic camera calibration, local scene reconstruction, and global optimization based on combining hand-eye calibration methods with standard SfM. For this purpose, we propose a novel method for extrinsic calibration based on rigid motion constraints that uses visual measurements directly instead of motion correspondences. Only a single calibration pattern visible within the view of one camera is needed to provide an accurate reconstruction with absolute scale.


  1. 1.
    Andreff, N., Horaud, R., Espiau, B.: On-line hand-eye calibration. In: 2nd International Conference on 3D Digital Imaging and Modeling, pp. 430–436 (1999)Google Scholar
  2. 2.
    Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools 25(11), 120–126 (2000)Google Scholar
  3. 3.
    Caspi, Y., Irani, M.: Alignment of non-overlapping sequences. Int. J. Comput. Vision 48(1), 39–51 (2002)CrossRefGoogle Scholar
  4. 4.
    Chen, H.H.: A screw motion approach to uniqueness analysis of head-eye geometry. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 145–151 (1991)Google Scholar
  5. 5.
    Dornaika, F., Chung, R.: Stereo geometry from 3D ego-motion streams. IEEE Trans. Syst. Man Cybern. B Cybern. 33(2), 308–323 (2003)CrossRefGoogle Scholar
  6. 6.
    Esquivel, S., Koch, R.: Structure from motion using rigidly coupled cameras without overlapping views. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 11–20. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  7. 7.
    Esquivel, S., Woelk, F., Koch, R.: Calibration of a multi-camera rig from non-overlapping views. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 82–91. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  8. 8.
    Farenzena, M., Fusiello, A., Gherardi, R.: Structure-and-motion pipeline on a hierarchical cluster tree. In: IEEE International Conference on Computer Vision Workshops, pp. 1489–1496 (2009)Google Scholar
  9. 9.
    Frahm, J.-M., Köser, K., Koch, R.: Pose estimation for multi-camera systems. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 286–293. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  10. 10.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
  11. 11.
    Hesch, J.A., Mourikis, A.I., Roumeliotis, S.I.: Mirror-based extrinsic camera calibration. In: Workshop on the Algorithmic Foundations of Robotics, pp. 285–299 (2008)Google Scholar
  12. 12.
    Kim, J.H., Chung, M.J.: Absolute motion and structure from stereo image sequences without stereo correspondence and analysis of degenerate cases. Pattern Recogn. 39(9), 1649–1661 (2006)CrossRefGoogle Scholar
  13. 13.
    Kumar, R.K., Ilie, A., Frahm, J.M., Pollefeys, M.: Simple calibration of non-overlapping cameras with a mirror. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2008)Google Scholar
  14. 14.
    Lébraly, P., Royer, E., Ait-Aider, O., Deymier, C., Dhome, M.: Fast calibration of embedded non-overlapping cameras. In: IEEE International Conference on Robotics and Automation, pp. 221–227 (2011)Google Scholar
  15. 15.
    Li, B., Heng, L., Köser, K., Pollefeys, M.: A multiple-camera system calibration toolbox using a feature descriptor-based calibration pattern. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1301–1307 (2013)Google Scholar
  16. 16.
    Lourakis, M.I.A.: Sparse non-linear least squares optimization for geometric vision. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 43–56. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  17. 17.
    Lourakis, M.I.A., Argyros, A.A.: The design and implementation of a generic sparse bundle adjustment software package based on the Levenberg-Marquardt algorithm. Technical report #340, Institute of Computer Science, Foundation for Research and Technology - Hellas (FORTH) (2004)Google Scholar
  18. 18.
    Moré, J.J., Garbow, B.S., Hillstrom, K.E.: User guide for MINPACK-1. Technical report ANL-80-74, Argonne National Laboratory (1980)Google Scholar
  19. 19.
    Newcombe, R.A., Lovegrove, S., Davison, A.J.: DTAM: Dense tracking and mapping in real-time. In: IEEE International Conference on Computer Vision, pp. 2320–2327 (2011)Google Scholar
  20. 20.
    Pagel, F.: Calibration of non-overlapping cameras in vehicles. In: IEEE Intelligent Vehicles Symposium, pp. 1178–1183 (2010)Google Scholar
  21. 21.
    Rodrigues, R., Barreto, J.P., Nunes, U.: Camera pose estimation using images of planar mirror reflections. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 382–395. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  22. 22.
    Rodríguez, A.L., de Teruel, P.E.L., Ruiz, A.: GEA optimization for live structureless motion estimation. In: IEEE International Conference on Computer Vision, pp. 715–718 (2011)Google Scholar
  23. 23.
    Strobl, K.H., Hirzinger, G.: Optimal hand-eye calibration. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4647–4653 (2006)Google Scholar
  24. 24.
    Terzakis, G., Culverhouse, P., Bugmann, G., Sharma, S., Sutton, R.: A recipe on the parameterization of rotation matrices for non-linear optimization using quaternions. Technical report MIDAS.SMSE.2012.TR.004, Marine and Industrial Dynamic Analysis School of Marine Science and Engineering, Plymouth University (2012)Google Scholar
  25. 25.
    Tsai, R.Y., Lenz, R.K.: A new technique for fully autonomous and efficient 3d robotics hand/eye calibration. IEEE Trans. Robot. Autom. 5(3), 345–358 (1989)CrossRefGoogle Scholar
  26. 26.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1330–1334 (2000)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (, which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  1. 1.Christian-Albrechts-UniversityKielGermany

Personalised recommendations