International Journal of Computer Vision

, Volume 87, Issue 3, pp 266–283 | Cite as

Camera Network Calibration and Synchronization from Silhouettes in Archived Video

Article
First Online:
Received:
Accepted:

Abstract

In this paper we present an automatic method for calibrating a network of cameras that works by analyzing only the motion of silhouettes in the multiple video streams. This is particularly useful for automatic reconstruction of a dynamic event using a camera network in a situation where pre-calibration of the cameras is impractical or even impossible. The key contribution of this work is a RANSAC-based algorithm that simultaneously computes the epipolar geometry and synchronization of a pair of cameras only from the motion of silhouettes in video.

Our approach involves first independently computing the fundamental matrix and synchronization for multiple pairs of cameras in the network. In the next stage the calibration and synchronization for the complete network is recovered from the pairwise information. Finally, a visual-hull algorithm is used to reconstruct the shape of the dynamic object from its silhouettes in video. For unsynchronized video streams with sub-frame temporal offsets, we interpolate silhouettes between successive frames to get more accurate visual hulls. We show the effectiveness of our method by remotely calibrating several different indoor camera networks from archived video streams.

Keywords

Camera calibration Epipolar geometry Silhouettes Frontier points Epipolar tangents Visual hulls Camera networks Camera network synchronization 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ballan, L., & Cortelazzo, G. M. (2006). Multimodal 3d shape recovery from texture, silhouette and shadow information. In 3DPVT’06: proceedings of the third international symposium on 3D data processing, visualization, and transmission (3DPVT’06) (pp. 924–930). Los Alamitos: IEEE Computer Society. CrossRefGoogle Scholar
  2. Bolles, R. C., & Fischler, M. A. (1981). A ransac-based approach to model fitting and its application to finding cylinders in range data. In Proc. of the 7th IJCAI, Vancouver, Canada (pp. 637–643). Google Scholar
  3. Bouguet, J. (2000). Matlab camera calibration toolbox. Google Scholar
  4. Boyer, E. (2006). On using silhouettes for camera calibration. In ACCV (1) (pp. 1–10). Google Scholar
  5. Brostow, G. J., Essa, I., Steedly, D., & Kwatra, V. (2004). Novel skeletal representation for articulated creatures. In ECCV04 (Vol. III, pp. 66–78). Google Scholar
  6. Carranza, J., Theobalt, C., Magnor, M. A., & Seidel, H.-P. (2003). Free-viewpoint video of human actors. In SIGGRAPH’03: ACM SIGGRAPH 2003 papers (pp. 569–577). New York: ACM. CrossRefGoogle Scholar
  7. Cheung, G. K. M., Baker, S., & Kanade, T. (2003). Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture. In CVPR (Vol. I, pp. 77–84). Google Scholar
  8. Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In SIGGRAPH’96: proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 303–312). New York: ACM. CrossRefGoogle Scholar
  9. Franco, J.-S., & Boyer, E. (2003). Exact polyhedral visual hulls. In Proceedings of the fourteenth British machine vision conference (pp. 329–338). Norwich, UK, September 2003. Google Scholar
  10. Franco, J.-S., Lapierre, M., & Boyer, E. (2006). Visual shapes of silhouette sets. In Proceedings of the 3rd international symposium on 3d data processing, visualization and transmission. Chapel Hill (USA). Google Scholar
  11. Furukawa, Y., Sethi, A., Ponce, J., & Kriegman, D. (2006). Robust structure and motion from outlines of smooth curved surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2), 302–315. CrossRefGoogle Scholar
  12. Hartley, R., & Zisserman, A. (2005). Multiple view geometry in computer vision (Vol. 23). Cambridge: Cambridge University Press. Google Scholar
  13. Hernández, C., Schmitt, F., & Cipolla, R. (2007). Silhouette coherence for camera calibration under circular motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(2), 343–349. CrossRefGoogle Scholar
  14. Joshi, T., Ahuja, N., & Ponce, J. (1995). Structure and motion estimation from dynamic silhouettes under perspective projection. In ICCV (pp. 290–295). Google Scholar
  15. Kanade, T., Rander, P., & Narayanan, P. J. (1997). Virtualized reality: constructing virtual worlds from real scenes. IEEE MultiMedia, 4(1), 34–47. CrossRefGoogle Scholar
  16. Laurentini, A. (1994). The visual hull concept for silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2), 150–162. CrossRefGoogle Scholar
  17. Lazebnik, S., Boyer, E., & Ponce, J. (2001). On computing exact visual hulls of solids bounded by smooth surfaces. In CVPR (Vol. I. pp. 156–161). Google Scholar
  18. Lazebnik, S., Sethi, A., Schmid, C., Kriegman, D. J., Ponce, J., & Hebert, M. (2002). On pencils of tangent planes and the recognition of smooth 3d shapes from silhouettes. In ECCV (3) (pp. 651–665). Google Scholar
  19. Levi, N., & Werman, M. (2003). The viewing graph. Computer Vision and Pattern Recognition, 01, 518–522. Google Scholar
  20. Matusik, W., Buehler, C., Raskar, R., Gortler, S. J., & McMillan, L. (2000). Image-based visual hulls. In K. Akeley (Ed.), Siggraph 2000, computer graphics proceedings (pp. 369–374). New York/Reading/Harlow: ACM/Addison Wesley/Longman. Google Scholar
  21. Matusik, W., Buehler, C., & Mcmillan, L. (2001). Polyhedral visual hulls for real-time rendering. In Proceedings of Eurographics workshop on rendering (pp. 115–126). Google Scholar
  22. Mendonça, P. R. S., Wong, K.-Y. K., & Cipolla, R. (2001). Epipolar geometry from profiles under circular motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 604–616. CrossRefGoogle Scholar
  23. Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., & Koch, R. (2004). Visual modeling with a hand-held camera. International Journal of Computer Vision, 59(3), 207–232. CrossRefGoogle Scholar
  24. Porrill, J., & Pollard, S. (1991). Curve matching and stereo calibration. Image and Vision Computing, 9(1), 45–50. CrossRefGoogle Scholar
  25. Sand, P., McMillan, L., & Popović, J. (2003). Continuous capture of skin deformation. In SIGGRAPH’03: ACM SIGGRAPH 2003 papers (pp. 578–586). New York: ACM. CrossRefGoogle Scholar
  26. Sinha, S. N., & Pollefeys, M. (2004a). Synchronization and calibration of camera networks from silhouettes. In ICPR’04: proceedings of the pattern recognition, 17th international conference on (ICPR’04) (Vol. 1, pp. 116–119). Los Alamitos: IEEE Computer Society. CrossRefGoogle Scholar
  27. Sinha, S. N., & Pollefeys, M. (2004b). Visual-hull reconstruction from uncalibrated and unsynchronized video streams. In 3dpvt (pp. 349–356). Google Scholar
  28. Sinha, S. N., Pollefeys, M., & McMillan, L. (2004). Camera network calibration from dynamic silhouettes. Computer Vision and Pattern Recognition, 01, 195–202. Google Scholar
  29. Starck, J., & Hilton, A. (2007). Surface capture for performance-based animation. IEEE Computer Graphics and Applications, 27(3), 21–31. CrossRefGoogle Scholar
  30. Triggs, B., McLauchlan, P., Hartley, R., & Fitzgibbon, A. (2000). Bundle adjustment—a modern synthesis. In W. Triggs, A. Zisserman, & R. Szeliski (Eds.), LNCS. Vision algorithms: theory and practice (pp. 298–375). Berlin: Springer. CrossRefGoogle Scholar
  31. Vijayakumar, B., Kriegman, D. J., & Ponce, J. (1996). Structure and motion of curved 3d objects from monocular silhouettes. In CVPR’96: proceedings of the 1996 conference on computer vision and pattern recognition (CVPR’96) (p. 327). Los Alamitos: IEEE Computer Society. CrossRefGoogle Scholar
  32. Wong, K. Y. K., & Cipolla, R. (2001). Structure and motion from silhouettes. In ICCV (Vol. II, pp. 217–222). Google Scholar
  33. Yezzi, A. J., & Soatto, S. (2003). Structure from motion for scenes without features. In CVPR (1) (pp. 525–532). Google Scholar
  34. Zhang, Z. (1998). Determining the epipolar geometry and its uncertainty: A review. International Journal of Computer Vision, 27(2), 161–195. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Computer ScienceUNC Chapel HillChapel HillUSA

Personalised recommendations