Skip to main content
Book cover

European Conference on Computer Vision

ECCV 1998: Computer Vision — ECCV'98 pp 55–71Cite as

Multi viewpoint stereo from uncalibrated video sequences

Part of the Lecture Notes in Computer Science book series (LNCS,volume 1406)

Abstract

This contribution describes an automatic 3D surface modeling system that extracts dense metric 3D surfaces from an uncalibrated video sequence. A static 3D scene is observed from multiple viewpoints by freely moving a video camera around the object. No restrictions on camera movement and internal camera parameters like zoom are imposed, as the camera pose and intrinsic parameters are calibrated from the sequence.

Dense surface reconstructions are obtained by first treating consecutive images of the sequence as stereoscopic pairs and computing dense disparity maps for all image pairs. All viewpoints are then linked by controlled correspondence linking for each image pixel. The correspondence linking algorithm allows for accurate depth estimation as well as image texture fusion from all viewpoints simultaneously. By keeping track of surface visibility and measurement uncertainty it can cope with occlusions and measurement outliers. The correspondence linking is applied to increase the robustness and geometrical resolution of surface depth as well as to remove highlights and specular reflections, and to create super-resolution texture maps for increased realism.

The major impact of this work is the ability to automatically generate geometrically correct and visually pleasing 3D surface models from image sequences alone, which allows the economic model generation for a wide range of applications. The resulting textured 3D surface model are highly realistic VRML representations of the scene.

Keywords

  • Fill Rate
  • Depth Estimate
  • Epipolar Line
  • Depth Error
  • Texture Enhancement

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Download conference paper PDF

References

  1. P. Beardsley, P. Torr and A. Zisserman: 3D Model Acquisition from Extended Image Sequences. In: B. Buxton, R. Cipolla(Eds.) Computer Vision — ECCV 96, Cambridge, UK., vol.2, pp.683–695. Lecture Notes in Computer Science, Vol. 1064. Springer Verlag, 1996.

    Google Scholar 

  2. Y. Chen and G. Medioni: Object Modeling by Registration of Multiple Range Images. Proceedings 1991 IEEE International Conference on Robotics and Automation, pp. 2724–2729, Sacramento (CA), 1991.

    Google Scholar 

  3. I. J. Cox, S. L. Hingorani, and S. B. Rao: A Maximum Likelihood Stereo Algorithm. Computer Vision and Image Understanding, Vol. 63, No. 3, May 1996.

    Google Scholar 

  4. L.Falkenhagen: Hierarchical Block-Based Disparity Estimation Considering Neighborhood Constraints. International Workshop on SNHC and 3D Imaging, September 5–9, 1997, Rhodes, Greece.

    Google Scholar 

  5. O.Faugeras: Three-Dimensional Computer Vision — a geometric viewpoint. MIT-Press, 1993.

    Google Scholar 

  6. O. Faugeras: What can be seen in three dimensions with an uncalibrated stereo rig. Proc. ECCV'92, pp.563–578.

    Google Scholar 

  7. O. Faugeras, Q.-T. Luong and S. Maybank: Camera self-calibration — Theory and experiments. Proc. ECCV'92, pp.321–334.

    Google Scholar 

  8. G.L.Gimel'farb: Symmetrical approach to the problem of automating stereoscopic measurements in photogrammetry. Cybernetics, 1979, 15(2), 235–247; Consultants Bureau, N.Y.

    MATH  Google Scholar 

  9. G.L.Gimel'farb and R.M.Haralick: Terrain reconstruction from multiple views. Proc. 7th Int. Conf. on Computer Analysis of Images and Patterns (CAIP'97), Kiel, Germany, Sept. 1997. (G.Sommer, K.Daniilidis, and J.Pauli, Eds.). Lecture Notes in Computer Science 1296, Springer: Berlin e.a., 1997, pp.694–701.

    Google Scholar 

  10. C.G. Harris and J.M. Pike: 3D Positional Integration from Image Sequences. 3rd Alvey Vision Conf, pp. 233–236, 1987.

    Google Scholar 

  11. R. Hartley: Estimation of relative camera positions for uncalibrated cameras. Proc. ECCV'92, pp.579–87.

    Google Scholar 

  12. A. Heyden and K. åström: Euclidean Reconstruction from Image Sequences with Varying and Unknown Focal Length and Principal Point. Proc. CVPR'97.

    Google Scholar 

  13. M. Irani and S. Peleg: Super resolution from image sequences. Tenth International Conference on Pattern Recognition (Atlantic City, NJ, June 16–21, 1990), IEEE Catalog No. 90CH2898-5, 1990, subconference C, 115–220.

    Google Scholar 

  14. R. Koch: 3-D Modeling of Human Heads from Stereoscopic Image Sequences. Proc. DAGM'96, Informatik Aktuell, Springer Heidelberg, Germany, Sept. 1996.

    Google Scholar 

  15. R. Koch: Surface Segmentation and Modeling of 3-D Polygonal Objects from Stereoscopic Image Pairs. Proc. ICPR'96, Vienna 1996.

    Google Scholar 

  16. R. Koch: Automatische Oberflächenmodellierung starrer dreidimensionaler Objekte aus stereoskopischen Bildfolgen. Ph.D. Thesis, Fortschr.-Ber. 10/499. Düsseldorf: VDI Verlag 1997.

    Google Scholar 

  17. A. Kochan: Eine Methodenbank zur Evaluierung von Stereo-Vision-Verfahren. Ph.D. Thesis, TU Berlin, June 1991.

    Google Scholar 

  18. L. Matthies, T. Kanade, and R. Szeliski: Kaiman filter-based algorithms for estimating depth from image sequences. International Journal of Computer Vision, vol. 3, 1989, 209–236.

    CrossRef  Google Scholar 

  19. M. Okutomi and T. Kanade: A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 4, 1993, 353–363.

    CrossRef  Google Scholar 

  20. E. Ofek, E. Shilat, A. Rappoport, M. Werman: Highlight and Reflection-Independent Multiresolution Textures from Image Sequences. IEEE Computer Graphics and Applications vol. 17 (2), March–April 1997.

    Google Scholar 

  21. M. Pollefeys, L. Van Gool and M. Proesmans: Euclidean 3D Reconstruction from Image Sequences with Variable Focal Lengths. Proc. ECCV'96, vol.1, pp. 31–42.

    Google Scholar 

  22. M. Pollefeys and L. Van Gool: Self-calibration from the absolute conic on the plane at infinity. Proc. CAIP'97.

    Google Scholar 

  23. M. Pollefeys, R. Koch and L. Van Gool: Self-Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters. Proc. ICCV'98, Bombay, India, 1998.

    Google Scholar 

  24. P.H.S. Torr: Motion Segmentation and Outlier Detection. PhD thesis, Dept. Eng. Science, University of Oxford, UK, 1995.

    Google Scholar 

  25. B. Triggs: The Absolute Quadric. Proc. CVPR'97.

    Google Scholar 

  26. Z. Zhang and O. Faugeras: 3D Dynamic Scene Analysis. Springer Verlag, 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Katholieke Universiteit Leuven, Kardinaal Mercierlaan 94, B-3001, Leuven, Belgium

    Reinhard Koch, Marc Pollefeys & Luc Van Gool

Authors
  1. Reinhard Koch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marc Pollefeys
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luc Van Gool
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

    Rights and permissions

    Reprints and Permissions

    Copyright information

    © 1998 Springer-Verlag Berlin Heidelberg

    About this paper

    Cite this paper

    Koch, R., Pollefeys, M., Van Gool, L. (1998). Multi viewpoint stereo from uncalibrated video sequences. In: Burkhardt, H., Neumann, B. (eds) Computer Vision — ECCV'98. ECCV 1998. Lecture Notes in Computer Science, vol 1406. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0055659

    Download citation

    • DOI: https://doi.org/10.1007/BFb0055659

    • Published:

    • Publisher Name: Springer, Berlin, Heidelberg

    • Print ISBN: 978-3-540-64569-6

    • Online ISBN: 978-3-540-69354-3

    • eBook Packages: Springer Book Archive