Machine Vision and Applications

, Volume 16, Issue 4, pp 246–257

Photorealistic 3D reconstruction from handheld cameras

  • Tomás Rodríguez
  • Peter Sturm
  • Pau Gargallo
  • Nicolas Guilbert
  • Anders Heyden
  • Fernando Jauregizar
  • J. M. Menéndez
  • J. I. Ronda
Original Paper

Abstract

One of the major challenges in the fields of computer vision and computer graphics is the construction and representation of life-like virtual 3D scenarios within a computer. The VISIRE project attempts to reconstruct photo-realistic 3D models of large scenarios using as input multiple freehand video sequences, while rendering the technology accessible to the non-expert.

VISIRE is application oriented and hence must deal with multiple issues of practical relevance that were commonly overlooked in past experiences. The paper presents both an innovative approach for the integration of previously unrelated experiences, as well as a number of novel contributions, such as: an innovative algorithm to enforce closedness of the trajectories, a new approach to 3D mesh generation from sparse data, novel techniques dealing with partial occlusions and a method for using photo-consistency and visibility constrains to refine the 3D mesh.

Keywords

Photo-realistic 3D reconstruction Self calibration Structure from motion Image based rendering (IBR) Video analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Rodríguez, T., Sturm, P., Heyden, A., Menéndez, J.M., et al.: Visire. photorealistic 3d reconstruction from video sequences. In: IEEE International Conference on Image Processing, pp. 705–708. Barcelona, Spain (2003)Google Scholar
  2. 2.
    Pollefeys, M., Gool, L.V., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. IJCV 59(3), 207–232 (2004)CrossRefGoogle Scholar
  3. 3.
    Gortler, S., Grzeszczuk, R., Szeliski, R., Cohen, M.: The lumigraph. In: Proceedings of the 23rd Conference on Computer graphics and Interactive Techniques, pp. 43–54 (1996)Google Scholar
  4. 4.
    Matusik, W., Pfister, H., Ngan, A., Beardsley, P., Ziegler, R., McMillan, L.: Image-based 3d photography using opacity hulls. In: Proceedings of the ACM SIGGRAPH 2002, p. 427–437 (2002)Google Scholar
  5. 5.
    Takashi Machida, H.T.: Dense estimation of surface reflectance properties based on inverse global illumination rendering. In: ICPR'04, vol. 2, pp. 895–898. Cambridge, UK (2004)Google Scholar
  6. 6.
    Shi, J.C.T.: Good features to track. In: CVPR'94, pp. 593–600 (1994)Google Scholar
  7. 7.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge, UK (2000)Google Scholar
  8. 8.
    Heyden, A., Åström, K.: Euclidean reconstruction from image sequences with varying and unknown focal length and principal point. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 438–443 (1997)Google Scholar
  9. 9.
    Triggs, W., McLauchlan, P., Hartley, R., Fitzgibbon, A.: Bundle adjustment: A modern synthesis. In: Vision Algorithms: Theory and Practice. Springer, Berlin Heidelberg New York (2000)Google Scholar
  10. 10.
    Fitzgibbon, A., Zisserman, A.: Automatic camera recovery for closed or open image sequences. In: Proceedings of the European Conference on Computer Vision, vol. I, pp. 311–326. Freiburg, Germany (1998)Google Scholar
  11. 11.
    Triggs, B., McLauchlan, P., Ri, H., Fitzgibbon, A.: Bundle adjustment—a modern synthesis. In: Vision Algorithms'99, pp. 298–372. in conjunction with ICCV'99, Kerkyra, Greece (1999)Google Scholar
  12. 12.
    Reid, I., Murray, D.: Active tracking of foveated feature clusrters using affine structure. In: International Journal of Computer Vision, pp. 41–60. Seattle, WA (1996)Google Scholar
  13. 13.
    Sturm, P., Triggs, B.: A factorization based algorithm for multi-image projective structure and motion. In: Buxton, B., Cipolla, R. (eds.) Computer Vision – ECCV'96, Lecture Notes in Computer Science, vol. 1065, pp. 709–720. Springer, Berlin Heidelberg New York (1996)Google Scholar
  14. 14.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(2), 137–154 (1992)CrossRefGoogle Scholar
  15. 15.
    Guilbert, N., Bartoli, A.: Batch recovery of multiple views with missing data using direct sparse solvers. In: British Machine Vision Conference. Norwich, UK (2003)Google Scholar
  16. 16.
    Triggs, B.: Linear projective reconstruction from matching tensors. Image Vis. Comput. 15(8), 617–625 (1997)CrossRefGoogle Scholar
  17. 17.
    Hoppe, H.: Surface reconstruction from unorganized points. Ph.D. thesis, Department of Computer Science and Engineering, University of Washington (1994)Google Scholar
  18. 18.
    Petitjean, S., Boyer, E.: Regular and non-regular point sets: Properties and reconstruction. Comput. Geom.—Theor. Appl. 19 (2001)Google Scholar
  19. 19.
    Manessis, A., Hilton, A., Palmer, P., McLauchlan, P., Shen, X.: Reconstruction of scene models from sparse 3d structure. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. Hilton Head, USA (2000)Google Scholar
  20. 20.
    Morris, D., Kanade, T.: Image-consistent surface triangulation. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. Hilton Head, USA (2000)Google Scholar
  21. 21.
    Huber, P.: Robust Statistics. Wiley, New York (1981)Google Scholar
  22. 22.
    Press, W., Flannery, B., Teukolsky, S., Vetterling, W.: Numerical Recipes in C. Cambridge University Press, Cambridge, UK (1988)Google Scholar
  23. 23.
    Strecha, C., Fransens, R., Gool, L.V.: Wide-baseline stereo from multiple views: a probabilistic account. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 552–559 Washington, DC (2004)Google Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  • Tomás Rodríguez
    • 1
  • Peter Sturm
    • 2
  • Pau Gargallo
    • 2
  • Nicolas Guilbert
    • 3
  • Anders Heyden
    • 3
  • Fernando Jauregizar
    • 4
  • J. M. Menéndez
    • 4
  • J. I. Ronda
    • 4
  1. 1.Eptron SA. R&D Dpt.MadridSpain
  2. 2.INRIA Rhône-Alpes MontbonnotFrance
  3. 3.Centre for Mathematical ScienceLund UniversitySweden
  4. 4.E.T.S.I. TelecomunicacionesUniversidad Politécnica de MadridSpain

Personalised recommendations