3D Research

, 2:4 | Cite as

An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods

3DR Review

Abstract

This paper provides an introduction to photometric methods for image-based 3D shape reconstruction and a survey of photometric stereo techniques. We begin with taxonomy of active and passive shape acquisition techniques. Then we describe the methodical background of photometric 3D reconstruction, define the canonical setting of photometric stereo (Lambertian surface reflectance, parallel incident light, known illumination direction, known surface albedo, absence of cast shadows), discuss the 3D reconstruction of surfaces from local gradients, summarize the concept of the bidirectional reflectance distribution function (BRDF), and outline several important empirically and physically motivated reflectance models. We provide a detailed treatment of several generalizations of the canonical setting of photometric stereo, namely non-distant light sources, unknown illumination directions, and, in some detail, non-Lambertian surface reflectance functions. An important special case is purely specular reflections, where an extended light source allows capturing a surface that consists of perfectly specular surface patches. Linear combinations of purely Lambertian and purely specular reflectance components are favorably used for reconstructing smooth surfaces and also human skin. Nonuniform surface reflectance properties are estimated based on a simultaneous 3D reconstruction and determination of the locally variable parameters of the reflectance function based on a multitude of images. Assuming faceted surfaces, the effective resolution of the 3D reconstruction result can be increased to some extent beyond that of the underlying images. Other approaches separate specular and diffuse reflectance components based on polarization data or color information. The specular reflections can be used additionally to estimate the direction from which the surface is illuminated. Finally, we describe methods to combine photometric 3D reconstruction techniques with active and passive triangulation-based approaches.

Keywords

3D surface reconstruction shape analysis survey Photometric Stereo 

References

  1. 1.
    A. Agrawal, R. Raskar, R. Chellappa (2006) What is the range of surface reconstructions from a gradient field? Proceedings of the European Conference on Computer Vision (ECCV 2006), 1(TR2006-021): 578–591.Google Scholar
  2. 2.
    N. Alldrin, T. Zickler, D. Kriegman (2008) Photometric stereo with non-parametric and spatiallyvarying reflectance, CVPR’08.Google Scholar
  3. 3.
    N. G. Alldrin (2006) Reflectance estimation under natural illumination. Technical report, University of California, San Diego.Google Scholar
  4. 4.
    N. G. Alldrin and D. J. Kriegman (2007) Toward reconstructing surfaces with arbitrary isotropic reflectance: A stratified photometric stereo approach, ICCV’07.Google Scholar
  5. 5.
    N. G. Alldrin, S. P. Mallick, D. J. Kriegman (2007) Resolving the generalized bas-relief ambiguity by entropy minimization, CVPR’07, doi: http://dx.doi.org/10.1109/CVPR.2007.383208.
  6. 6.
    S. Barsky and M. Petrou (2003) The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows, PAMI’03, 25(10):1239–1252.CrossRefGoogle Scholar
  7. 7.
    R. Basri, D. W. Jacobs, I. Kemelmacher (2007) Photometric stereo with general, unknown lighting, IJCV’07, 72(3):239–257.CrossRefGoogle Scholar
  8. 8.
    P. Beckmann and A. Spizzichino (1987) The Scattering of Electromagnetic Waves from Rough Surfaces, Number ISBN-13: 987-0890062382, Artech House Radar Library.Google Scholar
  9. 9.
    P. N. Belhumeur, D. J. Kriegman, A. L. Yuille (1999) The bas-relief ambiguity, IJCV’99, 35(1):1040–1046, doi: http://dx.doi.org/10.1023/A:1008154927611.Google Scholar
  10. 10.
    M. K. Chandraker, F. Kahl, D. Kriegman (2005) Reflections on the generalized bas-relief ambiguity, CVPR’05, 1:788–795.Google Scholar
  11. 11.
    J. J. Clark (1992) Active photometric stereo, CVPR’92, pages 29–34, doi:http://dx.doi.org/10.1109/CVPR.1992.223231.
  12. 12.
    R. L. Cook and K. E. Torrance (1981) A reflectance model for computer graphics. Proceedings of the 8th annual conference on Computer graphics and interactive techniques, 15(3):307–316, doi: http://doi.acm.org/10.1145/800224.806819.Google Scholar
  13. 13.
    B. L. Curless (1997) New Methods for Surface Reconstruction from Range Images. PhD thesis, Stanford University.Google Scholar
  14. 14.
    d’Angelo and C. Wöhler (2008) Image-based 3d surface reconstruction by combination of photometric, geometric, and real-aperture models, ISPRS Journal of Photo-grammetry and Remote Sensing, 63(3):297–321, doi: http://dx.doi.org/10.1016/j.isprsjprs.2007.09.005.CrossRefGoogle Scholar
  15. 15.
    P. Debevec (1999) Modeling and rendering architecture from photographs, Technical report.Google Scholar
  16. 16.
    O. Drbohlav and M. Chantler (2005) Can two specular pixels calibrate photometric stereo? ICCV’05, 2:1850–1857.Google Scholar
  17. 17.
    O. Drbohlav and R. Sara (2002) Specularities reduce ambiguity of uncalibrated photometric stereo, ECCV’02, 2:46–60.Google Scholar
  18. 18.
    R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Recognition of surface reflectance properties from a single image under unknown real-world illumination, Proceedings of the IEEE Workshop on Identifying Objects Across Variations in Lighting: Psychophysics & Computation.Google Scholar
  19. 19.
    R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Surface reflectance estimation and natural illumination statistics, Proceedings of the IEEE Workshop on Statistical and Computational Theories of Vision.Google Scholar
  20. 20.
    R. O. Dror, E. H. Adelson, A. S. Willsky (2001) Estimating surface reflectance properties from images under unknown illumination, Proceedings of the SPIE Conference on Human Vision and Electronic Imaging IV, 4.Google Scholar
  21. 21.
    R. O. Dror, T. K. Leung, E. H. Adelson, A. S. Willsky (2001) Statistics of real-world illumination, CVPR’01, 2:164–171.Google Scholar
  22. 22.
    J.-D. Duroua, M. Falconeb, M. Sagona (2007) Numerical methods for shape-from-shading: A new survey with benchmarks, Computer Vision and Image Understanding, 109(1).Google Scholar
  23. 23.
    P. Fechteler, P. Eisert, J. Rurainsky (2007) Fast and high resolution 3d face scanning, ICIP’07, 3:81–84, doi:http://dx.doi.org/10.1109/ICIP.2007.4379251.Google Scholar
  24. 24.
    W. T. Freeman (1994) The generic viewpoint assumption in a framework for visual perception, Nature, 368:542–545.CrossRefGoogle Scholar
  25. 25.
    J. Garding (1992) Shape from texture for smooth curved surfaces in perspective projection. Journal of Mathematical Imaging and Vision, 2:630–638.CrossRefGoogle Scholar
  26. 26.
    A. S. Georghiades (2003) Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo, ICCV’03, 2:816–823.Google Scholar
  27. 27.
    A. S. Georghiades, P. N. Belhumeur, D. J. Kriegman (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose, PAMI’01, 23:643–660.CrossRefGoogle Scholar
  28. 28.
    D. B. Goldman, B. Curless, A. Hertzmann, S. Seitz (2005) Shape and spatially-varying BRDFs from photometric stereo, ICCV’05, 1:341–348.Google Scholar
  29. 29.
    B. Goldman, B. Curless, A. Hertzmann, S. Seitz (2010) Shape and spatially varying BRDFs from photometric stereo, PAMI’10, 32(6):1060–1071.CrossRefGoogle Scholar
  30. 30.
    A. Grumpe, S. Herbort, C. Wöhler (2011) 3D reconstruction of non-Lambertian surfaces with nonuniform reflectance parameters by fusion of photometrically estimated surface normal data with active range scanner data, Oldenburger 3D Tage 2011, 10.Google Scholar
  31. 31.
    H. Hayakawa (1994) Photometric stereo under a light source with arbitrary motion. Journal of Optical Society of America A (JOSA A), 11:3079–3089, doi:10.1364/JOSAA.11.003079.CrossRefGoogle Scholar
  32. 32.
    C. Hernandez and G. Vogiatzis (2010) Selfcalibrating a real-time monocular 3D facial capture system, Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT2010).Google Scholar
  33. 33.
    C. Hernandez, G. Vogiatzis, R. Cipolla (2008) Shadows in three-source photometric stereo, ECCV’08, pages 290–303, doi: http://dx.doi.org/10.1007/978-3-540-88682-2_23.
  34. 34.
    C. Hernández, G. Vogiatzis, G. J. Brostow, B. Stenger, R. Cipolla (2007) Non-rigid photometric stereo with colored lights, ICCV’07.Google Scholar
  35. 35.
    A. Hertzmann and S. M. Seitz (2003) Shape and materials by example: A photometric stereo approach, CVPR’03, 1:533–540.Google Scholar
  36. 36.
    A. Hertzmann and S. M. Seitz (2005) Example-based photometric stereo: Shape reconstruction with general, varying BRDFs, PAMI’05, 27(8):1254–1264.CrossRefGoogle Scholar
  37. 37.
    B. K. P. Horn (1970) Shape from shading: A method for obtaining the shape of a smooth opaque object from one view, Technical Report 232, MIT.Google Scholar
  38. 38.
    B. K. P. Horn (1975) Determining Shape from Shading.Google Scholar
  39. 39.
    B. K. P. Horn (1975) Image intensity understanding, Technical Report 335, MIT, Artificial Intelligence Laboratory.Google Scholar
  40. 40.
    B. K. P. Horn (1977) Understanding image intensities, Artificial Intelligence, 11(2):201–231.CrossRefMathSciNetGoogle Scholar
  41. 41.
    B. K. P. Horn (1989) Height and gradient from shading, Technical Report 1105A, MIT, Artificial Intelligence Laboratory.Google Scholar
  42. 42.
    B. K. P. Horn and R. W. Sjoberg (1978) Calculating the reflectance map, Technical Report 498, MIT, Artificial Intelligence Laboratory.Google Scholar
  43. 43.
    K. Ikeuchi (1980) Numerical shape from shading and occluding contours in a single view, Technical Report 566, MIT, Artificial Intelligence Laboratory.Google Scholar
  44. 44.
    K. Ikeuchi (1981) Determining surface orientations of specular surfaces by using the photometric stereo method, PAMI, 3(6):661–669, doi: http://dx.doi.org/10.1109/TPAMI.1981.4767167.CrossRefGoogle Scholar
  45. 45.
    Ikeuchi and B. K. P. Horn (1981) Numerical shape from shading and occluding boundaries, Artificial Intelligence, 17:141–184.CrossRefGoogle Scholar
  46. 46.
    A. B. Israel and T. N. E. Greville (2003) Generalized Inverses Theory & Applications, Springer, 2nd edition.Google Scholar
  47. 47.
    Y. Iwahori, H. Sugie, N. Ishii (1990) Reconstructing shape from shading images under point light source illumination, ICPR’90, 1:83–87.Google Scholar
  48. 48.
    N. Joshi and D. J. Kriegman (2007) Shape from varying illumination and viewpoint, ICCV’07.Google Scholar
  49. 49.
    R. Kimmel and J. A. Sethian (2001) Optimal algorithm for shape from shading and path planning, Journal of Mathematical Imaging and Vision, 14: 237–244.CrossRefMATHMathSciNetGoogle Scholar
  50. 50.
    J.-H. Lambert (1760) Photometria, sive de mensura et gradibus luminis, colorum et umbrae, Vidae Eberhardi Klett.Google Scholar
  51. 51.
    D. Lanman and G. Taubin (2009) Build your own 3D scanner: 3D photography for beginners, Technical Report, Brown University.Google Scholar
  52. 52.
    J. Lawrence, A. Ben-Artzi, C. DeCoro, W. Matusik, H. Pfister, R. Ramamoorthi, S. Rusinkiewicz (2006) Inverse shade trees for non-parametric material representation and editing, ACM Transactions on Graphics (TOG’06), 25(3):735–745, doi: http://doi.acm.org/10.1145/1141911.1141949.CrossRefGoogle Scholar
  53. 53.
    J. Lim, J. Ho, M.-H. Yang, D. Kriegman (2005) Passive photometric stereo from motion, ICCV’05, 2:1635–1642, doi:http://dx.doi.org/10.1109/ICCV.2005.185.Google Scholar
  54. 54.
    S. P. Mallick, T. Zickler, D. J. Kriegman, P. N. Belhumeur (2005) Beyond Lambert: Reconstructing specular surfaces using color, CVPR’05, 1:619–626.Google Scholar
  55. 55.
    S. Marschner, E. P. F. Lafortune, S. H. Westin, K. E. Torrance, D. P. Greenberg (1999) Image based BRDF measurement, Applied Optics, 39:16.Google Scholar
  56. 56.
    S. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, D. P. Green-berg (1999) Imagebased BRDF measurement including human skin, Proceedings of 10th Eurographics Workshop on Rendering, pages 139–152.Google Scholar
  57. 57.
    S. R. Marschner (1998) Inverse Rendering for Computer Graphics, PhD thesis, Cornell University.Google Scholar
  58. 58.
    W. Matusik, H. Pfister, M. Brand, L. McMillan (2003) A data-driven reflectance model, ACM Transactions on Graphics, 22(3):759–769.CrossRefGoogle Scholar
  59. 59.
    W. Matusik, H. Pfister, M. Brand, L. McMillan (2003) Efficient isotropic BRDF measurement, 14th Eurographics Workshop on Rendering, 44:241–247.Google Scholar
  60. 60.
    Morel, F. Meriaudeau, C. Stolz, P. Gorria (2005) Polarization imaging applied to 3D reconstruction of specular metallic surfaces.Google Scholar
  61. 61.
    S. K. Nayar, K. Ikeuchi, T. Kanade (1988) Extracting shape and reflectance of Lam-bertian, specular and hybrid surfaces, Technical Report CMU-FU-TR-88-14, The Robotics Institute, Carnegie Mellon University.Google Scholar
  62. 62.
    S. K. Nayar, K. Ikeuchi, T. Kanade (1990) Determining shape and reflectance of hybrid surfaces by photometric sampling, IEEE Transactions on Robotics and Automation, 6(1):418–431.CrossRefGoogle Scholar
  63. 63.
    S. K. Nayar, K. Ikeuchi, T. Kanade (1990) Shape from interreflections, Technical Report CMU-RI-TR-90-14, Carnegie-Mellon University of Pittsburgh, PA, Robotics Institute.Google Scholar
  64. 64.
    S. K. Nayar, K. Ikeuchi, T. Kanade (1991) Surface reflection: Physical and geometrical perspectives, PAMI’99, 13:611–634.CrossRefGoogle Scholar
  65. 65.
    S. K. Nayar, X.-S. Fang, T. Boult (1997) Separation of reflection components using color and polarization, IJCV’97, 21(3):163–186, doi: 10.1023/A:1007937815113.CrossRefGoogle Scholar
  66. 66.
    D. Nehab, S. Rusinkiewicz, J. Davis, R. Ramamoorthi (2005) Efficiently combining positions and normals for precise 3d geometry, SIGGRAPH’05, 24(3):536–543, doi: http://doi.acm.org/10.1145/1073204.1073226.Google Scholar
  67. 67.
    F. Nicodemus, J. Richmond, J. Hsia, I. Ginsberg, T. Limperis (1977) Geometrical considerations and nomenclature for reflectance, Technical report, U.S. Department of Commerce, National Bureau of Standards.Google Scholar
  68. 68.
    E. North Coleman, Jr. and R. Jain (1982) Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry, Computer Graphics and Image Processing, 18:309–328, doi: http://dx.doi.org/10.1016/0146-664X(82)90001-6.CrossRefGoogle Scholar
  69. 69.
    T. Peng (2006) Algorithms and models for 3-D shape measurement using digital fringe projections, PhD thesis, University of Maryland, Department for Mechanical Engineering.Google Scholar
  70. 70.
    R. Penrose (1955) A generalized inverse for matrices, Proceedings of the Cambridge Philosophical Society, 51:406–413.CrossRefMATHMathSciNetGoogle Scholar
  71. 71.
    B. T. Phong (1975) Illumination for computer generated pictures. Communications of the ACM, 18(6):311–317, doi: http://doi.acm.org/10.1145/360825.360839.CrossRefGoogle Scholar
  72. 72.
    E. Prados and O. Faugeras (2003) Perspective shape from shading and viscosity solutions, ICCV’03, 2:826–831.Google Scholar
  73. 73.
    R. Ramamoorthi (2002) A signal processing framework for forward and inverse rendering, PhD thesis, Stanford University.Google Scholar
  74. 74.
    Y. Sato, M. D. Wheeler, K. Ikeuchi (1997) Object shape and reflectance modeling from observation. Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 379–387, doi:http://doi.acm.org/10.1145/258734.258885.
  75. 75.
    M. Seitz (1999) An overview of passive vision techniques. Technical report, The Robotics Institute, Carnegie Mellon University.Google Scholar
  76. 76.
    L. Shen and P. Tan (2009) Photometric stereo and weather estimation using internet images, CVPR’09, 1:1850–1857.Google Scholar
  77. 77.
    B. Shi, Y. Matsushita, Y. Wei, C. Xu, P. Tan (2010) Self-calibrating photometric stereo, CVPR’10.Google Scholar
  78. 78.
    W. M. Silver (1980) Determining shape and reflectance using multiple images, Master’s thesis, MIT, Computer Science and Artificial Intelligence Laboratory.Google Scholar
  79. 79.
    D. Simakov, D. Frolova, R. Basri (2003) Dense shape reconstruction of a moving object under arbitrary, unknown lighting, ICCV’03, 2:1202.Google Scholar
  80. 80.
    M. M. Stark, J. Arvo, B. Smits (2005) Barycentric parameterizations for isotropic BRDFs, IEEE Transactions on Visualization and Computer Graphics, 11(2):126–138, doi: http://dx.doi.org/10.1109/TVCG.2005.26.CrossRefGoogle Scholar
  81. 81.
    R. Szeliski (2010) Computer Vision Algorithms and Applications, online course material.Google Scholar
  82. 82.
    P. Tan and T. Zickler (2009) A projective framework for radiometric image analysis, CVPR 2009, pages 2977–2984, doi: http://dx.doi.org/10.1109/CVPR.2009.5206731.
  83. 83.
    P. Tan, S. Lin, L. Quan, H.-Y. Shum (2003) Highlight removal by illumination-constraint inpainting, ICCV’03, 1:164–169.Google Scholar
  84. 84.
    P. Tan, S. Lin, L. Quan (2006) Resolution-enhanced photometric stereo, ECCV’06, 3:58–71.Google Scholar
  85. 85.
    P. Tan, S. P. Mallick, L. Quan, D. J. Kriegman, T. Zickler (2007) Isotropy, reciprocity and the generalized bas-relief ambiguity, CVPR 2007, pages 1–8.Google Scholar
  86. 86.
    P. Tan, S. Lin, L. Quan (2008) Subpixel photometric stereo, PAMI’08, 30(8):1460–1471.Google Scholar
  87. 87.
    R. T. Tan and K. Ikeuchi (2003) Separating reflection components of textured surfaces using a single image, ICCV 2003, 1:870–877.Google Scholar
  88. 88.
    R. T. Tan and K. Ikeuchi (2005) Separating reflection components of textured surfaces using a single image, PAMI’05, 27(2):179–193.Google Scholar
  89. 89.
    A. Tankus, N. Sochen, Y. Yeshurun (2005) Shapefrom-shading under perspective projection, IJCV’05, 63(1):21–43.CrossRefGoogle Scholar
  90. 90.
    D. Thomas and A. Sugimoto (2010) Range image registration of specular objects under complex illumination, Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT2010).Google Scholar
  91. 91.
    C. Tomasi and T. Kanade (1992) Shape and motion from image streams under orthography: a factorization method, IJCV’92, 9(2):137–154, doi: http://dx.doi.org/10.1007/BF00129684.CrossRefGoogle Scholar
  92. 92.
    E. Torrance and E. M. Sparrow (1967) Theory for off-specular reflection from roughened surfaces, Journal of the Optical Society of America A (JOSA A), 57(9):1105–1114.CrossRefGoogle Scholar
  93. 93.
    H. von Helmholtz (1924) Handbuch der Physiologischen Optik, Optical Society of America.Google Scholar
  94. 94.
    G. J. Ward (1992) Measuring and modeling anisotropic reflection, ACM SIGGRAPH’92, 26(2):265–272.CrossRefGoogle Scholar
  95. 95.
    T. Weise, B. Leibe, L. Van Gool (2007) Fast 3d scanning with automatic motion compensation, CVPR’07, pages 1–8.Google Scholar
  96. 96.
    T. Weyrich, J. Lawrence, H. Lensch, S. Rusinkiewicz, T. Zickler (2008) Principles of appearance acquisition and representation, ACM SIGGRAPH 2008 classes, none: 1–70, doi: http://doi.acm.org/10.1145/1401132.1401234.
  97. 97.
    D. R. White, P. Saunders, S. J. Bonsey, J. van de Ven, H. Edgar (1998) Reflectometer for measuring the bidirectional reflectance of rough surfaces, Applied Optics, 37(16):3450–3454, doi: doi:10.1364/AO.37.003450.CrossRefGoogle Scholar
  98. 98.
    C. Wöhler (2009) 3D Computer Vision — Efficient Methods and Applications, Springer, 1st edition.Google Scholar
  99. 99.
    C. Wöhler and P. d’Angelo (2009) Stereo image analysis of non-Lambertian surfaces, IJCV’09, 81(2):529–540, doi: http://dx.doi.org/10.1007/s11263-008-0157-1.Google Scholar
  100. 100.
    L. B. Wolff (1989) Using polarization to separate reflection components, CVPR’89, 1(1):363–369, doi: http://dx.doi.org/10.1109/CVPR.1989.37873.Google Scholar
  101. 101.
    L. B. Wolff and T. E. Boult (1991) Constraining object features using a polarization reflectance model, PAMI’91, 13(7):635–657, doi:http://dx.doi.org/10.1109/34.85655.CrossRefGoogle Scholar
  102. 102.
    R. J. Woodham (1980) Photometric method for determining surface orientation from multiple images, Optical Engineering, 19(1):139–144.Google Scholar
  103. 103.
    Y. Yu, P. Debevec, J. Malik, T. Hawkins (1999) Inverse global illumination: Recovering reflectance models of real scenes from photographs, SIGGRAPH1999, pages 215–224, doi: http://doi.acm.org/10.1145/311535.311559.
  104. 104.
    A. L. Yuille, J. M. Coughlan, S. Konishi (2000) The generic viewpoint constraint resolves the generalized bas relief ambiguity, Conference on Information Science and Systems.Google Scholar
  105. 105.
    L. Zhang, B. Curless, A. Hertzmann, S. M. Seitz (2003) Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo, ICCV’03, 1:618–626.Google Scholar
  106. 106.
    R. Zhang, P.-S. Tsai, J. E. Cryer, M. Shah (1999) Shape from shading: A survey, PAMI’99, 21(8):690–706.CrossRefGoogle Scholar
  107. 107.
    Zhou and P. Tan (2010) Ring-light photometric stereo, ECCV’10, pages 1–14.Google Scholar
  108. 108.
    T. Zickler, P. N. Belhumeur, D. J. Kriegman (2002) Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction, ECCV’02, 3:869–884.Google Scholar

Copyright information

© 3D Display Research Center and Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Image Analysis GroupTU Dortmund UniversityDortmundGermany

Personalised recommendations