Abstract
Photometric 3D-reconstruction techniques aim at inferring the geometry of a scene from one or several images, by inverting a physical model describing the image formation. This chapter presents an introductory overview of the main photometric 3D-reconstruction techniques which are shape-from-shading, photometric stereo and shape-from-polarisation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adelson EH, Pentland AP (1996) The perception of shading and reflectance. In: Perception as Bayesian inference. Cambridge University Press, Cambridge, pp 409–423
AliceVision. https://alicevision.org/
Atkinson GA, Hancock ER (2005) Multi-view surface reconstruction using polarization. In: Proceedings of the IEEE international conference on computer vision 1:309–316
Atkinson GA, Hancock ER (2006) Recovery of surface orientation from diffuse polarization. IEEE Trans Image Process 15(6):1653–1664
Atkinson GA, Hancock ER (2007) Shape estimation using polarization and shading from two views. IEEE Trans Pattern Anal Mach Intell 29(11):2001–2017
Atkinson GA, Hancock ER (2007) Surface reconstruction using polarization and photometric stereo. In: Proceedings of the international conference on computer analysis of images and patterns, pp 466–473
Bähr M, Breuß M, Quéau Y, Boroujerdi AS, Durou J-D (2017) Fast and accurate surface normal integration on non-rectangular domains. Comput Vis Media 3(2):107–129
Barles G (1994) Solutions de viscosité des équations de Hamilton-Jacobi. Mathématiques et Applications, vol 17. Springer, Berlin
Barron JT, Malik J (2015) Shape, illumination, and reflectance from shading. IEEE Trans Pattern Anal Mach Intell 37(8):1670–1687
Bartoli A, Gérard Y, Chadebecq F, Collins T, Pizarro D (2015) Shape-from-template. IEEE Trans Pattern Anal Mach Intell 37(10):2099–2118
Basri R, Jacobs D, Kemelmacher I (2007) Photometric stereo with general, unknown lighting. Int J Comput Vis 72(3):239–257
Belhumeur PN, Kriegman DJ, Yuille AL (1999) The Bas-relief ambiguity. Int J Comput Vis 35(1):33–44
Berger K, Voorhies R, Matthies L (2016) Incorporating polarization in stereo vision-based 3D perception of non-Lambertian scenes. In: Unmanned systems technology XVIII. Proceedings of the SPIE, vol 9837, p. 98370P
Berger K, Voorhies R, Matthies LH (2017) Depth from stereo polarization in specular scenes for urban robotics. In: Proceedings of the international conference on robotics and automation, pp 1966–1973
Brady M, Yuille AL (1984) An extremum principle for shape from contour. IEEE Trans Pattern Anal Mach Intell 6(3):288–301
Breuß M, Cristiani E, Durou J-D, Falcone M, Vogel O (2012) Perspective shape from shading: ambiguity analysis and numerical approximations. SIAM J Imaging Sci 5(1):311–342
Camilli F, Grüne L (2000) Numerical approximation of the maximal solutions for a class of degenerate Hamilton-Jacobi equations. SIAM J Numer Anal 38(5):1540–1560
Camilli F, Tozza S (2017) A unified approach to the well-posedness of some non-Lambertian models in shape-from-shading theory. SIAM J Imaging Sci 10(1):26–46
Chen G, Han K, Shi B, Matsushita Y, Wong K-YK (2019) Self-calibrating deep photometric stereo networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8739–8747
Chen L, Zheng Y, Subpa-Asa A, Sato I (2018) Polarimetric three-view geometry. In: Proceedings of the European conference on computer vision, pp 20–36
Clark JJ (2010) Photometric stereo using LCD displays. Image Vis Comput 28(4):704–714
Collins T, Bartoli A (2012) 3D-reconstruction in laparoscopy with close-range photometric stereo. In: International conference on medical image computing and computer-assisted intervention. Lecture notes in computer science, vol 7511, pp 634–642
Courteille F, Durou J-D, Morin G (2006) A global solution to the SFS problem using B-spline surface and simulated annealing. In: Proceedings of the international conference on pattern recognition (volume II), pp 332–335
Crandall MG, Lions P-L (1983) Viscosity solutions of Hamilton-Jacobi equations. Trans Am Math Soc 277(1):1–42
Cristiani E (2014) 3D printers: a new challenge for mathematical modeling. arXiv:1409.1714
Cristiani E, Falcone M (2007) Fast semi-Lagrangian schemes for the eikonal equation and applications. SIAM J Numer Anal 45(5):1979–2011
Crouzil A, Descombes X, Durou J-D (2003) A multiresolution approach for shape from shading coupling deterministic and stochastic optimization. IEEE Trans Pattern Anal Mach Intell 25(11):1416–1421
Cui Z, Gu J, Shi B, Tan P, Kautz J (2017) Polarimetric multi-view stereo. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1558–1567
Daniel P, Durou J-D (2000) From deterministic to stochastic methods for shape from shading. In: Proceedings of the Asian conference on computer vision, pp 187–192
Durou J-D, Maître H (1996) On convergence in the methods of Strat and of Smith for shape from shading. Int J Comput Vis 17(3):273–289
Durou J-D, Falcone M, Sagona M (2008) Numerical methods for shape-from-shading: a new survey with benchmarks. Comput Vis Image Underst 109(1):22–43
Falcone M, Ferretti R (2014) Semi-Lagrangian approximation schemes for linear and Hamilton-Jacobi equations, 1st edn. Society for Industrial and Applied Mathematics, Philadelphia
Falcone M, Ferretti R (2016) Numerical methods for Hamilton-Jacobi type equations. In: Handbook of numerical methods for hyperbolic problems. Handbook of numerical analysis, vol 17. Elsevier, Amsterdam, pp 603–626
Falcone M, Sagona M (1997) An algorithm for the global solution of the shape-from-shading model. In: International conference on image analysis and processing. Lecture notes in computer science, vol 1310, pp 596–603
Falcone M, Sagona M (2003) A scheme for the shape-from-shading model with “black shadows”. In: Numerical mathematics and advanced applications. Springer, Berlin, pp 503–512
Festa A, Falcone M (2014) An approximation scheme for an Eikonal equation with discontinuous coefficient. SIAM J Numer Anal 52(1):236–257
Frankot RT, Chellappa R (1988) A method for enforcing integrability in shape from shading algorithms. IEEE Trans Pattern Anal Mach Intell 10(4):439–451
Gallardo M, Collins T, Bartoli A (2017) Dense non-rigid structure-from-motion and shading with unknown albedos. In: Proceedings of the IEEE international conference on computer vision, pp 3884–3892
Geng J (2011) Structured-light 3D surface imaging: a tutorial. Adv Opt Photonics 3(2):128–160
Gotardo PFU, Simon T, Sheikh Y, Matthews I (2015) Photogeometric scene flow for high-detail dynamic 3D reconstruction. In: Proceedings of the IEEE international conference on computer vision, pp 846–854
Haefner B, Quéau Y, Möllenhoff T, Cremers D (2018) Fight ill-posedness with ill-posedness: single-shot variational depth super-resolution from shading. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 164–174
Haefner B, Ye Z, Gao M, Wu T, Quéau Y, Cremers D (2019) Variational uncalibrated photometric stereo under general lighting. In: Proceedings of the IEEE international conference on computer vision, pp 8539–8548
Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge
Hernández C (2004) Stereo and silhouette fusion for 3D object modeling from uncalibrated images under circular motion. Thèse de doctorat, École Nationale Supérieure des Télécommunications
Herrera SEM, Malti A, Morel O, Bartoli A (2013) Shape-from-polarization in laparoscopy. In: Proceedings of the international symposium on biomedical imaging, pp 1412–1415
Hertzmann A, Seitz SM (2005) Example-based photometric stereo: shape reconstruction with general, varying BRDFs. IEEE Trans Pattern Anal Mach Intell 27(8):1254–1264
Hold-Geoffroy Y, Gotardo P, Lalonde J-F (2019) Single day outdoor photometric stereo. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2962693
Horn BKP (1970) Shape from shading: a method for obtaining the shape of a smooth opaque object from one view. PhD thesis, MIT
Horn BKP, Brooks MJ (1986) The variational approach to shape from shading. Comput Vis Graph Image Process 33(2):174–208
Huynh CP, Robles-Kelly A, Hancock ER (2010) Shape and refractive index recovery from single-view polarisation images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1229–1236
Huynh CP, Robles-Kelly A, Hancock ER (2013) Shape and refractive index from single-view spectro-polarimetric images. Int J Comput Vis 101(1):64–94
Ikehata S, Wipf D, Matsushita Y, Aizawa K (2012) Robust photometric stereo using sparse regression. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 318–325
Ikeuchi K, Horn BKP (1981) Numerical shape from shading and occluding boundaries. Artif Intell 17(1–3):141–184
Jancosek M, Pajdla T (2011) Multi-view reconstruction preserving weakly-supported surfaces. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3121–3128
Johnson MK, Adelson EH (2011) Shape estimation in natural illumination. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2553–2560
Ju Y-C, Tozza S, Breuß M, Bruhn A, Kleefeld A (2013) Generalised perspective shape from shading with Oren-Nayar reflectance. In: Proceedings of the British machine vision conference, pp 42.1–42.11
Kadambi A, Taamazyan V, Shi B, Raskar R (2015) Polarized 3D: high-quality depth sensing with polarization cues. In: Proceedings of the IEEE international conference on computer vision, pp 3370–3378
Kadambi A, Taamazyan V, Shi B, Raskar R (2017) Depth sensing using geometrically constrained polarization normals. Int J Comput Vis 125:34–51
Kong N, Tai Y-W, Shin JS (2013) A physically-based approach to reflection separation: from physical modeling to constrained optimization. IEEE Trans Pattern Anal Mach Intell 36(2):209–221
Kontsevich LL, Petrov AP, Vergelskaya IS (1994) Reconstruction of shape from shading in color images. J Opt Soc Am A 11(3):1047–1052
Langguth F, Sunkavalli K, Hadap S, Goesele M (2016) Shading-aware multi-view stereo. In: Proceedings of the European conference on computer vision, pp 469–485
Leclerc YG, Bobick AF (1991) The direct computation of height from shading. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 552–558
Li M, Zhou Z, Wu Z, Shi B, Diao C, Tan P (2020) Multi-view photometric stereo: a robust solution and benchmark dataset for spatially varying isotropic materials. IEEE Trans Image Process pp 4159–4173
Lions P-L, Rouy E, Tourin A (1993) Shape-from-shading, viscosity solutions and edges. Numer Math 64(1):323–353
Logothetis F, Mecca R, Sgallari F, Cipolla R (2019) A differential approach to shape from polarisation: a level-set characterisation. Int J Comput Vis 127(11–12):1680–1693
Lyu Y, Cui Z, Li S, Pollefeys M, Shi B (2019) Reflection separation using a pair of unpolarized and polarized images. In: Advances in Neural Information Processing Systems 32, Curran Associates, Inc., pp 14559–14569
Mahmoud AH, El-Melegy MT, Farag AA (2012) Direct method for shape recovery from polarization and shading. In: Proceedings of the IEEE international conference on image processing, pp 1769–1772
Maurer D, Ju Y-C, Breuß M, Bruhn A (2018) Combining shape from shading and stereo: a joint variational method for estimating depth, illumination and albedo. Int J Comput Vis 126(12):1342–1366
Mecca R, Falcone M (2013) Uniqueness and approximation of a photometric shape-from-shading model. SIAM J Imaging Sci 6(1):616–659
Mecca R, Wetzler A, Bruckstein A, Kimmel R (2014) Near field photometric stereo with point light sources. SIAM J Imaging Sci 7(4):2732–2770
Mecca R, Quéau Y, Logothetis F, Cipolla R (2016) A single-lobe photometric stereo approach for heterogeneous material. SIAM J Imaging Sci 9(4):1858–1888
Mélou J, Quéau Y, Castan F, Durou J-D (2019) A splitting-based algorithm for multi-view stereopsis of textureless objects. In: Proceedings of the international conference on scale space and variational methods in computer vision, pp 51–63
Miyazaki D, Kagesawa M, Ikeuchi K (2004) Transparent surface modeling from a pair of polarization images. IEEE Trans Pattern Anal Mach Intell 26(1):73–82
Miyazaki D, Saito M, Sato Y, Ikeuchi K (2002) Determining surface orientations of transparent objects based on polarization degrees in visible and infrared wavelengths. J Opt Soc Am A 19(4):687–694
Miyazaki D, Shigetomi T, Baba M, Furukawa R, Hiura S, Asada N (2012) Polarization-based surface normal estimation of black specular objects from multiple viewpoints. In: Proceedings of the international conference on 3D imaging, modeling, processing, visualization and transmission, pp 104–111
Miyazaki D, Shigetomi T, Baba M, Furukawa R, Hiura S, Asada N (2016) Surface normal estimation of black specular objects from multiview polarization images. Opt Eng 56(4):041303
Miyazaki D, Tan RT, Hara K, Ikeuchi K (2003) Polarization-based inverse rendering from a single view. In: Proceedings of the IEEE international conference on computer vision, pp 982–987
Moons T, Van Gool L, Vergauwen M (2008) 3D reconstruction from multiple images, part 1: principles. Found Trends Comput Graph Vis 4(4):287–404
Morel O, Meriaudeau F, Stolz C, Gorria P (2005) Polarization imaging applied to 3D reconstruction of specular metallic surfaces. In: Machine vision applications in industrial inspection XIII. Proceedings of the SPIE, vol 5679, pp 178–186
Nayar S, Fang X, Boult T (1997) Separation of reflection components using color and polarization. Int J Comput Vis 21(3):163–186
Nayar SK, Nakagawa Y (1994) Shape from focus. IEEE Trans Pattern Anal Mach Intell 16(8):824–831
Nehab D, Rusinkiewicz S, Davis J, Ramamoorthi R (2005) Efficiently combining positions and normals for precise 3D geometry. ACM Trans Graph 24(3):536–543
Ngo TT, Nagahara H, Taniguchi R (2015) Shape and light directions from shading and polarization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2310–2318
Onn R, Bruckstein AM (1990) Integrability disambiguates surface recovery in two-image photometric stereo. Int J Comput Vis 5(1):105–113
Or-El R, Rosman G, Wetzler A, Kimmel R, Bruckstein A (2015) RGBD-fusion: real-time high precision depth recovery. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5407–5416
Osher S, Fedkiw R (2003) Level set methods and dynamic implicit surfaces. In: Applied mathematical sciences, vol 153. Springer, Berlin
Peng S, Haefner B, Quéau Y, Cremers D (2017) Depth super-resolution meets uncalibrated photometric stereo. In: Proceedings of the IEEE international conference on computer vision workshops, pp 2961–2968
Pintus R, Dulecha TG, Ciortan I, Gobbetti E, Giachetti A (2019) State-of-the-art in multi-light image collections for surface visualization and analysis. Comput Graph Forum 38(3):909–934
Prados E, Faugeras O (2005) Shape from shading: a well-posed problem? Proceedings of the IEEE conference on computer vision and pattern recognition 2:870–877
Quéau Y, Mecca R, Durou J-D (2016) Unbiased photometric stereo for colored surfaces: a variational approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4359–4368
Quéau Y, Mecca R, Durou J-D, Descombes X (2017) Photometric stereo with only two images: a theoretical study and numerical resolution. Image Vis Comput 57:175–191
Quéau Y, Mélou J, Castan F, Cremers D, Durou J-D (2017) A variational approach to shape-from-shading under natural illumination. In: Proceedings of the international workshop on energy minimization methods in computer vision and pattern recognition, pp 342–357
Quéau Y, Wu T, Lauze F, Durou J-D, Cremers D (2017) A non-convex variational approach to photometric stereo under inaccurate lighting. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 99–108
Quéau Y, Durix B, Wu T, Cremers D, Lauze F, Durou J-D (2018) LED-based photometric stereo: modeling, calibration and numerical solution. J Math Imaging Vis 60(3):313–340
Quéau Y, Durou J-D, Aujol J-F (2018) Normal integration: a survey. J Math Imaging Vis 60(4):576–593
Rahmann S, Canterakis N (2001) Reconstruction of specular surfaces using polarization imaging. In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 1
Robles-Kelly A, Huynh CP (2013) Imaging spectroscopy for scene analysis. Springer, Berlin
Santo H, Samejima M, Sugano Y, Shi B, Matsushita Y (2017) Deep photometric stereo network. In: Proceedings of the IEEE international conference on computer vision workshops, pp 501–509
Schechner YY (2011) Inversion by \(P^4\): polarization-picture post-processing. Philos Trans R Soc B: Biol Sci 366(1565):638–648
Schechner YY (2015) Self-calibrating imaging polarimetry. In: Proceedings of the IEEE international conference on computational photography
Schechner YY, Shamir J, Kiryati N (2000) Polarization and statistical analysis of scenes containing a semireflector. J Opt Soc Am A 17(2):276–284
Sengupta S, Kanazawa A, Castillo CD, Jacobs DW (2018) SfSNet: learning shape, reflectance and illuminance of faces in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6296–6305
Sethian JA (1999) Level set methods and fast marching methods. Cambridge monographs on applied and computational mathematics, vol 3, 2nd edn. Cambridge University Press, Cambridge
Shabayek AER, Demonceaux C, Morel O, Fofi D (2012) Vision based UAV attitude estimation: progress and insights. J Intell Robot Syst 65(1–4):295–308
Shafer SA, Kanade T (1983) Using shadows in finding surface orientations. Comput Vis Graph Image Process 22(1):145–176
Shi B, Tan P, Matsushita Y, Ikeuchi K (2013) Bi-polynomial modeling of low-frequency reflectances. IEEE Trans Pattern Anal Mach Intell 36(6):1078–1091
Shi B, Mo Z, Wu Z, Duan D, Yeung S, Tan P (2019) A benchmark dataset and evaluation for non-Lambertian and uncalibrated photometric stereo. IEEE Trans Pattern Anal Mach Intell 41(2):271–284
Shurcliff WA (1962) Polarized light, production and use. Harvard University Press, Harvard
Simchony T, Chellappa R, Shao M (1990) Direct analytical methods for solving Poisson equations in computer vision problems. IEEE Trans Pattern Anal Mach Intell 12(5):435–446
Smith WAP, Fang F (2016) Height from photometric ratio with model-based light source selection. Comput Vis Image Underst 145:128–138
Smith WAP, Ramamoorthi R, Tozza S (2016) Linear depth estimation from an uncalibrated, monocular polarisation image. In: European conference on computer vision. Lecture notes in computer science, vol 9912, pp 109–125
Smith WAP, Ramamoorthi R, Tozza S (2019) Height-from-polarisation with unknown lighting or albedo. IEEE Trans Pattern Anal Mach Intell 41(12):2875–2888
Soukup D, Huber-Mörk R (2014) Convolutional neural networks for steel surface defect detection from photometric stereo images. In: International symposium on visual computing. Lecture notes in computer science, vol 8887, pp 668–677
Sun J, Smith M, Smith L, Coutts L, Dabis R, Harland C, Bamber J (2008) Reflectance of human skin using colour photometric stereo: with particular application to pigmented lesion analysis. Skin Res Technol 14(2):173–179
Szeliski R (1991) Fast shape from shading. Comput Vis Graph Image Process: Image Underst 53(2):129–153
Taamazyan V, Kadambi A, Raskar R (2016) Shape from mixed polarization. arXiv:1605.02066
Tankus A, Sochen N, Yeshurun Y (2003) A new perspective [on] shape-from-shading. In: Proceedings of the IEEE international conference on computer vision 2:862–869
Tozza S, Falcone M (2016) Analysis and approximation of some shape-from-shading models for non-Lambertian surfaces. J Math Imaging Vis 55(2):153–178
Tozza S, Mecca R, Duocastella M, Del Bue A (2016) Direct differential photometric stereo shape recovery of diffuse and specular surfaces. J Math Imaging Vis 56(1):57–76
Tozza S, Smith WAP, Zhu D, Ramamoorthi R, Hancock ER (2017) Linear differential constraints for photo-polarimetric height estimation. In: Proceedings of the IEEE international conference on computer vision, pp 2298–2306
Tuchin VV, Wang L, Zimnyakov DA (2006) Optical polarization in biomedical applications. Springer Science & Business Media, New York
Vogiatzis G, Hernández C, Cipolla R (2006) Reconstruction in the round using photometric normals and silhouettes. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2:1847–1854
White R, Forsyth D (2006) Combining cues: shape from shading and texture. In: Proceedings of the IEEE conference on computer vision and pattern recognition 2:1809–1816
Wieschollek P, Gallo O, Gu J, Kautz J (2018) Separating reflection and transmission images in the wild. In: Proceedings of the European conference on computer vision, pp 89–104
Witkin AP (1981) Recovering surface shape and orientation from texture. Artif Intell 17(1–3):17–45
Wolff LB (1990) Surface orientation from two camera stereo with polarizers. In: Optics, illumination, and image sensing for machine vision IV. Proceedings of the SPIE, vol 1194, pp 287–298
Wolff LB (1997) Polarization vision: a new sensory approach to image understanding. Image Vis Comput 15(2):81–93
Wolff LB, Boult TE (1991) Constraining object features using a polarization reflectance model. IEEE Trans Pattern Anal Mach Intell 13(7):635–657
Woodham RJ (1980) Photometric method for determining surface orientation from multiple images. Opt Eng 19(1):134–144
Worthington PL, Hancock ER (1999) Needle map recovery using robust regularizers. Image Vis Comput 17(8):545–557
Wu L, Ganesh A, Shi B, Matsushita Y, Wang Y, Ma Y (2010) Robust photometric stereo via low-rank matrix completion and recovery. In: Proceedings of the Asian conference on computer vision, pp 703–717
Yang L, Tan F, Li A, Cui Z, Furukawa Y, Tan P (2018) Polarimetric dense monocular SLAM. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3857–3866
Yuille AL, Snow D, Epstein R, Belhumeur PN (1999) Determining generative models of objects under varying illumination: shape and albedo from multiple images using SVD and integrability. Int J Comput Vis 35(3):203–222
Zhang R, Tsai P-S, Cryer JE, Shah M (1999) Shape-from-shading: a survey. IEEE Trans Pattern Anal Mach Intell 21(8):690–706
Zhu D, Smith WAP (2019) Depth from a polarisation + RGB stereo pair. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7586–7595
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Durou, JD., Falcone, M., Quéau, Y., Tozza, S. (2020). A Comprehensive Introduction to Photometric 3D-Reconstruction. In: Durou, JD., Falcone, M., Quéau, Y., Tozza, S. (eds) Advances in Photometric 3D-Reconstruction. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-030-51866-0_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-51866-0_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-51865-3
Online ISBN: 978-3-030-51866-0
eBook Packages: Computer ScienceComputer Science (R0)