Skip to main content
SpringerLink
Log in

Direct Differential Photometric Stereo Shape Recovery of Diffuse and Specular Surfaces

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Recovering the 3D shape of an object from shading is a challenging problem due to the complexity of modeling light propagation and surface reflections. Photometric Stereo (PS) is broadly considered a suitable approach for high-resolution shape recovery, but its functionality is restricted to a limited set of object surfaces and controlled lighting setup. In particular, PS models generally consider reflection from objects as purely diffuse, with specularities being regarded as a nuisance that breaks down shape reconstruction. This is a serious drawback for implementing PS approaches, since most common materials have prominent specular components. In this paper, we propose a PS model that solves the problem for both diffuse and specular components aimed at shape recovery of generic objects with the approach being independent of the albedo values thanks to the image ratio formulation used. Notably, we show that by including specularities, it is possible to solve the PS problem for a minimal number of three images using a setup with three calibrated lights and a standard industrial camera. Even if an initial separation of diffuse and specular components is still required for each input image, experimental results on synthetic and real objects demonstrate the feasibility of our approach for shape reconstruction of complex geometries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Anderson, R., Stenger, B., Cipolla, R.: Color photometric stereo for multicolored surfaces. In: 13th International Conference on Computer Vision (ICCV 2011), Barcelona, Spain (2011)

  2. Artusi, A., Banterle, F., Chetverikov, D.: A survey of specularity removal methods. Comput. Graph. Forum 30(8), 2208–2230 (2011)

    Article  Google Scholar 

  3. Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1239–1252 (2003)

    Article  Google Scholar 

  4. Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72(3), 239–257 (2007)

    Article  Google Scholar 

  5. Belhumeur, P.N., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. Int. J. Comput. Vis. 35(1), 33–44 (1999)

    Article  Google Scholar 

  6. Blinn, J.F.: Models of light reflection for computer synthesized pictures. Comput. Graph. 11(2), 192–198 (1977)

    Article  Google Scholar 

  7. Chandraker, M., Bai, J., Ramamoorthi, R.: On differential photometric reconstruction for unknown, isotropic BRDFs. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2941–2955 (2013)

    Article  Google Scholar 

  8. Chung, H.S., Jia, J.: Efficient photometric stereo on glossy surfaces with wide specular lobes. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008)

  9. Cook, R.L., Torrance, K.E.: A reflectance model for computer graphics. ACM Trans. Graph. 1(1), 7–24 (1982)

    Article  Google Scholar 

  10. Del Bue, A., Xavier, J., Agapito, L., Paladini, M.: Bilinear modeling via augmented lagrange multipliers (balm). IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1496–1508 (2012)

    Article  Google Scholar 

  11. Frankot, R., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 439–451 (1988)

    Article  MATH  Google Scholar 

  12. Hayakawa, H.: Photometric stereo under a light source with arbitrary motion. JOSA A 11(11), 3079–3089 (1994)

    Article  MathSciNet  Google Scholar 

  13. Hernández, C., Vogiatzis, G., Brostow, G., Stenger, B., Cipolla, R.: Non-rigid photometric stereo with colored lights. In: 11th International Conference on Computer Vision (ICCV 2011), pp. 1–8. Rio de Janeiro, Brazil (2007)

  14. Higo, T., Matsushita, Y., Ikeuchi, K.: Consensus photometric stereo. In: The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, pp. 1157–1164. San Francisco, CA, USA, 13–18 June 2010

  15. Ikehata, S., Aizawa, K.: Photometric stereo using constrained bivariate regression for general isotropic surfaces. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2187–2194 (2014)

  16. Ikehata, S., Wipf, D., Matsushita, Y., Aizawa, K.: Robust photometric stereo using sparse regression. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 318–325 (2012)

  17. Julià, C., Lumbreras, F., Sappa, A.D.: A factorization-based approach to photometric stereo. Int. J. Imaging Syst. Technol. 21(1), 115–119 (2011)

    Article  Google Scholar 

  18. Kim, H., Jin, H., Hadap, S., Kweon, I.: Specular reflection separation using dark channel prior. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1460–1467 (2013)

  19. Lu, F., Matsushita, Y., Sato, I., Okabe, T., Sato, Y.: Uncalibrated photometric stereo for unknown isotropic reflectances. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp. 1490–1497. IEEE Computer Society, Washington, DC, USA (2013)

  20. Mecca, R., Falcone, M.: Uniqueness and approximation of a photometric shape-from-shading model. SIAM J. Imaging Sci. 6(1), 616–659 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Mecca, R., Tankus, A., Wetzler, A., Bruckstein, A.: A direct differential approach to photometric stereo with perspective viewing. SIAM J. Imaging Sci. 7(2), 579–612 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  22. Mecca, R., Tozza, S.: Shape reconstruction of symmetric surfaces using photometric stereo. In: Breuss, M., Bruckstein, A., Maragos, P. (eds.) Innovations for Shape Analysis: Models and Algorithms, pp. 219–243. Springer, Berlin (2013)

    Chapter  Google Scholar 

  23. Mecca, R., Wetzler, A., Kimmel, R., Bruckstein, A.M.: Direct shape recovery from photometric stereo with shadows. In: 3DV, pp. 382–389 (2013)

  24. Miyazaki, D., Ikeuchi, K.: Photometric stereo under unknown light sources using robust svd with missing data. In: 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 4057–4060. IEEE (2010)

  25. Nayar, S., Ikeuchi, K., Kanade, T.: Surface reflection: physical and geometrical perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 13(7), 611–634 (1991)

    Article  Google Scholar 

  26. Ngan, A., Durand, F., Matusik, W.: Experimental analysis of BRDF models. In: Proceedings of the Sixteenth Eurographics Conference on Rendering Techniques, EGSR ’05, pp. 117–126 (2005)

  27. Papadhimitri, T., Favaro, P.: A closed-form, consistent and robust solution to uncalibrated photometric stereo via local diffuse reflectance maxima. Int. J. Comput. Vis. 107(2), 139–154 (2014)

    Article  MATH  Google Scholar 

  28. Phong, B.T.: Illumination for computer generated pictures. Commun. ACM 18(6), 311–317 (1975)

    Article  Google Scholar 

  29. Qian, J., Zhang, Y.T., Zhao, H.K.: Fast sweeping methods for eikonal equations on triangular meshes. SIAM J. Numer. Anal. 45(1), 83–107 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  30. Quarteroni, A., Valli, A.: Numerical Approximation of Partial Differential Equations. Springer, Berlin (1994)

    MATH  Google Scholar 

  31. Salvi, J., Pages, J., Batlle, J.: Pattern codification strategies in structured light systems. Pattern Recognit. 37(4), 827–849 (2004)

    Article  MATH  Google Scholar 

  32. Shen, H.L., Zheng, Z.H.: Real-time highlight removal using intensity ratio. Appl. Opt. 52(19), 4483–4493 (2013)

    Article  Google Scholar 

  33. Shi, B., Matsushita, Y., Wei, Y., Xu, C., Tan, P.: Self-calibrating photometric stereo. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1118–1125 (2010)

  34. Sills, K., Capson, D., Bone, G.: Specular-reduced imaging for inspection of machined surfaces. In: 2012 Ninth Conference on Computer and Robot Vision (CRV), pp. 361–368 (2012)

  35. Strikwerda, J.: Finite Difference Schemes and Partial Differential Equations. SIAM, Philadelphia (2004)

    Book  MATH  Google Scholar 

  36. Wolff, L.: On the relative brightness of specular and diffuse reflection. In: Proceedings CVPR ’94, IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1994, pp. 369–376 (1994)

  37. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 134–144 (1980)

    Article  Google Scholar 

  38. Wu, C., Narasimhan, S.G., Jaramaz, B.: A multi-image shape-from-shading framework for near-lighting perspective endoscopes. IJCV 86, 211–228 (2009)

    Article  MathSciNet  Google Scholar 

  39. Yeung, S., Tsui, H., Yim, A.: Global shape from shading for an endoscope image. In: Taylor, C., Colchester, A. (eds.) Medical Image Computing and Computer-Assisted Intervention (MICCAI99). Lecture Notes in Computer Science, vol. 1679, pp. 318–327. Springer, Berlin (1999)

    Google Scholar 

  40. Zhang, L., Yip, A.M., Tan, C.L.: Shape from shading based on lax-friedrichs fast sweeping and regularization techniques with applications to document image restoration. In: CVPR (2007)

Download references

Acknowledgments

The first author acknowledges the support of INDAM under the GNCS research Project “Metodi numerici per la regolarizzazione nella ricostruzione feature-preserving di dati.”

Author information

Authors and Affiliations

  1. Dipartimento di Matematica, Sapienza - Università di Roma, Piazzale Aldo Moro, 5, 00185, Rome, Italy

    S. Tozza

  2. Engineering Department, University of Cambridge, Trumpington Street, Cambridge, CB2 1PZ, UK

    R. Mecca

  3. Pattern Analysis & Computer Vision Department, Istituto Italiano di Tecnologia, Via Morego, 30, 16163, Genova, Italy

    A. Del Bue

  4. Nanoscopy, Istituto Italiano di Tecnologia, Via Morego, 30, 16163, Genova, Italy

    M. Duocastella

Authors
  1. S. Tozza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Mecca
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Duocastella
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. Del Bue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Tozza.

Additional information

R. Mecca: Marie Curie fellow of the Istituto Nazionale di Alta Matematica.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tozza, S., Mecca, R., Duocastella, M. et al. Direct Differential Photometric Stereo Shape Recovery of Diffuse and Specular Surfaces. J Math Imaging Vis 56, 57–76 (2016). https://doi.org/10.1007/s10851-016-0633-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-016-0633-0

Keywords

Search

Navigation