Skip to main content
SpringerLink
Log in

Uncalibrated photometric stereo refined by polarization angle

  • Regular Paper
  • Published:
Optical Review Aims and scope Submit manuscript

Abstract

Photometric stereo allows us to estimate the surface normal of an object based on its shading. In uncalibrated photometric stereo, the light source direction is unknown; thus, both the light source direction and surface normal should be estimated, which establishes an ill-posed problem with ambiguity in its solution. Consequently, assumptions should be made to uniquely determine the solution; however, assumptions are not always satisfied in practice, which results in an estimated surface normal that differs from the true surface normal. To improve surface normal estimation, we analyze the polarization state of reflected light considering that polarization can constrain the possible orientation of the surface normal. Thereafter, through extensive experimental evaluations, we demonstrate that polarization effectively improves uncalibrated photometric stereo.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31

Similar content being viewed by others

References

  1. Woodham, R.J.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 191139 (1980). https://doi.org/10.1117/12.7972479

    Article  Google Scholar 

  2. Woodham, R.J., Iwahori, Y., Barman, R.A.: Photometric stereo: Lambertian reflectance and light sources with unknown direction and strength. In: Technical Reports, The University of British Columbia, TR-91-18, 11 pp. (1991)

  3. Hayakawa, H.: Photometric stereo under a light source with arbitrary motion. J. Opt. Soc. Am. A 11(11), 3079–3089 (1994). https://doi.org/10.1364/JOSAA.11.003079

    Article  ADS  Google Scholar 

  4. Yuille, A.L., Snow, D., Epstein, R., Belhumeur, P.N.: Determining generative models of objects under varying illumination: Shape and albedo from multiple images using SVD and integrability. Int. J. Comput. Vis. 35(3), 203–222 (1999). https://doi.org/10.1023/A:1008180726317

    Article  Google Scholar 

  5. Alldrin, N.G., Mallick, S.P., Kriegman, D.J.: Resolving the generalized bas-relief ambiguity by entropy minimization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2007). https://doi.org/10.1109/CVPR.2007.383208

  6. Belhumeur, P.N., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. Int. J. Comput. Vis. 35, 33–44 (1999). https://doi.org/10.1023/A:1008154927611

    Article  Google Scholar 

  7. Shi, B., Matsushita, Y., Wei, Y., Xu, C., Tan, P.: Self-calibrating photometric stereo. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1118–1125 (2010). https://doi.org/10.1109/CVPR.2010.5540091

  8. Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vis. 72(3), 239–257 (2007). https://doi.org/10.1007/s11263-006-8815-7

    Article  Google Scholar 

  9. Sato, I., Okabe, T., Yu, Q., Sato, Y.: Shape reconstruction based on similarity in radiance changes under varying illumination. In: Proceedings of International Conference on Computer Vision, pp. 1–8 (2007). [https://doi.org/10.1109/ICCV.2007.4409020]

  10. Drbohlav, O., Šára, R.: Specularities reduce ambiguity of uncalibrated photometric stereo. In: A. Heyden, G. Sparr, M. Nielsen, P. Johansen (eds.) Computer Vision —ECCV 2002, Lecture Notes in Computer Science, vol. 2351. Springer, Berlin, Heidelberg (2002). [https://doi.org/10.1007/3-540-47967-8_4]

  11. Georghiades, A.S.: Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo. In: Proceedings of IEEE International Conference on Computer Vision, vol. 2, pp. 816–825 (2003). https://doi.org/10.1109/ICCV.2003.1238432

  12. Chandraker, M.K., Kahl, F., Kriegman, D.J.: Reflections on the generalized bas-relief ambiguity. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 788–795 (2005). https://doi.org/10.1109/CVPR.2005.299

  13. Mo, Z., Shi, B., Lu, F., Yeung, S., Matsushita, Y.: Uncalibrated photometric stereo under natural illumination. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2936–2945 (2018). https://doi.org/10.1109/CVPR.2018.00310

  14. Favaro, P., Papadhimitri, T.: A closed-form solution to uncalibrated photometric stereo via diffuse maxima. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 821–828 (2012). https://doi.org/10.1109/CVPR.2012.6247754

  15. Wu, Z., Tan, P.: Calibrating photometric stereo by holistic reflectance symmetry analysis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1498–1505 (2013). https://doi.org/10.1109/CVPR.2013.197

  16. Wolff, L.B., Boult, T.E.: Constraining object features using a polarization reflectance model. IEEE Transact. Pattern Anal. Mach. Intell. 13(7), 635–657 (1991). https://doi.org/10.1109/34.85655

    Article  Google Scholar 

  17. Rahmann, S., Canterakis, N.: Reconstruction of specular surfaces using polarization imaging. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. I–I (2001). https://doi.org/10.1109/CVPR.2001.990468

  18. Atkinson, G.A., Hancock, E.R.: Shape estimation using polarization and shading from two views. IEEE Transact. Pattern Anal. Mach. Intell. 29(11), 2001–2017 (2007). https://doi.org/10.1109/TPAMI.2007.1099

    Article  Google Scholar 

  19. Atkinson, G.A., Hancock, E.R.: Recovery of surface orientation from diffuse polarization. IEEE Transact. Image Process. 15(6), 1653–1664 (2006). https://doi.org/10.1109/TIP.2006.871114

    Article  ADS  Google Scholar 

  20. Huynh, C.P., Robles-Kelly, A., Hancock, E.R.: Shape and refractive index from single-view spectro-polarimetric images. Int. J. Comput. Vis. 101, 64–94 (2013). https://doi.org/10.1007/s11263-012-0546-3

    Article  MathSciNet  Google Scholar 

  21. Atkinson, G.A., Hancock, E.R.: Surface reconstruction using polarization and photometric stereo. In: W. G. Kropatsch, M. Kampel, A. Hanbury (eds.) Computer Analysis of Images and Patterns, CAIP 2007, Lecture Notes in Computer Science, vol. 4673. Springer, Berlin, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74272-2_58

  22. Drbohlav, O., Sara, R.: Unambiguous determination of shape from photometric stereo with unknown light sources. In: Proceedings of IEEE International Conference on Computer Vision, vol. 1, pp. 581–586 (2001). https://doi.org/10.1109/ICCV.2001.9375702

  23. Ngo, T.T., Nagahara, H., Taniguchi, R.: Shape and light directions from shading and polarization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2310–2318 (2015). https://doi.org/10.1109/CVPR.2015.7298844

  24. Mukaigawa, Y., Ishii, Y., Shakunaga, T.: Analysis of photometric factors based on photometric linearization. J. Opt. Soc. Am. A 24(10), 3326–3334 (2007). https://doi.org/10.1364/JOSAA.24.003326

    Article  ADS  Google Scholar 

  25. Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. Int. J. Comput. Vis. 9, 137–154 (1992). https://doi.org/10.1007/BF00129684

    Article  Google Scholar 

  26. Shum, H., Ikeuchi, K., Reddy, R.: Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transact. Pattern Anal. Mach. Intell. 17(9), 854–867 (1995). https://doi.org/10.1109/34.406651

    Article  Google Scholar 

  27. Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometric stereo via low-rank matrix completion and recovery. In: R. Kimmel, R. Klette, A. Sugimoto (eds.) Computer Vision - ACCV 2010, Lecture Notes in Computer Science, vol. 6494. Springer, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-642-19318-7_55

  28. Ikehata, S., Wipf, D., Matsushita, Y., Aizawa, K.: Photometric stereo using sparse Bayesian regression for general diffuse surfaces. IEEE Transact. Pattern Anal. Mach. Intell. 39(9), 1816–1831 (2014). https://doi.org/10.1109/TPAMI.2014.2299798

    Article  Google Scholar 

  29. Mori, T., Taketa, R., Hiura, S., Sato, K.: Photometric linearization by robust PCA for shadow and specular removal. In: G. Csurka, M. Kraus, R. S. Laramee, P. Richard, J. Braz (eds.) Computer Vision, Imaging and Computer Graphics, Theory and Application, Communications in Computer and Information Science, vol. 359. Springer, Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38241-3_14

  30. Miyazaki, D., Ikeuchi, K.: Photometric stereo under unknown light sources using robust SVD with missing data. In: Proceedings of IEEE International Conference on Image Processing, pp. 4057–4060 (2010). https://doi.org/10.1109/ICIP.2010.5650067

  31. Wolff, L.B., Lundberg, A., Tang, R.: Image understanding from thermal emission polarization. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 625–631 (1998). https://doi.org/10.1109/CVPR.1998.698670

Download references

Author information

Author notes

  1. Shuhei Hashimoto

    Present address: Mitsubishi Electric Control Software Corporation, Kobe, Japan

Authors and Affiliations

  1. Hiroshima City University, 3-4-1 Ozukahigashi Asaminami-ku, Hiroshima, Japan

    Daisuke Miyazaki & Shuhei Hashimoto

Authors
  1. Daisuke Miyazaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuhei Hashimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daisuke Miyazaki.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miyazaki, D., Hashimoto, S. Uncalibrated photometric stereo refined by polarization angle. Opt Rev 28, 119–133 (2021). https://doi.org/10.1007/s10043-021-00640-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10043-021-00640-0

Keywords

Search

Navigation