Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Fast 3D face reconstruction based on uncalibrated photometric stereo

  • 507 Accesses

  • 11 Citations

Abstract

This paper proposes a fast algorithm for three-dimensional face reconstruction using uncalibrated Photometric Stereo. With a reference face model, lighting parameters are estimated from input face images lighted by unknown illumination, which can be used in classical photometric stereo to estimate surface normal and albedo. The estimated results are used in turn to refine the lighting parameters until an optimal estimation of the surface normal is achieved. Differing from traditional optimization algorithms, the iteration method used in this paper is a unified process thus results accurate lighting estimation. The proposed method relaxes lighting constraints and simplifies the image acquisition procedure. The reconstructed results tested on YaleB and BU3D databases show the effectiveness of our method.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Alain K, Dipanda A, Claire Bourgeois R (2012) Evolutionary-based 3D Reconstruction Using an uncalibrated stereovision system: application of building a panoramic object view. Multimed Tools Appl 57(3):565–586

  2. 2.

    Alldrin NG, Mallick SP, Kriegman DJ (2007) Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization. Conf on Comp. Vision and Pattern Recognition (CVPR):1–7.

  3. 3.

    Basri R, Jacobs D, Ira Kemelmacher I (2007) Photometric stereo with general, unknown lighting. Int J Comp Vision 72(3):239–257

  4. 4.

    Belhumeur PN, Kriegman DJ, Yuille AL (1999) The Bas-relief ambiguity. Int J Comp Vision 35(1):33–44

  5. 5.

    Black MJ, Anandan P (1996) The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comp Vis Image Underst 63(1):75–104

  6. 6.

    Castelan M, Smith WAP, Hancock ER (2007) A coupled statistical model for face shape recovery from brightness images. IEEE Trans Image Process 16(4):1139–1151

  7. 7.

    Chellappa R, Raskar R (2005) An algebraic approach to surface reconstruction from gradient fields. Proc Tenth IEEE Int Conf Comput Vis 01:174–181

  8. 8.

    Chen C-P, Chen C-S (2006) The 4-Source Photometric Stereo under General Unknown Lighting. ECCV, pp: 72–83

  9. 9.

    Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From Few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

  10. 10.

    Hertzmann A, Seitz SM (2003) Shape and Materials by Example: A Photometric Stereo Approach. Comp Vision Pattern Recognit, pp:533–540

  11. 11.

    Jang IY, Cho J-H, Lee KH (2012) 3D human modeling from a single depth image dealing with self-occlusion. Multimed Tools Appl 58(1):267–288

  12. 12.

    Jian M, Dong J (2011) Capture and fusion of 3D Surface texture. Multimed Tools Appl 53(1):237–251

  13. 13.

    Kemelmacher-Shlizerman I, Basri R (2011) 3D Face Reconstruction from a Single Image Using a Single Reference Face Shape. IEEE Trans Pattern Anal Mach Intell 33(2):394–405

  14. 14.

    Kim S-J, Koh K, Lustig M et al (2007) An interior-point method for large-scale 1-regularized least squares. IEEE J Sel Top Sign Process 1(4):606–617

  15. 15.

    Lee SW, Wang PSP, Yanushkevich SN (2008) Noniterative 3D face reconstruction based on photometric stereo. Int J Pattern Recog Artif Intell 22(3):389–410

  16. 16.

    Li A, Shan S, Chen X, Chai X, W Gao (2008) Recovering 3-D facial shape via coupled 2D/3D space learning. IEEE Int Conf Autom Face Gesture Recognit, pp:1–6

  17. 17.

    Malzbender T, Wilburn B, Gelb D et al (2006) Surface enhancement using real-time photometric stereo and reflectance transformation. Proc EGSR 245–250

  18. 18.

    McGunnigle G, Dong J (2011) Augmenting photometric stereo with coaxial illumination. Comp Vision IET 5(1):33–49

  19. 19.

    Metz CE, Herman BA, Shen JH (1998) Maximum-likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. Stat Med, in press.

  20. 20.

    Nehab D, Rusinkiewicz S, Davis J et al (2005) Efficiently combining positions and normals for precise 3D geometry. Proc SIGGRAPH 24(3):536–543

  21. 21.

    Puerto-Souza G, Van Horebeek J (2009) Using subspace multiple linear regression for 3-D face shape prediction from a single image. Int Symp Visual Comput, pp: 662–673

  22. 22.

    Ramamoorthi R, Hanrahan P (2001) An Efficient Representation for Irradiance Environment Maps. Proceedings of the 28th annual conference on Computer graphics and interactive techniques, 497–500.

  23. 23.

    Seitz S, Curless B, Diebel J et al. (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. Proc. of Computer Vision and Pattern Recognition, pp:519–526

  24. 24.

    Shashua A (1992) Geometry and Photometry in 3D visual Recognition. Ph.D. thesis, MIT.

  25. 25.

    Song M, Tao D, Huang X (2012) Three-dimensional face reconstruction from a single image by a coupled RBF network. IEEE Trans Image Process 21(5):2887–2897

  26. 26.

    Wegner A, Gardner A, Tschou C, Unger J, Hawkins T, Debevec T (2005) Performance relighting and reflectance transformation with time multiplexed illumination. Proc Siggraph:756–764

  27. 27.

    Woodham RJ (1980) Photometric Method for Determining Surface Orientation from Multiple Images. Opt Eng 19(1):139–144

  28. 28.

    Woodham RJ (1994) Gradient and curvature from the photometric stereo method, including local confidence estimation. J Opt Soc Am 11(11):3050–3068

  29. 29.

    Xiaofeng Z, Caiming Z, Wenjing T (2012) Medical image segmentation using improved FCM. Sci China Inf Sci 55(4):1052–1061

  30. 30.

    Yin L, Wei X, Sun Y (2006) A 3D facial expression database for facial behavior research. IEEE Int Conf Autom Face Gesture Recog 10(12):211–216

  31. 31.

    Zitnick CL, Kang SB, Uyttendaele M et al (2004) High-quality video view interpolation using a layered representation. ACM Trans Graph 23(3):600–608

Download references

Author information

Correspondence to Junyu Dong.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Sun, Y., Dong, J., Jian, M. et al. Fast 3D face reconstruction based on uncalibrated photometric stereo. Multimed Tools Appl 74, 3635–3650 (2015). https://doi.org/10.1007/s11042-013-1791-3

Download citation

Keywords

  • Photometric stereo
  • Face surface normal
  • Face albedo
  • Lambertian model