Estimating Facial Reflectance Properties Using Shape-from-Shading

Article

Abstract

In this paper we show how to estimate facial surface reflectance properties (a slice of the BRDF and the albedo) in conjunction with the facial shape from a single image. The key idea underpinning our approach is to iteratively interleave the two processes of estimating reflectance properties based on the current shape estimate and updating the shape estimate based on the current estimate of the reflectance function. For frontally illuminated faces, the reflectance properties can be described by a function of one variable which we estimate by fitting a curve to the scattered and noisy reflectance samples provided by the input image and estimated shape. For non-frontal illumination, we fit a smooth surface to the scattered 2D reflectance samples. We make use of a novel statistical face shape constraint which we term ‘model-based integrability’ which we use to regularise the shape estimation. We show that the method is capable of recovering accurate shape and reflectance information from single grayscale or colour images using both synthetic and real world imagery. We use the estimated reflectance measurements to render synthetic images of the face in varying poses. To synthesise images under novel illumination, we show how to fit a parametric model of reflectance to the estimated reflectance function.

Keywords

Shape-from-shading BRDF estimation Reflectance modelling Face recognition Colour constancy 

References

  1. Ahmed, A. H., & Farag, A. A. (2006). A new formulation for shape from shading for non–Lambertian surfaces. In Proc. CVPR (Vol. 2, pp. 1817–1824). Google Scholar
  2. Atick, J. J., Griffin, P. A., & Redlich, A. N. (1996). Statistical approach to SFS: Reconstruction of 3D face surfaces from single 2D images. Neural Computation, 8(6), 1321–1340. CrossRefGoogle Scholar
  3. Atkinson, G., & Hancock, E. R. (2008). Two-dimensional BRDF estimation from polarisation. Computer Vision and Image Understanding, 111(2), 126–141. CrossRefGoogle Scholar
  4. Baranoski, G. V. G., & Krishnaswamy, A. (2004). An introduction to light interaction with human skin. Revista de Informá, tica Teórica e Aplicada, XI(1), 33–62. Google Scholar
  5. Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233. CrossRefGoogle Scholar
  6. Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1063–1074. CrossRefGoogle Scholar
  7. Claridge, E., Cotton, S., Hall, P., & Moncrieff, M. (2003). From colour to tissue histology: physics based interpretation of images of pigmented skin lesions. Medical Image Analysis Journal, 7, 489–502. CrossRefGoogle Scholar
  8. Cootes, T. F., Edwards, G. J., & Taylor, C. J. (1998). Active appearance models. In Proc. ECCV (pp. 484–498). Google Scholar
  9. Dana, K. J., van Ginneken, B., Nayar, S. K., & Koenderink, J. J. (1999). Reflectance and texture of real–world surfaces. ACM Transactions on Graphics, 18(1), 1–34. CrossRefGoogle Scholar
  10. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., & Sarokin, W. (2000). Acquiring the reflectance field of a human face. In Proc. SIGGRAPH (2000). Google Scholar
  11. D’Errico, J. (2005). Surface fitting using gridfit. MATLAB Central File Exchange. Google Scholar
  12. Donner, C., & Wann Jensen, H. (2006). A spectral BSSRDF for shading human skin. In Rendering techniques 2006: 17th eurographics workshop on rendering (pp. 409–418). Google Scholar
  13. Dovgard, R., & Basri, R. (2004). Statistical symmetric shape from shading for 3D structure recovery of faces. In Proc. ECCV (Vol. 2, pp. 99–113). Google Scholar
  14. Durou, J.-D., Falcone, M., & Sagona, M. (2008). Numerical methods for shape-from-shading: A new survey with benchmarks. Computer Vision and Image Understanding, 109(1), 22–43. CrossRefGoogle Scholar
  15. Frankot, R. T., & Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 439–451. MATHCrossRefGoogle Scholar
  16. Fuchs, M., Blanz, V., Lensch, H., & Seidel, H.-P. (2005). Reflectance from images: A model–based approach for human faces. IEEE Transactions on Visualization and Computer Graphics, 11(3), 296–305. CrossRefGoogle Scholar
  17. Georghiades, A. (2003). Recovering 3–d shape and reflectance from a small number of photographs. In Eurographics symposium on rendering (pp. 230–240). Google Scholar
  18. Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660. CrossRefGoogle Scholar
  19. Hertzmann, A., & Seitz, S. M. (2005). Example–based photometric stereo: Shape reconstruction with general, varying BRDFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1254–1264. CrossRefGoogle Scholar
  20. Horn, B. K. P., & Sjoberg, R. W. (1979). Calculating the reflectance map. Applied Optics, 18(11), 1770–1779. CrossRefGoogle Scholar
  21. Kemelmacher, I., & Basri, R. (2006). Molding face shapes by example. In Proc. ECCV (pp. 277–288). Google Scholar
  22. Koenderink, J., & Pont, S. (2003). The secret of velvety skin. Machine Vision and Applications, 14, 260–268. CrossRefGoogle Scholar
  23. Marschner, S. R., Westin, S. H., Lafortune, E. P. F., Torrance, K. E., & Greenberg, D. P. (1999). Reflectance measurements of human skin (Technical Report PCG–99–2). Cornell University. Google Scholar
  24. Oliensis, J. (1991). Uniqueness in shape from shading. International Journal of Computer Vision, 6(2), 75–104. CrossRefGoogle Scholar
  25. Oren, M., & Nayar, S. K. (1995). Generalization of the Lambertian model and implications for machine vision. International Journal of Computer Vision, 14(3), 227–251. CrossRefGoogle Scholar
  26. Phong, B. T. (1975). Illumination for computer generated images. Communications of the ACM, 18(6), 311–317. CrossRefGoogle Scholar
  27. Prados, E., & Faugeras, O. (2005). A generic and provably convergent shape-from-shading method for orthographic and pinhole cameras. International Journal of Computer Vision, 65(1–2), 97–125. CrossRefGoogle Scholar
  28. Ragheb, H., & Hancock, E. R. (2003). A probabilistic framework for specular shape–from–shading. Pattern Recognition, 36(2), 407–427. CrossRefGoogle Scholar
  29. Robles-Kelly, A., & Hancock, E. R. (2005). Estimating the surface radiance function from single images. Graphical Models, 67(6), 518–548. MATHCrossRefGoogle Scholar
  30. Sim, T., Baker, S., & Bsat, M. (2003). The CMU pose, illumination, and expression database. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1615–1618. CrossRefGoogle Scholar
  31. Smith, W. A. P., & Hancock, E. R. (2007). A new framework for grayscale and colour non-Lambertian shape-from-shading. In Proc. ACCV (pp. 869–880). Google Scholar
  32. Smith, W. A. P., & Hancock, E. R. (2008). Facial shape–from–shading and recognition using principal geodesic analysis and robust statistics. International Journal of Computer Vision, 76(1), 71–91. CrossRefGoogle Scholar
  33. Torrance, K., & Sparrow, E. (1967). Theory for off–specular reflection from roughened surfaces. Journal of the Optical Society of America, 57(9), 1105–1114. CrossRefGoogle Scholar
  34. USF HumanID 3D Face Database (1998). Courtesy of Sudeep Sarkar, University of South Florida, Tampa, FL. Google Scholar
  35. van Gemert, M. J. C., Jacques, S. L., Sterenborg, H. J. C. M., & Star, W. M. (1989). Skin optics. IEEE Transactions on Biomedical Engineering, 36(12), 1146–1154. CrossRefGoogle Scholar
  36. Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Wann Jensen, H., & Gross, M. (2006). Analysis of human faces using a measurement-based skin reflectance model. ACM Transactions on Graphics, 25(3), 1013–1024. CrossRefGoogle Scholar
  37. Zhang, L., & Samaras, D. (2006). Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(3), 351–363. CrossRefGoogle Scholar
  38. Zhang, R., Tsai, P. S., Cryer, J. E., & Shah, M. (1999). Shape–from–shading: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 690–706. CrossRefGoogle Scholar
  39. Zhao, W. Y., & Chellappa, R. (2001). Symmetric shape–from–shading using self–ratio image. International Journal of Computer Vision, 45, 55–75. MATHCrossRefGoogle Scholar
  40. Zickler, T., Mallick, S. P., Kriegman, D. J., & Belhumeur, P. N. (2008). Color subspaces as photometric invariants. International Journal of Computer Vision, 79(1), 13–30 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations