International Journal of Computer Vision

, Volume 76, Issue 1, pp 71–91 | Cite as

Facial Shape-from-shading and Recognition Using Principal Geodesic Analysis and Robust Statistics

  • William A. P. Smith
  • Edwin R. Hancock


The aim in this paper is to use principal geodesic analysis to model the statistical variations for sets of facial needle maps. We commence by showing how to represent the distribution of surface normals using the exponential map. Shape deformations are described using principal geodesic analysis on the exponential map. Using ideas from robust statistics we show how this deformable model may be fitted to facial images in which there is significant self-shadowing. Moreover, we demonstrate that the resulting shape-from-shading algorithm can be used to recover accurate facial shape and albedo from real world images. In particular, the algorithm can effectively fill-in the facial surface when more than 30% of its area is subject to self-shadowing. To investigate the utility of the shape parameters delivered by the method, we conduct experiments with illumination insensitive face recognition. We present a novel recognition strategy in which similarity is measured in the space of the principal geodesic parameters. We also use the recovered shape information to generate illumination normalized prototype images on which recognition can be performed. Finally we show that, from a single input image, we are able to generate the basis images employed by a number of well known illumination-insensitive recognition algorithms. We also demonstrate that the principal geodesics provide an efficient parameterization of the space of harmonic basis images.


Shape-from-shading Face recognition Principal geodesic analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. USF HumanID. 3D Face Database, Courtesy of Sudeep. Sarkar, University of South Florida, Tampa, FL. Google Scholar
  2. Atick, J. J., Griffin, P. A., & Redlich, A. N. (1996). Statistical approach to SFS: Reconstruction of 3D face surfaces from single 2D images. Neural Computation, 8(6), 1321–1340. CrossRefGoogle Scholar
  3. Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233. CrossRefGoogle Scholar
  4. Belhumeur, P.N., Kriegman, D.J., & Yuille, A.L. (1999). The bas-relief ambiguity. International Journal of Computer Vision, 35(1), 33–44. CrossRefGoogle Scholar
  5. Blanz, V., Basso, C., Poggio, T., & Vetter, T. (2003). Reanimating faces in images and video. In Proceedings of EUROGRAPHICS (pp. 641–650). Google Scholar
  6. Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3D faces. In Proceedings of SIGGRAPH (pp. 187–194). Google Scholar
  7. Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1063–1074. CrossRefGoogle Scholar
  8. Bronstein, A., Bronstein, M., & Kimmel, R. (2005). Three-dimensional face recognition. International Journal of Computer Vision, 64(1), 5–30. CrossRefGoogle Scholar
  9. Castelán, M., & Hancock, E. R. (2006). Acquiring height data from a single image of a face using local shape indicators. Computer Vision and Image Understanding, 103(1), 64–79. CrossRefGoogle Scholar
  10. Dovgard, R., & Basri, R. (2004). Statistical symmetric shape from shading for 3D structure recovery of faces. In Proceedings of ECCV (Vol. 2, pp. 99–113). Google Scholar
  11. Dupuis, P., & Oliensis, J. (1994). An optimal control formulation and related numerical methods for a problem in shape reconstruction. Annals of Applied Probability, 4(2), 287–346. MathSciNetzbMATHGoogle Scholar
  12. Finlayson, G. D., Drew, M. S., & Lu, C. (2004). Intrinsic images by entropy minimization. In Proceedings of ECCV (pp. 582–595). Google Scholar
  13. Fisher, N. I. (1985). Spherical medians. Journal of the Royal Statistical Society, Series B, 47(2), 342–348. MathSciNetzbMATHGoogle Scholar
  14. Fletcher, P. T., Joshi, S., Lu, C., & Pizer, S. M. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging, 23(8), 995–1005. CrossRefGoogle Scholar
  15. Frankot, R. T., & Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 439–451. CrossRefzbMATHGoogle Scholar
  16. Fua, P., & Leclerc, Y. G. (1994). Using 3-dimensional meshes to combine image-based and geometry-based constraints. In Proceedings of ECCV (pp. 281–291). Google Scholar
  17. Georghiades, A. (2003). Recovering 3-d shape and reflectance from a small number of photographs. In Eurographics symposium on rendering (pp. 230–240). Google Scholar
  18. Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660. CrossRefGoogle Scholar
  19. Giles, P. T. (2001). Remote sensing and cast shadows in mountainous terrain. Photogrammetric Engineering & Remote Sensing, 67(7), 833–840. Google Scholar
  20. Gregory, R. L. (1997). Knowledge in perception and illusion. Philosophical Transactions of the Royal Society of London, Series B, 352, 1121–1128. CrossRefGoogle Scholar
  21. Hill, H., & Bruce, V. (1996). Effects of lighting on the perception of facial surfaces. Journal of Experimental Psychology: Human Perception and Performance, 22(4), 986–1004. CrossRefGoogle Scholar
  22. Huber, P. (1981). Robust statistics. Chichester: Wiley. zbMATHGoogle Scholar
  23. Lee, K., Ho, J., & Kriegman, D. (2005). Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 1–15. Google Scholar
  24. Lenglet, C., Rousson, M., Deriche, R., Faugeras, O., Lehericy, S., & Ugurbil, K. (2005). A Riemannian approach to diffusion tensor images segmentation. In Proceedings of the information processing in medical imaging (pp. 591–602). Google Scholar
  25. Levine, M. D., & Bhattacharyya, J. (2005). Removing shadows. Pattern Recognition Letters, 26(3), 251–265. CrossRefGoogle Scholar
  26. Mardia, K. V., & Jupp, P. E. (2000). Directional statistics. New York: Wiley. zbMATHGoogle Scholar
  27. Marr, D. (1982). Vision. San Francisco: Freeman. Google Scholar
  28. Nishino, K., Belhumeur, P. N., & Nayar, S. K. (2005). Using eye reflections for face recognition under varying illumination. In Proceedings of ICCV (Vol. 1, pp. 519–526). Google Scholar
  29. Pennec, X. (1999). Probabilities and statistics on Riemannian manifolds: basic tools for geometric measurements. In Proceedings of the IEEE workshop on nonlinear signal and image processing. Google Scholar
  30. Pennec, X. (2004). Probabilities and statistics on Riemannian manifolds: a geometric approach (Technical Report RR-5093). INRIA. Google Scholar
  31. Prados, E., & Faugeras, O. (2004). A rigorous and realistic shape from shading method and some of its applications (Technical Report RR-5133). INRIA. Google Scholar
  32. Prados, E., & Faugeras, O. D. (2004). Unifying approaches and removing unrealistic assumptions in shape from shading: mathematics can help. In Proceedings of ECCV (pp. 141–154). Google Scholar
  33. Samaras, D., & Metaxas, D. (2003). Illumination constraints in deformable models for shape and light direction estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 247–264. CrossRefGoogle Scholar
  34. Smith, W. A. P., & Hancock, E. R. (2006). Recovering facial shape using a statistical model of surface normal direction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 1914–1930. CrossRefGoogle Scholar
  35. Turk, M., & Pentland, A. (1991). Face recognition using eigenfaces. In Proceedings of CVPR (pp. 586–591). Google Scholar
  36. Worthington, P. L., & Hancock, E. R. (1999). New constraints on data-closeness and needle map consistency for shape-from-shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12), 1250–1267. CrossRefGoogle Scholar
  37. Zhang, L., Wang, S., & Samaras, D. (2005). Face synthesis and recognition under arbitrary unknown lighting using a spherical harmonic basis morphable model. In Proceedings of CVPR (pp. 209–216). Google Scholar
  38. Zhao, W. Y., & Chellappa, R. (2000). Illumination-insensitive face recognition using symmetric SFS. In Proceedings of CVPR (pp. 286–293). Google Scholar
  39. Zhao, W. Y., & Chellappa, R. (2001). Symmetric shape-from-shading using self-ratio image. International Journal of Computer Vision, 45, 55–75. CrossRefzbMATHGoogle Scholar
  40. Zhou, S., & Chellappa, R. (2003). Rank constrained recognition under unknown illuminations. In Proceedings of the IEEE international workshop on analysis and modeling of faces and gestures (pp. 11–18). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations