The Visual Computer

, Volume 25, Issue 10, pp 899–909 | Cite as

3D face computational photography using PCA spaces

  • Jesús P. Mena-ChalcoEmail author
  • Ives Macêdo
  • Luiz Velho
  • Roberto M. CesarJr.
Original Article


In this paper, we present a 3D face photography system based on a facial expression training dataset, composed of both facial range images (3D geometry) and facial texture (2D photography). The proposed system allows one to obtain a 3D geometry representation of a given face provided as a 2D photography, which undergoes a series of transformations through the texture and geometry spaces estimated. In the training phase of the system, the facial landmarks are obtained by an active shape model (ASM) extracted from the 2D gray-level photography. Principal components analysis (PCA) is then used to represent the face dataset, thus defining an orthonormal basis of texture and another of geometry. In the reconstruction phase, an input is given by a face image to which the ASM is matched. The extracted facial landmarks and the face image are fed to the PCA basis transform, and a 3D version of the 2D input image is built. Experimental tests using a new dataset of 70 facial expressions belonging to ten subjects as training set show rapid reconstructed 3D faces which maintain spatial coherence similar to the human perception, thus corroborating the efficiency and the applicability of the proposed system.


3D face reconstruction Principal components analysis Computer vision Computational photography 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blanz, V.: Face recognition based on a 3D morphable model. In: International Conference on Automatic Face and Gesture Recognition, pp. 617–624 (2006) Google Scholar
  2. 2.
    Bregler, C., Covell, M., Slaney, M.: Video rewrite: driving visual speech with audio. In: SIGGRAPH ’97: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 353–360. ACM/Addison–Wesley, New York/Reading (1997) CrossRefGoogle Scholar
  3. 3.
    Chai, J.-X., Xiao, J., Hodgins, J.: Vision-based control of 3D facial animation. In: Breen, D., Lin, M. (eds.) Eurographics/SIGGRAPH Symposium on Computer Animation, pp. 193–206. Eurographics Association, San Diego (2003) Google Scholar
  4. 4.
    Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models: Their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995) CrossRefGoogle Scholar
  5. 5.
    Dedeoglu, G., Baker, S., Kanade, T.: Resolution-aware fitting of active appearance models to low resolution images. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV (2). Lecture Notes in Computer Science, vol. 3952, pp. 83–97. Springer, Berlin (2006) Google Scholar
  6. 6.
    Dryden, I.L., Mardia, K.V. (eds.): Statistical Shape Analysis. Wiley, Chichester (1998) zbMATHGoogle Scholar
  7. 7.
    Elyan, E., Ugail, H.: Reconstruction of 3D human facial images using partial differential equations. J. Comput. 2(8), 1–8 (2007) Google Scholar
  8. 8.
    Golovinskiy, A., Matusik, W., Pfister, H., Rusinkiewicz, S., Funkhouser, T.: A statistical model for synthesis of detailed facial geometry. ACM Trans. Graph. 25(3), 1025–1034 (2006) CrossRefGoogle Scholar
  9. 9.
    Goodall, C.: Procrustes methods in the statistical analysis of shape. J. R. Stat. Soc. Ser. B 53(2), 285–339 (1991) zbMATHMathSciNetGoogle Scholar
  10. 10.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Berlin (2003) Google Scholar
  11. 11.
    Hong, P., Wen, Z., Huang, T.S., Shum, H.Y.: Real-time speech-driven 3D face animation. In: 3D Data Processing Visualization and Transmission, pp. 713–716 (2002) Google Scholar
  12. 12.
    Jiang, D.L., Hu, Y.X., Yan, S.C., Zhang, L., Zhang, H.J., Gao, W.: Efficient 3D reconstruction for face recognition. Pattern Recognit. 38(6), 787–798 (2005) CrossRefGoogle Scholar
  13. 13.
    Kanade, T.: Picture processing system by computer complex and recognition of human faces. Doctoral Dissertation, Kyoto University, November 1973 Google Scholar
  14. 14.
    Kittler, J.V., Hilton, A., Hamouz, M., Illingworth, J.: 3D assisted face recognition: A survey of 3D imaging, modelling and recognition approaches. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 3, p. 114 (2005) Google Scholar
  15. 15.
    Lee, M.W., Ranganath, S.: 3D deformable face model for pose determination and face synthesis. In: International Conference on Image Analysis and Processing, pp. 260–265 (1999) Google Scholar
  16. 16.
    Leyvand, T., Cohen-Or, D., Dror, G., Lischinski, D.: Data-driven enhancement of facial attractiveness. ACM Trans. Graph. 27(3), 1–9 (2008) CrossRefGoogle Scholar
  17. 17.
    Litke, N., Droske, M., Rumpf, M., Schröder, P.: An image processing approach to surface matching. In: SGP ’05: Proceedings of the Third Eurographics Symposium on Geometry Processing, p. 207. Eurographics Association, Aire-la-Ville (2005) Google Scholar
  18. 18.
    Macêdo, I., Brazil, E.V., Velho, L.: Expression transfer between photographs through multilinear AAM’s. In: SIBGRAPI, pp. 239–246. IEEE Computer Society, Los Alamitos (2006) Google Scholar
  19. 19.
    Mena-Chalco, J.P., Macêdo, I., Velho, L., Cesar, R.M. Jr.: PCA-based 3D face photography. In: Brazilian Symposium on Computer Graphics and Image Processing, pp. 313–320. IEEE Computer Society, Los Alamitos (2008) CrossRefGoogle Scholar
  20. 20.
    Onofrio, D., Tubaro, S.: A model based energy minimization method for 3D face reconstruction. In: ICME, pp. 1274–1277. IEEE Press, New York (2005) Google Scholar
  21. 21.
    Schölkopf, B., Smola, A., Müller, K.-R.: Kernel principal component analysis. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, J.-D. (eds.) International Conference on Artificial Neural Networks—ICANN. Lecture Notes in Computer Science, vol. 1327, pp. 583–588. Springer, Berlin (1997) Google Scholar
  22. 22.
    Soyel, H., Demirel, H.: Facial expression recognition using 3D facial feature distances. In: International Conference on Image Analysis and Recognition, pp. 831–838 (2007) Google Scholar
  23. 23.
    Twining, C.J., Taylor, C.J.: Kernel principal component analysis and the construction of non-linear active shape models. In: Cootes, T.F., Taylor, C.J. (eds.) British Machine Vision Conference. British Machine Vision Association (2001) Google Scholar
  24. 24.
    Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. ACM Trans. Graph. 24(3), 426–433 (2005) CrossRefGoogle Scholar
  25. 25.
    Vollmer, J., Mencl, R., Muller, H.: Improved Laplacian smoothing of noisy surface meshes. Comput. Graph. Forum 18(3), 131–138 (1999) CrossRefGoogle Scholar
  26. 26.
    Wang, Y.J., Chua, C.S.: Face recognition from 2D and 3D images using 3D Gabor filters. Image Vis. Comput. 23(11), 1018–1028 (2005) CrossRefGoogle Scholar
  27. 27.
    Yabui, T., Kenmochi, Y., Kotani, K.: Facial expression analysis from 3D range images; comparison with the analysis from 2D images and their integration. In: International Conference on Image Processing, pp. 879–882 (2003) Google Scholar
  28. 28.
    Zhang, Y., Xu, S.: Data-driven feature-based 3D face synthesis. In: 3-D Digital Imaging and Modeling, pp. 39–46 (2007) Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  • Jesús P. Mena-Chalco
    • 1
    Email author
  • Ives Macêdo
    • 2
  • Luiz Velho
    • 2
  • Roberto M. CesarJr.
    • 1
  1. 1.IME—Universidade de São PauloSão PauloBrazil
  2. 2.Instituto Matemática Pura e AplicadaRio de JaneiroBrazil

Personalised recommendations