International Journal of Computer Vision

, Volume 28, Issue 2, pp 103–116 | Cite as

Synthesis of Novel Views from a Single Face Image

  • Thomas Vetter


Images formed by a human face change with viewpoint. A new technique is described for synthesizing images of faces from new viewpoints, when only a single 2D image is available. A novel 2D image of a face can be computed without explicitly computing the 3D structure of the head. The technique draws on a single generic 3D model of a human head and on prior knowledge of faces based on example images of other faces seen in different poses. The example images are used to “learn” a pose-invariant shape and texture description of a new face. The 3D model is used to solve the correspondence problem between images showing faces in different poses.

The proposed method is interesting for view independent face recognition tasks as well as for image synthesis problems in areas like teleconferencing and virtualized reality.

image synthesis face recognition rotation invariance flexible templates 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aizawa, K. Harashima, H. and Saito, T. 1989. Model-based analysis synthesis image coding (MBASIC) system for a person'sface. Signal Processing: Image Communication, 1:139–152.Google Scholar
  2. Akimoto, T., Suenaga, Y. and Wallace, R.S. 1993. Automatic creation of 3D facial models. IEEE Computer Graphics and Applications, 13(3):16–22.Google Scholar
  3. Bergen, J.R. Anandan, P., Hanna, K.J. and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, Santa Margherita Ligure, Italy, pp. 237–252.Google Scholar
  4. Bergen, J.R. and Hingorani, R. 1990. Hierarchical motion-based frame rate conversion. Technical report, David Sarnoff Research Center, Princeton, NJ.Google Scholar
  5. Beymer, D. 1993. Face recognition under varying pose. A.I. Memo No. 1461, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.Google Scholar
  6. Beymer, D. and Poggio, T. 1995. Face recognition from one model view. In Proceedings of the 5th International Conference on Computer Vision.Google Scholar
  7. Beymer, D. and Poggio, T. 1996. Image representation for visual learning. Science, 272:1905–1909.Google Scholar
  8. Beymer, D., Shashua, A. and Poggio, T. 1993. Example-based image analysis and synthesis. A.I. Memo No. 1431, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.Google Scholar
  9. Burt, P.J. and Adelson, E.H. 1983. The Laplacian pyramide as a compact image code. IEEE Transactions on Communications, 31:532–540.Google Scholar
  10. Burt, P.J. and Adelson, E.H. 1985. Merging images through pattern decomposition. Applications of Digital Image Processing VIII, 575:73–181. SPIE The International Society for Optical Engeneering.Google Scholar
  11. Choi, C.S., Okazaki, T., Harashima, H. and Takebe, T. 1991. A system of analyzing and synthesizing facial images. In Proc. IEEE Int. Symposium of Circuit and Syatems (ISCAS91), pp. 2665– 2668.Google Scholar
  12. Cootes, T.F., Taylor, C.J., Cooper, D.H. and Graham, J. 1995. Active shape models-their training and application. Computer Vision and Image Understanding, 61:38–59.Google Scholar
  13. Craw, I. and Cameron, P. 1991. Parameterizing images for recognition and reconstruction. In Proc. British Machine Vision Conference, Springer, pp. 367–370.Google Scholar
  14. Hallinan, P.W. 1995. A deformable model for the recognition of human faces under arbitrary illumination. Doctoral thesis, Harvard University, Cambridge, MA.Google Scholar
  15. Horn, B.K.P. 1987. Robot Vision. MIT Press: Cambridge, MA.Google Scholar
  16. Huang, T.S., and Lee, C.H. 1989. Motion and structure from orthographic projections. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2(5):536–540.Google Scholar
  17. Jones, M., and Poggio, T. 1995. Model-based matching of line drawings by linear combination of prototypes. In Proceedings of the 5th International Conference on Computer Vision.Google Scholar
  18. Lanitis, A., Taylor, C.J., Cootes, T.F., and Ahmad, T. 1995. Automatic interpretation of human faces and hand gestures using flexible models. In Proc. InternationalWorkshop on Face and Gesture Recognition, Zurich, Switzerland, pp. 98–103.Google Scholar
  19. O'Toole, A.J., Deffenbacher, K.A., Valentin, D. and Abdi, H. 1994. Structural aspects of face recognition and the other-race effect. Memory and Cognition, 22:208–224.Google Scholar
  20. Poggio, T. and Brunelli, R. 1992. A novel approach to graphics. Technical report 1354, MIT Media Laboratory Perceptual Computing Section.Google Scholar
  21. Press, Teukolsky, Vetterling and Flannery. 1992. Numerical recipes in C: the art of scientific computing. Cambridge University Press: Cambridge.Google Scholar
  22. C.A. Rothwell, D.A. Forsyth, Zissermann, A. and Mundy, J.L. 1993. Extracting projective structure from single perspective views of 3D point sets. In Proceedings of the International Conference on Computer Vision (ICCV), Berlin, Germany, pp. 573–582.Google Scholar
  23. Terzopoulos, D. and Waters, K. 1993 Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(6):569–579.Google Scholar
  24. Thalmann, N.D. and Thalmann, D. 1995. Digital actors for interactive television. In Proceedings of the IEEE, 83(7):1022–1031.Google Scholar
  25. Vetter, T., Jones, M. and Poggio, T. 1997. A bootstrapping algorithm for learning linearized models of object classes. in IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  26. Vetter, T. and Poggio, T. 1994. Symmetric 3D objects are an easy case for 2D object recognition. Spatial Vision, 8(4):443–453.Google Scholar
  27. Vetter, T. and Poggio, T. 1996. Image synthesis from a single example image. In volume 1065 of LNCS,Computer Vision – ECCV'96, Cambridge UK. Springer.Google Scholar
  28. Wolberg, G. 1990. Image Warping. IEEE Computer Society Press: Los Alamitos, CA.Google Scholar
  29. Xu, W. and Hauske, G. 1994. Picture quality evaluation based on error segmentation. In Proc. SPIE, Visual Communications and Image Processing, 2308:1–12.Google Scholar

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Thomas Vetter
    • 1
  1. 1.Max-Planck-Institut für Biologische KybernetikTüubingenGermany. E-mail

Personalised recommendations