Abstract
How to synthesize photorealistic images of human faces has been a fascinating yet difficult problem in computer graphics. Here, the term “face synthesis” refers to synthesis of still images as well as synthesis of facial animations. In this chapter, we focus more on the synthesis of still images and skip most of the aspects that mainly involve the motion over time. We review recent advances on face synthesis including 3D face modeling, face relighting, and facial expression synthesis.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Akimoto, T., Suenaga, Y., Wallace, R.S.: Automatic 3d facial models. IEEE Comput. Graph. Appl. 13(5), 16–22 (1993)
Badler, N., Platt, S.: Animating facial expressions. In: Computer Graphics, pp. 245–252. Siggraph, August 1981
Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. In: Proc. ICCV’01, pp. 383–390 (2001)
Beier, T., Neely, S.: Feature-based image metamorphosis. In: Computer Graphics, pp. 35–42. Siggraph, July 1992
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d faces. In: Computer Graphics, Annual Conference Series, pp. 187–194. Siggraph, August 1999
Bregler, C., Covell, M., Slaney, M.: Video rewrite: Driving visual speech with audio. In: Computer Graphics, pp. 353–360. Siggraph, August 1997
Chuang, E., Bregler, C.: Mood swings: expressive speech animation. ACM Trans. Graph. 24(2), 331–347 (2005)
Dariush, B., Kang, S.B., Waters, K.: Spatiotemporal analysis of face profiles: Detection, segmentation, and registration. In: Proc. of the 3rd International Conference on Automatic Face and Gesture Recognition, April 1998, pp. 248–253. IEEE, New York (1998)
Debevec, P.E.: Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Computer Graphics, Annual Conference Series, pp. 189–198. Siggraph, July 1998
Debevec, P.E., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: Computer Graphics, Annual Conference Series, pp. 145–156. Siggraph, July 2000
Deng, Z., Neumann, U.: Data-Driven 3D Facial Animation. Springer, Berlin (2007)
Dimitrijevic, M., Ilic, S., Fua, P.: Accurate face models from uncalibrated and ill-lit video sequences. In: Computer Vision and Pattern Recognition, vol. II, pp. 1034–1041 (2004)
Fua, P., Miccio, C.: From regular images to animated heads: A least squares approach. In: Eurographics of Computer Vision, pp. 188–202 (1996)
Fua, P., Miccio, C.: Animated heads from ordinary images: A least-squares approach. Comput. Vis. Image Underst. 75(3), 247–259 (1999)
Georghiades, A., Belhumeur, P., Kriegman, D.: Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting. In: IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, pp. 47–54 (1999)
Guenter, B., Grimm, C., Wood, D., Malvar, H., Pighin, F.: Making faces. In: Computer Graphics, Annual Conference Series, pp. 55–66. Siggraph, July 1998
Horn, B.K.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)
Ip, H.H.S., Yin, L.: Constructing a 3d individualized head model from two orthogonal views. Vis. Comput. 12, 254–266 (1996)
Jiang, X., Kong, Y., Huang, J., Zhao, R., Zhang, Y.: Learning from real images to model lighting variations for face images. In: European Conference on Computer Vision (ECCV’2008), vol. IV, pp. 284–297 (2008)
Joshi, P., Tien, W.C., Desbrun, M., Pighin, F.: Learning controls for blend shape based realistic facial animation. In: Proc. Symposium on Computer Animation (SCA’03), pp. 187–192, July 2003
Kemelmacher, I., Basri, R.: Molding face shapes by example. In: European Conference on Computer Vision, pp. I:277–288 (2006)
Lau, M., Chai, J., Xu, Y.-Q., Shum, H.-Y.: Face poser: Interactive modeling of 3D facial expressions using facial priors. ACM Trans. Graph. 29(1), 1–17 (2009)
Lee, Y., Terzopoulos, D., Waters, K.: Realistic modeling for facial animation. In: Computer Graphics, pp. 55–62. Siggraph, August 1995
Lee, J., Moghaddam, B., Pfister, H., Machiraju, R.: A bilinear illumination model for robust face recognition. In: International Conference on Computer Vision (ICCV’05), vol. II, pp. 1177–1184 (2005)
Lei, Z., Bai, Q., He, R., Li, S.: Face shape recovery from a single image using cca mapping between tensor spaces. In: Computer Vision and Pattern Recognition (2008)
Li, S.Z., Gu, L.: Real-time multi-view face detection, tracking, pose estimation, alignment, and recognition. In: IEEE Conf. on Computer Vision and Pattern Recognition Demo Summary (2001)
Li, H., Weise, T., Pauly, M.: Example-based facial rigging. In: ACM Transactions on Graphics. Siggraph, July 2010
Litwinowicz, P., Williams, L.: Animating images with drawings. In: Computer Graphics, pp. 235–242. Siggraph, August 1994
Liu, Z.: A fully automatic system to model faces from a single image. Microsoft Research Technical Report: MST-TR-2003-55 (2003)
Liu, Z., Zhang, Z.: Robust head motion computation by taking advantage of physical properties. In: IEEE Workshop on Human Motion (HUMO), pp. 73–77 (2000)
Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In: Computer Graphics, Annual Conference Series, pp. 271–276. Siggraph, August 2001
Liu, Z., Zhang, Z., Jacobs, C., Cohen, M.: Rapid modeling of animated faces from video. J. Vis. Comput. Animat. 12(4), 227–240 (2001)
Ma, W.-C., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., Debevec, P.: Facial performance synthesis using deformation-driven polynomial displacement maps. In: SIGGRAPH Asia ’08, pp. 1–10. ACM, New York (2008)
Marschner, S.R., Greenberg, D.P.: Inverse lighting for photography. In: IST/SID Fifth Colort Imaging Conference, November 1997
Marschner, S.R., Westin, S., Lafortune, E., Torance, K., Greenberg, D.: Image-based brdf measurement including human skin. In: Rendering Techniques (1999)
Marschner, S.R., Guenter, B., Raghupathy, S.: Modeling and rendering for realistic facial animation. In: Rendering Techniques, pp. 231–242. Springer, New York (2000)
Noh, J.J., Neumann, U.: Expression cloning. In: Computer Graphics, Annual Conference Series, pp. 277–288. Siggraph, August 2001
Parke, F.I.: Computer generated animation of faces. In: ACM National Conference, November 1972
Patel, A., Smith, W.: 3d morphable face models revisited. In: Computer Vision and Pattern Recognition (CVPR’09), pp. 1327–1334 (2009)
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: Computer Graphics, Annual Conference Series, pp. 75–84. Siggraph, July 1998
Pyun, H., Kim, Y., Chae, W., Kang, H.W., Shin, S.Y.: An example-based approach for facial expression cloning. In: Proc. Symposium on Computer Animation (SCA’03), pp. 167–176, July 2003
Qing, L., Shan, S., Gao, W.: Face recognition with harmonic de-lighting. In: Asian Conference on Computer Vision (ACCV), January 2004
Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environment maps. In: Proc. SIGGRAPH 2001, pp. 497–500, August 2001
Riklin-Raviv, T., Shashua, A.: The quotient image: Class based re-rendering and recognition with varying illuminations. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 566–571, June 1999
Romdhani, S., Vetter, T.: Estimating 3d shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: Computer Vision and Pattern Recognition, vol. II, pp. 986–993 (2005)
Romdhani, S., Blanz, V., Vetter, T.: Face identification by fitting a 3d morphable model using linear shape and texture error functions. In: European Conference on Computer Vision (ECCV’2002), vol. IV, pp. 3–19 (2002)
Shan, Y., Liu, Z., Zhang, Z.: Modle-based bundle adjustment with application to face modeling. In: International Conference on Computer Vision (ICCV’01), vol. II, pp. 644–651 (2001)
Shim, H., Luo, J., Chen, T.: A subspace model-based approach to face relighting under unknown lighting and poses. IEEE Trans. Image Process. 17(8), 1331–1341 (2008)
Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. In: Face and Gesture’02 (2002)
Smith, W., Hancock, E.: Recovering facial shape using a statistical model of surface normal direction. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 1914–1930 (2006)
Stoschek, A.: Image-based re-rendering of faces for continuous pose and illumination directions. In: Computer Vision and Pattern Recognition (CVPR’00), pp. 582–587 (2000)
Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. In: Computer Graphics, Annual Conference Series, pp. 399–405. Siggraph (2004)
Terzopoulos, D., Waters, K.: Physically-based facial modeling and animation. J. Vis. Comput. Animat. 1(4), 73–80 (1990)
Wang, Y., Huang, X., Lee, C.-S., Zhang, S., Li, Z., Samaras, D., Metaxas, D., Elgammal, A., Huang, P.: High resolution acquisition, learning and transfer of dynamic 3-d facial expressions. Comput. Graph. Forum 23, 677–686 (2004)
Wang, Y., Zhang, L., Liu, Z., Hua, G., Wen, Z., Zhang, Z., Samaras, D.: Face re-lighting from a single image under arbitrary unknown lighting conditions. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 1968–1984 (2009)
Waters, K.: A muscle model for animating three-dimensional facial expression. Comput. Graph. 22(4), 17–24 (1987)
Wen, Z., Liu, Z., Huang, T.S.: Face relighting with radiance environment maps. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. II, pp. 158–165, June 2003
Williams, L.: Performance-driven facial animation. In: Computer Graphics, pp. 235–242. Siggraph, August 1990
Wolberg, G.: Digital Image Warping. IEEE Computer Society Press, Los Alamitos (1990)
Yan, S., Li, M., Zhang, H., Cheng, Q.: Ranking prior likelihood distributions for Bayesian shape localization framework. In: International Conference on Computer Vision (ICCV’03) (2003)
Yu, Y., Debevec, P.E., Malik, J., Hawkins, T.: Inverse global illumination: Recovering reflectance models of real scenes from photographs. In: Proc. SIGGRAPH 99, pp. 215–224, July 1999
Zhang, L., Samaras, D.: Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 351–363 (2006)
Zhang, Q., Liu, Z., Guo, B., Shum, H.: Geometry-driven photorealistic facial expression synthesis. In: Proc. Symposium on Computer Animation (SCA’03), pp. 177–186, July 2003
Zhang, Q., Liu, Z., Guo, B., Terzopoulos, D., Shum, H.-Y.: Geometry-driven photorealistic facial expression synthesis. IEEE Trans. Vis. Comput. Graph. 12(1), 48–60 (2006)
Zhou, S.K., Aggarwal, G., Chellappa, R., Jacobs, D.: Appearance characterization of linear Lambertian objects, generalized photometric stereo and illumination-invariant face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 330–245 (2007)
Acknowledgements
We thank Ying-Li Tian for carefully reading our manuscripts and providing critical reviews. We also thank Zhengyou Zhang, Alex Acero, and Heung-Yeung Shum for their support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag London Limited
About this chapter
Cite this chapter
Wang, Y., Liu, Z., Guo, B. (2011). Face Synthesis. In: Li, S., Jain, A. (eds) Handbook of Face Recognition. Springer, London. https://doi.org/10.1007/978-0-85729-932-1_20
Download citation
DOI: https://doi.org/10.1007/978-0-85729-932-1_20
Publisher Name: Springer, London
Print ISBN: 978-0-85729-931-4
Online ISBN: 978-0-85729-932-1
eBook Packages: Computer ScienceComputer Science (R0)