Bilinear Kernel Reduced Rank Regression for Facial Expression Synthesis
Conference paper
Abstract
In the last few years, Facial Expression Synthesis (FES) has been a flourishing area of research driven by applications in character animation, computer games, and human computer interaction. This paper proposes a photo-realistic FES method based on Bilinear Kernel Reduced Rank Regression (BKRRR). BKRRR learns a high-dimensional mapping between the appearance of a neutral face and a variety of expressions (e.g. smile, surprise, squint). There are two main contributions in this paper: (1) Propose BKRRR for FES. Several algorithms for learning the parameters of BKRRR are evaluated. (2) Propose a new method to preserve subtle person-specific facial characteristics (e.g. wrinkles, pimples). Experimental results on the CMU Multi-PIE database and pictures taken with a regular camera show the effectiveness of our approach.
Keywords
Facial Expression Neutral Face Facial Animation Alternate Little Square Ground Truth Image
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download
to read the full conference paper text
References
- 1.Gratch, J., Rickel, J., Andre, E., Cassell, J., Petajan, E., Badler, N.: Creating interactive virtual humans: some assembly required. IEEE Intelligent Systems 17, 54–63 (2002)Google Scholar
- 2.Choi, C., Aizawa, K., Harashima, H., Takebe, T.: Analysis and synthesis of facial image sequences in model-based image coding. IEEE Trans. CSVT 4, 257–275 (1994)Google Scholar
- 3.Breen, D., Lin, M.: Vision-based control of 3D facial animation. In: SCA, pp. 193–206 (2003)Google Scholar
- 4.Noh, J., Neumann, U.: Expression cloning. SIGGRAPH 1, 277–288 (2001)Google Scholar
- 5.Keeve, E., Girod, S., Kikinis, R., Girod, B.: Deformable modeling of facial tissue for craniofacial surgery simulation. Computer Aided Surgery 3, 223–228 (1998)CrossRefGoogle Scholar
- 6.Liu, Z., Shan, Y., Zhang, Z.: Expressive expression mapping with ratio images. In Proc. of Ann. Conf. on Computer Graphics and Interactive Techniques (2001)Google Scholar
- 7.Zhang, Q., Liu, Z., Guo, B., Shum, H.: Geometry-driven photorealistic facial expression synthesis. IEEE Trans. VCG 12, 48–60 (2006)Google Scholar
- 8.Chung, K.: Gross Anatomy (Board Review). Lippincott Williams & Wilkins, Hagerstown (2005)Google Scholar
- 9.Nguyen, M., Lalonde, J., Efros, A., De la Torre, F.: Image-based shaving. Computer Graphics Forum (Eurographics) 27, 627–635 (2008)CrossRefGoogle Scholar
- 10.Vasilescu, M., Terzopoulos, D.: Multilinear analysis of image ensembles: Tensorfaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 447–460. Springer, Heidelberg (2002)CrossRefGoogle Scholar
- 11.Tenenbaum, J., Freeman, W.: Separating style and content with bilinear models. Neural Computation 12, 1247–1283 (2000)CrossRefGoogle Scholar
- 12.Wang, H., Ahuja, N.: Facial expression decomposition. In: ICCV (2003)Google Scholar
- 13.Abboud, B., Davoine, F.: Appearance factorization for facial expression analysis. In: BMVC (2004)Google Scholar
- 14.Vlasic, D., Brand, M., Pfister, H., Popovic, J.: Face transfer with multiliner models. ACM Trans. Graphics 24, 426–433 (2005)CrossRefGoogle Scholar
- 15.Macedo, I., Brazil, E., Velho, L.: Expression transfer between photographs through multilinear aam’s. In: SIBGRAPI, pp. 239–246 (2006)Google Scholar
- 16.Bettinger, F., Cootes, T., Taylor, C.: Modelling facial behaviours. In: BMVC, vol. 2 (2002)Google Scholar
- 17.Zalewski, L., Gong, S.: Synthesis and recognition of facial expressions in virtual 3D views. In: AFGR (2004)Google Scholar
- 18.Chang, Y., Hu, C., Ferisa, R., Turk, M.: Manifold based analysis of facial expression. Image and Vision Computing 24, 605–614 (2005)CrossRefGoogle Scholar
- 19.Kouadio, C., Poulin, P., Lachapelle, P.: Real-time facial animation based upon a bank of 3D facial expressions. In: Proc. of Computer Animation (1998)Google Scholar
- 20.Pighin, F., Szeliski, R., Salesin, D.: Resynthesizing facial animation through 3D model-based tracking. In: ICCV (1999)Google Scholar
- 21.Pyun, H., Kim, Y., Chae, W., Kang, H., Shin, S.: An example-based approach for facial expression cloning. In: SIGGRAPH/Eurographics SCA, pp. 167–176 (2003)Google Scholar
- 22.Parke, F.I., Waters, K.: Computer facial animation. AK Peters, Wellesley (1996)Google Scholar
- 23.Anderson, T.: Estimating linear restrictions on regression coefficients for multivariate normal distributions. Annals of Mathematics Statistics 12, 327–351 (1951)CrossRefGoogle Scholar
- 24.Scharf, L.: The SVD and reduced rank signal processing. Signal Processing 25, 113–133 (2002)CrossRefGoogle Scholar
- 25.Diamantaras, K.: Principal Component Neural Networks (Therory and Applications). John Wiley & Sons, Chichester (1996)Google Scholar
- 26.De la Torre, F., Black, M.: Dynamic coupled component analysis. In: CVPR (2001)Google Scholar
- 27.Baldi, P., Hornik, K.: Neural networks and principal component analysis: Learning from examples without local minima. Neural Networks 2, 53–58 (1989)CrossRefGoogle Scholar
- 28.Bathe, K., Wilson, E.: Numerical Methods in Finite Element. Prentice-Hall, Englewood Cliffs (1971)Google Scholar
- 29.De la Torre, F., Gross, R., Baker, S., Kumar, V.: Representational oriented component analysis for face recognition with one sample image per training class. In: CVPR (2005)Google Scholar
- 30.Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: The CMU multi-pose, illumination, and expression (multi-pie) face database. Tech. rep., Robotics Institute, Carnegie Mellon University,TR-07-08 (2007)Google Scholar
- 31.Weinberger, K., Tesauro, G.: Metric learning for kernel regression. In: AISTATS (2007)Google Scholar
- 32.Huang, D., De la Torre, F.: Bilinear kernel reduced rank regression for facial expression synthesis. Tech. rep., Robotics Institute, Carnegie Mellon University,TR-10-23 (2010)Google Scholar
Copyright information
© Springer-Verlag Berlin Heidelberg 2010