View-Constrained Latent Variable Model for Multi-view Facial Expression Classification

  • Stefanos Eleftheriadis
  • Ognjen Rudovic
  • Maja Pantic
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8888)


We propose a view-constrained latent variable model for multi-view facial expression classification. In this model, we first learn a discriminative manifold shared by multiple views of facial expressions, followed by the expression classification in the shared manifold. For learning, we use the expression data from multiple views, however, the inference is performed using the data from a single view. Our experiments on data of posed and spontaneously displayed facial expressions show that the proposed approach outperforms the state-of-the-art methods for multi-view facial expression classification, and several state-of-the-art methods for multi-view learning.


Facial Expression Discrete Cosine Transform Local Binary Pattern Scale Invariant Feature Transform Multiple View 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zeng, Z., Pantic, M., Roisman, G., Huang, T.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. on PAMI 31, 39–58 (2009)CrossRefGoogle Scholar
  2. 2.
    Zhu, Z., Ji, Q.: Robust real-time face pose and facial expression recovery. In: IEEE Int’l Conf. on CVPR, pp. 681–688 (2006)Google Scholar
  3. 3.
    Moore, S., Bowden, R.: Local binary patterns for multi-view facial expression recognition. CVIU 115, 541–558 (2011)Google Scholar
  4. 4.
    Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: ICPR, pp. 1–4 (2008)Google Scholar
  5. 5.
    Hesse, N., Gehrig, T., Gao, H., Ekenel, H.K.: Multi-view facial expression recognition using local appearance features. In: ICPR, pp. 3533–3536 (2012)Google Scholar
  6. 6.
    Rudovic, O., Pantic, M., Patras, I.: Coupled gaussian processes for pose-invariant facial expression recognition. IEEE Trans. on PAMI 35, 1357–1369 (2013)CrossRefGoogle Scholar
  7. 7.
    Rudovic, O., Patras, I., Pantic, M.: Regression-based multi-view facial expression recognition. In: Proc. of ICPR, pp. 4121–4124 (2010)Google Scholar
  8. 8.
    Zheng, W., Tang, H., Lin, Z., Huang, T.S.: Emotion recognition from arbitrary view facial images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 490–503. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Tariq, U., Yang, J., Huang, T.S.: Multi-view facial expression recognition analysis with generic sparse coding feature. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part III. LNCS, vol. 7585, pp. 578–588. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. on PAMI 24, 971–987 (2002)CrossRefGoogle Scholar
  11. 11.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning, 273–297 (1995)Google Scholar
  12. 12.
    Cootes, T.F., Edwards, G.J., Taylor, C.J., et al.: Active appearance models. IEEE Trans. on PAMI 23, 681–685 (2001)CrossRefGoogle Scholar
  13. 13.
    Lowe, D.: Object recognition from local scale-invariant features. In: ICCV, vol. 2, pp. 1150–1157 (1999)Google Scholar
  14. 14.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conf. on CVPR, pp. 1794–1801 (2009)Google Scholar
  15. 15.
    Eleftheriadis, S., Rudovic, O., Pantic, M.: Shared gaussian process latent variable model for multi-view facial expression recognition. In: Bebis, G., et al. (eds.) ISVC 2013, Part I. LNCS, vol. 8033, pp. 527–538. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  16. 16.
    Shon, A., Grochow, K., Hertzmann, A., Rao, R.: Learning shared latent structure for image synthesis and robotic imitation. NIPS 18, 1233 (2006)Google Scholar
  17. 17.
    Bertsekas, D.P.: Constrained optimization and lagrange multiplier methods. Computer Science and Applied Mathematics. Academic Press, Boston (1982)zbMATHGoogle Scholar
  18. 18.
    Ek, C., Lawrence, P.: Shared Gaussian Process Latent Variable Models. PhD thesis. Oxford Brookes University (2009)Google Scholar
  19. 19.
    Rasmussen, C., Williams, C.: Gaussian processes for machine learning, vol. 1. MIT Press, Cambridge (2006)zbMATHGoogle Scholar
  20. 20.
    Urtasun, R., Darrell, T.: Discriminative gaussian process latent variable model for classification. In: ICML, pp. 927–934. ACM (2007)Google Scholar
  21. 21.
    Sundararajan, S., Keerthi, S.S.: Predictive approaches for choosing hyperparameters in gaussian processes. Neural Computation 13, 1103–1118 (2001)CrossRefzbMATHGoogle Scholar
  22. 22.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image and Vision Computing 28, 807–813 (2010)CrossRefGoogle Scholar
  23. 23.
    Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. In: IEEE Conf. on CVPR, pp. 545–552 (2011)Google Scholar
  24. 24.
    Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: A semi-automatic methodology for facial landmark annotation. In: CVPR-W 2013 (2013)Google Scholar
  25. 25.
    Bishop, C.M.: Pattern recognition and machine learning, vol. 4. Springer (2006)Google Scholar
  26. 26.
    Zheng, Z., Yang, F., Tan, W., Jia, J., Yang, J.: Gabor feature-based face recognition using supervised locality preserving projection. Signal Processing 87 (2007)Google Scholar
  27. 27.
    Zhong, G., Li, W.-J., Yeung, D.-Y., Hou, X., Liu, C.-L.: Gaussian process latent random field. In: AAAI Conference on Artificial Intelligence (2010)Google Scholar
  28. 28.
    Kan, M., Shan, S., Zhang, H., Lao, S., Chen, X.: Multi-view discriminant analysis. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part I. LNCS, vol. 7572, pp. 808–821. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  29. 29.
    Sharma, A., Kumar, A., Daume, H., Jacobs, D.W.: Generalized multiview analysis: A discriminative latent space. In: IEEE Conf. on CVPR, pp. 2160–2167 (2012)Google Scholar
  30. 30.
    Niyogi, X.: Locality preserving projections. In: NIPS, vol. 16, p. 153 (2004)Google Scholar
  31. 31.
    Pantic, M., Patras, I.: Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans. on SMCB - Part B 36, 433–449 (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Stefanos Eleftheriadis
    • 1
  • Ognjen Rudovic
    • 1
  • Maja Pantic
    • 1
    • 2
  1. 1.Comp. Dept.Imperial College LondonUK
  2. 2.EEMCSUniversity of TwenteThe Netherlands

Personalised recommendations