Weight-Based Facial Expression Recognition from Near-Infrared Video Sequences

  • Matti Taini
  • Guoying Zhao
  • Matti Pietikäinen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5575)


This paper presents a novel weight-based approach to recognize facial expressions from the near-infrared (NIR) video sequences. Facial expressions can be thought of as specific dynamic textures where local appearance and motion information need to be considered. The face image is divided into several regions from which local binary patterns from three orthogonal planes (LBP-TOP) features are extracted to be used as a facial feature descriptor. The use of LBP-TOP features enables us to set different weights for each of the three planes (appearance, horizontal motion and vertical motion) inside the block volume. The performance of the proposed method is tested in the novel NIR facial expression database. Assigning different weights to the planes according to their contribution improves the performance. NIR images are shown to deal with illumination variations comparing with visible light images.


Local binary pattern region based weights illumination invariance support vector machine 


  1. 1.
    Ahonen, T., Hadid, A., Pietikäinen, M.: Face Description with Local Binary Patterns: Application to Face Recognition. IEEE PAMI 28(12), 2037–2041 (2006)CrossRefzbMATHGoogle Scholar
  2. 2.
    Feng, X., Hadid, A., Pietikäinen, M.: A Coarse-to-Fine Classification Scheme for Facial Expression Recognition. In: Campilho, A.C., Kamel, M.S. (eds.) ICIAR 2004. LNCS, vol. 3212, pp. 668–675. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Shan, C., Gong, S., McOwan, P.W.: Robust Facial Expression Recognition Using Local Binary Patterns. In: 12th IEEE ICIP, pp. 370–373 (2005)Google Scholar
  4. 4.
    Liao, S., Fan, W., Chung, A.C.S., Yeung, D.-Y.: Facial Expression Recognition Using Advanced Local Binary Patterns, Tsallis Entropies and Global Appearance Features. In: 13rd IEEE ICIP, pp. 665–668 (2006)Google Scholar
  5. 5.
    Bassili, J.: Emotion Recognition: The Role of Facial Movement and the Relative Importance of Upper and Lower Areas of the Face. Journal of Personality and Social Psychology 37, 2049–2059 (1979)CrossRefGoogle Scholar
  6. 6.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face Recognition: A Literature Survey. ACM Computing Surveys 35(4), 399–458 (2003)CrossRefGoogle Scholar
  7. 7.
    Adini, Y., Moses, Y., Ullman, S.: Face Recognition: The Problem of Compensating for Changes in Illumination Direction. IEEE PAMI 19(7), 721–732 (1997)CrossRefGoogle Scholar
  8. 8.
    Li, S.Z., Chu, R., Liao, S., Zhang, L.: Illumination Invariant Face Recognition Using Near-Infrared Images. IEEE PAMI 29(4), 627–639 (2007)CrossRefGoogle Scholar
  9. 9.
    Taini, M., Zhao, G., Li, S.Z., Pietikäinen, M.: Facial Expression Recognition from Near-Infrared Video Sequences. In: 19th ICPR (2008)Google Scholar
  10. 10.
    Zhao, G., Pietikäinen, M.: Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions. IEEE PAMI 29(6), 915–928 (2007)CrossRefGoogle Scholar
  11. 11.
    Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley & Sons, New York (2001)zbMATHGoogle Scholar
  12. 12.
    Zhao, G., Pietikäinen, M.: Principal Appearance and Motion from Boosted Spatiotemporal Descriptors. In: 1st IEEE Workshop on CVPR4HB, pp. 1–8 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Matti Taini
    • 1
  • Guoying Zhao
    • 1
  • Matti Pietikäinen
    • 1
  1. 1.Machine Vision Group, Infotech Oulu and Department of Electrical and Information EngineeringUniversity of OuluFinland

Personalised recommendations