Facial Expression Recognition Using Learning Vector Quantization

  • Gert-Jan de Vries
  • Steffen Pauws
  • Michael Biehl
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9257)


Although the detection of emotions from facial video or images has been topic of intense research for several years, the set of applied classification techniques seems limited to a few popular methods. Benchmark datasets facilitate direct comparison of methods. We used one such dataset, the Cohn-Kanade database, to build classifiers for facial expression recognition based upon Local Binary Patterns (LBP) features. We are interested in the application of Learning Vector Quantization (LVQ) classifiers to this classification task. These prototype-based classifiers allow to inspect of prototypical features of the emotion classes, are conceptually intuitive and quick to train. For comparison we also consider Support Vector Machine (SVM) and observe that LVQ performances exceed those reported in literature for methods based upon LBP features and are amongst the overall top performing methods. Most prominent features were found to originate, primarily, from the mouth region and eye regions. Finally, we explored the specific LBP features that were found most influential within these regions.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bashyal, S., Venayagamoorthy, G.K.: Recognition of facial expressions using Gabor wavelets and learning vector quantization. Engineering Applications of Artificial Intelligence 21(7), 1056–1064 (2008)CrossRefGoogle Scholar
  2. 2.
    van den Broek, E., Janssen, J., Westerink, J., Healey, J.: Prerequisites for affective signal processing (asp). In: Encarnao, P., Veloso, A. (eds.) Biosignals 2009: Proceedings of the Second International Conference on Bio-Inspired Systems and Signal Processing, pp. 426–433. INSTICC Press, Portugal (2009)Google Scholar
  3. 3.
    Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System [E-book]. Research Nexus, Salt Lake City (2002)Google Scholar
  4. 4.
    Ekman, P.: About brows: emotional and conversational signals. In: von Cranach, M., Foppa, K., Lepenies, W., Ploog, D. (eds.) Human Ethology: Claims and limits of a new discipline, pp. 169–248. Cambridge University Press (1979)Google Scholar
  5. 5.
    Gritti, T., Shan, C., Jeanne, V., Braspenning, R.: Local features based facial expression recognition with face registration errors. In: 8th IEEE International Conference on Automatic Face Gesture Recognition, FG 2008, pp. 1–8 (2008)Google Scholar
  6. 6.
    Jabid, T., Kabir, M., Chae, O.: Facial expression recognition using local directional pattern (LDP). In: 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 1605–1608 (2010)Google Scholar
  7. 7.
    Kanade, T., Cohn, J., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000, pp. 46–53 (2000)Google Scholar
  8. 8.
    Kohonen, T.: Improved versions of learning vector quantization. In: International Joint Conference on Neural Networks, vol. 1, pp. 545–550. IEEE (1990)Google Scholar
  9. 9.
    Lajevardi, S.M., Hussain, Z.M.: Novel higher-order local autocorrelation-like feature extraction methodology for facial expression recognition. IET Image Processing 4(2), 114–119 (2010)CrossRefGoogle Scholar
  10. 10.
    Lisetti, C.L., Nasoz, F.: Affective intelligent car interfaces with emotion recognition. In: Proceedings of the 11th International Conference on Human Computer Interaction, p. 41. ACM, Las Vegas (2005)Google Scholar
  11. 11.
    Lu, H., Wang, Z., Liu, X.: Facial expression recognition using NKFDA method with gabor features. In: The Sixth World Congress on Intelligent Control and Automation, WCICA 2006, vol. 2, pp. 9902–9906 (2006)Google Scholar
  12. 12.
    Neural Networks Research Centre, Helsinki: Bibliography on the self-organizing maps (SOM) and learning vector quantization (LVQ). Otaniemi: Helsinki Univ. of Technology (2002).
  13. 13.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)CrossRefGoogle Scholar
  14. 14.
    Pantic, M., Sebe, N., Cohn, J.F., Huang, T.: Affective multimodal human-computer interaction. In: Proceedings of the 13th Annual ACM International Conference on Multimedia MULTIMEDIA 2005, pp. 669–676. ACM, New York (2005)Google Scholar
  15. 15.
    Peter, C., Beale, R. (eds.): Affect and Emotion in Human-Computer Interaction. LNCS, vol. 4868. Springer, Heidelberg (2008) Google Scholar
  16. 16.
    Schneider, P., Biehl, M., Hammer, B.: Adaptive relevance matrices in learning vector quantization. Neural Computation 21(12), 3532–3561 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    Schneider, P., Biehl, M., Hammer, B.: Distance learning in discriminative vector quantization. Neural computation 21(10), 2942–2969 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    Seo, S., Obermayer, K.: Soft learning vector quantization. Neural Computation 15, 1589–1604 (2003)zbMATHCrossRefGoogle Scholar
  19. 19.
    Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing 27(6), 803–816 (2009)CrossRefGoogle Scholar
  20. 20.
    Swerts, M., Krahmer, E.: Facial expression and prosodic prominence: Effects of modality and facial area. Journal of Phonetics 36(2), 219–238 (2008)CrossRefGoogle Scholar
  21. 21.
    Thomson Reuters: Web of science (2014).
  22. 22.
    Tian, Y.L.: Evaluation of face resolution for expression analysis. In: Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2004, pp. 82–82 (2004)Google Scholar
  23. 23.
    Vapnik, V.: Statistical learning theory. Wiley (1998)Google Scholar
  24. 24.
    Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer-Verlag New York Inc., New York (1995)zbMATHCrossRefGoogle Scholar
  25. 25.
    Wan, S., Aggarwal, J.K.: Spontaneous facial expression recognition: A robust metric learning approach. Pattern Recognition 47(5), 1859–1868 (2014)CrossRefGoogle Scholar
  26. 26.
    Witoelar, A.W., Ghosh, A., de Vries, J.J.G., Hammer, B., Biehl, M.: Window-based example selection in learning vector quantization. Neural Computation 22(11), 2924–2961 (2011)CrossRefGoogle Scholar
  27. 27.
    Zavaschi, T.H., Britto Jr, A.S., Oliveira, L.E., Koerich, A.L.: Fusion of feature sets and classifiers for facial expression recognition. Expert Systems with Applications 40(2), 646–655 (2013)CrossRefGoogle Scholar
  28. 28.
    Zhao, X., Zhang, S.: Facial expression recognition based on local binary patterns and kernel discriminant isomap. Sensors (Basel, Switzerland) 11(10), 9573–9588 (2011)CrossRefGoogle Scholar
  29. 29.
    van der Zwaag, M.D., Janssen, J.H., Westerink, J.H.: Directing physiology and mood through music: Validation of an affective music player. IEEE Transactions on Affective Computing 4(1), 57–68 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Gert-Jan de Vries
    • 1
    • 2
  • Steffen Pauws
    • 1
  • Michael Biehl
    • 2
  1. 1.Philips Research - HealthcareEindhovenThe Netherlands
  2. 2.Johann Bernoulli Institute for Mathematics and Computer ScienceUniversity of GroningenGroningenThe Netherlands

Personalised recommendations