Abstract
Although the detection of emotions from facial video or images has been topic of intense research for several years, the set of applied classification techniques seems limited to a few popular methods. Benchmark datasets facilitate direct comparison of methods. We used one such dataset, the Cohn-Kanade database, to build classifiers for facial expression recognition based upon Local Binary Patterns (LBP) features. We are interested in the application of Learning Vector Quantization (LVQ) classifiers to this classification task. These prototype-based classifiers allow to inspect of prototypical features of the emotion classes, are conceptually intuitive and quick to train. For comparison we also consider Support Vector Machine (SVM) and observe that LVQ performances exceed those reported in literature for methods based upon LBP features and are amongst the overall top performing methods. Most prominent features were found to originate, primarily, from the mouth region and eye regions. Finally, we explored the specific LBP features that were found most influential within these regions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bashyal, S., Venayagamoorthy, G.K.: Recognition of facial expressions using Gabor wavelets and learning vector quantization. Engineering Applications of Artificial Intelligence 21(7), 1056–1064 (2008)
van den Broek, E., Janssen, J., Westerink, J., Healey, J.: Prerequisites for affective signal processing (asp). In: Encarnao, P., Veloso, A. (eds.) Biosignals 2009: Proceedings of the Second International Conference on Bio-Inspired Systems and Signal Processing, pp. 426–433. INSTICC Press, Portugal (2009)
Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System [E-book]. Research Nexus, Salt Lake City (2002)
Ekman, P.: About brows: emotional and conversational signals. In: von Cranach, M., Foppa, K., Lepenies, W., Ploog, D. (eds.) Human Ethology: Claims and limits of a new discipline, pp. 169–248. Cambridge University Press (1979)
Gritti, T., Shan, C., Jeanne, V., Braspenning, R.: Local features based facial expression recognition with face registration errors. In: 8th IEEE International Conference on Automatic Face Gesture Recognition, FG 2008, pp. 1–8 (2008)
Jabid, T., Kabir, M., Chae, O.: Facial expression recognition using local directional pattern (LDP). In: 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 1605–1608 (2010)
Kanade, T., Cohn, J., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000, pp. 46–53 (2000)
Kohonen, T.: Improved versions of learning vector quantization. In: International Joint Conference on Neural Networks, vol. 1, pp. 545–550. IEEE (1990)
Lajevardi, S.M., Hussain, Z.M.: Novel higher-order local autocorrelation-like feature extraction methodology for facial expression recognition. IET Image Processing 4(2), 114–119 (2010)
Lisetti, C.L., Nasoz, F.: Affective intelligent car interfaces with emotion recognition. In: Proceedings of the 11th International Conference on Human Computer Interaction, p. 41. ACM, Las Vegas (2005)
Lu, H., Wang, Z., Liu, X.: Facial expression recognition using NKFDA method with gabor features. In: The Sixth World Congress on Intelligent Control and Automation, WCICA 2006, vol. 2, pp. 9902–9906 (2006)
Neural Networks Research Centre, Helsinki: Bibliography on the self-organizing maps (SOM) and learning vector quantization (LVQ). Otaniemi: Helsinki Univ. of Technology (2002). http://liinwww.ira.uka.de/bibliography/Neural/SOM.LVQ.html
Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)
Pantic, M., Sebe, N., Cohn, J.F., Huang, T.: Affective multimodal human-computer interaction. In: Proceedings of the 13th Annual ACM International Conference on Multimedia MULTIMEDIA 2005, pp. 669–676. ACM, New York (2005)
Peter, C., Beale, R. (eds.): Affect and Emotion in Human-Computer Interaction. LNCS, vol. 4868. Springer, Heidelberg (2008)
Schneider, P., Biehl, M., Hammer, B.: Adaptive relevance matrices in learning vector quantization. Neural Computation 21(12), 3532–3561 (2009)
Schneider, P., Biehl, M., Hammer, B.: Distance learning in discriminative vector quantization. Neural computation 21(10), 2942–2969 (2009)
Seo, S., Obermayer, K.: Soft learning vector quantization. Neural Computation 15, 1589–1604 (2003)
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing 27(6), 803–816 (2009)
Swerts, M., Krahmer, E.: Facial expression and prosodic prominence: Effects of modality and facial area. Journal of Phonetics 36(2), 219–238 (2008)
Thomson Reuters: Web of science (2014). http://apps.webofknowledge.com/
Tian, Y.L.: Evaluation of face resolution for expression analysis. In: Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2004, pp. 82–82 (2004)
Vapnik, V.: Statistical learning theory. Wiley (1998)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer-Verlag New York Inc., New York (1995)
Wan, S., Aggarwal, J.K.: Spontaneous facial expression recognition: A robust metric learning approach. Pattern Recognition 47(5), 1859–1868 (2014)
Witoelar, A.W., Ghosh, A., de Vries, J.J.G., Hammer, B., Biehl, M.: Window-based example selection in learning vector quantization. Neural Computation 22(11), 2924–2961 (2011)
Zavaschi, T.H., Britto Jr, A.S., Oliveira, L.E., Koerich, A.L.: Fusion of feature sets and classifiers for facial expression recognition. Expert Systems with Applications 40(2), 646–655 (2013)
Zhao, X., Zhang, S.: Facial expression recognition based on local binary patterns and kernel discriminant isomap. Sensors (Basel, Switzerland) 11(10), 9573–9588 (2011)
van der Zwaag, M.D., Janssen, J.H., Westerink, J.H.: Directing physiology and mood through music: Validation of an affective music player. IEEE Transactions on Affective Computing 4(1), 57–68 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
de Vries, GJ., Pauws, S., Biehl, M. (2015). Facial Expression Recognition Using Learning Vector Quantization. In: Azzopardi, G., Petkov, N. (eds) Computer Analysis of Images and Patterns. CAIP 2015. Lecture Notes in Computer Science(), vol 9257. Springer, Cham. https://doi.org/10.1007/978-3-319-23117-4_65
Download citation
DOI: https://doi.org/10.1007/978-3-319-23117-4_65
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23116-7
Online ISBN: 978-3-319-23117-4
eBook Packages: Computer ScienceComputer Science (R0)