Neural Computing and Applications

, Volume 29, Issue 9, pp 425–443 | Cite as

An exposition of facial expression recognition techniques

  • Somia Saeed
  • M. Khalid Mahmood
  • Yaser Daanial Khan


Automated facial expression recognition schemes have been a subject of interest ever since the inception of its idea. The initial efforts required supervised and controlled environments to get realistic and convincing experimental results. Various improved methods have been suggested to detect the facial expressions of a human. Some of the popularly used methods are automatic expression recognition system (AERS), graph-preserving sparse nonnegative matrix factorization (GSNMF) algorithm, two-phase test sample representation (TPTSR) technique and temporal template method. The current state-of-the-art techniques are able to detect facial expressions in difficult and obscured environments. Advancements in this technique have helped researchers to use video sequences as images to detect expressions. A single image is extracted from a video sequence, and elaborate techniques are applied to detect the expression. This article endeavors to discuss these intricate techniques and critically analyzes them. It helps the reader to understand the contemporary problems in facial recognition systems and how various researchers have employed different models to overcome these challenges. In this paper, the performance of various techniques such as AERS, GSNMF algorithm, TPTSR, performance-based character animation, temporal template method, feature extractions using Gabor filter and image sequencing-based methods has been scrutinized in terms of their efficiency, accuracy and effectiveness. The efficiency and accuracy of the techniques have been compared using various benchmarks such as leave one out, cross-validation and receiver operating characteristics. Each technique bears its own advantages and disadvantages in terms of accuracy and efficiency. The highest accuracy rate is exhibited by the technique using canny edge detection algorithm and chamfer image method.


Facial expression Expression recognition Emotion recognition 


Compliance with ethical standards

Conflict of interest

The authors declare that there is no conflict of interest regarding this publication.


  1. 1.
    Ahad MAR, Tan JK, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23(2):255–281. doi: 10.1007/s00138-010-0298-4
  2. 2.
    Alirezaee S et al (2006) An efficient algorithm for face localization. Int J Inf Technol 12(7):30–36Google Scholar
  3. 3.
    Bartlett MS, Whitehill J (2010) Automated facial expression measurement: recent applications to basic research in human behavior, learning, and education. In: Calder A, Rhodes G, Haxby JV (eds) Handbook of face perception, vol 7. Oxford University Press, San DiegoGoogle Scholar
  4. 4.
    Brunelli R, Poggio T (1993) Face recognition: features vs templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052Google Scholar
  5. 5.
    Carlson NR (1986) Physiology of behavior. Allyn & Bacon, BostonGoogle Scholar
  6. 6.
    Chakrabarti D, Dutta D (2013) Facial expression recognition using eigenspaces. In: International conference on computational intelligence: modeling techniques and applications, IndiaGoogle Scholar
  7. 7.
    Dailey MN, Joyce C, Lyons MJ, Kamachi M, Ishi H, Gyoba J, Cottrell GW (2010) Evidence and a computational explanation of cultural differences in facial expression recognition. Emotion 10(6):874–893CrossRefGoogle Scholar
  8. 8.
    Deng HB, Jin LW, Zhen LX, Huang JC (2005) A new facial expression recognition method based on local Gabor filter bank and PCA plus LDA. Int J Inf Technol 86–96Google Scholar
  9. 9.
    Dornaika F, Raducanu B (2009) Facial expression recognition for HCI applications. Encycl Artif Intell 2(2009):625–631Google Scholar
  10. 10.
    Ekman EL (1993) Facial expression and emotion. Encycl Neurosci (Suppl. 3):51–52Google Scholar
  11. 11.
    Ekman P (2006) Darwin and facial expression: a century of research in review. Ishk, San Jose, CAGoogle Scholar
  12. 12.
    Ekman P, Rolls ET, Perrett DI, Ellis HD (1992) Facial expressions of emotion: an old controversy and new findings [and discussion]. Philos Trans R Soc B Biol Sci 335(1273):63–69CrossRefGoogle Scholar
  13. 13.
    Fasel B, Luettin J (2003) Automatic facial expression analysis: a survey. Pattern Recognit 259–275Google Scholar
  14. 14.
    Fogel A, Thelen E (1987) Development of early expressive and communicative action: reinterpreting the evidence from a dynamic systems perspective. Dev Psychol 747–761Google Scholar
  15. 15.
    Fridlund AJ (1994) Human facial expression: an evolutionary view. Academic Press, San DeigoGoogle Scholar
  16. 16.
    Frith C (2009) Role of facial expressions in social interactions. Comput Emot Man Mach 3453–3458Google Scholar
  17. 17.
    Jack RE, Garrod OGB, Yu H, Caldara R, Schyns PG (2012) Facial expressions of emotion are not culturally universal. Proc Natl Acad Sci 109(19):7241–7244CrossRefGoogle Scholar
  18. 18.
    Kanade T, Yamada A (2003) Multi-subregion based probabilistic approach toward poseinvariant face recognition. In: Proceedings of the 2003 IEEE international symposium on computational intelligence in robotics and automation, 2003, vol 2. IEEE, pp 954–959Google Scholar
  19. 19.
    Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: Proceedings of the fourth IEEE international conference on automatic face and gesture recognition. IEEEGoogle Scholar
  20. 20.
    Kimura MY (2010) Facial expression recognition and its degree estimation. The Kansai Electric Power, OsakaGoogle Scholar
  21. 21.
    Kumbhar M, Jadhav A, Patil M (2012) Facial expression recognition based on image feature. Int J Comput Commun Eng 1(2):117Google Scholar
  22. 22.
    Lucey PC (2010) The extended Cohn–Kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society conference on computer vision. IEEEGoogle Scholar
  23. 23.
    Lyons MJ, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. In: Proceedings of the third IEEE international conference on automatic face and gesture recognition, IEEE Computer Society, Nara, Japan, 14–16 April 1998, pp 200–205Google Scholar
  24. 24.
    Malatesta CZ, Culver C, Tesman JR, Shepard B, Fogel A, Reimers M, Zivin G (1989) The development of emotion expression during the first two years of life. Monogr Soc Res Child Dev 1–136Google Scholar
  25. 25.
    Matsumoto D, Ekman P (2008) Facial expression analysis. Scholarpedia 3(5):4237Google Scholar
  26. 26.
    Mehrabian A (1981) Silent messages: implicit communication of emotions and attitudes. Wadsworth, Belmont, CAGoogle Scholar
  27. 27.
    Meshgini S, Aghagolzadeh A, Seyedarabi H (2012) Face recognition using Gabor filter bank, kernel principle component analysis and support vector machine. Int J Comput Theory Eng 4(5):767Google Scholar
  28. 28.
    Moore S, Ong EJ, Bowden R (2010) Facial expression recognition using spatiotemporal boosted discriminatory classifiers. Int Conf Image Anal Recognit 1(1):405–414Google Scholar
  29. 29.
    Pantic M, Rothkrantz LJM (2000) Automatic analysis of facial expressions: the state art. IEEE Trans Pattern Analysis and Mach Intell 22(12):1424–1445CrossRefGoogle Scholar
  30. 30.
    Pitas KS (1996) Face localization and facial feature extraction based on shape and color information. In: International conference on image processing, Lausanne, pp 483–486Google Scholar
  31. 31.
    Samaria FS (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of the second IEEE workshop on applications of computer vision. IEEE, pp 138–142Google Scholar
  32. 32.
    Shergill GS et al (2008) Computerized sales assistants: the application of computer technology to measure consumer interest—a conceptual framework. J Electron Commer Res 9(2):176Google Scholar
  33. 33.
    Tian D (2013) A review on image feature extraction and representation techniques. Int J Multimed Ubiquitous Eng 385–396Google Scholar
  34. 34.
    Trevarthen C, Aitken CT (2001) Infant intersubjectivity: research, theory, and clinical applications. J Child Psychol Psychiatry 3–48Google Scholar
  35. 35.
    Trujillo L, Olague G, Hammoud R, Hernandez B (2005) Automatic feature localization in thermal images for facial expression recognition. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPR ‘05)-workshops, pp 14–25. IEEEGoogle Scholar
  36. 36.
    Valstar MF, Jiang B, Mehu M, Pantic M, Scherer K (2011) The first facial expression recognition and analysis challenge. In: 9th IEEE international conference on face and gesture recognition, Santa Barbara, CA, USAGoogle Scholar
  37. 37.
    Weise T, Bouaziz S, Li H, Pauly M (2011) Realtime performance-based facial animation. ACM Trans Graph 30(4):9CrossRefGoogle Scholar
  38. 38.
    Xu Y et al (2011) A two-phase test sample sparse representation method for use with face recognition. IEEE Trans Circuits Syst Video Technol 21(9):1255–1262Google Scholar
  39. 39.
    Zhi R et al (2011) Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition. IEEE Trans Syst Man Cybern B 41(1):38–52Google Scholar

Copyright information

© The Natural Computing Applications Forum 2016

Authors and Affiliations

  • Somia Saeed
    • 1
  • M. Khalid Mahmood
    • 2
  • Yaser Daanial Khan
    • 1
  1. 1.School of Systems and TechnologyUniversity of Management and TechnologyLahorePakistan
  2. 2.Department of MathematicsUniversity of PunjabLahorePakistan

Personalised recommendations