Skip to main content
SpringerLink
Log in

Robust facial expression recognition system based on hidden Markov models

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

Facial expressions are considered important communicative tools. In this paper, we present a new system able to recognize facial expressions. This system utilizes one-dimensional Hidden Markov Models (1D-HMMs) with small number of states involving fast computational time during the learning and recognition steps. First, face features are obtained using Gabor wavelets. Then, the Fisher’s Discriminant Analysis method is employed to reduce the features dimensions to eliminate redundancy and overlap in the information. The proposed system differs from previous methods using 1D-HMMs, by the fact that it employs 1D-HMMs with only three states without any prior segmentation step of interest regions in the face (hair, forehead, eyes, etc). The proposed system is evaluated using the JAFFE and KDEF facial expressions data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Pantic M, Rothkrantz LJM (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22(12):1424–1445

    Article  Google Scholar 

  2. Bartlett MS, Hager JC, Ekman P, Sejnowski TJ (1999) Measuring facial expressions by computer image analysis. Psychophysiology 36(02):253–263

    Article  Google Scholar 

  3. Wu X, Zhao J (Aug 2010) Curvelet feature extraction for face recognition and facial expression recognition. In: 2010 Sixth International Conference on Natural Computation, vol 3, pp 1212–1216

  4. Hjelmås E, Low BK (2001) Face detection: a survey. Comput Vis Image Underst 83(3):236–274

  5. Ekman P, Rosenberg EL (1997) What the face reveals: basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA

    Google Scholar 

  6. Zhang Z, Lyons M, Schuster M, Akamatsu S (1998) Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron. In: Automatic Face and Gesture Recognition, 1998. Proceedings. Third IEEE International Conference on, pp 454–459. IEEE

  7. Ghimire D, Lee J (2013) Geometric feature-based facial expression recognition in image sequences using multi-class adaboost and support vector machines. Sensors 13(6):7714–7734

    Article  Google Scholar 

  8. Moore S, Bowden R (2011) Local binary patterns for multi-view facial expression recognition. Comput Vis Image Underst 115(4):541–558

    Article  Google Scholar 

  9. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803–816

    Article  Google Scholar 

  10. Zhao G, Huang X, Taini M, Li SZ, Pietikäinen M (2011) Facial expression recognition from near-infrared videos. Image Vis Comput 29(9):607–619

    Article  Google Scholar 

  11. Ghimire D, Lee J (2012) Histogram of orientation gradient feature-based facial expression classification using bagging with extreme learning machine. Adv Sci Lett 17(1):156–161

    Article  Google Scholar 

  12. Lin DT (2006) Facial expression classification using pca and hierarchical radial basis function network. J Inf Sci Eng 22(5):1033–1046

    Google Scholar 

  13. Cox IJ, Ghosn J, Yianilos PN (1996) Feature-based face recognition using mixture-distance. In: Computer Vision and Pattern Recognition, 1996. Proceedings CVPR ’96, 1996 IEEE Computer Society Conference on, pp 209–216

  14. Verma M, Rani P, Kundra H (2010) A hybrid approach to human face detection. Int J Comput Appl 1(13):67–69

  15. Littlewort G, Bartlett MS, Fasel IR, Susskind J, Movellan JR (2006) Dynamics of facial expression extracted automatically from video. Image Vis Comput 24(6):615–625

    Article  Google Scholar 

  16. Cohen I, Sebe N, Sun Y, Lew MS, Huang TS (2003) Evaluation of expression recognition techniques. In: Image and Video Retrieval, Second International Conference, CIVR 2003, Urbana-Champaign, IL, USA, July 24–25, 2003, Proceedings, pp 184–195

  17. Wang J, Wang S, Ji Q (2014) Early facial expression recognition using hidden markov models. In: 2014 22nd International Conference on Pattern Recognition (ICPR), pp 4594–4599. IEEE

  18. He L, Wang X, Yu C, Wu K (2009) Facial expression recognition using embedded hidden markov model. In: Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on, pp 1568–1572. IEEE

  19. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  20. Marciniak T, Chmielewska A, Weychan R, Parzych M, Dabrowski A (2013) Influence of low resolution of images on reliability of face detection and recognition. Multimed Tools Appl 74(12):4329–4349

    Article  Google Scholar 

  21. Jesorsky O, Kirchberg KJ, Frischholz RW (2001) Robust face detection using the hausdorff distance. In: Audio-and video-based biometric person authentication, pp 90–95. Springer

  22. Marciniak T, Chmielewska A, Weychan R, Parzych M, Dabrowski A (2015) Influence of low resolution of images on reliability of face detection and recognition. Multimed Tools Appl 74(12):4329–4349

    Article  Google Scholar 

  23. Park U, Tong Y, Jain AK (2010) Age-invariant face recognition. Pattern Anal Mach Intell IEEE Trans 32(5):947–954

    Article  Google Scholar 

  24. Sung J, Kim D (2008) Pose-robust facial expression recognition using view-based 2d 3d aam. Syst Man Cybernet Part A Syst Hum IEEE Trans 38(4):852–866

  25. Dongcheng S, Fang C, Guangyi D (2013) Facial expression recognition based on gabor wavelet phase features. In: Image and Graphics (ICIG), 2013 Seventh International Conference on, pp 520–523. IEEE

  26. Yang MH, Kriegman DJ, Ahuja N (2002) Detecting faces in images: a survey. Pattern Anal Mach Intell IEEE Trans 24(1):34–58

    Article  Google Scholar 

  27. Zhang C, Zhang Z (2010) A survey of recent advances in face detection, vol 132077, MSR-TR-2010-66. Technical report of Microsoft Research, Redmond

  28. Kotsia I, Buciu I, Pitas I (2008) An analysis of facial expression recognition under partial facial image occlusion. Image Vis Comput 26(7):1052–1067

    Article  Google Scholar 

  29. Lyons MJ, Budynek J, Akamatsu S (1999) Automatic classification of single facial images. IEEE Trans Pattern Anal Mach Intell 12:1357–1362

    Article  Google Scholar 

  30. Feng X, Pietikainen M, Hadid A (2005) Facial expression recognition with local binary patterns and linear programming. Pattern Recogn Image Anal C/C Raspoznavaniye Obrazov I Analiz Izobrazhenii 15(2):546

    Google Scholar 

  31. Zhao G, Pietikainen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. Pattern Anal Mach Intell IEEE Trans 29(6):915–928

    Article  Google Scholar 

  32. Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: Computer Vision and Pattern Recognition, 1991. Proceedings CVPR’91., IEEE Computer Society Conference on, pp 586–591. IEEE

  33. Uddin MZ, Lee JJ, Kim TS (2009) An enhanced independent component-based human facial expression recognition from video. Consumer Electron IEEE Trans 55(4):2216–2224

    Article  Google Scholar 

  34. Savič T, Pavešić N (2007) Personal recognition based on an image of the palmar surface of the hand. Pattern Recogn 40(11):3152–3163

    Article  MATH  Google Scholar 

  35. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. Pattern Anal Mach Intell IEEE Trans 19(7):711–720

    Article  Google Scholar 

  36. Wenjing H, Haifeng L, Lin M (2011) Estimating continuous-valued emotion of real-life speech. J Converg Inf Technol 6(6):308–316

    Article  Google Scholar 

  37. Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  38. Lyons MJ, Akamatsu S, Kamachi M, Gyoba J, Budynek J (1998) The japanese female facial expression (jaffe) database. In: Proceedings of third international conference on automatic face and gesture recognition, 14-16 April 1998, Nara, Japan. IEEE Computer Society, pp 200–205

  39. Lundqvist D, Flykt A, Öhman A (1998) The karolinska directed emotional faces (kdef). CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet

  40. Praseeda Lekshmi V, Sasikumar M (2009) Analysis of facial expression using gabor and svm. Int J Recent Trends Eng 1(2):1–43

    Google Scholar 

  41. Peng X, Zou B, Tang L, Luo P (2009) Research on dynamic facial expressions recognition. Mod Appl Sci 3(5):31

    Article  Google Scholar 

  42. Mu J, Chen F, Wang C (2015) Research on emotion recognition for facial expression images based on hidden markov model. Int J Mod Trends Eng Res 2(1):683–688

  43. Adaidi W, Lablack A, Bilasco IM (2014) Caractérisation locale des changements de texture pour la reconnaissance des expressions faciales spontanées. In: Colloque National Compression et Représentation des Signaux Audiovisuels (CORESA)

  44. Liew CF, Yairi T (2013) A comparison study of feature spaces and classification methods for facial expression recognition. In: Robotics and Biomimetics (ROBIO), 2013 IEEE International Conference on, pp 1294–1299. IEEE

Download references

Author information

Authors and Affiliations

  1. LAVETE Laboratory, FST of Hassan 1st University Settat, Settat, Morocco

    Zineb Elgarrai & Hakim Allali

  2. LabSIE Laboratory, FPO of Ibn Zohr University, Agadir, Morocco

    Othmane El Meslouhi

  3. Département d’Informatique, Université de Moncton, Moncton, Canada

    Mustapha Kardouchi

Authors
  1. Zineb Elgarrai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Othmane El Meslouhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mustapha Kardouchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hakim Allali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Othmane El Meslouhi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elgarrai, Z., Meslouhi, O.E., Kardouchi, M. et al. Robust facial expression recognition system based on hidden Markov models. Int J Multimed Info Retr 5, 229–236 (2016). https://doi.org/10.1007/s13735-016-0113-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-016-0113-8

Keywords

Search

Navigation