Multimedia Tools and Applications

, Volume 76, Issue 6, pp 7803–7821 | Cite as

Facial expression recognition based on local region specific features and support vector machines

  • Deepak Ghimire
  • Sunghwan Jeong
  • Joonwhoan Lee
  • San Hyun Park
Article

Abstract

Facial expressions are one of the most powerful, natural and immediate means for human being to communicate their emotions and intensions. Recognition of facial expression has many applications including human-computer interaction, cognitive science, human emotion analysis, personality development etc. In this paper, we propose a new method for the recognition of facial expressions from single image frame that uses combination of appearance and geometric features with support vector machines classification. In general, appearance features for the recognition of facial expressions are computed by dividing face region into regular grid (holistic representation). But, in this paper we extracted region specific appearance features by dividing the whole face region into domain specific local regions. Geometric features are also extracted from corresponding domain specific regions. In addition, important local regions are determined by using incremental search approach which results in the reduction of feature dimension and improvement in recognition accuracy. The results of facial expressions recognition using features from domain specific regions are also compared with the results obtained using holistic representation. The performance of the proposed facial expression recognition system has been validated on publicly available extended Cohn-Kanade (CK+) facial expression data sets.

Keywords

Facial expressions Local representation Appearance features Geometric features Support vector machines 

References

  1. 1.
    Agrawal S, Mukherjee DP (2015) Facial expression recognition through adaptive learning of local motion descriptors. Multimed Tools Appl. doi:10.1007/s11042-015-3103-6, online firstGoogle Scholar
  2. 2.
    Bradski G (2000) “The OpenCV library,” Dr. Gobb’s. J Softw ToolsGoogle Scholar
  3. 3.
    Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods and their applications. IEEE Trans Affect Comput 1(1):18–37CrossRefGoogle Scholar
  4. 4.
    Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol, pp. 2:27:1—27:27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
  5. 5.
    Chiranjeevi P, Gopalakrishna V, Moogi P (2015) Neutral face classification using personalized appearance models for fast and robust emotion detection. IEEE Trans Image Process 24(9):2701–2711MathSciNetCrossRefGoogle Scholar
  6. 6.
    Cruz AC, Bhanu B, Thakoor NS (2014) Vision and attention theory based sampling for continuous facial emotion recognition. IEEE Trans Affect Comput 5(4):418–431CrossRefGoogle Scholar
  7. 7.
    Danelakis A, Theoharis T, Pratikakis I (2014) A survey on facia expression recognition in 3D video sequences. Multimed Tools Appl 74(15):5577–5615CrossRefGoogle Scholar
  8. 8.
    Ekman P (1989) “The argument and evidence about universals in facial expressions of emotions,” Handbook of Social Psychophysiology. Wiley, Chichester, pp 143–164Google Scholar
  9. 9.
    Ekman P, Friesen W (1978) Facial action coding system (FACS). Consult. Psychol. Press, Palo AltoGoogle Scholar
  10. 10.
    Ekman P, Friesen WV, Hager JC (2002) Facial action coding system. A Human Face, Salt Lake CityGoogle Scholar
  11. 11.
    Ghimire D, Lee J (2013) Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines. Sensors 13:7714–7734CrossRefGoogle Scholar
  12. 12.
    Ghimire D, Lee J (2014) Extreme learning machine ensemble using bagging for facial expression recognition. J Inf Process Syst 10(3):443–458CrossRefGoogle Scholar
  13. 13.
    Hsu CW, Chang CC, Lin CJ (2010) A practical guide to support vector classification. Technical Report; Department of Computer Science, National Taiwan University, TaiwanGoogle Scholar
  14. 14.
    Jiang B, Martinez B, Valster MF, Pantic M (2014) Decision level fusion of domain specific regions for facial action recognition. Int. Conf. on Pattern Recog., p 1776–1781, 24–28Google Scholar
  15. 15.
    Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, 23–28 June 2014, p 1867–1874Google Scholar
  16. 16.
    King DE (2009) Dlib-ml: a machine learning toolkit. J Mach Learn Res 10:1755–1758Google Scholar
  17. 17.
    Kotisa I, Pitas I (2007) Facial expression recognition in image sequence using geometric deformation features and support vector machines. IEEE Trans Image Process 16(1):172–187MathSciNetCrossRefGoogle Scholar
  18. 18.
    Li Y, Wang S, Zhao Y, Ji Q (2013) Simultaneous facial feature tracking and facial expression recognition. IEEE Trans Image Process 22:2559–2573CrossRefGoogle Scholar
  19. 19.
    Liu P, Han S, Meng Z, Tong Y (2014) Facial expression recogniton via a boosted deep belief network. In: Proc. IEEE Conf. on CVPR, p 1805–1812, 23–28Google Scholar
  20. 20.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray scale and rotation invarient texture analysis with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987CrossRefGoogle Scholar
  21. 21.
    Pantic M, Valster M, Rademaker R, Maat L (2010) The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion –specific expressions. In: Proc. of 3rd IEEE Workshop on CVPR for Human Communicatin Behaviour Analysis, p 94–101Google Scholar
  22. 22.
    Poursaberi A, Noubari HA, Gavrilova M, Yanushkevich SN (2012) Gauss-Laguerre wavelet textural feature fusion with geometrical information for facial expression identification. EURASIP J Image Video Process 2012:17, pp. 1–13CrossRefGoogle Scholar
  23. 23.
    Rudovic O, Pantic M, Patras I (2013) Coupled Gaussian processes for pose-invarient facial expression recognition. IEEE Trans Pattern Anal Mach Intell 35(6):1357–1369CrossRefGoogle Scholar
  24. 24.
    Saeed A, Al-Hamadi A, Niese R, Elzobi M (2014) Frame-based facial expression recognition using geometric features. Adv Hum Comput Interact 2014:1–13CrossRefGoogle Scholar
  25. 25.
    Schels M, Schwenker F (2010) A multiple classifier system approach for facial expressions in image sequence utilizing GMM Supervectors. In: Proc. of the 2010 10th Int. Conf. on Pattern Recog., p 4251–4254Google Scholar
  26. 26.
    Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27:803–816CrossRefGoogle Scholar
  27. 27.
    Siddiqi MH, Ali R, Khan AM, Park Y-T, Lee S (2015) Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE Trans Image Process 24(4):1386–1398MathSciNetCrossRefGoogle Scholar
  28. 28.
    Siddiqi MH, Lee S, Lee Y-K, Khan AM, Truc PTH (2013) Hierarchical recognitoin scheme for human facial expression recognition systems. Sensors 13:16682–16713CrossRefGoogle Scholar
  29. 29.
    Soyel H, Demirel H (2011) Improved SIFT matching for pose robust facial expression recognition. In: Prof. IEEE Int. Conf. on FG, p 585–590, 21–25Google Scholar
  30. 30.
    Sun X, Xu H, Zhao C, Yang J (2008) Facial expression recognition based on histogram sequence of local Gabor binary patterns. In: Proc. IEEE Conf. on Cybernatics and Intell. Systems, p 158–163, 21–24Google Scholar
  31. 31.
    Susskind JM, Hinton GE, Movellan JR, Anderson AK (2008) Generating facial expressions with deep belief nets. In: Kordic V, (ed) Affective computing, emotion modeling, synthesis and recognition, p 421–440Google Scholar
  32. 32.
    Tu Y-H, Hsu C-T (2012) “Dual subspace nonnegative matrix factorization for person-invarient facial expression recognition,” 21st Int. Conf. on Pattern Recognition (ICPR 2012), p 2391–2394Google Scholar
  33. 33.
    Uddin MZ, Hassan MM (2013) A depth video-based facial expression recognition system using radon transforn, generalized discriminant analysis, and hidden Markov model. Multimed Tools Appl 74(11):3675–3690CrossRefGoogle Scholar
  34. 34.
    Uddin MZ, Lee J, Kim T (2009) An enhanced independent component-based human facial expression recognition from videos. IEEE Trans Consum Electron 55(4):2216–2224CrossRefGoogle Scholar
  35. 35.
    Yeasin M, Bullot B, Sharma R (2006) Recognition of facial expressions and measurements of levels of interest from videos. IEEE Trans Multimedia 8(3):500–508CrossRefGoogle Scholar
  36. 36.
    Zhang S, Zhao X, Lei B (2012) Robust facial expression recognition via compressive sensing. Sensors 12:3747–3761CrossRefGoogle Scholar
  37. 37.
    Zhao G, Pietikäinen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928CrossRefGoogle Scholar
  38. 38.
    Zhao X, Zhang S (2011) Facial expression recognition based on local binary pattern and kernel discriminant isomap. Sensors 11:9573–9588MathSciNetCrossRefGoogle Scholar
  39. 39.
    Zhi R, Flierl M, Ruan Q, Kleijn WB (2011) Graph-preserving sparse nonnegative matrix factorization with applications to facial expression recognition. IEEE Trans Syst Man Cybern B Cybern 41(1):38–52CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Deepak Ghimire
    • 1
  • Sunghwan Jeong
    • 1
  • Joonwhoan Lee
    • 2
  • San Hyun Park
    • 1
  1. 1.Korea Electronics Technology InstituteJeonju-siRepublic of Korea
  2. 2.Division of Computer EngineeringJeonbuk National UniversityJeonju-siRepublic of Korea

Personalised recommendations