Skip to main content
SpringerLink
Log in

Facial expression recognition based on local region specific features and support vector machines

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Facial expressions are one of the most powerful, natural and immediate means for human being to communicate their emotions and intensions. Recognition of facial expression has many applications including human-computer interaction, cognitive science, human emotion analysis, personality development etc. In this paper, we propose a new method for the recognition of facial expressions from single image frame that uses combination of appearance and geometric features with support vector machines classification. In general, appearance features for the recognition of facial expressions are computed by dividing face region into regular grid (holistic representation). But, in this paper we extracted region specific appearance features by dividing the whole face region into domain specific local regions. Geometric features are also extracted from corresponding domain specific regions. In addition, important local regions are determined by using incremental search approach which results in the reduction of feature dimension and improvement in recognition accuracy. The results of facial expressions recognition using features from domain specific regions are also compared with the results obtained using holistic representation. The performance of the proposed facial expression recognition system has been validated on publicly available extended Cohn-Kanade (CK+) facial expression data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Agrawal S, Mukherjee DP (2015) Facial expression recognition through adaptive learning of local motion descriptors. Multimed Tools Appl. doi:10.1007/s11042-015-3103-6, online first

    Google Scholar 

  2. Bradski G (2000) “The OpenCV library,” Dr. Gobb’s. J Softw Tools

  3. Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods and their applications. IEEE Trans Affect Comput 1(1):18–37

    Article  Google Scholar 

  4. Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol, pp. 2:27:1—27:27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

  5. Chiranjeevi P, Gopalakrishna V, Moogi P (2015) Neutral face classification using personalized appearance models for fast and robust emotion detection. IEEE Trans Image Process 24(9):2701–2711

    Article  MathSciNet  Google Scholar 

  6. Cruz AC, Bhanu B, Thakoor NS (2014) Vision and attention theory based sampling for continuous facial emotion recognition. IEEE Trans Affect Comput 5(4):418–431

    Article  Google Scholar 

  7. Danelakis A, Theoharis T, Pratikakis I (2014) A survey on facia expression recognition in 3D video sequences. Multimed Tools Appl 74(15):5577–5615

    Article  Google Scholar 

  8. Ekman P (1989) “The argument and evidence about universals in facial expressions of emotions,” Handbook of Social Psychophysiology. Wiley, Chichester, pp 143–164

    Google Scholar 

  9. Ekman P, Friesen W (1978) Facial action coding system (FACS). Consult. Psychol. Press, Palo Alto

    Google Scholar 

  10. Ekman P, Friesen WV, Hager JC (2002) Facial action coding system. A Human Face, Salt Lake City

    Google Scholar 

  11. Ghimire D, Lee J (2013) Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines. Sensors 13:7714–7734

    Article  Google Scholar 

  12. Ghimire D, Lee J (2014) Extreme learning machine ensemble using bagging for facial expression recognition. J Inf Process Syst 10(3):443–458

    Article  Google Scholar 

  13. Hsu CW, Chang CC, Lin CJ (2010) A practical guide to support vector classification. Technical Report; Department of Computer Science, National Taiwan University, Taiwan

  14. Jiang B, Martinez B, Valster MF, Pantic M (2014) Decision level fusion of domain specific regions for facial action recognition. Int. Conf. on Pattern Recog., p 1776–1781, 24–28

  15. Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, 23–28 June 2014, p 1867–1874

  16. King DE (2009) Dlib-ml: a machine learning toolkit. J Mach Learn Res 10:1755–1758

    Google Scholar 

  17. Kotisa I, Pitas I (2007) Facial expression recognition in image sequence using geometric deformation features and support vector machines. IEEE Trans Image Process 16(1):172–187

    Article  MathSciNet  Google Scholar 

  18. Li Y, Wang S, Zhao Y, Ji Q (2013) Simultaneous facial feature tracking and facial expression recognition. IEEE Trans Image Process 22:2559–2573

    Article  Google Scholar 

  19. Liu P, Han S, Meng Z, Tong Y (2014) Facial expression recogniton via a boosted deep belief network. In: Proc. IEEE Conf. on CVPR, p 1805–1812, 23–28

  20. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray scale and rotation invarient texture analysis with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987

    Article  Google Scholar 

  21. Pantic M, Valster M, Rademaker R, Maat L (2010) The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion –specific expressions. In: Proc. of 3rd IEEE Workshop on CVPR for Human Communicatin Behaviour Analysis, p 94–101

  22. Poursaberi A, Noubari HA, Gavrilova M, Yanushkevich SN (2012) Gauss-Laguerre wavelet textural feature fusion with geometrical information for facial expression identification. EURASIP J Image Video Process 2012:17, pp. 1–13

    Article  Google Scholar 

  23. Rudovic O, Pantic M, Patras I (2013) Coupled Gaussian processes for pose-invarient facial expression recognition. IEEE Trans Pattern Anal Mach Intell 35(6):1357–1369

    Article  Google Scholar 

  24. Saeed A, Al-Hamadi A, Niese R, Elzobi M (2014) Frame-based facial expression recognition using geometric features. Adv Hum Comput Interact 2014:1–13

    Article  Google Scholar 

  25. Schels M, Schwenker F (2010) A multiple classifier system approach for facial expressions in image sequence utilizing GMM Supervectors. In: Proc. of the 2010 10th Int. Conf. on Pattern Recog., p 4251–4254

  26. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27:803–816

    Article  Google Scholar 

  27. Siddiqi MH, Ali R, Khan AM, Park Y-T, Lee S (2015) Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE Trans Image Process 24(4):1386–1398

    Article  MathSciNet  Google Scholar 

  28. Siddiqi MH, Lee S, Lee Y-K, Khan AM, Truc PTH (2013) Hierarchical recognitoin scheme for human facial expression recognition systems. Sensors 13:16682–16713

    Article  Google Scholar 

  29. Soyel H, Demirel H (2011) Improved SIFT matching for pose robust facial expression recognition. In: Prof. IEEE Int. Conf. on FG, p 585–590, 21–25

  30. Sun X, Xu H, Zhao C, Yang J (2008) Facial expression recognition based on histogram sequence of local Gabor binary patterns. In: Proc. IEEE Conf. on Cybernatics and Intell. Systems, p 158–163, 21–24

  31. Susskind JM, Hinton GE, Movellan JR, Anderson AK (2008) Generating facial expressions with deep belief nets. In: Kordic V, (ed) Affective computing, emotion modeling, synthesis and recognition, p 421–440

  32. Tu Y-H, Hsu C-T (2012) “Dual subspace nonnegative matrix factorization for person-invarient facial expression recognition,” 21st Int. Conf. on Pattern Recognition (ICPR 2012), p 2391–2394

  33. Uddin MZ, Hassan MM (2013) A depth video-based facial expression recognition system using radon transforn, generalized discriminant analysis, and hidden Markov model. Multimed Tools Appl 74(11):3675–3690

    Article  Google Scholar 

  34. Uddin MZ, Lee J, Kim T (2009) An enhanced independent component-based human facial expression recognition from videos. IEEE Trans Consum Electron 55(4):2216–2224

    Article  Google Scholar 

  35. Yeasin M, Bullot B, Sharma R (2006) Recognition of facial expressions and measurements of levels of interest from videos. IEEE Trans Multimedia 8(3):500–508

    Article  Google Scholar 

  36. Zhang S, Zhao X, Lei B (2012) Robust facial expression recognition via compressive sensing. Sensors 12:3747–3761

    Article  Google Scholar 

  37. Zhao G, Pietikäinen M (2007) Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928

    Article  Google Scholar 

  38. Zhao X, Zhang S (2011) Facial expression recognition based on local binary pattern and kernel discriminant isomap. Sensors 11:9573–9588

    Article  MathSciNet  Google Scholar 

  39. Zhi R, Flierl M, Ruan Q, Kleijn WB (2011) Graph-preserving sparse nonnegative matrix factorization with applications to facial expression recognition. IEEE Trans Syst Man Cybern B Cybern 41(1):38–52

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Korea Electronics Technology Institute, Jeonju-si, Jeollabuk-do, 561-844, Republic of Korea

    Deepak Ghimire, Sunghwan Jeong & San Hyun Park

  2. Division of Computer Engineering, Jeonbuk National University, Jeonju-si, Jeollabuk-do, 561-756, Republic of Korea

    Joonwhoan Lee

Authors
  1. Deepak Ghimire
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sunghwan Jeong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joonwhoan Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. San Hyun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joonwhoan Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghimire, D., Jeong, S., Lee, J. et al. Facial expression recognition based on local region specific features and support vector machines. Multimed Tools Appl 76, 7803–7821 (2017). https://doi.org/10.1007/s11042-016-3418-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3418-y

Keywords

Search

Navigation