Skip to main content

Advertisement

SpringerLink
Log in

Edge-enhanced bi-dimensional empirical mode decomposition-based emotion recognition using fusion of feature set

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Emotion recognition has been of great interest in psychology, machine intelligence, human–machine interaction and biomedical fields. This paper proposes a novel soft computing technique for facial emotion recognition by introducing edge- enhanced bidimensional empirical mode decomposition (EEBEMD) as a feature extraction tool for facial emotion recognition. Facial images are subjected to optimized cost function-based self-guided edge enhancement algorithm. BEMD has been applied on the edge- enhanced facial images, and the first four intrinsic mode functions (IMFs) and the residue have been calculated. On the basis of an empirical analysis, the first IMF is selected for further analysis. A proposed fusion model that consists of selected features from the gray-level co-occurrence matrix, the histogram of oriented gradients and the local ternary pattern of the IMF response is fed to a recursive feature elimination-based algorithm to select the appropriate feature subsets for classification. These feature vectors have been trained in three machine learning algorithms namely multi-class SVM, ELM with RBF kernel and k-NN classifier independently. The IMFs have been subjected to principal component analysis and linear discriminant analysis (LDA) algorithm successively for dimensionality reduction, and the facial images with different emotions have been clustered in different zones in the LDA subspace. The proposed method demonstrates promising accuracy when tested on the JAFFE database, Cohn–Kanade database and the eNTERFACE database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Ali H (2015) Facial emotion recognition using empirical mode decomposition. Expert Syst Appl 42(3):1261–1277

    Article  Google Scholar 

  • Bhattacharya A, Choudhury D, Dey D (2016) Emotion recognition from facial image analysis Using composite similarity measure aided bidimensional empirical mode decomposition. In: IEEE conference on control, measurement and instrumentation, Jadavpur, Kolkata

  • Chakraborti T, Chatterjee A, Konar A, Roy AH (2015) Automated emotion recognition employing a novel modified binary quantum-behaved gravitational search algorithm with differential mutation. Expert Syst Appl 32(4):552–530

    Google Scholar 

  • Chatterjee S , Shi H (2010) A novel neuro fuzzy approach to human emotion determination. In: Digital image computing: techniques and applications, pp 282–287

  • Clausi David A (2002) An analysis of co-occurrence texture statistics as a function of grey level quantization. Can J Remote Sens 28(1):45–62

    Article  Google Scholar 

  • Dahmane M, Meunier J (2011) Emotion recognition using dynamic grid-based HOG features. In: IEEE gesture recognition, Santa Barbara, CA, USA

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Internal computer society conference on computer visions and pattern recognition

  • Deng HB, Jin LW, Zhen LX, Huang JC (2005) A new facial expression recognition method based on local Gabor filter bank and PCA plus LDA. Int J Inform Technol 11(11):86–96

    Google Scholar 

  • Ekman P, Frisen W (1978) Facial action coding system (FACS). Consulting Psychologists Press, Palo Alto

    Google Scholar 

  • Eleyan A, Demirel H (2011) Co-occurrence matrix and its statistical features as a new approach of face recognition. Turk J ElecEng Comp Sci 19(1):97–107

    Google Scholar 

  • Gomathi V, Ramar K, Jeevakumar AS (2009) Human facial expression recognition using MANFIS model. Int J Comput Sci Eng 3(2):93–97

    Google Scholar 

  • Gu WF, Venkatesh YV, Xiang C (2010) A novel application of self-organizing network for facial expression recognition from radial encoded contours. Soft Comput 14:113–122

    Article  Google Scholar 

  • Gupta SK, Agarwal S, Meena YK, Nain N (2011) A hybrid method of feature extraction for facial expression recognition. In: 7th international conference on signal image technology & internet-based systems, pp 422–425

  • Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern 3(6):610–621

    Article  Google Scholar 

  • He K, Sun J, Tang X (2013) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397–1409

    Article  Google Scholar 

  • Huang et al (2002) Emotion recognition from cauchy naive based classifier. In: 16th international conference on pattern recognition, vol 1

  • Huang N et al (1998) The empirical mode decomposition and the Hilbert spectrum for non-linear and non-stationary time series analysis. Proc R Soc Lond A 454:903–995

    Article  MathSciNet  MATH  Google Scholar 

  • Huang CL, Wang CJ (2006) A GA-based feature selection and parameters optimization for support vector machines. Expert Syst Appl 31(2):231–240

    Article  Google Scholar 

  • Hussain S, Triggs B (2010) Feature sets and dimensionality reduction for visual object detection. In: Proceedings of the British machine vision conference, pp 112.1–112.10

  • Ilbeygi M, Hosseini HS (2012) A novel fuzzy facial expression recognition system based on facial feature extraction from color face images. Eng Appl Artif Intell 25:130–146

    Article  Google Scholar 

  • Jin Y, Ruan QQ (2009) Face recognition using Gabor-based improved supervised locality preserving projections. Comput Inform 28:81–95

    MATH  Google Scholar 

  • Kanade T, Cohn JF, Tian YL (2000) Comprehensive database for facial expression analysis. In: Proceedings of 4th IEEE international conference on automatic face and gesture recognition (FG’00), pp 46–53

  • Kazmi SB, Qurat-ul-Ain, Jaffar MA (2012) Wavelet-based facial expression recognition using a bank of support vector machines. Soft Comput 16(3):369–379

  • Kharat GU, Dudul SV (2009) Emotion recognition from facial expression using neural networks. In: Hippe ZS, Kulikowski JL (eds) Human–computer systems interaction, AISC 60, pp 207–219

  • Kim DJ, Song WK, Han JS, Zenn Bien Z (2003) Soft computing based intention reading techniques as a means of human-robot interaction for human centered system. Soft Comput Fusion Found Methodol Appl 7(3):160–166

    MATH  Google Scholar 

  • Koley B, Dey D (2012) An ensemble system for automatic sleep stage classification using single channel EEG signal. Comput Biol Med 42(12):1186–1195

    Article  Google Scholar 

  • Lee CM, Narayanan S, Pieraccini R (2001) Recognition of negative emotions from the speech signal. In: IEEE workshop on automatic speech recognition and understanding, 2001

  • Lin HJ, Wang SY, Yen SH, Kao YT (2005) Face detection based on skin color segmentation and neural network. In: International conference on neural network and brain. Beijing, pp 1144 – 1149

  • Lin X, Yang F, Zhou L, Yin P, Kong H, Xing W, Lu X, Jia L, Wang Q, Xu G (2012) A support vector machine-recursive feature elimination feature selection method based on artificial contrast variables and mutual information. J Chromatogr 910:149–155

    Google Scholar 

  • Liu Z, Wang S (2011) Emotion recognition using hidden markov models from facial temperature sequence. In: Affective computing and intelligent interaction. vol 6975. Springer, Heidelberg, pp 240–247

  • Martin O, Kotsia I, Macq B, Pitas I (2005) The eNTERFACE’05 audio-visual emotion database. In: Proceedings of the first IEEE workshop on multimedia database management, Atlanta

  • Matsuno K, Lee CW, Kimura S , Tsuji S (1995) Automatic recognition of human facial expressions. In: Proceeding of the fifth international conference on computer vision (ICCV’ 95)

  • Murthy GRS, Jadon RS (2009) Effectiveness of Eigenspaces for facial expressions recognition. Int J Comput Theory Eng 1(5):1793–8201

    Google Scholar 

  • Nunes JC, Bouaoune Y, Delechelle E, Niang O, Bunel P (2003) Image analysis by bidimensional empirical mode decomposition. Image Vis Comput 21(2003):1019–1026

    Article  MATH  Google Scholar 

  • Oliveira LS, Koerich RL, Mansano M (2011) 2D Principal Component Analysis for Face and Facial-Expression Recognition. J Comput Sci Eng 13(3):9–13. doi: 10.1109/MCSE.2010.149

  • Rahulamathavan Y, Phan RCW, Chambers JA, Parish DJ (2013) Facial expression recognition in the encrypted domain based on local fisher discriminant analysis. IEEE Trans Affect Comput 4(1):83–92

    Article  Google Scholar 

  • Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27:803–816

    Article  Google Scholar 

  • Shih FY, Chuang CF, Wang PSP (2008) Performance comparisons of facial expression recognition in JAFFE database. Int J Pattern Recognit 22(3):445–459

    Article  Google Scholar 

  • Soh L, Tsatsoulis C (1999) Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices. IEEE Trans Geosci Remote Sens 37(2):780–795

    Article  Google Scholar 

  • Sugiyama M (2006) Local fisher discriminant analysis for supervised dimensionality reduction. In: Proceedings of the 23rd international conference on machine learning, Pittsburgh

  • Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19:1635–1650

    Article  MathSciNet  MATH  Google Scholar 

  • Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  • Visutsak P (2013) Emotion classification through lower facial expressions using adaptive support vector machines. J Man Mach Technol 2(1):12–20

    Google Scholar 

  • Wang Z, Ruan Q (2010) Facial expression based orthogonal local fisher discriminant analysis. In: Proceedings of ICSP 2010, pp 1358–1361

  • Xu X, Liang J, Lv S, Wu Q (2014) Human facial expression analysis based on granule LPP. Int J Mach Learn Cybern 5:907–921

    Article  Google Scholar 

  • Zhang S, Zhao X, Lei B (2012) Facial expression recognition based on local binary patterns and LFDA. WSEAS Trans Signal Process 8(1):21–31

    Google Scholar 

  • Zhang L, Tjondronegoro D (2011) Facial expression recognition using facial movement features. IEEE Trans Affect Comput 2(4):219–229

    Article  Google Scholar 

  • Zhao X, Zhang S (2011) Facial expression recognition based on local binary patterns and kernel discriminant isomap. Sensors 11(10):9573–9588

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Jadavpur University, Kolkata, India

    Arghya Bhattacharya, Dwaipayan Choudhury & Debangshu Dey

Authors
  1. Arghya Bhattacharya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dwaipayan Choudhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Debangshu Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arghya Bhattacharya.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhattacharya, A., Choudhury, D. & Dey, D. Edge-enhanced bi-dimensional empirical mode decomposition-based emotion recognition using fusion of feature set. Soft Comput 22, 889–903 (2018). https://doi.org/10.1007/s00500-016-2395-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-016-2395-4

Keywords

Search

Navigation