Skip to main content

Advertisement

SpringerLink
Log in

Face feature extraction for emotion recognition using statistical parameters from subband selective multilevel stationary biorthogonal wavelet transform

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Facial expression recognition is an extensive aspect in the field of pattern recognition and affective computing. Recognizing emotions by facial expression is an imperative action to design control-oriented and human computer interactive applications. Facial expression recognition is probable by the motion of facial muscles resulting in the appearance variation of face features. Accurate feature extraction is one of the extreme challenges that should be scrutinized for an admirable facial expression recognition system. One of the extensive key techniques used for feature extraction mechanism in facial expression recognition is wavelet transform. The features extracted from the wavelet transform incorporate both spatial and spectral domain information which is best adequate for identifying human emotions through facial expressions. In this paper, the statistical parameters from the proposed subband selective multilevel stationary biorthogonal wavelet transform are estimated and are used as features for effective recognition of emotion. The potency of the feature extraction algorithm is boosted by calculating the mean and maximum local energy wavelet subband of stationary biorthogonal wavelet transform. SVM classifier is used for classification of emotion using the preferred chosen features. Protracted experiments with well-known database for facial expression such as JAFEE database, CK + database, FEED database, SFEW database and RAF database demonstrates a better promising results in emotion classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Alfakih A, Yang S, Hu T (2020) Multi-view cooperative deep convolutional network for facial recognition with small samples learning. Advances in intelligent systems and computing, vol 1003. Springer, Cham. https://doi.org/10.1007/978-3-030-23887-2_24

    Book  Google Scholar 

  • Ali H, Hariharan M, Yaacob S, Adom AH (2015) Facial emotion recognition based on higher-order spectra using support vector based on higher-order spectra using support vector machines. J Med Imag Health Inf 5:1272–1277

    Google Scholar 

  • Bhattacharya A, Choudhury D, Dey D (2018) Edge-enhanced bi-dimensional empirical mode decomposition-based emotion recognition using fusion of feature set. Soft Comput 22:889–903. https://doi.org/10.1007/s00500-016-2395-4

    Article  Google Scholar 

  • Cabada RZ, Rangel HR, Estrada MLB, Lopez HMC (2019) Hyperparameter optimization in CNN for learning centered emotion recognition for intelligent tutoring systems. Soft Comput. https://doi.org/10.1007/s00500-019-04387-4

    Article  Google Scholar 

  • Cambria E (2016) Affective computing and sentiment analysis. IEEE Intell Syst 31(2):102–107

    Article  Google Scholar 

  • Crouse MS, Nowak RD, Baraniuk RG (1998) Wavelet-based statistical signal processing using hidden Markov models. IEEE Trans Signal Proc 46(4):886–902

    Article  MathSciNet  Google Scholar 

  • Darwin C (1872) The expression of the emotions in man and animals. J. Murray, London

    Book  Google Scholar 

  • Dhall A, Goecke R, Lucey S, Gedeon T (2011) Static facial expression analysis in tough conditions: data, evaluation protocol and benchmark. In: Proceedings of IEEE international conference on computer vision workshops, pp 2106–2112

  • Edwards T (1992) Discrete wavelet transforms: theory and implementation. Technical report, Stanford University, 1991

  • Ekman P, Friesen WV (1971) Constant across cultures in face and emotions. J Pers Soc Psychol 17(2):124–129

    Article  Google Scholar 

  • Fan X, Tjahjadi T (2019) Fusing dynamic deep learned features and handcrafted features for facial expression recognition. J Vis Commun Image Represent. https://doi.org/10.1016/j.jvcir.2019.102659

    Article  Google Scholar 

  • Gan Y, Chen J, Yang Z, Luhui X (2019) Multiple attention network for facial expression recognition. IEEE Access. https://doi.org/10.1109/ACCESS.2020.2963913

    Article  Google Scholar 

  • Gavrilescu M (2015) Recognizing emotions from videos by studying facial expressions, body postures and hand gestures. In: 23rd Telecommunications Forum Telfor, Belgrade, SERBIA, pp 720–723

  • Ghimire G, Lee J (2013) Geometric feature-based facial expression recognition in image sequences using multi-class adaboost and support vector machines. J Sens 13:7714–7734

    Article  Google Scholar 

  • Goh KM, Ng CH, Lim LL, Sheikh UU (2018) Micro-expression recognition: an updated review of current trends, Challenges and Solutions. Vis Comput Springer 2018:1–24

    Google Scholar 

  • Goyani M, Patel N (2017) Template matching and machine learning-based robust facial expression recognition system using multi-level Haar wavelet. Int J Comput Appl. https://doi.org/10.1080/1206212X.2017.1395134

    Article  Google Scholar 

  • Goyani M, Patel N (2018) Robust facial expression recognition using local haar mean binary pattern. J Inf Sci Eng 34:1237–1249. https://doi.org/10.6688/JISE.201809_34(5)0008

    Article  Google Scholar 

  • Iqbal MTB, Abdullah-Al-Wadud M, Ryu B, Makhmudkhujaev F, Chae O (2018) Facial expression recognition with neighborhood-aware edge directional pattern (NEDP). IEEE Trans Affect Comput 11(1):125–137. https://doi.org/10.1109/taffc.2018.2829707

    Article  Google Scholar 

  • Jamshidnezhad A, Nordin MJ (2013) Bee royalty offspring algorithm for improvement of facial expressions classification model. Int J Bio-Inspired Comput 5(3):175–191

    Article  Google Scholar 

  • Kazmi SB, Arfan QJ (2012) Wavelets-based facial expression recognition using a bank of support vector machines. Soft Comput 16:369–379. https://doi.org/10.1007/s00500-011-0721-4

    Article  Google Scholar 

  • Khan RA, Meyer A, Konik H, Bouakaz S (2013) Framework for reliable, real-time facial expression recognition for low resolution images. Pattern Recogn Lett 34:1159–1168

    Article  Google Scholar 

  • Li W, Zhang Y, Fu Y (2007) Speech emotion recognition in Elearning system based on affective computing. In: Proceedings of natural computation, 2007, (ICNC 2007), pp 809–813

  • Li S, Weihong D, JunPing D (2017) Reliable crowdsourcing and deep locality preserving learning for expression recognition in the wild. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2584–2593

  • Li K, Jin Y, Akram MW (2020) Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy. Vis Comput 36:391–404. https://doi.org/10.1007/s00371-019-01627-4

    Article  Google Scholar 

  • Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z (2010) The extended Cohn-Kanade Dataset (CK +): a complete dataset for action unit and emotion-specified expression. In: Proceedings of the third international workshop on CVPR for Human communicative behaviour analysis (CVPR4HB 2010), pp 94–101

  • Lyons MJ, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. In: 3rd IEEE international conference on automatic face and gesture recognition, pp 200–205

  • Ma J, Fan X, Yang SX, Zhang X, Zhu X (2018) Contrast limited adaptive histogram equalization based fusion in YIQ and HIS color spaces for underwater image enhancement. Int J Pattern Recognit Artif Intell 32(07):1854018

    Article  Google Scholar 

  • Makhmudkhujaev F, Abdullah-Al-Wadud M, Iqbal MTB, Ryu B, Chae O (2019) Facial expression recognition with local prominent directional pattern. Signal Process Image Commun. https://doi.org/10.1016/j.image.2019.01.002

    Article  Google Scholar 

  • Meena HK, Joshi SD, Sharma KK (2019) Facial expression recognition using graph signal processing on HOG. IETE J Res. https://doi.org/10.1080/03772063.2019.1565952

    Article  Google Scholar 

  • Nason GP, Silverman BW (1995) The stationary wavelet transform and some statistical applications. Wavelet at Statistics, Lecture Notes in statistics, Vol. 103, Springer, New York. pp 281–299

  • Pan X (2020) Fusing HOG and convolutional neural network spatial–temporal features for video-based facial expression recognition. IET Image Process 14(1):176–182. https://doi.org/10.1049/iet-ipr.2019.0293

    Article  Google Scholar 

  • Qayyum H, Majid M, Anwar SM, Khan B (2017) Facial expression recognition using stationary wavelet transform features. Hindawi Math Probl Eng, Vol 2017

  • Reza AM (2004) Realization of the contrast limited adaptive histogram equalization (CLAHE) for real time image enhancement. J VLSI Signal Process Syst Signal Image Video Technol 38(1):35–44

    Article  Google Scholar 

  • Ryu B, Rivera AR, Kim J, Chae O (2017) Local directional ternary pattern for facial expression recognition. IEEE Trans Image Process 26(12):6006–6018. https://doi.org/10.1109/TIP.2017.2726010

    Article  MathSciNet  Google Scholar 

  • Sadeghi H, Raie AA (2019) Histogram distance metric learning for facial expression recognition. J Vis Commun Image Represent 62:152–165. https://doi.org/10.1016/j.jvcir.2019.05.004

    Article  Google Scholar 

  • Sanket NJ, Vyshak AV, Manikantan K, Ramachandran S (2014) Face recognition using adaptive filter wavelet transform based feature extraction. In: International conference on Science Engineering and Management Research, (ICSEMR) 2014 IEEE Stanford University

  • Shoumya NJ, Angb L-M, Sengc KP, Motiur Rahamana DM, Ziaa T (2019) Multimodal big data affective analytics: a comprehensive survey using text, audio, visual and physiological signals. J Netw Comput Appl. https://doi.org/10.1016/j.jnca.2019.102447

    Article  Google Scholar 

  • Sun X, Zheng S, Fu H (2020) ROI-attention vectorized CNN model for static facial expression recognition. IEEE Access 8:7183–7194. https://doi.org/10.1109/ACCESS.2020.2964298

    Article  Google Scholar 

  • Tian Y, Cheng J, Li Y, Wang S (2019) Secondary information aware facial expression recognition. IEEE Signal Process Lett 26(12):1753–1757. https://doi.org/10.1109/LSP.2019.2942138

    Article  Google Scholar 

  • Tsai HH, Chang YC (2017) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22(13):4389–4405. https://doi.org/10.1007/s00500-017-2634-3

    Article  Google Scholar 

  • Uma Maheswari V, Varaprasad G, Viswanadha Raju S (2020) Local directional maximum edge patterns for facial expression recognition. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-020-018863

    Article  Google Scholar 

  • Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  • Wallhoff F, Schuller B, Hawellek M, Rigoll G (2006) Efficient recognition of authentic dynamic facial expressions on the feedtum database. In: IEEE international conference on multimedia and expo, IEEE Computer Society, pp 493–496

  • Wang S, Zhuo Z, Yang H, Li H (2013) An approach to facial expression recognition integrating radial basis function kernel and multidimensional scaling analysis. Soft Comput 18(7):1363–1371. https://doi.org/10.1007/s00500-013-1149-9

    Article  Google Scholar 

  • Wang S-H, Phillips P, Dong Z-C, Zhang Y-D (2017) Intelligent facial emotion recognition based on stationary wavelet entropy and jaya algorithm. Neurocomputing 272:668–676

    Article  Google Scholar 

  • Wang K, Peng X, Yang J, Meng D, Qiao Yu (2020) Region attention networks for pose and occlusion robust facial expression recognition. IEEE Trans Image Process 29:4057–4069

    Article  Google Scholar 

  • Xu Y, Liu J, Zhai Y et al (2020) Weakly supervised facial expression recognition via transferred DAL-CNN and active incremental learning. Soft Comput 24:5971–5985. https://doi.org/10.1007/s00500-019-04530-1

    Article  Google Scholar 

  • Yan W, Ming L, Congxuan Z, Hao C, Yuming L (2019) Weighted-fusion feature of MB-LBPUH and HOG for facial expression recognition. Soft Comput. https://doi.org/10.1007/s00500-019-04380-x

    Article  Google Scholar 

  • Yanga J, Wanga R, Guanb X, Hassanc MM, Almogrenc A, Alsanadc A (2019) AI-enabled emotion-aware robot: the fusion of smart clothing, edge clouds and robotics. Future Gener Comput Syst 102:701–709. https://doi.org/10.1016/j.future.2019.09.029

    Article  Google Scholar 

  • Ye Y, Song Z, Guo J, Qiao Y (2020) SIAT-3DFE: a high-resolution 3D facial expression dataset. IEEE Access 8:48205–48211. https://doi.org/10.1109/ACCESS.2020.2979518

    Article  Google Scholar 

  • Yu M, Zheng H, Peng Z, Dong J, Du H (2020) Facial expression recognition based on a multi-task global-local network. Pattern Recognition Lett. https://doi.org/10.1016/j.patrec.2020.01.016

    Article  Google Scholar 

  • Zhang S, Zhao X, Lei B (2012) Facial expression recognition based on local binary patterns and local fisher discriminant analysis. WSEAS Trans Signal Process 8:21–31

    Google Scholar 

  • Zhang Y-D, Yang Z-J, Hui-Min L, Zhou X-X, Phillips P, Liu Q-M, Wang S-H (2016) Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access 4(2016):8375–8385

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bethlahem Institute of Engineering, Karungal, 629157, India

    R. Jeen Retna Kumar

  2. VSB Engineering College, Karur, 639111, India

    M. Sundaram & V. Kavitha

  3. National Engineering College, Kovilpatti, 628 503, India

    N. Arumugam

Authors
  1. R. Jeen Retna Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Sundaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. N. Arumugam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. V. Kavitha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. Jeen Retna Kumar.

Ethics declarations

Conflict of interest

All authors of the paper declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was not required as no humans or animals were involved.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jeen Retna Kumar, R., Sundaram, M., Arumugam, N. et al. Face feature extraction for emotion recognition using statistical parameters from subband selective multilevel stationary biorthogonal wavelet transform. Soft Comput 25, 5483–5501 (2021). https://doi.org/10.1007/s00500-020-05550-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-020-05550-y

Keywords

Search

Navigation