Skip to main content
SpringerLink
Log in

A novel Leaky Rectified Triangle Linear Unit based Deep Convolutional Neural Network for facial emotion recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In numerous fields, Facial Expression Recognitions (FER) is employed, which is a vital topic. The Facial Expressions (FE) is categorized by the FER into human emotions. Most networks are formed for facial Emotion Recognitions (ER); however, they all still possess some challenges like performance degradation together with the lowest accuracy. A novel Leaky Rectified Triangle Linear Unit (LRTLU) Activation Function (AF) based Deep Convolutionals Neural Networks (DCNN) is proposed for achieving better CA. To pre-process the input images, the unique filtering technique Adaptive Bilateral Filter Contourlet Transform (ABFCT) is used. The Chehra face detector was then used to detect the face in the filtered image. The Facial landmarks are recovered from the facial detected image using a cascaded regression tree, and essential features are extracted based on the identified Facial LandMarks. The recovered feature set is then fed into the Leaky Rectified Triangle Linear Unit AF-based Deep Convolutional Neural Networks (LRTLU-DCNN). It classifies the expressions of the inputted image into ‘6’ emotions, say happy, sad, neutral, angry, disgust, together with surprise. The experimentation is performed utilizing the CK+ and JAFFE datasets. The proposed work attains the classification’s accuracy of 99.67347% for the CK+ dataset together with 99.65986% for the JAFFE dataset. The experimental outcome exhibits that the LRTLU-DCNN is better analogized to other prevailing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1:
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data avalibility

In this project, datasets are CK+ and JAFFE which are taken from the internet source and the link is https://www.kaggle.com/datasets/shawon10/ckplus and https://www.kaggle.com/code/mohamedberrimi/jaffe-ck-48/data.

References

  1. Aamir M, Ali T, Shaf A, Irfan M, Saleem MQ (2020) ML-DCNNet: multi-level deep convolutional neural network for facial expression recognition and intensity estimation. Arab J Sci Eng 45(12):10605–10620

    Article  Google Scholar 

  2. Alphonse AS, Shankar K, Jeyasheela Rakkini MJ, Ananthakrishnan S, Athisayamani S, Robert Singh A, Gobi R (2021) A multi-scale and rotation-invariant phase pattern (MRIPP) and a stack of restricted Boltzmann machine (RBM) with preprocessing for facial expression classification. J Ambient Intell Humaniz Comput 12(3):3447–3463

    Article  Google Scholar 

  3. Altameem T, Altameem A (2020) Facial expression recognition using human machine interaction and multi-modal visualization analysis for healthcare applications. Image Vis Comput 103:104044

    Article  Google Scholar 

  4. Arora M, Kumar M, Garg NK (2018) Facial emotion recognition system based on PCA and gradient features. Natl Acad Sci Lett 41(6):365–368

    Article  Google Scholar 

  5. Asthana A, Zafeiriou S, Cheng S, Pantic M (2014) Incremental face alignment in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1859–1866

    Google Scholar 

  6. Bougourzi F, Dornaika F, Mokrani K, Taleb-Ahmedd A, Ruicheke Y (2020) Fusing transformed deep and shallow features (ftds) for image-based facial expression recognition. Expert Syst Appl 156:1–9

    Article  Google Scholar 

  7. Boutorh A, Guessoum A (2016) Complex diseases SNP selection and classification by hybrid association rule mining and artificial neural network—based evolutionary algorithms. Eng Appl Artif Intell 51:58–70

    Article  Google Scholar 

  8. Cai Y, Guo Y, Jiang H, Huang M-C (2018) Machine-learning approaches for recognizing muscle activities involved in facial expressions captured by multi-channels surface electromyogram. Smart Health 5:15–25

    Article  Google Scholar 

  9. Chowdary MK, Nguyen TN, Hemanth DJ (2021) Deep learning-based facial emotion recognition for human–computer interaction applications. Neural Comput Appl:1–18

  10. Happy SL, Dantcheva A, Bremond F (2019) A weakly supervised learning technique for classifying facial expressions. Pattern Recogn Lett 128:162–168

    Article  Google Scholar 

  11. Janu N, Mathur P, Gupta SK, Agrwal SL (2017) Performance analysis of frequency domain based feature extraction techniques for facial expression recognition. In: IEEE 7th international conference on cloud computing, data science & engineering-confluence, pp 591–594

    Google Scholar 

  12. Jiana J, Lin J, Zhou X-h, Haoa LU (2011) Inversion of neural network rayleigh wave dispersion based on LM algorithm. Adv Control Eng Inf Sci 15:5126–5132

    Google Scholar 

  13. Kas M, Ruichek Y, Messoussi R (2020) New framework for person-independent facial expression recognition combining textural and shape analysis through new feature extraction approach. Inf Sci 549:200–220

    Article  MathSciNet  Google Scholar 

  14. Kumar P, Roy PP, Dogra DP (2018) Independent bayesian classifier combination based sign language recognition using facial expression. Inf Sci 428:30–48

    Article  MathSciNet  Google Scholar 

  15. Kurup AR, Ajith M, Ramón MM (2019) Semi-supervised facial expression recognition using reduced spatial features and deep belief networks. Neurocomputing 367:188–197

    Article  Google Scholar 

  16. Lakshmi D, Ponnusamy R (2021) Facial emotion recognition using modified HOG and LBP features with deep stacked autoencoders. Microprocess Microsyst 82:103834

    Article  Google Scholar 

  17. Lekdioui K, Messoussi R, Ruichek Y, Chaabi Y, Touahni R (2017) Facial decomposition for expression recognition using texture/shape descriptors and SVM classifier. Signal Process Image Commun 58:300–312

    Article  Google Scholar 

  18. Li H, Xu H (2020) Deep reinforcement learning for robust emotional classification in facial expression recognition. Knowl-Based Syst 204:106172

    Article  Google Scholar 

  19. Li J, Jin K, Zhou D, Kubota N, Ju Z (2020) Attention mechanism-based CNN for facial expression recognition. Neurocomputing 411:340–350

    Article  Google Scholar 

  20. Li Z, Wang C, Liu X, Wang Y (2021) Facial expression description and recognition based on fuzzy semantic concepts. Futur Gener Comput Syst 114:619–628

    Article  Google Scholar 

  21. Lin C-H (2016) Novel application of continuously variable transmission system using composite recurrent Laguerre orthogonal polynomials modified PSO NN control system. ISA Trans 64:405–417

    Article  MathSciNet  Google Scholar 

  22. Liu Y-u, Yuan X, Gong X, Xie Z, Fang F, Luo Z (2018) Conditional convolution neural network enhanced random forest for facial expression recognition. Pattern Recogn 84:251–261

    Article  Google Scholar 

  23. Mehendale N (2020) Facial emotion recognition using convolutional neural networks (FERC). SN App Sci 2(3):1–8

    Google Scholar 

  24. Michael Revina I, Sam Emmanuel WR (2019) Face expression recognition with the optimization based multi-SVNN classifier and the modified LDP features. J Vis Commun Image Represent 62:43–55

    Article  Google Scholar 

  25. Mishra S, Joshi B, Paudyal R, Chaulagain D, Shakya S (2022) Deep residual learning for facial emotion recognition. In: Mobile computing and sustainable informatics. Springer, Singapore, pp 301–313

    Chapter  Google Scholar 

  26. Mlakar U, Fister I, Brest J, Potočnik B (2017) Multi-objective differential evolution for feature selection in facial expression recognition systems. Expert Syst Appl 89:129–137

    Article  Google Scholar 

  27. Rescigno M, Spezialetti M, Rossi S (2020) Personalized models for facial emotion recognition through transfer learning. Multimed Tools Appl 79(47):35811–35828

    Article  Google Scholar 

  28. Sánchez D, Melin P, Castillo O (2017) Optimization of modular granular neural networks using a firefly algorithm for human recognition. Eng Appl Artif Intell 64:172–186

    Article  Google Scholar 

  29. Shao J, Cheng Q (2021) E-FCNN for tiny facial expression recognition. Appl Intell 51(1):549–559

    Article  Google Scholar 

  30. Singh A, Khan MA, Baghel N (2020) Face emotion identification by fusing neural network and texture features: facial expression. In: IEEE international conference on contemporary computing and applications (IC3A), pp 187–190

    Google Scholar 

  31. Sreedharan NPN, Ganesan B, Raveendran R, Sarala P, Dennis B (2018) Grey wolf optimisation-based feature selection and classification for facial emotion recognition. IET Biom 7(5):490–499

    Article  Google Scholar 

  32. Sun X, Xia P, Ren F (2021) Multi-attention based deep neural network with hybrid features for dynamic sequential facial expression recognition. Neurocomputing 444:378–389

    Article  Google Scholar 

  33. Wang X-H, Liu A, Zhang S-Q (2015) New facial expression recognition based on FSVM and KNN. Optik 126(21):3132–3134

    Article  Google Scholar 

  34. Wang S, Yuan Y, Zheng X, Lu X (2020) Local and correlation attention learning for subtle facial expression recognition. Neurocomputing 453:742–753

    Article  Google Scholar 

  35. Wang Y, Li Y, Song Y, Rong X (2020) The influence of the activation function in a convolution neural network model of facial expression recognition. Appl Sci 10(5):1–20

    Article  Google Scholar 

  36. Yang M, Liu Y, You Z (2017) The Euclidean embedding learning based on convolutional neural network for stereo matching. Neurocomputing 267:195–200

    Article  Google Scholar 

  37. Yu H, He F, Pan Y (2020) A scalable region-based level set method using adaptive bilateral filter for noisy image segmentation. Multimed Tools Appl 79(9):5743–5765

    Article  Google Scholar 

  38. Zheng H, Wang R, Ji W, Zong M, Wong WK, Lai Z, Lv H (2020) Discriminative deep multi-task learning for facial expression recognition. Inf Sci 533:60–71

    Article  Google Scholar 

  39. Zhou L, Liu M, Ye B, Wang X, Liu Q (2020) Sad expressions during encoding enhance facial identity recognition in visual working memory in depression: behavioural and electrophysiological evidence. J Affect Disord 279:630–639

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Jawaharlal Nehru Technological University, Kakinada, India

    Anjani Suputri Devi D & Suneetha Eluri

Authors
  1. Anjani Suputri Devi D
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Suneetha Eluri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anjani Suputri Devi D.

Ethics declarations

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

D, A.S.D., Eluri, S. A novel Leaky Rectified Triangle Linear Unit based Deep Convolutional Neural Network for facial emotion recognition. Multimed Tools Appl 82, 18669–18689 (2023). https://doi.org/10.1007/s11042-022-14186-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14186-z

Keywords

Search

Navigation