Skip to main content

Advertisement

SpringerLink
Log in

An artificial intelligence driven facial emotion recognition system using hybrid deep belief rain optimization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Facial expression recognition is a process of identifying the different facial expressions of the individuals to categorize the mental health of the individual. This system is used in most of the fields but is vastly used in the medical field to identify the mental health issues. In this paper, a novel approach has been proposed to identify the facial expressions of the individuals and categorizing it into seven different emotions. Initially, the images collected from the dataset are subjected to pre-processing for de-noising. Then, the major geometric and appearance-based features are extracted from the images. The most relevant features are selected from the extracted feature set. Finally, based on the selected features, the classification is performed where the input images get labelled into seven different emotions. The classification is carried out with the use of the hybrid strategy called the Deep Belief Rain Optimization (DBRO) technique. The efficiency of the proposed model is proved through the simulations and it is identified to outperform the other existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

References

  1. Alreshidi A, Ullah M (2020) Facial emotion recognition using hybrid features. In: Informatics Multidisciplinary Digital Publishing Institute 7(1):1–13

  2. Anguraju K, Kumar NS, Kumar SJ, Anandhan K, Preethi P (2020) Adaptive feature selection based learning model for emotion recognition. J Critic Rev

  3. Arora M, Kumar M (2021) AutoFER: PCA and PSO based automatic facial emotion recognition. Multimed Tools Appl 80(2):3039–3049

  4. Chen Y, He F, Li H, Zhang D, Wu Y (2020) A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration. Appl Soft Comput 93:106335

    Article  Google Scholar 

  5. Choudhary D, Shukla J (2020) Feature extraction and feature selection for emotion recognition using facial expression. In: 2020 IEEE sixth international conference on multimedia big data (BigMM), pp 125–133

  6. Dhiman G, Singh KK, Soni M, Nagar A, Dehghani M, Slowik A, Kaur A, Sharma A, Houssein EH, Cengiz K (2021 Apr 1) MOSOA: a new multi-objective seagull optimization algorithm. Expert Syst Appl 167:114150

    Article  Google Scholar 

  7. Garg D, Goel P, Pandya S, Ganatra A, Kotecha K (2018) A deep learning approach for face detection using YOLO. In: 2018 IEEE Punecon, pp 1–4

  8. Graumann L, Duesenberg M, Metz S, Schulze L, Wolf OT, Roepke S, Otte C, Wingenfeld K (2021 Jan 1) Facial emotion recognition in borderline patients is unaffected by acute psychosocial stress. J Psychiatr Res 132:131–135

    Article  Google Scholar 

  9. Hassan AK, Mohammed SN (2020 Oct 1) A novel facial emotion recognition scheme based on graph mining. Defence Technology 16(5):1062–1072

    Article  Google Scholar 

  10. Hosny KM, Kassem MA, Fouad MM (2020) Classification of skin lesions into seven classes using transfer learning with AlexNet. J Digit Imaging 33(5):1325–1334

    Article  Google Scholar 

  11. Hou N, He F, Zhou Y, Chen Y (2020) An efficient GPU-based parallel tabu search algorithm for hardware/software co-design. Front Comput Sci 14(5):1–18

    Article  Google Scholar 

  12. Jahanjoo A, Naderan M, Rashti MJ (2020) Detection and multi-class classification of falling in elderly people by deep belief network algorithms. J Ambient Intell Humaniz Comput 11(10):4145–4165

  13. Jain DK, Shamsolmoali P, Sehdev P (2019) Extended deep neural network for facial emotion recognition. Pattern Recogn Lett 120:69–74

    Article  Google Scholar 

  14. Li B, Lima D (2021) Facial expression recognition via ResNet-50. Int J Cogn Comput Eng 2:57–64

    Google Scholar 

  15. Li H, He F, Chen Y, Luo J (2020) Multi-objective self-organizing optimization for constrained sparse array synthesis. Swarm Evol Comput 58:100743

    Article  Google Scholar 

  16. Liang Y, He F, Zeng X (2020) 3D mesh simplification with feature preservation based on whale optimization algorithm and differential evolution. Integrated computer-aided engineering preprint, pp 1–19

  17. Luo J, He F, Yong J (2020) An efficient and robust bat algorithm with fusion of opposition-based learning and whale optimization algorithm. Intell Data Anal 24(3):581–606

    Article  Google Scholar 

  18. Ma T, Benon K, Arnold B, Yu K, Yang Y, Hua Q, Wen Z, Paul AK (2020) Bottleneck feature extraction-based deep neural network model for facial emotion recognition. In: International Conference on Mobile Networks and Management, Springer, Cham, pp 30–46

  19. Mehendale N (2020 Mar) Facial emotion recognition using convolutional neural networks (FERC). SN Appl Sci 2(3):1–8

    Article  Google Scholar 

  20. Mistry K, Rizvi B, Rook C, Iqbal S, Zhang L, Joy CP (2020) A multi-population FA for automatic facial emotion recognition. In: 2020 international joint conference on neural networks (IJCNN), IEEE, pp 1–8

  21. Moazzeni AR, Khamehchi E (2020 Dec 1) Rain optimization algorithm (ROA): a new metaheuristic method for drilling optimization solutions. J Pet Sci Eng 195:107512

    Article  Google Scholar 

  22. Nawaz R, Cheah KH, Nisar H, Yap VV (2020 Jul 1) Comparison of different feature extraction methods for EEG-based emotion recognition. Biocybern Biomed Eng 40(3):910–926

    Article  Google Scholar 

  23. Nguyen TD (n.d.) Multimodal emotion recognition using deep learning techniques (Doctoral dissertation, Queensland University of Technology), pp 1–138

  24. Patwari M, Gutjahr R, Raupach R, Maier A (2020) Low dose CT Denoising via joint bilateral filtering and intelligent parameter optimization, pp 1–4. arXiv preprint arXiv:2007.04768

  25. Rahul M, Shukla R, Goyal PK, Siddiqui ZA, Yadav V (2021) Gabor filter and ICA-based facial expression recognition using two-layered hidden Markov model. In: Advances in computational intelligence and communication technology. Springer, Singapore, pp 511–518

    Chapter  Google Scholar 

  26. Saha S, Ghosh M, Ghosh S, Sen S, Singh PK, Geem ZW, Sarkar R (2020 Jan) Feature selection for facial emotion recognition using cosine similarity-based harmony search algorithm. Appl Sci 10(8):2816

    Article  Google Scholar 

  27. Sepas-Moghaddam A, Etemad A, Pereira F, Correia PL (2020) Facial emotion recognition using light field images with deep attention-based bidirectional LSTM. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 3367–3371

  28. Siddiqui MF, Javaid AY (2020 Sep) A multimodal facial emotion recognition framework through the fusion of speech with visible and infrared images. Multimodal Technol Interact 4(3):46

    Article  Google Scholar 

  29. Simcock G, McLoughlin LT, De Regt T, Broadhouse KM, Beaudequin D, Lagopoulos J, Hermens DF (2020 Jan) Associations between facial emotion recognition and mental health in early adolescence. Int J Environ Res Public Health 17(1):330

    Article  Google Scholar 

  30. Slimani K, Kas M, El Merabet Y, Ruichek Y, Messoussi R (2020 Aug 1) Local feature extraction based facial emotion recognition: a survey. Int J Electr Comput Eng 10(4):4080

    Google Scholar 

  31. Staff AI, Luman M, Van der Oord S, Bergwerff CE, van den Hoofdakker BJ, Oosterlaan J (2021 Jan 7) Facial emotion recognition impairment predicts social and emotional problems in children with (subthreshold) ADHD. Eur Child Adolesc Psychiatry 31:1–3

    Google Scholar 

  32. Ulusoy SI, Gülseren ŞA, Özkan N, Bilen C (2020 Jul 1) Facial emotion recognition deficits in patients with bipolar disorder and their healthy parents. Gen Hosp Psychiatry 65:9–14

    Article  Google Scholar 

  33. Wang S-H, Phillips P, Dong Z-C, Zhang Y-D (2018) Intelligent facial emotion recognition based on stationary wavelet entropy and Jaya algorithm. Neurocomputing 272:668–676

    Article  Google Scholar 

  34. Wang K, Su G, Liu L, Wang S (2020 Jul 20) Wavelet packet analysis for speaker-independent emotion recognition. Neurocomputing 398:257–264

    Article  Google Scholar 

  35. Wieckowski AT, White SW (2020 Jan) Attention modification to attenuate facial emotion recognition deficits in children with autism: a pilot study. J Autism Dev Disord 50(1):30–41

    Article  Google Scholar 

  36. Yang M, Xiao X, Liu Z, Sun L, Guo W, Cui L, Sun D, Zhang P, Yang G (2020) Deep RetinaNet for dynamic left ventricle detection in multiview echocardiography classification. Sci Program 2020:1–6

    Google Scholar 

  37. Yarasca FA, Henríquez SD (2020) Intelligent system based on wavelets for automatic facial emotion recognition. In: 2020 IEEE engineering international research conference (EIRCON), pp 1–4

  38. Yildirim S, Kaya Y, Kılıç F (2021 Feb) A modified feature selection method based on metaheuristic algorithms for speech emotion recognition. Appl Acoust 173:107721

    Article  Google Scholar 

  39. Yin Z, Liu L, Chen J, Zhao B, Wang Y (2020) Locally robust EEG feature selection for individual-independent emotion recognition. Exp Sys Appl 162:113768

    Article  Google Scholar 

  40. Zhou W, Gao S, Zhang L, Lou X (2020 Mar 13) Histogram of oriented gradients feature extraction from raw Bayer pattern images. IEEE Trans Circuits Syst II Express Briefs 67(5):946–950

    Google Scholar 

Download references

Funding

No funding is provided for the preparation of manuscript.

Author information

Authors and Affiliations

  1. Department of Electrical & Electronic Engineering, University of Dhaka, Dhaka, Bangladesh

    Fakir Mashuque Alamgir & Md. Shafiul Alam

Authors
  1. Fakir Mashuque Alamgir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Md. Shafiul Alam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fakir Mashuque Alamgir.

Ethics declarations

Ethics approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of interest

Authors Fakir Mashuque Alamgir, Md. Shafiul Alam declares that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alamgir, F.M., Alam, M.S. An artificial intelligence driven facial emotion recognition system using hybrid deep belief rain optimization. Multimed Tools Appl 82, 2437–2464 (2023). https://doi.org/10.1007/s11042-022-13378-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13378-x

Keywords

Search

Navigation