Skip to main content

Advertisement

SpringerLink
Log in

An optimized facial emotion recognition architecture based on a deep convolutional neural network and genetic algorithm

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Emotion recognition is one of the most interesting subjects in machine learning and computer vision fields, which is recognized by body language, speech, and face. Automatic emotion recognition is used in a variety of applications. In practice, recognizing human emotions with high accuracy is a challenging task. For this purpose, in this paper, we have recognized emotion from facial images using convolutional neural network architecture as one of the deep learning networks that used inception modules and dense blocks. The new proposed architecture is represented as GA-Dense-FaceliveNet, in which a genetic algorithm is expressed to tune the hyperparameters of the deep convolutional neural network. The proposed model is evaluated using three well-known datasets: CK + (extended Cohn–Kanade), JAFFE (Japanese Female Facial Expression), and KDEF (Karolinska Directed Emotional Faces). In the experiment, the accuracy of using CK + , JAFFE, and KDEF datasets is 99.96%, 98.92%, and 99.17%, respectively. The results demonstrate that the proposed method has higher performance compared to the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The data used to support the findings of this study are available from the corresponding author upon request.

References

  1. Shen, F., Dai, G., Lin, G., Zhang, J., Kong, W., Zeng, H.: EEG-based emotion recognition using 4D convolutional recurrent neural network. Cogn. Neurodyn. 14, 815–828 (2020)

    Article  PubMed  PubMed Central  Google Scholar 

  2. Wang, J., Wang, M.: Review of the emotional feature extraction and classification using EEG signals. Cogn. Robot. J. 1, 29–40 (2021)

    Article  Google Scholar 

  3. Topic, A., Russo, M.: Emotion recognition based on EEG feature maps through deep learning network. Eng. Sci. Technol. Int. J. 24(6), 1442–1454 (2021)

    Google Scholar 

  4. Jain, N., Kumar, S., Kumar, A., Shamsolmoali, P., Zareapoor, M.: Hybrid deep neural networks for face emotion recognition. Pattern Recognit. Lett. 115, 101–106 (2018)

    Article  ADS  Google Scholar 

  5. Rani, P., Muneeswaran, K.: Emotion recognition based on facial components. Sādhanā 43, 1–16 (2018)

    Article  MathSciNet  Google Scholar 

  6. Ekman, P., Davidson, R.: The Nature of Emotion: Fundamental. Oxford University Press, Oxford (1994)

    Google Scholar 

  7. Hung, J.C., Lin, K.-C., Lai, N.-X.: Recognizing learning emotion based on convolutional neural networks and transfer learning. Appl. Soft Comput. J. 84, 105724 (2019)

    Article  Google Scholar 

  8. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2017)

    Google Scholar 

  9. Khan, A., Sohail, A., Zahoora, U., Saeed Qureshi, A.: A survey of the recent architectures of deep convolutional neural networks. Artif. Intell. Rev. 53, 5455–5516 (2020)

    Article  Google Scholar 

  10. Banharnsakun, A.: Towards improving the convolutional neural networks for deep learning using the distributed artificial bee colony method. Int. J. Mach. Learn. Cybern. 10, 1301–1311 (2018)

    Article  Google Scholar 

  11. Alzubaidi, L., Zhang, J., Humaidi, A.J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Farhan, L.: Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J. Big Data 8, 1–74 (2021)

    Article  Google Scholar 

  12. Bhatti, Y., Jamil, A., Nida, N., Yousaf, M., Viriri, S., Velastin, S.: Facial expression recognition of instructor using deep features and extreme learning machine. Comput. Intell. Neurosci. 2021, 1–17 (2021)

    Article  Google Scholar 

  13. Chen, L., Li, M., Lai, X., Hirota, K., Pedrycz, W.: CNN-based broad learning with efficient incremental reconstruction model for facial emotion recognition. IFAC-PapersOnLine 53(2), 10236–10241 (2020)

    Article  Google Scholar 

  14. Sun, X., Zheng, S., Fu, H.: ROI-attention vectorized CNN model for static facial expression recognition. IEEE Access 8, 7183–7194 (2020)

    Article  Google Scholar 

  15. Li, K., Jin, Y., Akram, M., Han, R., Chen, J.: Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy. Vis. Comput. 36, 391–404 (2019)

    Article  CAS  Google Scholar 

  16. Jain, D., Shamsolmoali, P., Sehdev, P.: Extended deep neural network for facial emotion recognition. Pattern Recognit. Lett. 120, 69–74 (2019)

    Article  ADS  Google Scholar 

  17. Bendjillali, R., Belad, M., Merit, K., Taleb-Ahmed, A.: Improved facial expression recognition based on DWT feature for deep CNN. Electronics 8(3), 324 (2019)

    Article  Google Scholar 

  18. Boughida, A., Kouahla, M., Lafifi, Y.: A novel approach for facial expression recognition based on Gabor filters and genetic algorithm. Evol. Syst. 13(2), 331–345 (2022)

    Article  Google Scholar 

  19. Kumar, R., Sundaram, M., Arumugam, N.: Facial emotion recognition using subband selective multilevel stationary wavelet gradient transform and fuzzy support vector machine. Vis. Comput. 37(8), 2315–2329 (2021)

    Article  Google Scholar 

  20. Lucey, P., Cohn, J., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: (2010) The extended Cohn–Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition–Workshops, pp. 13–18 (2010)

  21. Lundqvist, D., Flykt A.: The Karolinska directed emotional faces KDEF. Retrieved from http://www.emotionlab.se/resourses/kdef (1998)

  22. Lyons, M.J., Kamachi, M.: Japanese female facial expressions (JAFFE). Database Digi. Images 3, 1997 (1997)

    Google Scholar 

  23. Revina, I., Emmanuel, W.: A survey on human face expression recognition techniques. J. King Saud Univ. Comput. Inf. Sci. 33(6), 619–628 (2018)

    Google Scholar 

  24. Mehendale, N.: Facial emotion recognition using convolutional neural networks (FERC). SN Appl. Sci. 2(3), 446 (2020)

    Article  Google Scholar 

  25. Verma, G., Verma, H.: Hybrid-deep learning model for emotion recognition using facial expressions. Rev. Socionetw. Strateg. 14, 171–180 (2020)

    Article  Google Scholar 

  26. Chakraborty, K., Bhattacharyya, S., Bag, R., Hassanien, A.: Sentiment analysis on a set of movie reviews using deep learning techniques. In: Social Network Analytics Computational Research Methods and Techniques, pp. 127–147. Academic Press, Cambridge (2019)

    Chapter  Google Scholar 

  27. Chung, H., Joo Lee, S., Gue Park, J.: Deep neural network using trainable activation functions. In: 2016 International Joint Conference on Neural Networks (IJCNN). Vancouver, BC, Canada (2016)

  28. Klambauer, G., Unterthiner, T., Mayr, A.: Self-normalizing neural networks (2017)

  29. Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolution network (2015)

  30. Faradars: Retrieved 25 May 2020 from https://blog.faradars.org/

  31. Park, S.: A 2021 guide to improving CNNs-network architectures: historical network architectures (medium). Retrieved 8 Nov 2021 from https://medium.com/geekculture/a-2021-guide-to-improving-cnns-network-architectures-historical-network-architectures-d23f32afb1bd

  32. Huang, G., Liu, Z., Maaten, L., Weinberger, K.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA (2017)

  33. Sanii Abadeh, M., Jabal Amelian, Z.: Evolutionary Algorithms and Biological Computing. Niaz Danesh, Tehran (2016)

    Google Scholar 

  34. Abdel-Basset, M., Abdel-Fatah, L., Kumar-Sangaiah, A.: Metaheuristic Algorithms: a comprehensive review. In: Computational Intelligence for Multimedia Big Data on the Cloud with Engineering Applications. Elsevier, Amsterdam (2018)

    Google Scholar 

  35. Katoch, S., Chauhan, S., Kumar, V.: A review on genetic algorithm: past, present, and future. Multimed. Tools Appl. 80, 8091–8126 (2020)

    Article  PubMed  PubMed Central  Google Scholar 

  36. Maas, A., Hannun, A., Ng, A.: Rectifier nonlinearities improve neural network acoustic models. In: The 30th International Conference on Machine Learning, Atlanta, Georgia, USA (2013)

  37. Ruder, S.: An overview of gradient descent optimization algorithms. arXiv:1609.04747 (2017)

  38. Ramachandran, K.M., Tsokos, C.P.: Chapter 12–nonparametric statistics. In: Mathematical Statistics with Applications in R (Third Edition), pp. 491–530. Academic Press, Cambridge (2021)

    Chapter  Google Scholar 

  39. Kohavi, R.: A study of cross validation and bootstrap for accuracy estimation and model selection. In: Appears in the International Joint Conference on Artifcial Intelligence (IJCAI) (1995)

Download references

Funding

The authors have not received any financial support from any person, institution, or organization for this research work.

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Information Technology, Arak Branch-Islamic Azad University, Arak, Iran

    Fereshteh Aghabeigi, Sara Nazari & Nafiseh Osati Eraghi

Authors
  1. Fereshteh Aghabeigi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sara Nazari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nafiseh Osati Eraghi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Aghabeigi wrote the main manuscript text, Osati edited the language of the text, and Aghabeigi and Nazari prepared figures and tables.

Corresponding author

Correspondence to Sara Nazari.

Ethics declarations

Competing interests

The authors reported no potential competing interests with respect to the research, writing, or publication of this article.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors. Results are gotten through simulation with available datasets. All authors declare that they have no competing interests.

Consent to participate

All authors declare that they have the consent to participate.

Consent to publish

All authors declare that they have consent to publish.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aghabeigi, F., Nazari, S. & Osati Eraghi, N. An optimized facial emotion recognition architecture based on a deep convolutional neural network and genetic algorithm. SIViP 18, 1119–1129 (2024). https://doi.org/10.1007/s11760-023-02764-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02764-z

Keyword

Search

Navigation