Skip to main content
SpringerLink
Log in

Human Emotion Recognition from Facial Images Using Convolutional Neural Network

  • Conference paper
  • First Online:
Machine Intelligence and Emerging Technologies (MIET 2022)

Abstract

Recognizing human emotions from facial images has drawn widespread attention in different kinds of applications. Detecting emotions from face has become one of the important research areas as it has a significant impact on the area of emotional communication between people and machines. The goal of this project is to develop a convolutional neural network-based face emotion identification system through the analysis of facial expressions. Convolution Neural Network also known as CNN has been recognized as the best algorithm for image classification. To begin the implementation of our system, each image was subjected to a pre-processing method as part of the image processing procedure. Then, the Convolution, Pooling, and Dropout layers work on the pre-processed image to extract its features. For classification of the image, a fully connected layer with a classifier is used here. To achieve a better performance, we have implemented our system using different optimizers, classifiers, and deep learning techniques. The entire dataset from the Facial Emotion Recognition FER-2013 with Kaggle was used to test the model. Using the FER-2013 dataset, the assessed performance displays the greatest training and testing accuracy, which are, respectively, 77 and 68%. This method aids in categorizing the seven emotions-angry, disgusted, fearful, happy, neutral, sad, and surprised from facial images.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmed, T.U., Jamil, M.N., Hossain, M.S., Andersson, K., Hossain, M.S.: An integrated real-time deep learning and belief rule base intelligent system to assess facial expression under uncertainty. In: 2020 Joint 9th International Conference on Informatics, Electronics & Vision (ICIEV) and 2020 4th International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 1–6. IEEE (2020)

    Google Scholar 

  2. Alharbi, S.T., Hossain, M.S., Monrat, A.A.: A belief rule based expert system to assess autism under uncertainty. In: Proceedings of the World Congress on Engineering and Computer Science, vol. 1 (2015)

    Google Scholar 

  3. Alizadeh, S., Fazel, A.: Convolutional neural networks for facial expression recognition (2017)

    Google Scholar 

  4. Chowdhury, R.R., Hossain, M.S., ul Islam, R., Andersson, K., Hossain, S.: Bangla handwritten character recognition using convolutional neural network with data augmentation. In: 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 318–323. IEEE (2019)

    Google Scholar 

  5. Darwin, C.: The Expression of the Emotions in Man and Animals. University of Chicago Press, Chicago (2015)

    Google Scholar 

  6. Fumo, D.: Types of machine learning algorithms you should know. Towards Data Sci. 15 (2017)

    Google Scholar 

  7. Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8228, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42051-1_16

    Chapter  Google Scholar 

  8. Gunes, H., Schuller, B.: Categorical and dimensional affect analysis in continuous input: current trends and future directions. Image Vis. Comput. 31(2), 120–136 (2013)

    Article  Google Scholar 

  9. Horseman, A.: SVM for facial expression recognition. A demonstrate project using SVM (2007)

    Google Scholar 

  10. Kabir, S., Islam, R.U., Hossain, M.S., Andersson, K.: An integrated approach of belief rule base and deep learning to predict air pollution. Sensors 20(7), 1956 (2020)

    Article  Google Scholar 

  11. Ko, B.C.: A brief review of facial emotion recognition based on visual information. Sensors 18(2), 401 (2018)

    Article  Google Scholar 

  12. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25 (2012)

    Google Scholar 

  13. Latha, C.P., Priya, M.: A review on deep learning algorithms for speech and facial emotion recognition. APTIKOM J. Comput. Sci. Inf. Technol. 1(3), 92–108 (2016)

    Article  Google Scholar 

  14. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  15. Li, F., Ma, J., Huang, D.: MFCC and SVM based recognition of Chinese vowels. In: Hao, Y., et al. (eds.) CIS 2005. LNCS (LNAI), vol. 3802, pp. 812–819. Springer, Heidelberg (2005). https://doi.org/10.1007/11596981_118

    Chapter  Google Scholar 

  16. Li, S., Deng, W.: Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. 13(3), 1195–1215 (2020)

    Article  MathSciNet  Google Scholar 

  17. Liliana, D.Y.: Emotion recognition from facial expression using deep convolutional neural network. In: Journal of Physics: Conference Series, vol. 1193, p. 012004. IOP Publishing (2019)

    Google Scholar 

  18. Mehendale, N.: Facial emotion recognition using convolutional neural networks (FERC). SN Appl. Sci. 2(3), 1–8 (2020)

    Article  Google Scholar 

  19. Mohammadpour, M., Khaliliardali, H., Hashemi, S.M.R., AlyanNezhadi, M.M.: Facial emotion recognition using deep convolutional networks. In: 2017 IEEE 4th International Conference on Knowledge-based Engineering and Innovation (KBEI), pp. 0017–0021. IEEE (2017)

    Google Scholar 

  20. Otroshi-Shahreza, H.: Frame-based face emotion recognition using linear discriminant analysis. In: 2017 3rd Iranian Conference on Intelligent Systems and Signal Processing (ICSPIS), pp. 141–146. IEEE (2017)

    Google Scholar 

  21. Phan-Xuan, H., Le-Tien, T., Nguyen-Tan, S.: FPGA platform applied for facial expression recognition system using convolutional neural networks. Procedia Comput. Sci. 151, 651–658 (2019)

    Article  Google Scholar 

  22. Raihan, S., Zisad, S.N., Islam, R.U., Hossain, M.S., Andersson, K.: A belief rule base approach to support comparison of digital speech signal features for Parkinson’s disease diagnosis. In: Mahmud, M., Kaiser, M.S., Vassanelli, S., Dai, Q., Zhong, N. (eds.) BI 2021. LNCS (LNAI), vol. 12960, pp. 388–400. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86993-9_35

    Chapter  Google Scholar 

  23. Roopa, N.: Emotion recognition from facial expression using deep learning. Int. J. Eng. Adv. Technol. (IJEAT) (2019). ISSN 2249–8958

    Google Scholar 

  24. Sajja, T.K., Kalluri, H.K.: Gender classification based on face images of local binary pattern using support vector machine and back propagation neural networks. Adv. Modell. Anal. B 62(1), 31–35 (2019). https://iieta.org/journals/ama_b

  25. Saravanan, A., Perichetla, G., Gayathri, D.K.S.: Facial emotion recognition using convolutional neural networks (2019)

    Google Scholar 

  26. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  27. Toumi, T., Zidani, A.: From human-computer interaction to human-robot social interaction. arXiv preprint arXiv:1412.1251 (2014)

  28. Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018 (2018)

    Google Scholar 

  29. Zadeh, M.M.T., Imani, M., Majidi, B.: Fast facial emotion recognition using convolutional neural networks and Gabor filters. In: 2019 5th Conference on Knowledge Based Engineering and Innovation (KBEI), pp. 577–581. IEEE (2019)

    Google Scholar 

  30. Zisad, S.N., Chowdhury, E., Hossain, M.S., Islam, R.U., Andersson, K.: An integrated deep learning and belief rule-based expert system for visual sentiment analysis under uncertainty. Algorithms 14(7), 213 (2021)

    Article  Google Scholar 

  31. Zisad, S.N., Hossain, M.S., Andersson, K.: Speech emotion recognition in neurological disorders using convolutional neural network. In: Mahmud, M., Vassanelli, S., Kaiser, M.S., Zhong, N. (eds.) BI 2020. LNCS (LNAI), vol. 12241, pp. 287–296. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59277-6_26

    Chapter  Google Scholar 

  32. Zisad, S.N., Hossain, M.S., Hossain, M.S., Andersson, K.: An integrated neural network and SEIR model to predict COVID-19. Algorithms 14(3), 94 (2021)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Chittagong, Chittagong, Bangladesh

    Saima Sultana, Rashed Mustafa & Mohammad Sanaullah Chowdhury

Authors
  1. Saima Sultana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rashed Mustafa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Sanaullah Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saima Sultana .

Editor information

Editors and Affiliations

  1. Noakhali Science and Technology University, Noakhali, Bangladesh

    Md. Shahriare Satu

  2. The University of Queensland, St. Lucia, QLD, Australia

    Mohammad Ali Moni

  3. Jahangirnagar University, Dhaka, Bangladesh

    M. Shamim Kaiser

  4. Daffodil International University, Dhaka, Bangladesh

    Mohammad Shamsul Arefin

Rights and permissions

Reprints and permissions

Copyright information

© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sultana, S., Mustafa, R., Chowdhury, M.S. (2023). Human Emotion Recognition from Facial Images Using Convolutional Neural Network. In: Satu, M.S., Moni, M.A., Kaiser, M.S., Arefin, M.S. (eds) Machine Intelligence and Emerging Technologies. MIET 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 491. Springer, Cham. https://doi.org/10.1007/978-3-031-34622-4_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34622-4_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34621-7

  • Online ISBN: 978-3-031-34622-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation