Emotion Recognition Through Facial Gestures - A Deep Learning Approach

  • Shrija Mishra
  • Geeta Ramani Bala Prasada
  • Ravi Kant Kumar
  • Goutam Sanyal
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10682)


As defined by some theorists, human emotions are discrete and consistent responses to internal or external events which have significance for an organism. They constitute a major part of our non-verbal communication. Among the human emotions, happy, sad, fear, anger, surprise, disgust and neutral are the seven basic emotions. Facial expressions are the best way to exhibit emotions. In this era of booming human-computer interaction, enabling the machines to recognize these emotions is a paramount task. There is an amalgamation of emotions in every facial expression. In this paper, we identified the different emotions and their intensity level in a human face by implementing deep learning approach through our proposed Convolution Neural Network (CNN). The architecture and the algorithm here yield appreciable results that can be used as a motivation for further research in computer based emotion recognition system.


Face detection Emotion recognition Human-computer interaction Convolutional Neural Network (CNN) Deep learning Cross validation SVM 


  1. 1.
    Carton, J.S., Kessler, E.A., Pape, C.L.: Nonverbal decoding skills and relationship well-being in adults. J. Nonverbal Behav. 23(1), 91–100 (1999)CrossRefGoogle Scholar
  2. 2.
    Izard, C.E.: Human Emotions. Springer, New York (2013)Google Scholar
  3. 3.
    Happy, S.L., George, A., Routray, A.: A real time facial expression classification system using Local Binary Patterns. In: Intelligent Human Computer Interaction (IHCI), 4th International Conference, pp. 1–5. IEEE (2012)Google Scholar
  4. 4.
    Lee, C.C., Mower, E., Busso, C., Lee, S., Narayanan, S.: Emotion recognition using a hierarchical binary decision tree approach. Speech Commun. 53(9), 1162–1171 (2011)CrossRefGoogle Scholar
  5. 5.
    Lopes, A.T., de Aguiar, E., De Souza, A.F., Oliveira-Santos, T.: Facial expression recognition with Convolutional Neural Networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017)CrossRefGoogle Scholar
  6. 6.
    Hu, T., De Silva, L.C., Sengupta, K.: A hybrid approach of NN and HMM for facial emotion classification. Pattern Recogn. Lett. 23(11), 1303–1310 (2002)CrossRefzbMATHGoogle Scholar
  7. 7.
    Sebe, N., Cohen, I., Gevers, T., Huang, T.S.: Emotion recognition based on joint visual and audio cues. In: 18th International Conference on Pattern Recognition, ICPR, vol. 1, pp. 1136–1139. IEEE, August 2006Google Scholar
  8. 8.
    Liu, M., Wang, R., Li, S., Shan, S., Huang, Z., Chen, X.: Combining multiple kernel methods on riemannian manifold for emotion recognition in the wild. In: Proceedings of the 16th ACM International Conference on Multimodal Interaction, pp. 494–501 (2014)Google Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  10. 10.
    FERC 2013: Form 714 – Annual Electric Balancing Authority Area and Planning Area Report (Part 3 Schedule 2) 2006–2012 Form 714 Database, Federal Energy Regulatory Commission (2013)Google Scholar
  11. 11.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRefGoogle Scholar
  12. 12.
    Weston, J., Watkins, C.: Multi-class support vector machines. Technical report CSD-TR-98-04, Department of Computer Science, Royal Holloway, University of London (1998)Google Scholar
  13. 13.
    LeCun, Y.: LeNet-5, Convolutional Neural Networks (2015). http://yann.lecun.Com/exdb/lenet

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringNational Institute of Technology DurgapurDurgapurIndia

Personalised recommendations