Acceleration of CNN-Based Facial Emotion Detection Using NVIDIA GPU

  • Bhakti SonawaneEmail author
  • Priyanka Sharma
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 673)


Emotions often mediate and facilitate interactions among human beings and are conveyed by speech, gesture, face, and physiological signal. Facial expression is a form of nonverbal communication. Failure of correct interpretation of emotion may cause for interpersonal and social conflict. Automatic FER is an active research area and has extensive scope in medical field, crime investigation, marketing, etc. Performance of classical machine learning techniques used for emotion detection is not well when applied directly to images, as they do not consider the structure and composition of the image. In order to address the gaps in traditional machine learning techniques, convolutional neural networks (CNNs) which are a deep Learning algorithm are used. This paper comprises of results and analysis of facial expression for seven basic emotion detection using multiscale feature extractors which are CNNs. Maximum accuracy got using one CNN as 96.5% on JAFFE database. Implementation exploited Graphics Processing Unit (GPU) computation in order to expedite the training process of CNN using GeForce 920 M. In future scope, detection of nonbasic expression can be done using CNN and GPU processing.


CBIR CNN GPU processing Emotion detection 


  1. 1.
    Face Expression Recognition and Analysis: The State of the Art Vinay Kumar Bettadapura Columbia University.Google Scholar
  2. 2.
    Nazia Perveen, Nazir Ahmad, M. Abdul Qadoos Bilal Khan, Rizwan Khalid, Salman Qadri, Facial expression recognition Through Machine Learning, International Journal of Scientific & Technology Research, Volume 5, Issue 03, March 2016, ISSN 2277-8616.Google Scholar
  3. 3.
    S L Happy, Aurobinda Routray Automatic Facial expression recognition Using Features of Salient Facial Patches, IEEE transactions on Affective Computing, VOL. 6, NO. 1, January–March 2015.Google Scholar
  4. 4.
    Muzammil Abdulrahman, Alaa Eleyan, Facial expression recognition Using Support Vector Machines, in Proceeding of 23nd Signal Processing and Communications Applications Conference, PP. 276–279, May 2015.Google Scholar
  5. 5.
    Debishree Dagar, Abir Hudait, H. K. Tripathy, M. N. Das Automatic Emotion Detection Model from Facial Expression, International Conference on Advanced Communication Control and Computing Technologies (ICACCCT), ISBN No. 978-1-4673-9545-8,2016.Google Scholar
  6. 6.
    G. Ramkumar E. Logashanmugam, An Effectual Facial expression recognition using HMM, International Conference on Advanced Communication Control and Computing Technologies (ICACCCT), ISBN No. 978-1-4673-9545-8,2016.Google Scholar
  7. 7.
    S. Tivatansakul, S. Puangpontip, T. Achalakul, and M. Ohkura, “Emotional healthcare system: Emotion detection by facial expressions using Japanese database,” in Proceeding of 6th Computer Science and Electronic Engineering Conference (CEEC), PP. 41–46, Colchester, UK, Sept. 2014.Google Scholar
  8. 8.
    A.R. Syafeeza, M. Khalil-Hani, S.S. Liew, and R. Bakhteri, Convolutional neural network for face recognition with pose and illumination variation, Int. J. of Eng. and Technology, vol. 6(1), pp. 44–57, 2014.Google Scholar
  9. 9.
    Vedantham Ramachandran, E Srinivasa Reddy, Facial Expression Recognition with enhanced feature extraction using PSO & EBPNN. International Journal of Applied Engineering Research, 11(10):69116915, 2016.Google Scholar
  10. 10.
    Boughrara, Hayet, et al. “Facial expression recognition based on a mlp neural network using constructive training algorithm.” Multimedia Tools and Applications 75.2 (2016): 709–731.Google Scholar
  11. 11.
    Hai, Tran Son, and Nguyen Thanh Thuy. “Facial expression classification using artificial neural network and k-nearest neighbor.” International Journal of Information Technology and Computer Science (IJITCS) 7.3 (2015): 27.Google Scholar
  12. 12.
    Shih, Frank Y., Chao-Fa Chuang, and Patrick SP Wang. “Performance comparisons of facial expression recognition in JAFFE database”. International Journal of Pattern Recognition and Artificial Intelligence 22.03 (2008): 445–459.Google Scholar
  13. 13.
    C. Garcia and M. Delakis, “Convolutional face finder: a neural architecture for fast and robust face detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, pp. 1408–1423, 2004.Google Scholar
  14. 14.
    S. Chopra, R. Hadsell, and Y. LeCun, “Learning a Similarity Metric Discriminatively, with Application to Face Verification,” in In Proceedings of CVPR (1) 2005, 2005, pp. 539–546.Google Scholar
  15. 15.
    T. Fok Hing Chi and A. Bouzerdoum, “A Gender Recognition System using Shunting Inhibitory Convolutional Neural Networks,” in International Joint Conference on Neural Networks, 2006, pp. 5336–5341.Google Scholar
  16. 16.
    Caifeng Shan, Shaogang Gong, Peter W. McOwanb, Facial expression recognition based on Local Binary Patterns: A comprehensive study, Image and Vision Computing, V. 27 n. 6, pp. 803–816, May 2009.].
  17. 17.
    Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural Comput., 1(4):541551, Dec. 1989.Google Scholar
  18. 18.
    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. arXiv preprint arXiv:1409.0575, 2014.
  19. 19.
    C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014.
  20. 20.
    Mollahosseini, Ali, David Chan, and Mohammad H. Mahoor. “Going deeper in Facial expression recognition using deep neural networks.” Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on. IEEE, 2016.Google Scholar
  21. 21.
    Jung, Heechul, et al. “Joint fine-tuning in deep neural networks for facial expression recognition.” Proceedings of the IEEE International Conference on Computer Vision. 2015.Google Scholar
  22. 22.
    Dhall A, Murthy OVR, Goecke R, Joshi J, Gedeon T (2015) Video and image based emotion recognition challenges in the wild: Emotiw 2015. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, ACM, pp 423–426.Google Scholar
  23. 23.
    B. Kim, J. Roh, S. Dong, and S. Lee, Hierarchical committee of deep convolutional neural networks for robust facial expression recognition, Journal on Multimodal User Interfaces, pp. 117, 2016.Google Scholar
  24. 24.
    P. Ekman and W. Friesen. Constants Across Cultures in the Face and Emotion. Journal of Personality and Social Psychology, 17(2):124129, 1971.Google Scholar
  25. 25.
    Michael J. Lyons, Shigeru Akemastu, Miyuki Kamachi, Jiro Gyoba. Coding Facial Expressions with Gabor Wavelets, 3rd IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200–205 (1998).Google Scholar
  26. 26.
    Lyons M., Akamatsu S., Kamachi M., and Gyoba J. Coding Facial Expressions with Gabor Wavelets. In Third IEEE International Conference on Automatic Face and Gesture Recognition, pages 200205, April 1998.Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Nirma UniversityAhmedabadIndia

Personalised recommendations