Skip to main content
SpringerLink
Log in

Specific Designed Facial Expression Recognition System for Interactive Film Applications

  • Conference paper
  • First Online:
Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD 2020)

Part of the book series: Lecture Notes on Data Engineering and Communications Technologies ((LNDECT,volume 88))

  • 55 Accesses

Abstract

Emotion Artificial Intelligence (AI) is a novel technology for advanced human machine interaction in real-world applications. In interactive films, variant contents can be adjusted and displayed in the film based on audiences’ interaction such as voice, hand gesture or body movement, etc. In this paper, a specific emotion detection system is designed and implemented that can detect emotion continuously through the audience’s facial expression and give the feedback to the film immediately. Then the film can change its contents accordingly. In this system, A pre-trained convolutional neural network is used for feature extraction from video frames and then the emotion is predicted by a support vector regression model. The environmental noise is reduced in the pre-processing stage and the final prediction is smoothed in the post-processing. A database is recorded for this particular scene and the proposed system is trained on it. The experimental results demonstrate the effectiveness of the system and the built interactive film “RIOT” has been exhibited on several occasions with good performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alshamsi, H., Kepuska, V., Meng, H.: Automated facial expression recognition app development on smart phones using cloud computing. In: Proceedings of 8th IEEE Annual Conference on Ubiquitous Computing, Electronics and Mobile Communication (UEMCON), pp. 577–583 (2017)

    Google Scholar 

  2. Alshamsi, H., Meng, H., Li, M.: Real time facial expression recognition app development on mobile phones. In: Processing on 2016 12th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pp. 1750–1755 (2016)

    Google Scholar 

  3. Bound, K.: Future of filmmaking: Artificial intelligent and biofeedback interactive movies. https://www.linkedin.com/pulse/future-filmmaking-artificial-intelligent-biofeedback-keith

  4. Bradski, G.R.: Real time face and object tracking as a component of a perceptual user interface. In: Proceedings of Fourth IEEE Workshop on Applications of Computer Vision 1998, WACV 1998, pp. 214–219 (1998)

    Google Scholar 

  5. Chen, S.D., Ramli, A.R.: Minimum mean brightness error bi-histogram equalization in contrast enhancement. IEEE Trans. Consum. Electron. 49(4), 1310–1319 (2003)

    Google Scholar 

  6. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Processing on IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893 (2005)

    Google Scholar 

  7. Gala, P., Shah, R., Shah, V., Shah, Y., Rane, M.S.: Moodyplayer: a music player based on facial expression recognition. Int. Res. J. Eng. Technol. (IRJET) 5(4), 3703–3707 (2018)

    Google Scholar 

  8. Georgescu, M.I., Ionescu, R.T., Popescu, M.: Local learning with deep and handcrafted features for facial expression recognition. IEEE Access 7 64827–64836 (2019)

    Google Scholar 

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  10. Rebecca Hills-Duty. darksides. https://www.vrfocus.com/2017/10/mindtree-pictures-developing-interactive-vr-movies/

  11. Jan, A., Meng, H., Gaus, Y.F.A., Zhang, F., Turabzadeh, S.: Automatic depression scale prediction using facial expression dynamics and regression. In: Proceedings on the 4th International Workshop on Audio/Visual Emotion Challenge, pp. 73–80. ACM (2014)

    Google Scholar 

  12. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  13. Li, S., Deng, W.: Deep facial expression recognition: a survey. IEEE Trans. Affect. Comput. https://doi.org/10.1109/TAFFC.2020.2981446

  14. Liu, Y., Yuan, X., Gong, X., Xie, Z., Fang, F., Luo, Z.: Conditional convolution neural network enhanced random forest for facial expression recognition. Pattern Recogn. 84, 251–261 (2018)

    Article  Google Scholar 

  15. Lopes, A.T., de Aguiar, E., De Souza, A.F., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017)

    Google Scholar 

  16. Meng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., Wang, Y.: Depression recognition based on dynamic facial and vocal expression features using partial least square regression. In: Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge, pp. 21–30. ACM (2013)

    Google Scholar 

  17. Minaee, S., Abdolrashidi, A.: Deep-emotion: facial expression recognition using attentional convolutional network. arXiv preprint arXiv:1902.01019 (2019)

  18. Morishima, S.: Real-time talking head driven by voice and its application to communication and entertainment. In: AVSP 1998 International Conference on Auditory-Visual Speech Processing (1998)

    Google Scholar 

  19. Nakatsu, R., Nicholson, J., Tosa, N.: Emotion recognition and its application to computer agents with spontaneous interactive capabilities. In: Proceedings of the Seventh ACM International Conference on Multimedia (Part 1), pp. 343–351. ACM (1999)

    Google Scholar 

  20. Palmer, K.: RIOT - KAREN PALMER. http://karenpalmer.uk/portfolio/riot/

  21. Marie-Laure, R.: Narrative as virtual reality: Immersion and Interactivity in Literature and Electronic Media. Johns Hopkins University Press, Baltimore (2001)

    Google Scholar 

  22. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI 2017), vol. 4, p. 12 (2017)

    Google Scholar 

  23. Turabzadeh, S., Meng, H., Swash, R.M., Pleva, M., Juhar, J.: Facial expression emotion detection for real-time embedded systems. Technologies 6(1), 17 (2018)

    Google Scholar 

Download references

Acknowledgement

We’d like to thank Karen Palmer, for the design and coordination of the RIOT prototype. We’d also like to thank The National Theatre Immersive Storytelling Studio, The Perception Institute and Crossover Labs for the support.

Author information

Authors and Affiliations

  1. Brunel University London, London, UB8 3PH, UK

    Rui Qin, Jingxin Liu & Hongying Meng

  2. Southwest University, Chongqing, 400715, China

    Tong Chen

Authors
  1. Rui Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingxin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongying Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongying Meng .

Editor information

Editors and Affiliations

  1. College of Engineering, Design and Physical Sciences, Brunel University London, Uxbridge, UK

    Hongying Meng

  2. School of Electronical Information and Artificial Engineering, Shaanxi University of Science and Technology, Xi’an, China

    Tao Lei

  3. College of Engineering, Design and Physical Sciences, Brunel University London, Uxbridge, UK

    Maozhen Li

  4. College of Electrical and Information, Hunan University, Changsha, China

    Kenli Li

  5. Division of Intelligent Future Technologies, Mälardalen University, Västerås, Västmanlands Län, Sweden

    Ning Xiong

  6. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore

    Lipo Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Qin, R., Liu, J., Meng, H., Chen, T. (2021). Specific Designed Facial Expression Recognition System for Interactive Film Applications. In: Meng, H., Lei, T., Li, M., Li, K., Xiong, N., Wang, L. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2020. Lecture Notes on Data Engineering and Communications Technologies, vol 88. Springer, Cham. https://doi.org/10.1007/978-3-030-70665-4_70

Download citation

Publish with us

Policies and ethics

Search

Navigation