Skip to main content
SpringerLink
Log in

Investigating the Effect of Noise on Facial Expression Recognition

  • Conference paper
  • First Online:
Advances in Computer Vision (CVC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 944))

Included in the following conference series:

Abstract

In this paper we investigate the effect of noise on automated recognition of facial expressions. We take images from a publicly available data set; corrupt them with different noise levels and use them for training and testing algorithms for expression recognition. We do recognition using a variant of AlexNet. We do training and testing on same noise levels and also, train on clean images and test on images with added noise. We show that the recognition performance is fairly robust for reasonable levels of noise, however it degrades considerably after that.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anderson, K., McOwan, P.W.: A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 36(1), 96–105 (2006)

    Article  Google Scholar 

  2. Banda, N., Robinson, P.: Noise analysis in audio-visual emotion recognition. In: Proceedings of the International Conference on Multimodal Interaction, pp. 1–4 (2011)

    Google Scholar 

  3. Bartlett, M.S., Littlewort, G., Braathen, B., Sejnowski, T.J., Movellan, J.R.: A prototype for automatic recognition of spontaneous facial actions. In: Advances in Neural Information Processing Systems, pp. 1295–1302. MIT (1998, 2003)

    Google Scholar 

  4. Berretti, S., Del Bimbo, A., Pala, P., Amor, B.B., Daoudi, M.: A set of selected sift features for 3D facial expression recognition. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 4125–4128. IEEE (2010)

    Google Scholar 

  5. Chang, Y., Hu, C., Feris, R., Turk, M.: Manifold based analysis of facial expression. Image Vis. Comput. 24(6), 605–614 (2006)

    Article  Google Scholar 

  6. Chen, R.-H., Lin, R.-J., Yang, P.-C.: The relationships between eCRM, innovation, and customer value - an empirical study. In: 2011 IEEE International Summer Conference of Asia Pacific Business Innovation and Technology Management, pp. 299–302, July 2011

    Google Scholar 

  7. Chen, Y., McBain, R., Norton, D.: Specific vulnerability of face perception to noise: a similar effect in schizophrenia patients and healthy individuals. Psychiatry Res. 225(3), 619–624 (2015)

    Article  Google Scholar 

  8. Cherifi, D., Radji, N., Nait-Ali, A.: Effect of noise, blur and motion on global appearance face recognition based methods performance. Int. J. Comput. Their. Appl. 16(6), 4–13 (2011)

    Google Scholar 

  9. Corneanu, C.A., Simon, M.O., Cohn, J.F., Guerrero, S.E.: Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: history, trends, and affect-related applications. IEEE Trans. Pattern Anal. Mach. Intell. 38(8), 1548–1568 (2016)

    Article  Google Scholar 

  10. Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using PHOG and LPQ features. In: 2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG 2011), pp. 878–883. IEEE (2011)

    Google Scholar 

  11. Donato, G., Bartlett, M.S., Hager, J.C., Ekman, P., Sejnowski, T.J.: Classifying facial actions. IEEE Trans. Pattern Anal. Mach. Intell. 21(10), 974–989 (1999)

    Article  Google Scholar 

  12. Dutta, A., Veldhuis, R.N.J., Spreeuwers, L.J.: The impact of image quality on the performance of face recognition. In: 33rd WIC Symposium on Information Theory in the Benelux. Centre for Telematics and Information Technology (CTIT) (2012)

    Google Scholar 

  13. Ebrahimi Kahou, S., Michalski, V., Konda, K., Memisevic, R., Pal, C.: Recurrent neural networks for emotion recognition in video. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 467–474. ACM (2015)

    Google Scholar 

  14. Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)

    Article  Google Scholar 

  15. Grm, K., Štruc, V., Artiges, A., Caron, M., Ekenel, H.K.: Strengths and weaknesses of deep learning models for face recognition against image degradations. IET Biometrics 7(1), 81–89 (2018)

    Article  Google Scholar 

  16. Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)

    Article  Google Scholar 

  17. Hu, Y., Zeng, Z., Yin, L., Wei, X., Tu, J., Huang, T.S.: A study of non-frontal-view facial expressions recognition. In: 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008)

    Google Scholar 

  18. Krizhevsky, A., Sutskever, I., Hinton, G. E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  19. Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 503–510. ACM (2015)

    Google Scholar 

  20. Lucey, S., Ashraf, A.B., Cohn, J.F.: Investigating spontaneous facial action recognition through AAM representations of the face. INTECH Open Access Publisher (2007)

    Google Scholar 

  21. Molavi, M., Yunus, J.: The effect of noise removing on emotional classification. In: 2012 International Conference on Computer Information Science (ICCIS), vol. 1, pp. 485–489, June 2012

    Google Scholar 

  22. Moore, S., Bowden, R.: Local binary patterns for multi-view facial expression recognition. Comput. Vis. Image Underst. 115(4), 541–558 (2011)

    Article  Google Scholar 

  23. Moore, S., Bowden, R.: The effects of pose on facial expression recognition. In: Proceedings of the British machine vision conference, pp. 1–11 (2009)

    Google Scholar 

  24. Ng, H.-W., Nguyen, V.D., Vonikakis, V., Winkler, S.: Deep learning for emotion recognition on small datasets using transfer learning. In: Proceedings of the 2015 ACM on international conference on multimodal interaction, pp. 443–449. ACM (2015)

    Google Scholar 

  25. Oh, D., Osherson, D.N., Todorov, A.: Robustness of emotional expression recognition under low visibility

    Google Scholar 

  26. Pantic, M., Bartlett, M.S.: Machine analysis of facial expressions. I-Tech Education and Publishing (2007)

    Google Scholar 

  27. Reed, S., Sohn, K., Zhang, Y., Lee, H.: Learning to disentangle factors of variation with manifold interaction. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 1431–1439 (2014)

    Google Scholar 

  28. Rifai, S., Bengio, Y., Courville, A., Vincent, P., Mirza, M.: Disentangling factors of variation for facial expression recognition. In: European Conference on Computer Vision, pp. 808–822. Springer (2012)

    Google Scholar 

  29. Rudovic, O., Pantic, M., Patras, I.: Coupled gaussian processes for pose-invariant facial expression recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1357–1369 (2013)

    Article  Google Scholar 

  30. Rudovic, O., Patras, I., Pantic, M.: Regression-based multi-view facial expression recognition. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 4121–4124. IEEE (2010)

    Google Scholar 

  31. Sariyanidi, E., Gunes, H., Cavallaro, A.: Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1113–1133 (2015)

    Article  Google Scholar 

  32. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  33. Tang, H., Hasegawa-Johnson, M., Huang, T.: Non-frontal view facial expression recognition based on ergodic hidden markov model supervectors. In: 2010 IEEE International Conference on Multimedia and Expo (ICME), pp. 1202–1207. IEEE (2010)

    Google Scholar 

  34. Tang, Y.: Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239 (2013)

  35. Tariq, U., Yang, J., Huang, T.S.: Maximum margin GMM learning for facial expression recognition. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6. IEEE (2013)

    Google Scholar 

  36. Tariq, U., Yang, J., Huang, T.S.: Supervised super-vector encoding for facial expression recognition. Pattern Recogn. Lett. 46, 89–95 (2014)

    Article  Google Scholar 

  37. Tong, Y., Shen, Y., Gao, B., Sun, F., Chen, R., Xu, Y.: A noisy-robust approach for facial expression recognition. KSII Trans. Internet Inf. Syst. (TIIS) 11(4), 2124–2148 (2017)

    Google Scholar 

  38. Tsin, Y., Ramesh, V., Kanade, T.: Statistical calibration of ccd imaging process. In: Proceedings Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 1, pp. 480–487 (2001)

    Google Scholar 

  39. Valstar, M.F., Gunes, H., Pantic, M.: How to distinguish posed from spontaneous smiles using geometric features. In: Proceedings of the 9th International Conference on Multimodal Interfaces, pp. 38–45. ACM (2007)

    Google Scholar 

  40. Weerachai, S., Mizukawa, M.: Human behavior recognition via top-view vision for intelligent space. In: ICCAS 2010, pp. 1687–1690 (2010)

    Google Scholar 

  41. Wimmer, M., MacDonald, B.A., Jayamuni, D., Yadav, A.: Facial expression recognition for human-robot interaction–a prototype. In: International Workshop on Robot Vision, pp. 139–152. Springer (2008)

    Google Scholar 

  42. Zafeiriou, S., Papaioannou, A., Kotsia, I., Nicolaou, M., Zhao, G.: Facial affect “in-the-wild": a survey and a new database. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2016

    Google Scholar 

  43. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)

    Article  Google Scholar 

  44. Zheng, W., Tang, H., Huang, T.S.: Emotion recognition from non-frontal facial images. In: Konar, A., Charkraborty, A. (eds.) Emotion Recognition: A Pattern Analysis Approach, 1 edn., pp. 183–213 (2014)

    Google Scholar 

  45. Zheng, W., Tang, H., Lin, Z., Huang, T.S.: A novel approach to expression recognition from non-frontal face images. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1901–1908. IEEE (2009)

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by two Faculty Research Grants; FRG15-R-42 and FRG-17-R-44 from the American University of Sharjah, UAE.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, American University of Sharjah, PO Box 26666, Sharjah, UAE

    Muhannad Alkaddour & Usman Tariq

Authors
  1. Muhannad Alkaddour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Usman Tariq
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Usman Tariq .

Editor information

Editors and Affiliations

  1. Saga University, Saga, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alkaddour, M., Tariq, U. (2020). Investigating the Effect of Noise on Facial Expression Recognition. In: Arai, K., Kapoor, S. (eds) Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, vol 944. Springer, Cham. https://doi.org/10.1007/978-3-030-17798-0_55

Download citation

Publish with us

Policies and ethics

Search

Navigation