Advertisement

Recognition of Facial Expressions in VR an Experiment of Still Photos Versus Three Dimensional Computer Graphic Images

Conference paper
  • 117 Downloads
Part of the Progress in IS book series (PROIS)

Abstract

Modern techniques, such as photogrammetry, allows for capturing humans and convert them into realistic looking three-dimensional digital humans. Scanning human faces through photogrammetry and applying them into realistic virtual environments becomes more affordable and easy to use. However, it is unclear whether people differ in recognizing human expressions from 3D photogrammetry faces compared to those captured through traditional media such as photographs. In this study, we compared recognition of ten facial expressions (irritation, hot anger, sadness, despair, disgust, contempt, happiness, elated joy, panic fear, and anxiety) with two intensity levels taken from computer graphic (CG) scanned faces and photographs of two actors. The results, taken from a hundred participants aged between 18–55 years old, showed no differences in facial expression recognition between traditional photographs and computer graphic images. In addition, in line with previous research, the overall recognition of expressions was relatively low (around 50%). These results suggest that CG scanned faces, without any optimization, can already be used within VR environments without risking a loss of expression recognition. However, the results also advocate developers to invest time and money in optimizing the realism in photogrammetry scanned faces to increase the chance of recognizing the right facial expressions when communicating through human faces in virtual reality.

Keywords

Photogrammetry Photographs Expressions Recognition Computer graphics 

References

  1. Aber, J. S., Marzolff, I., & Ries, J. B. (2010). Small-format aerial photography: Principles, techniques and geoscience applications. Amsterdam: Elsevier.  https://doi.org/10.1016/B978-0-444-53260-2.10003-1.CrossRefGoogle Scholar
  2. Bänziger, T., Grandjean, D., & Scherer, K. R. (2009). Emotion recognition from expressions in face, voice, and body: The Multimodal Emotion Recognition Test (MERT). Emotion, 9(5), 691–704.  https://doi.org/10.1037/a0017088.CrossRefGoogle Scholar
  3. Cahill, L. (2006). Why sex matters for neuroscience. Nature Reviews Neuroscience, 7(6), 477–484Google Scholar
  4. Community BUFF. (2018, December 6). The 3 best developments in gaming over the last decade. Retrieved from https://medium.com/buff-game/the-3-best-developments-in-gaming-over-the-last-decade-4e874ce7b778
  5. Cowie et al. (2001) Emotion recognition in human-computer interaction.Google Scholar
  6. D’Apuzzo, N. (1998). Automated photogrammetric measurement of human faces. International Archives of the Photogrammetry, Remote SensingGoogle Scholar
  7. D’Apuzzo, N. (2002). Modeling human faces with multi-image photogrammetry. Three-Dimensional Image Capture and Applications V.  https://doi.org/10.1117/12.460168.CrossRefGoogle Scholar
  8. Dyck, M., Winbeck, M., Leiberg, S., Chen, Y., Gur, R. C., & Mathiak, K. (2008). Recognition profile of emotions in natural and virtual faces. PLoS ONE, 3, e3628.CrossRefGoogle Scholar
  9. Ekman, P., & Davidson, R. (1994). The nature of emotion: Fundamental questions (pp. 291–93). New York: Oxford University Press. ISBN 978-0195089448. Emotional processing, but not emotions, can occur unconsciously.Google Scholar
  10. Elezaj, R. (2018, November 30). The future of content: Can AI replace humans. Retrieved from https://hackernoon.com/the-future-of-content-can-ai-replace-humans-9c1ab2d21cb4
  11. Ey-Chmielewska, H., Chruściel-Nogalska, M., & Frączak, B. (2015). Photogrammetry and its potential application in medical science on the basis of selected literature. Advances in Clinical and Experimental Medicine, 24(4), 737–741.  https://doi.org/10.17219/acem/58951.CrossRefGoogle Scholar
  12. Fabri, M., Moore, D., & Hobbs, D. (2004). Mediating the expression of emotion in educational collaborative virtual environments: an experimental study. Virtual Real, 7, 66–81.CrossRefGoogle Scholar
  13. Freitas-Magalhães, A. (2011). Facial expression of emotion: From theory to application. Porto: FEELab Science Books. ISBN 978-972-99700-3-0.Google Scholar
  14. Gutiérrez-Maldonado, J., Rus-Calafell, M., & González-Conde, J. (2014). Creation of a new set of dynamic virtual reality faces for the assessment and training of facial emotion recognition ability. Virtual Reality, 18(1), 61–71Google Scholar
  15. Ho, C.-C., & Macdorman, K. F. (2010). Revisiting the uncanny valley theory: Developing and validating an alternative to the godspeed indices. Computers in Human Behavior, 26(6), 1508–1518.  https://doi.org/10.1016/j.chb.2010.05.015.CrossRefGoogle Scholar
  16. LaValle, S. M. (2017). Virtual reality. Cambridge University Press: University of llinois.Google Scholar
  17. Linder, W. (2014). Digital photogrammetry. Berlin, Germany: Springer.Google Scholar
  18. Liong, S.-T., See, J., Phan, R.C.-W., Oh, Y.-H., Ngo, A. C. L., Wong, K., & Tan, S.-W. (2016). Spontaneous subtle expression detection and recognition based on facial strain. Signal Processing: Image Communication, 47, 170–182.  https://doi.org/10.1016/j.image.2016.06.004.CrossRefGoogle Scholar
  19. Loeffler, J. (2019, February 23). Meet the world’s first female AI news anchor, Xin Xiaopeng. Retrieved from https://interestingengineering.com/meet-the-worlds-first-femaleai-news-anchor
  20. Luhmann, T., Robson, S., Kyle, S., & Boehm, J. (2013). Close-range photogrammetry and 3D imaging. Berlin: De Gruyter.CrossRefGoogle Scholar
  21. Matsumoto, D., & Hwang, H. S. (2011). Reading facial expressions of emotion. PsycEXTRA Dataset.  https://doi.org/10.1037/e574212011-002.CrossRefGoogle Scholar
  22. Mori, M. (1970). The uncanny valley. Energy, 7(4), 33–35.Google Scholar
  23. Noël, S., Dumoulin, S., & Lindgaard, G. (2009, August). Interpreting human and avatar facial expressions. In IFIP Conference on Human-Computer Interaction (pp. 98–110). Springer, Berlin, HeidelbergGoogle Scholar
  24. Prinz, J. (2004) Which emotions are basic? Emotion, Evolution, and Rationality, 69–88.  https://doi.org/10.1093/acprof:oso/9780198528975.003.0004
  25. Project VIBE. (2017, November 3). Retrieved from https://www.mind-labs.nl/news/projectvibe
  26. Qu, F., Yan, W.-J., Chen, Y.-H., Li, K., Zhang, H., & Fu, X. (2017). You should have seen the look on your face…: Self-awareness of facial expressions. Frontiers in Psychology, 8.  https://doi.org/10.3389/fpsyg.2017.00832
  27. Seyama, J. I., & Nagayama, R. S. (2007). The uncanny valley: Effect of realism on the impression of artificial human faces. Presence: Teleoperators and Virtual Environments, 16(4), 337–351.Google Scholar
  28. Slijkhuis, P. J. (2017). The uncanny valley phenomenon. Faculty of Behavioural, Management & Social Sciences.Google Scholar
  29. Statham, N. (2018). Use of Photogrammetry in video games: A historical overview. Games and Culture.  https://doi.org/10.1177/1555412018786415.CrossRefGoogle Scholar
  30. Van Gisbergen, M. S., Kovacs, M. H., Campos, F., van der Heeft, M., & Vugts, V. (2019). What we don’t know. The effect of realism in virtual reality on experience and behaviour. In M. tom Dieck, & T. Jung (Eds.), Augmented reality and virtual reality. Progress in IS (pp. 45–59). Cham: Springer.Google Scholar
  31. Van Gisbergen, M. S., Sensagir, I., & Relouw, J. (2020). How real do you see yourself in VR? The effect of user-avatar resemblance on virtual reality experiences and behaviour. In T. Jung, M. C. tom Dieck, & P. A. Rauschnabel (Eds.), Augmented reality and virtual reality: Changing realities in a dynamic world (pp. 401–409). Cham: Springer. https://link-springer-com.proxy1.dom1.nhtv.nl/book/10.1007/978-3-030-37869-1
  32. Wingenbach, T. S. H., Ashwin, C., & Brosnan, M. (2018). Sex differences in facial emotion recognition across varying expression intensity levels from videos. Plos One, 13(1).  https://doi.org/10.1371/journal.pone.0190634

Copyright information

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.Academy of Games and MediaBreda University of Applied SciencesBredaThe Netherlands

Personalised recommendations