Cinemacraft: exploring fidelity cues in collaborative virtual world interactions

Abstract

The research presented in this paper explores the contribution of avatar fidelity to social interaction in virtual environments and how sensory fusion can improve these interactions. Specifically, we vary levels of interaction fidelity to investigate how responsiveness and behavioural realism affect people’s experience of interacting with virtual humans. This is accomplished through the creation of Cinemacraft, a technology-mediated immersive platform for collaborative human–computer interaction. Cinemacraft leverages a voxel game engine similar to Minecraft to facilitate collaborative interaction in a virtual 3D world and incorporates sensory fusion to improve the fidelity of real-time collaboration. The primary hypothesis of the study is that embodied interactions result in a higher degree of presence, and that sensory fusion can improve the quality of presence and co-presence. We tested our hypothesis through a user-study of 24 participants. Based on suggestions from existing literature, we sidestep the uncanny valley effect through the use of low fidelity avatars (a la Minecraft) and identify cues that impact users ratings of presence, co-presence and successful collaboration. The findings and ensuing data in this research can be applied to produce a more compelling platform for live collaborative interactions, performances, and empathetic storytelling. This research contributes to the field of immersive, collaborative interaction by making transparent the platform, methodology, instruments and code accessible for team members with less technological expertise, as well as developers aspiring to use interactive 3D media to promote further experimentation and conceptual discussions.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

References

  1. Ahmaniemi T (2010) Gesture controlled virtual instrument with dynamic vibrotactile feedback. In: NIME, pp 485–488

  2. Albaum G (1997) The likert scale revisited. Mark Res Soc J 39(2):1–21

    Article  Google Scholar 

  3. Anderson A, Dossick CS, Iorio J, Taylor JE (2017) The impact of avatars, social norms and copresence on the collaboration effectiveness of aec virtual teams. J Inf Technol Constr (ITcon) 22(15):287–304

    Google Scholar 

  4. Animesh SB, Pinsonneault OH (2011) An odyssey into virtual worlds: exploring the impacts of technological and spatial environments on intention to purchase virtual products. Mis Q 35(3):789–810

    Article  Google Scholar 

  5. Argelaguet F, Hoyet (2016) The role of interaction in virtual embodiment: effects of the virtual hand representation. In: Virtual reality (VR), 2016 IEEE. IEEE, pp 3–10

  6. Bailey J, Bailenson JN, Won AS, Flora J, Armel KC (2012) Presence and memory: immersive virtual reality effects on cued recall. In: Proceedings of the international society for presence research annual conference, Oct, Citeseer, pp 24–26

  7. Bainbridge WS (2007) The scientific research potential of virtual worlds. Science 317(5837):472–476

    Article  Google Scholar 

  8. Barnes B, Elsi G, Kiseleva M (2016) Cinemacraft: virtual minecraft presence using operacraft. Inst Creat Arts Technol (ICAT), pp 11–32

  9. Bianchi-Berthouze N (2013) Understanding the role of body movement in player engagement. Hum Comput Interact 28(1):40–75

    Google Scholar 

  10. Bianchi-Berthouze N, Kim WW, Patel D (2007) Does body movement engage you more in digital game play? and why? In: International conference on affective computing and intelligent interaction. Springer, pp 102–113

  11. Biocca F, Harms (2001) The networked minds measure of social presence: pilot test of the factor structure and concurrent validity. In: 4th annual international workshop on presence, Philadelphia, pp 1–9

  12. Bray DA, Konsynski BR (2007) Virtual worlds: multi-disciplinary research opportunities. SIGMIS Database 38(4):17–25. https://doi.org/10.1145/1314234.1314239

    Article  Google Scholar 

  13. Buecheler C (2010) Character: the next great gaming frontier? http://www.chicagotribune.com/sns-gamereview-feature-characters-story.html. Accessed 22 July 2018

  14. Bukvic I (2012) A behind-the-scenes peek at world’s first linux-based laptop orchestra–the design of l2ork infrastructure and lessons learned. In: Linux audio conference, Stanford, California, pp 55–60

  15. Bukvic II, Cahoon C, Wyatt A (2014) Operacraft: blurring the lines between real and virtual. In: ICMC, pp 6–7

  16. Carey B, Ulas B (2016) Vr’space opera’: mimetic spectralism in an immersive starlight audification system. arXiv preprint arXiv:161103081, pp 4–5

  17. Carlson PJ, Davis GB (1998) An investigation of media selection among directors and managers: from“self” to“other” orientation. MIS Q 22(3):335–362. https://doi.org/10.2307/249669

    Article  Google Scholar 

  18. Choney S (2016) Microsoft stores offering free minecraft vr demos on oculus rift. https://blogs.microsoft.com/firehose/2016/09/29/microsoft-stores-offering-free-minecraft-vr, http://dl.acm.org/citation.cfm?id=2910632. Accessed 4 Oct 2018

  19. Collingwoode-Williams T, Gillies M (2017) The effect of lip and arm synchronization on embodiment: a pilot study. In: Virtual reality (VR), 2017 IEEE. IEEE, pp 253–254

  20. Denisova A, Cairns P (2015) First person vs. third person perspective in digital games: do player preferences affect immersion? In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, pp 145–148

  21. Duncan SC (2011) Minecraft, beyond construction and survival. Well Played J Video Games Value Mean 1(1):1–22

    Google Scholar 

  22. Fritsch T, Ritter H, Schiller J (2005) The effect of latency and network limitations on mmorpgs: a field study of everquest2. In: Proceedings of 4th ACM SIGCOMM workshop on Network and system support for games. ACM, pp 1–9

  23. FUDI (2017) Fudi. https://en.wikipedia.org/wiki/FUDI. Accessed 1 June 2018

  24. Garrelts N (2014) Understanding Minecraft: essays on play, community and possibilities. McFarland, Jefferson

    Google Scholar 

  25. Heidicker P, Langbehn E, Steinicke F (2017) Influence of avatar appearance on presence in social vr. In: IEEE symposium on 3D user interfaces (3DUI), 2017. IEEE, pp 233–234

  26. Hirose M, Schmalstieg D, Wingrave CA, Nishimura K (2009) Higher levels of immersion improve procedure memorization performance. In: Proceedings of the 15th joint virtual reality eurographics conference on virtual environments, pp 121–128

  27. Huh Y, Duarte GT, El Zarki M (2018) Minebike: Exergaming with minecraft. In: 2018 IEEE 20th International conference on e-health networking, applications and services (Healthcom). IEEE, pp 1–6

  28. Institute for Creativity A, Technology (2016) Icat day 2016. https://www.icat.vt.edu/content/icat-day-2016

  29. Kastelein R (2013) The rise of machinima, the artform. http://insights.wired.com/profiles/blogs/the-rise-of-machinima-the-artform. Accessed 12 May 2018

  30. Kätsyri J, de Gelder B (2018) Uncanny slope instead of an uncanny valley: testing the uncanny valley hypothesis in painted, computer-rendered, and human faces, pp 4–9

  31. Kinect M (2017) Kinect kinect. http://www.xbox.com/en-US/xbox-one/accessories/kinect. Accessed 2 Jan 2019

  32. Kokkinara E, Slater M (2015) The effects of visuomotor calibration to the perceived space and body, through embodiment in immersive virtual reality. ACM Trans Appl Percept (TAP) 13(1):3

    Google Scholar 

  33. Lay S, Brace N, Pike G, Pollick F (2016) Circling around the uncanny valley: design principles for research into the relation between human likeness and eeriness. i-Perception 7(6):2–6. https://doi.org/10.1177/2041669516681309

    Article  Google Scholar 

  34. Lecuyer A (2017) Playing with senses in vr: alternate perceptions combining vision and touch. IEEE Comput Gr Appl 37(1):20–26. https://doi.org/10.1109/MCG.2017.14

    Article  Google Scholar 

  35. Lombard M, Ditton T (1997) At the heart of it all: the concept of presence. J Comput Mediat Commun 3(2):0–0

    Article  Google Scholar 

  36. Maister L, Slater M (2015) Changing bodies changes minds: owning another body affects social cognition. Trends Cogn Sci 19(1):6–12

    Article  Google Scholar 

  37. Makled E, Abdelrahman (2018) I like to move it: investigating the effect of head and body movement of avatars in vr on user’s perception. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems, ACM, New York, CHI EA ’18, pp LBW099:1–LBW099:6. https://doi.org/10.1145/3170427.3188573

  38. Mäntymäki M, Riemer K (2011) How social are social virtual worlds? an investigation of hedonic, utilitarian, social and normative usage drivers. In: PACIS, p 126

  39. Microsoft (2016) Impact minecraft education edition is making in classrooms. https://education.minecraft.net/. Accessed 2 August 2018

  40. Microsoft (2017a) Kinect 360. https://support.xbox.com/en-US/xbox-on-windows/accessories/kinect-for-windows-info. Accessed 3 July 2018

  41. Microsoft (2017b) Windows presentation foundation (wpf) is a next-generation presentation system for building windows client applications. https://msdn.microsoft.com/en-us/library/aa970268(v=vs.100).aspx. Accessed 3 July 2018

  42. Minetest (2016) Meet minetest. http://www.minetest.net/. Accessed 6 August

  43. Mori M, MacDorman KF, Kageki N (2012) The uncanny valley [from the field]. IEEE Robot Autom Mag 19(2):98–100

    Article  Google Scholar 

  44. Mousas C, Anastasiou D, Spantidi O (2018) The effects of appearance and motion of virtual characters on emotional reactivity. Comput Hum Behav 86:99–108

    Article  Google Scholar 

  45. Nah FFH, Eschenbrenner B, DeWester D (2011) Enhancing brand equity through flow and telepresence: a comparison of 2d and 3d virtual worlds. MIs Q 35(3):731–747

    Article  Google Scholar 

  46. Narang S, Best A, Manocha D (2018) Simulating movement interactions between avatars & agents in virtual worlds using human motion constraints. In: 2018 IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, pp 9–16

  47. Nash EB, Edwards GW, Thompson JA, Barfield W (2000) A review of presence and performance in virtual environments. Int J Hum Comput Interact 12(1):1–41

    Article  Google Scholar 

  48. Nowak KL, Biocca F (2003) The effect of the agency and anthropomorphism on users’ sense of telepresence, copresence, and social presence in virtual environments. Presence Teleoper Virtual Environ 12(5):481–494

    Article  Google Scholar 

  49. Park N, Lee KM, Jin SAA, Kang S (2010) Effects of pre-game stories on feelings of presence and evaluation of computer games. Int J Hum Comput Stud 68(11):822–833

    Article  Google Scholar 

  50. Parker JR (2008) Buttons, simplicity, and natural interfaces. Loading 2(2)

  51. Polyak E (2012) Virtual impersonation using interactive glove puppets. In: SIGGRAPH Asia 2012 posters, ACM, New York, SA ’12, pp 31:1–31:1. https://doi.org/10.1145/2407156.2407191

  52. Polys NF, Knapp B, Bukvic I (2015) Fusality: an open framework for cross-platform mirror world installations. In: Proceedings of the 20th international conference on 3D web technology. ACM, pp 171–179

  53. Ragan ED, Sowndararajan A, Kopper R, Bowman DA (2010) The effects of higher levels of immersion on procedure memorization performance and implications for educational virtual environments. Presence Teleoper Virtual Environ 19(6):527–543

    Article  Google Scholar 

  54. Ratcliffe J (2014) Hand motion-controlled audio mixing interface. Proc New Interfaces Musical Expr (NIME) 2014:136–139

    Google Scholar 

  55. Roth D, Lugrin JL, Galakhov D, Hofmann (2016) Avatar realism and social interaction quality in virtual reality. In: Virtual reality (VR), 2016 IEEE. IEEE, pp 277–278

  56. Sacau A, Laarni J (2008) Influence of individual factors on presence. Comput Hum Behav 24(5):2255–2273

    Article  Google Scholar 

  57. Saunders C, Rutkowski AF, van Genuchten M, Vogel D, Orrego JM (2011) Virtual space and place: theory and test. MIS Q 35(4):1079–1098

    Article  Google Scholar 

  58. Schroeder R (2012) The social life of avatars: presence and interaction in shared virtual environments. Springer, Berlin

    Google Scholar 

  59. Schroeder R, Steed (2001) Collaborating in networked immersive spaces: as good as being there together? Comput Gr 25(5):781–788

    Article  Google Scholar 

  60. Schultze U (2011) The avatar as sociomaterial entanglement: a performative perspective on identity, agency and world-making in virtual worlds. In: Proceedings of the international conference on information systems, ICIS 2011, Shanghai, China

  61. Schultze, Orlikowski (2010) Virtual worlds: a performative perspective on globally distributed, immersive work. Inf Syst Res 21(4):810–821

    Article  Google Scholar 

  62. Seymour M, Riemer K, Kay J (2017) Interactive realistic digital avatars–revisiting the uncanny valley. In: Hawaii international conference on system sciences, HICSS-50, Honolulu

  63. Seymour M, Riemer K, Kay J (2018) Actors, avatars and agents: potentials and implications of natural face technology for the creation of realistic visual presence. J Assoc Inf Syst 19(10):953–981

    Google Scholar 

  64. Shin D (2018) Empathy and embodied experience in virtual environment: to what extent can virtual reality stimulate empathy and embodied experience? Comput Hum Behav 78:64–73

    Article  Google Scholar 

  65. Sia CL, Tan BC, Wei KK (2002) Group polarization and computer-mediated communication: effects of communication cues, social presence, and anonymity. Inf Syst Res 13(1):70–90

    Article  Google Scholar 

  66. Sims K (1994) Evolving virtual creatures. In: Proceedings of the 21st annual conference on computer graphics and interactive techniques. ACM, pp 15–22

  67. Slater M (1999) Measuring presence: a response to the witmer and singer presence questionnaire. Presence Teleoper Virtual Environ 8(5):560–565

    Article  Google Scholar 

  68. Slater M, Sadagic (2000) Small-group behavior in a virtual and real environment: a comparative study. Presence Teleoper Virtual Environ 9(1):37–51

    Article  Google Scholar 

  69. Slater M, Spanlang B, Sanchez-Vives MV, Blanke O (2010) First person experience of body transfer in virtual reality. PloS one 5(5):e10564

    Article  Google Scholar 

  70. Spante M, Heldal (2003) Is there a tradeoff between presence and copresence. In: Proceedings of presence 2003: 6th international workshop on presence

  71. Swainston A, Jeanneret N, et al (2015) Wot opera: a joyful, creative and immersive experience. In: Music: educating for life. ASME XXth national conference proceedings, australian society for music education, p 99

  72. Tech V (2016) South by southwest 2016. http://www.vt.edu/sxsw.html. Accessed 7 May 2018

  73. Tech V (2017) Science museum of western virginia. http://www.icat.vt.edu/smwv. Accessed 7 May 2018

  74. Thon J-N (2008) Immersion revisited: on the value of a contested concept. In: Leino O, Wirman H, Fernandez A (eds) Extending experiences: structure, analysis and design of computer game player experience. Lapland University Press, Lapland, pp 29–43

    Google Scholar 

  75. Tinwell A, Grimshaw M, Nabi DA, Williams A (2011) Facial expression of emotion and perception of the uncanny valley in virtual characters. Comput Hum Behav 27(2):741–749

    Article  Google Scholar 

  76. Viniconis N (2011) Minecraft + kinect : building worlds! http://www.orderofevents.com/MineCraft/KinectInfo.htm. Accessed 14 Oct 2018

  77. Vivecraft (2016) Vivecraft. http://www.vivecraft.org/. Accessed 24 Oct 2018

  78. Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence Teleoper Virtual Environ 7(3):225–240

    Article  Google Scholar 

  79. Wright M, Freed A, Momeni A (2003) Opensound control: state of the art 2003. In: Proceedings of the 2003 conference on new interfaces for musical expression. National University of Singapore, pp 153–160

  80. Yoo Y, Alavi M (2001) Media and group cohesion: relative influences on social presence, task participation, and group consensus. MIS Q 25(3):371–390

    Article  Google Scholar 

  81. Zhu L, Benbasat I, Jiang Z (2010) Let’s shop online together: an empirical investigation of collaborative online shopping support. Inf Syst Res 21(4):872–891

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Siddharth Narayanan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1 : Questionnaires

All 24 participants recorded their responses to each experimental interaction exercise using a presence questionnaire based on Slater’s (Slater 1999) presence questionnaire, a co-presence questionnaire based on the Networked Minds (Biocca and Harms 2001) and Nowak’s (Nowak and Biocca 2003) co-presence questionnaires and finally, the immersive tendencies questionnaire (Witmer and Singer 1998).

1.1 Presence questionnaire

Please rate your sense of being in the virtual environment, on a scale of 1 to 7, where 7 represents your normal experience of being in a place. How much were you able to control events? How responsive was the environment to actions that you initiated (or performed)? How natural did your interactions with the environment seem? How completely were all of your senses engaged? How much did the visual aspects of the environment involve you? How much did the auditory aspects of the environment involve you? How natural was the mechanism which controlled movement through the environment? How aware were you of events occurring in the real world around you? How aware were you of your display and control devices? How compelling was your sense of objects moving through space? How inconsistent or disconnected was the information coming from your various senses? How much did your experiences in the virtual environment seem consistent with your real-world experiences ? Were you able to anticipate what would happen next in response to the actions that you performed ? How completely were you able to actively survey or search the environment using vision? How well could you identify sounds? How well could you localize sounds? How compelling was your sense of moving around inside the virtual environment? How closely were you able to examine objects? How well could you examine objects from multiple viewpoints? How well could you move or manipulate objects in the virtual environment? To what degree did you feel confused or disoriented at the beginning of breaks or at the end of the experimental session? How involved were you in the virtual environment experience? How distracting was the control mechanism? How much delay did you experience between your actions and expected outcomes? How quickly did you adjust to the virtual environment experience? How proficient in moving and interacting with the virtual environment did you feel at the end of the experience? How much did the visual display quality interfere or distract you from performing assigned interaction exercises or required activities? How much did the control devices interfere with the performance of assigned interaction exercises or with other activities? How well could you concentrate on the assigned interaction exercises or required activities rather than on the mechanisms used to perform those interaction exercises or activities? Did you learn new techniques that enabled you to improve your performance? Were you involved in the experimental interaction exercise to the extent that you lost track of time? To what extent were there times during the experience when the virtual environment was the reality for you? When you think back to the experience, do you think of the virtual environment more as images that you saw or more as somewhere that you visited?

1.2 Co-presence questionnaire

I often felt as if I was all alone. I think the other individual often felt alone. I hardly noticed another individual. The other individual didn’t notice me in the room. I was often aware of others in the environment. Others were often aware of me in the room. I think the other individual often felt alone. I often felt as if I was all alone. I sometimes pretended to pay attention to the other individual. The other individual paid close attention to me I paid close attention to the other individual. My partner was easily distracted when other things were going on around us. I was easily distracted when other things were going on around me When I was happy, the other was happy. When the other was happy, I was happy. My interaction partner seemed to find our interaction stimulating. My interaction partner communicated coldness rather than warmth. My interaction partner seemed detached during our interaction. My interaction partner was unwilling to share personal information with me. My interaction partner created a sense of closeness between us. My interaction partner was interested in talking to me. I wanted to maintain a sense of distance between us. I was interested in talking to my interaction partner I perceive that I am in the presence of another person in the room with me. I feel that the person is watching me and is aware of my presence. The thought that the person is not a real person crossed my mind often. The person appears to be sentient (conscious and alive) to me. I perceive the person as being only a computerized image, not as a real person.

1.3 Immersive tendencies questionnaire

Do you easily become deeply involved in movies or tv dramas? Do you ever become so involved in a television program or book that people have problems getting your attention? How mentally alert do you feel at the present time? Do you ever become so involved in a movie that you are not aware of things happening around you? How frequently do you find yourself closely identifying with the characters in a story line? Do you ever become so involved in a video game that it is as if you are inside the game rather than moving a joystick and watching the screen? How physically fit do you feel today? How good are you at blocking out external distractions when you are involved in something? When watching sports, do you ever become so involved in the game that you react as if you were one of the players? Do you ever become so involved in a daydream that you are not aware of things happening around you? Do you ever have dreams that are so real that you feel disoriented when you awake? When playing sports, do you become so involved in the game that you lose track of time? How well do you concentrate on enjoyable activities? How often do you play arcade or video games? (OFTEN should be taken to mean every day or every two days, on average.)

Appendix 2: List of body expressions

Participants were given the interaction exercise sheets for each experimental interaction exercise and were expected to enact out and guess the body expressions within a fixed amount of time for each experimental interaction exercise, without stating or explicitly alluding to the caption on the list. The expectation was that the number of body expressions successfully guessed and enacted from their designated lists, would increase in interaction exercises with higher interaction fidelity.

figurea

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Narayanan, S., Polys, N. & Bukvic, I.I. Cinemacraft: exploring fidelity cues in collaborative virtual world interactions. Virtual Reality 24, 53–73 (2020). https://doi.org/10.1007/s10055-019-00382-0

Download citation

Keywords

  • Information systems
  • Software engineering
  • Virtual worlds software
  • Computing methodologies
  • Motion capture