Skip to main content
SpringerLink
Log in
Book cover

Everyday Virtual and Augmented Reality pp 147–179Cite as

Networked Virtual Reality and Enhanced Sensing for Remote Classes and Presentations

  • Chapter
  • First Online:

  • 790 Accesses

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

Networked virtual reality is gaining recognition as a way to conduct remote classes or meetings when in-person meetings are difficult or risky. This chapter summarizes our ongoing work to develop and assess VR techniques for remote education. We first present two case studies of remote teaching in VR: a classroom-embedded virtual field trip of an energy center guided by a remote teacher, and a remote university class conducted for several weeks in a social VR tool. We then summarize our ongoing research to enhance remote educational VR interfaces using enhanced sensing, for example, to visualize or detect student attention based on eye-tracked gaze. Finally, we identify several practical considerations that will need to be addressed for the long-term success of educational deployments of virtual reality. This can help educators, researchers, and VR developers make informed decisions about how to best use VR technology for designing and deploying educational VR in everyday contexts such as schools and homes.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Agrawal P, Sahana H, De’ R (2017) Digital distraction. In: Proceedings of the 10th international conference on theory and practice of electronic governance, pp 191–194

    Google Scholar 

  • Ahn SJG, Levy L, Eden A, Won AS, MacIntyre B, Johnsen K (2021) IEEEVR2020: exploring the first steps toward standalone virtual conferences. Front Virtual Reality 2:28

    Article  Google Scholar 

  • Asish SM, Hossain E, Kulshreshth AK, Borst CW (2021) Deep learning on eye gaze data to classify student distraction level in an educational vr environment. In: ICAT-EGVE 2021—international conference on artificial reality and telexistence and Eurographics symposium on virtual environments

    Google Scholar 

  • Bailenson JN (2021) Nonverbal overload: a theoretical argument for the causes of zoom fatigue. Technol Mind Behav 2(1)

    Google Scholar 

  • Baumgartner T, Speck D, Wettstein D, Masnari O, Beeli G, Jäncke L (2008) Feeling present in arousing virtual reality worlds: prefrontal brain regions differentially orchestrate presence experience in adults and children. Front Human Neurosci 2:8

    Article  Google Scholar 

  • Bevilacqua F, Engström H, Backlund P (2019) Game-calibrated and user-tailored remote detection of stress and boredom in games. Sensors 19(13):2877

    Article  Google Scholar 

  • Biocca F, Tang A, Owen C, Xiao F (2006) Attention funnel: omnidirectional 3d cursor for mobile augmented reality platforms. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1115–1122

    Google Scholar 

  • Borst CW, Kinsland L, Gary (2005) Examples from the chicxulub impact crater. Visualization and interpretation of 3-d geological and geophysical data in heterogeneous virtual reality displays. Trans Gulf Coast Assoc Geol Societies 55:284–293

    Google Scholar 

  • Borst CW, Lipari NG, Woodworth JW (2018) Teacher-guided educational VR: assessment of live and prerecorded teachers guiding virtual field trips. In: Proceedings of the 2018 IEEE conference on Virtual Reality and 3D user interfaces (VR)

    Google Scholar 

  • Borst CW, Ritter KA, Chambers TL (2016) Virtual energy center for teaching alternative energy technologies. In: 2016 IEEE Virtual Reality (VR)

    Google Scholar 

  • Bousefsaf F, Maaoui C, Pruski A (2013) Remote assessment of the heart rate variability to detect mental stress. In: 7th international conference on pervasive computing technologies for healthcare and workshops, pp 348–351

    Google Scholar 

  • Bower GH, Sahgal A, Routh DA (1983) Affect and cognition. Philos Trans R Soc; Ser B 302(1110):387–402

    Google Scholar 

  • Bradley MM, Lang PJ (1999) International affective digitized sounds (iads): stimuli, instruction manual and affective ratings (tech. rep. no. b-2). The Center for Research in Psychophysiology, University of Florida, Gainesville, FL

    Google Scholar 

  • Broussard DM, Rahman Y, Kulshreshth AK, Borst CW (2021a) An interface for enhanced teacher awareness of student actions and attention in a VR classroom. In: 2021 IEEE conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp 284–290

    Google Scholar 

  • Broussard DM, Rahman Y, Kulshreshth AK, Borst CW (2021b) Visual indicators for monitoring students in a VR class. In: 2021 IEEE conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp 502–503

    Google Scholar 

  • Cacioppo J, Petty R, Losch M, Kim H (1986) Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. J Personality Soc Psychol 50:260–268

    Article  Google Scholar 

  • Chambers T, Raush J, Russo B (2014) Installation and operation of parabolic trough organic Rankine cycle solar thermal power plant in South Louisiana. Energy Proc 49:1107–1116

    Article  Google Scholar 

  • Chen Y-T, Hsu C-H, Chung C-H, Wang Y-S, BabuSV (2019) ivrnote: design, creation and evaluation of an interactive note-taking interface for study and reflection in VR learning environments. In: 2019 IEEE conference on Virtual Reality and 3D User Interfaces (VR). IEEE, pp 172–180

    Google Scholar 

  • Coan JA, Allen JJ (2007) Handbook of emotion elicitation and assessment. Oxford University Press

    Google Scholar 

  • David P, Kim J-H, Brickman JS, Ran W, Curtis CM (2015) Mobile phone distraction while studying. New Media Soc 17(10):1661–1679

    Article  Google Scholar 

  • David-John B, Hosfelt D, Butler K, Jain E (2021) A privacy-preserving approach to streaming eye-tracking data. IEEE Trans Visualization Comput Graph 27(5):2555–2565

    Article  Google Scholar 

  • Dill K, Pursel ER, Garrity P, Fragomeni G, Quantico V (2012) Design patterns for the configuration of utility-based AI. In: Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), number 12146 in I/ITSEC, pp 1–12

    Google Scholar 

  • Dillman KR, Mok TTH, Tang A, Oehlberg L, Mitchell A (2018) A visual interaction cue framework from video game environments for augmented reality. In: Proceedings of the 2018 CHI conference on human factors in computing systems, pp 1–12

    Google Scholar 

  • D’Mello S, Olney A, Williams C, Hays P (2012) Gaze tutor: a gaze-reactive intelligent tutoring system. Int J Human-Comput Stud 70(5):377–398

    Article  Google Scholar 

  • Ekong S, Borst CW, Woodworth J, Chambers TL (2016) Teacher-student VR telepresence with networked depth camera mesh and heterogeneous displays. International symposium on visual computing. Springer, Heidelberg, pp 246–258

    Google Scholar 

  • Engzell P, Frey A, Verhagen MD (2021) Learning loss due to school closures during the covid-19 pandemic. Proc Natl Acad Sci 118(17)

    Google Scholar 

  • Falk JH, Dierking LD (1997) School field trips: assessing their long-term impact. Curator: Museum J 40(3):211–218

    Google Scholar 

  • Fowler A, Nesbitt K, Canossa A (2019) Identifying cognitive load in a computer game: an exploratory study of young children. In: 2019 IEEE Conference on Games (CoG), pp 1–6

    Google Scholar 

  • Gnacek M, Broulidakis J, Mavridou I, Fatoorechi M, Seiss E, Kostoulas T, Balaguer-Ballester E, Kiprijanovska I, Rosten C, Nduka C (2022) emteqpro—fully integrated biometric sensing array for non-invasive biomedical research in virtual reality. Front Virtual Reality 3

    Google Scholar 

  • Grant MM, Cheon J (2007) The value of using synchronous conferencing for instruction and students. Journal of Interactive Online Learning 6(3):211–226

    Google Scholar 

  • Grubert J, Witzani L, Ofek E, Pahud M, Kranz M, Kristensson PO (2018) Text entry in immersive head-mounted display-based virtual reality using standard keyboards. In: 2018 IEEE conference on Virtual Reality and 3D User Interfaces (VR). IEEE, pp 159–166

    Google Scholar 

  • Huston P, McFarlane B (2016) Health benefits of tai chi: What is the evidence? Can Family Physician 62(11):881–890

    Google Scholar 

  • Robertson A (2018) How much vr user data is oculus giving to facebook? https://www.theverge.com/2018/4/9/17206650/oculus-facebook-vr-user-data-mining-privacy-policy-advertising

  • Jennings M, Bronack SC (2001) The use of desktop video conferencing as a medium for collaboration between beginning instructional designers and intern teachers. Int J Educ Telecommun 7(2):91–107

    Google Scholar 

  • Kern F, Kullmann P, Ganal E, Korwisi K, Stingl R, Niebling F, Latoschik ME (2021) Off-the-shelf stylus: using xr devices for handwriting and sketching on physically aligned virtual surfaces. Front Virtual Reality 2:69

    Article  Google Scholar 

  • Khokhar A, Borst CW (2022) Modifying pedagogical agent spatial guidance sequences to respond to eye-tracked student gaze in VR. In: ACM Symposium on Spatial User Interaction (SUI 2022), Article No. 15

    Google Scholar 

  • Khokhar A, Yoshimura A, Borst CW (2019) Pedagogical agent responsive to eye tracking in educational VR. In: 2019 IEEE conference on Virtual Reality and 3D User Interfaces (VR). IEEE, pp 1018–1019

    Google Scholar 

  • Khokhar A, Yoshimura A, Borst CW (2020) Modified playback of avatar clip sequences based on student attention in educational VR. In: 2020 IEEE conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, pp 850–851

    Google Scholar 

  • Lang PJ, Bradley MM, Cuthbert BN et al (1997) International affective picture system (IAPS): technical manual and affective ratings. NIMH Center Stud Emotion Attention 1(39–58):3

    Google Scholar 

  • Lascio ED, Gashi S, Santini S (2018) Unobtrusive assessment of students’ emotional engagement during lectures using electrodermal activity sensors. In: Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, vol 2. issue 3, pp 1–21

    Google Scholar 

  • Le DA, Maclntyre B, Outlaw J (2020) Enhancing the experience of virtual conferences in social virtual environments. In: 2020 IEEE conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, pp 485–494

    Google Scholar 

  • Liao D, Shu L, Liang G, Li Y, Zhang Y, Zhang W, Xu X (2019) Design and evaluation of affective virtual reality system based on multimodal physiological signals and self-assessment manikin. IEEE J Electromagnetics RF Microwaves Med Biol 4(3):216–224

    Article  Google Scholar 

  • Madary M, Metzinger TK (2016) Real virtuality: a code of ethical conduct. Recommendations for good scientific practice and the consumers of VR-technology. Front Robot AI 3:3

    Google Scholar 

  • Marín-Morales J, Higuera-Trujillo JL, Greco A, Guixeres J, Llinares C, Scilingo EP, Alcañiz M, Valenza G (2018) Affective computing in virtual reality: emotion recognition from brain and heartbeat dynamics using wearable sensors. Sci Rep 8(1):1–15

    Article  Google Scholar 

  • Mauss IB, McCarter L, Levenson RW, Wilhelm FH, Gross JJ (2005) The tie that binds? Coherence among emotion experience, behavior, and physiology. Emotion 5(2):175–190

    Article  Google Scholar 

  • McDuff D, Gontarek S, Picard R (2014) Remote measurement of cognitive stress via heart rate variability. In: 36th annual international conference of the IEEE engineering in medicine and biology society, pp 2957–2960

    Google Scholar 

  • McDuff DJ, Hernandez J, Gontarek S, Picard RW (2016) Cogcam: contact-free measurement of cognitive stress during computer tasks with a digital camera. In: Proceedings of the 2016 CHI conference on human factors in computing systems. Association for Computing Machinery, New York, USA, pp 4000–4004

    Google Scholar 

  • Meyer OA, Omdahl MK, Makransky G (Oct2019) Investigating the effect of pre-training when learning through immersive virtual reality and video: a media and methods experiment. Comput Educ 140:103603

    Google Scholar 

  • Miehlbradt J, Cuturi LF, Zanchi S, Gori M, Micera S (2021) Immersive virtual reality interferes with default head-trunk coordination strategies in young children. Sci Rep 11(1):1–13

    Article  Google Scholar 

  • Nguyen TTh, Duval T (2014) A survey of communication and awareness in collaborative virtual environments. In: 2014 international workshop on collaborative virtual environments (3DCVE). IEEE, pp 1–8

    Google Scholar 

  • Nourbakhsh N, Wang Y, Chen F (2013) GSR and blink features for cognitive load classification. In: Kotzé P, Marsden G, Lindgaard G, Wesson J, Winckler M (eds) Human-computer interaction-INTERACT 2013. Springer, Heidelberg, pp 159–166

    Chapter  Google Scholar 

  • Peper E, Wilson V, Martin M, Rosegard E, Harvey R (2021) Avoid zoom fatigue, be present and learn. NeuroRegulation 8(1):47–47

    Article  Google Scholar 

  • Pham D-M, Stuerzlinger W (2019) Hawkey: efficient and versatile text entry for virtual reality. In: 25th ACM symposium on virtual reality software and technology, pp 1–11

    Google Scholar 

  • Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I (2020) A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput Educ 147:103778

    Article  Google Scholar 

  • Rahman Y, Asish SM, Fisher NP, Bruce EC, Kulshreshth AK, Borst CW (2020) Exploring eye gaze visualization techniques for identifying distracted students in educational VR. In: 2020 IEEE conference on Virtual Reality and 3D User Interfaces (VR). IEEE, pp 868–877

    Google Scholar 

  • Ramlogan S, Raman V, Sweet J (2014) A comparison of two forms of teaching instruction: video vs. live lecture for education in clinical periodontology. Eur J Dental Educ 18(1):31–38

    Google Scholar 

  • Renner P, Pfeiffer T (2017) Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems. In: 2017 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, pp 186–194

    Google Scholar 

  • Ritter III K (2016) Virtual solar energy center: a case study of the use of advanced visualization techniques for the comprehension of complex engineering products and processes. PhD thesis, University of Louisiana at Lafayette

    Google Scholar 

  • Rivu R, Jiang R, Mäkelä V, Hassib M, Alt F (2021) Emotion elicitation techniques in virtual reality. IFIP conference on human-computer interaction. Springer, Heidelberg, pp 93–114

    Google Scholar 

  • Russell J (1980) A circumplex model of affect. J Personality Soc Psychol 39:1161–1178

    Article  Google Scholar 

  • Russell JA (1978) Evidence of convergent validity on the dimensions of affect. J Personality Soc Psychol 36(10):1152

    Article  Google Scholar 

  • Segovia KY, Bailenson JN (2009) Virtually true: children’s acquisition of false memories in virtual reality. Media Psychol 12(4):371–393

    Article  Google Scholar 

  • Steinicke F, Meinecke A, Lehmann-Willenbrock N (2020) A first pilot study to compare virtual group meetings using video conferences and (immersive) virtual reality. In: ACM symposium on Spatial User Interaction (SUI) 2020

    Google Scholar 

  • Villanueva I, Campbell BD, Raikes AC, Jones SH, Putney LG (2018) A multimodal exploration of engineering students’ emotions and electrodermal activity in design activities. J Eng Educ 107(3):414–441

    Article  Google Scholar 

  • Wang H, Chignell M, Ishizuka M (2006) Empathic tutoring software agents using real-time eye tracking. In: Proceedings of the 2006 symposium on eye tracking research & applications, pp 73–78

    Google Scholar 

  • Woodworth JW, Broussard D, Borst CW (2020) Designing tools to improve collaborative interaction in a VR environment for teaching geosciences interpretation. In: Mensch und Computer 2020-Workshopband

    Google Scholar 

  • Woodworth JW, Borst CW (2017) Design of a practical tv interface for teacher-guided VR field trips. In: 2017 IEEE 3rd Workshop on Everyday Virtual Reality (WEVR). IEEE, pp 1–6

    Google Scholar 

  • Woodworth JW, Borst CW (2021) Designing immersive affective tasks for emotion elicitation in virtual reality. In: 2nd momentary emotion elicitation and capture workshop

    Google Scholar 

  • Woodworth JW, Broussard D, Borst CW (2022) Redirecting desktop interface input to animate cross-reality avatars. In: 2022 IEEE conference on Virtual Reality and 3D User Interfaces (VR), pp 843–851

    Google Scholar 

  • Woodworth RS (1899) Accuracy of voluntary movement. Psychol Rev: Monograph Suppl 3(3):i

    Google Scholar 

  • Yao P, Lympouridis V, Zyda M (2021) Virtual equipment system: face mask and voodoo doll for user privacy and self-expression options in virtual reality. In: 2021 IEEE conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, pp 747–748

    Google Scholar 

  • Yoshimura A, Borst CW (2020a) Evaluation and comparison of desktop viewing and headset viewing of remote lectures in VR with mozilla hubs. In: ICAT-EGVE 2020—international conference on artificial reality and telexistence and Eurographics symposium on virtual environments

    Google Scholar 

  • Yoshimura A, Borst CW (2020b) Remote instruction in virtual reality: a study of students attending class remotely from home with VR headsets. In: Hansen C, Nürnberger A, Preim B (eds) Mensch und Computer 2020 - Workshopband. Bonn, Gesellschaft für Informatik e.V

    Google Scholar 

  • Yoshimura A, Borst CW (2021) A study of class meetings in VR: student experiences of attending lectures and of giving a project presentation. Front Virtual Reality 2:34

    Article  Google Scholar 

  • Yoshimura A, Khokhar A, Borst CW (2019a) Eye-gaze-triggered visual cues to restore attention in educational VR. In: 2019 IEEE conference on Virtual Reality and 3D User Interfaces (VR), IEEE

    Google Scholar 

  • Yoshimura A, Khokhar A, Borst CW (2019b) Visual cues to restore student attention based on eye gaze drift, and application to an offshore training system. In: Symposium on spatial user interaction, pp 1–2

    Google Scholar 

  • Youngblut C (1998) Educational uses of virtual reality technology. Technical report, Institute for defense analysis, Alexendria VA

    Google Scholar 

  • Yu K, Gorbachev G, Eck U, Pankratz F, Navab N, Roth D (2021) Avatars for teleconsultation: effects of avatar embodiment techniques on user perception in 3d asymmetric telepresence. IEEE Trans Visualization Comput Graph 27(11):4129–4139

    Article  Google Scholar 

  • Zhang Y, Pfeuffer K, Chong MK, Alexander J, Bulling A, Gellersen H (2017) Look together: using gaze for assisting co-located collaborative search. Personal Ubiquitous Comput 21(1):173–186

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the co-authors of cited work presented in this chapter, who have also provided pictures, tables, and other information: David Broussard, Jason Woodworth, Andrew Yoshimura, Adil Khokhar, Yitoshee Rahman, Ekram Hossain, Sarker Asish, and Ethan Bruce. We are also grateful to collaborators of our earlier related work, and to school and community personnel who supported deployment of Kvasir-VR to Lafayette classrooms and to the Chattanooga Public Library. This material is based upon work supported by the National Science Foundation under Grants No. 1451833 and 1815976, grants from the Mozilla Gigabit Community Fund, US Ignite Application Development Awards, and by the Louisiana Board of Regents under contracts No. LEQSF(2015–16)-ENH-TR-30 and LEQSF(2019-20)-ENH-DE-22.

Author information

Authors and Affiliations

  1. University of Louisiana at Lafayette, Lafayette, USA

    Christoph W. Borst & Arun K. Kulshreshth

Authors
  1. Christoph W. Borst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arun K. Kulshreshth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph W. Borst .

Editor information

Editors and Affiliations

  1. KU Leuven, Leuven, Belgium

    Adalberto Simeone

  2. Human-Computer Interaction Group, University of Trier, Trier, Germany

    Benjamin Weyers

  3. Liverpool Business School, Liverpool John Moores University, Liverpool, UK

    Svetlana Bialkova

  4. HIT Lab NZ, University of Canterbury, Christchurch, New Zealand

    Robert W. Lindeman

Appendix

Appendix

Preliminary results from two ongoing studies are presented here, in support of Sects. 3.1 (Attention Restoration Cues) and 3.3 (Gaze-responsive Presentation).

Figure 17 summarizes performance of 9 attention restoration cues and a no-cue baseline. 65 subjects in a virtual oil rig looked at numbered barrels in sequential order. Distraction spheres appeared in a randomized manner, requiring the subject to glance at the spheres and then return to the barrel task. Visual restoration cues appeared during the distraction. Two conditions, wide and narrow, had the barrels spaced out by different distances. The figure shows the time taken for subjects to look back at the barrel from which they were distracted. A notable result, besides showing the relative performance of cues, is that it differs substantially from prior studies, which focused on guidance to new targets rather than attention restoration. A likely cause is the subjects’ memory of the prior attention target. Statistical analysis, and analysis of other metrics, is ongoing.

Fig. 17
figure 17

Time taken for attention restoration with various cues, for wide and narrow target layouts. The median is plotted for 65 subjects. The error bars are computed by bootstrapping and show the range containing 68 percent of 5000 median estimates

Full size image
Fig. 18
figure 18

Ratings of three pedagogical agent behaviors during distraction

Full size image

Figure 18 summarizes ratings given by 37 subjects who viewed the three possible pedagogical agent behaviors during pre-recorded student distractions (Khokhar and Borst 2022). The subjects used a VR headset to watch the student and agent, with an “over-the-shoulder” view behind the student. Three different distraction events were viewed, each with three possible agent behaviors. For each of the resulting 9 combinations, subjects ranked behavior appropriateness, naturalness, and strangeness. Per subject, ratings were averaged across the three events, giving one score per behavior per subject. Preliminary analysis results were that ratings of behavior appropriateness were significantly different between all three behavior pairs. Also, the respond behavior differed significantly from continue and pause in ratings of naturalness and strangeness. Other experiment phases considered reviews of longer presentation sequences and of first-person experiences with the subjects as students. There was some evidence of a respond behavior benefit in each phase.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Borst, C.W., Kulshreshth, A.K. (2023). Networked Virtual Reality and Enhanced Sensing for Remote Classes and Presentations. In: Simeone, A., Weyers, B., Bialkova, S., Lindeman, R.W. (eds) Everyday Virtual and Augmented Reality. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-031-05804-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05804-2_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05803-5

  • Online ISBN: 978-3-031-05804-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation