Abstract
Before caring for patients, video instruction is commonly used for undergraduate medical students, and 360° virtual reality (VR) videos have gained increasing interest in clinical medical education. Therefore, the effect of immersive 360° VR video learning compared with two-dimensional (2D) VR video learning in clinical skills acquisition should be evaluated. This randomized, intervention-controlled clinical trial was aimed to assess whether immersive 360° VR video improves undergraduate medical students' learning effectiveness and reduces the cognitive load in history taking and physical examination (H&P) training. From May 1 2018 to October 30 2018, 64 senior undergraduate medical students in a tertiary academic hospital were randomized to receive a 10-min immersive 360° (360° VR video group; n = 32) or 2D VR instructional video (2D VR video group; n = 32), including essential knowledge and competency of H&P. The demographic characteristics of the two groups were comparable for age, sex, and cognitive style. The total procedure skill score, physical examination score, learner’s satisfaction score, and total cognitive load in the 360° VR video group were significantly higher than those in the 2D VR video group (effect sizes [95% confidence interval]: 0.72 [0.21–1.22], 0.63 [0.12–1.13], 0.56 [0.06–1.06], and 0.53 [0.03–1.03], respectively). This study suggested that a10-minute 360° VR video instruction helped undergraduate medical students perform fundamental H&P skills as effectively as 2D VR video. Furthermore, the 360° VR video might result in significantly better procedural metrics of physical examinations with higher learner satisfaction despite the higher cognitive load.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
History taking and physical examination (H&P) are critical core competencies of undergraduate medical education (UME). Performing H&P can deepen the relationship between students and patients, direct clinical reasoning, and lead to further assessments for a differential diagnosis (von Fragstein et al. 2008). Therefore, H&P instruction is essential for UME students to learn how to provide comprehensive care to patients (Noble, Scott-Smith, O'Neill, Salisbury, & Education 2018). In addition to traditional instructional methods such as scripts, classroom lectures, and faculty teaching in a clinical setting to learn basic H&P knowledge and skills, UME students can also use modern learning methods such as standardized patients, videos, e-learning, small group workshops, simulations, and virtual reality (VR) to improve these core competencies (Danielson et al. 2019; Keifenheim et al. 2015; Lee et al. 2018a, b, 2018a; Letterie 2002).
VR can create vivid memories and emotionally engaging experiences (Riva et al. 2007). There has been a large volume of research on VR applications in an educational context (Abich et al. 2021), e.g., medical knowledge, examinations, communication, procedures, treatment, and surgery (Checa, Miguel-Alonso, & Bustillo, 2021; Chen et al. 2021; Lohre et al. 2020; Nas et al. 2020; Wu et al. 2021). Immersive VR learning can target a high level of simulation fidelity (Lungu et al. 2021) and impose a low level of cognitive load (Andersen et al. 2016b), which is beneficial when applying this education to the real world. Recently, 360° VR videos have been used to create practical, highly immersive, three-dimensional (3D) educational systems (Izard, et al. 2018; Pulijala et al. 2018; Yoganathan et al. 2018). In these systems, learners can watch a 360° VR video from a first-person perspective at a time of their choosing. Notably, a first-person perspective has been shown to improve learning outcomes (Fiorella et al. 2017).
Recent evidence-based studies have demonstrated that VR instruction is effective for teaching and assessments (Alaker et al. 2016; Kyaw et al. 2019). For example, medical students using immersive VR learning have a higher knowledge gain than non-immersive VR learning (Gutiérrez et al 2007). Furthermore, immersive VR training allows the learners to perform clinical techniques (such as administering the anesthesia) more accurately and confidently (Collaço et al. 2021). Moreover, VR learning has been shown to help acquire complex skills outside the classroom due to reduced working hours, fewer learning sessions, and patient safety issues (Alaker, et al. 2016). Although the effect of the first-person perspective in 360° VR video may be more robust for complex tasks regarding accuracy and time, it may not be as effective as imitating actions while learning or explaining how to perform H&P skills during clinical assessments (Fiorella et al. 2017). Some medical educators have highlighted the importance of improving the current evidence level of VR-based learning (Khan et al. 2018). Thus, further studies with a high level of evidence for the effect of 360° VR video on learning outcomes are needed.
VR instructional applications need to apply cognitive load theory to reduce cognitive load to improve learning and skill acquisition (Andersen et al. 2016b). For example, VR learning with enhancing cognitive load has been shown to improve the performance of a surgical task compared to solitary VR learning (Sankaranarayanan et al. 2020). In contrast, immersive VR learning, which induces a higher cognitive load, has been associated with a more unsatisfactory performance than conventional VR learning (Frederiksen et al. 2020). However, the cognitive load depends on the user’s prior experience and self-efficacy within the learning environment (Vasile et al. 2011). Therefore, evaluating the cognitive load in VR-based learning is of increasing interest for medical education.
The global COVID-19 pandemic is having an enormous impact on medical education. With medical institutions temporarily closed, lectures switched to online, clinical rotations delayed, clinical skill evaluations compromised or even canceled (Tariq et al. 2020), in-person educational activities have been significantly reduced for infection control. Consequently, virtual pedagogical activities for teaching H&P skills (Sukumar et al. 2021) are increasing. VR is one of the most effective direct instructions (Lee, Lim, Jeon, Song 2020). However, the effects of 360º video, compared with two-dimensional (2D) video, through a VR headset (and more generally immersive versus non-immersive VR) on measures of learning and cognitive load are not well understood.
1.1 The present study
This study aimed to compare the effects of 360° VR video learning with 2D VR video learning on H&P skills, cognitive load, and learning satisfaction in the real world using a randomized controlled trial with allocation concealment, blinding of outcome assessment, and intention-to-treat analysis. For achieving these purposes, both VR videos' source recording was the same and came from a 360º camera held by the physician. Then the investigators offered the student the possibility to have a full 360º experience via a VR headset and controllers, with a first-person perspective but de-coupled from that of the physician, or to have a limited experience in which the field of view was restricted to 120º and the video perspective was entirely controlled by the physician and not by the student in a 2D theater-like projection.
Hypothesis
The null hypothesis of this study was that "360° VR video improves learner's H&P performance and cognitive load in the same way as 2D VR video".
2 Methods
2.1 Ethical considerations
We conducted this prospective, intervention-controlled clinical trial (Supplement 1) from August 1 2017 to July 31 2020 at an academic teaching hospital (Department of Otorhinolaryngology, Head and Neck Surgery [ORL-HNS], Linkou Chang Gung Memorial Hospital, Taoyuan, Taiwan). This study was approved by the Institutional Review Board of Chang Gung Medical Foundation (No: 201601821B0), and all procedures were conducted in compliance with the Declaration of Helsinki 1975. All the participants were informed about the study aims and provided written informed consent. The study followed the Consolidated Standards of Reporting Trials (CONSORT) guidelines (Moher et al. 2010).
2.2 Participants
We invited final-year UME students older than 20 years and novices of ORL-HNS from our department to participate. The exclusion criteria were: (1) contraindications for VR such as recent motion sickness, heart conditions, epileptic symptoms, and (2) refusing to participate. One of the authors (Lee LA) held information meetings about the study. All participants had been shown the practical aspects of using VR headsets and controllers and confirmed to have the essential ability to use the head-mounted display and controllers. Furthermore, the cognitive style of each learner was determined using the Group Embedded Figures Test (GEFT) (Lee et al. 2018a, b; Scott et al. 1981): “field-dependence” (GEFT score ≤ 12) and “field-independence” (GEFT score > 12) (Witkin et al. 1971). The study flowchart is shown in Fig. 1.
2.3 Randomization and blinding
According to the literature, age (Plancher et al. 2010), sex (Tippett et al. 2009), and cognitive style (Lee et al. 2018a, b; Scott et al. 1981) were covariates of new educational technology such as VR and mobile e-learning. For controlling sample size and covariates, the stratified randomization method was applied (Suresh KP. 2011). The Random Number Generator in IBM SPSS software (version 25; IBM, Armonk, NY, USA) was used to create a list of random numbers to allocate the students with a fixed block size of 8 in parallel groups. After the participants had been identified and assigned into blocks (matched for age, sex, and cognitive style), we performed the simple randomization within each block to allocate the participants (1:1) to a 360° VR video group and 2D video group. The allocation sequence was concealed before the implementation of the video module.
2.4 Instructional materials of video learning
A 10-min instructional video (video encoder: H264/Advanced Video Coding; resolution: 3840 pixels × 2160 pixels; framerate: 30 frames/s), including essential knowledge and procedural skills of H&P according to the guidelines of the American Board of Otolaryngology (Tsue 2014), was made with analysis, design, development, implementation, and evaluation models (Morrison et al. 2013) (Supplement 2). Briefly, the video showed what the learners needed to know about H&P skills using multimedia demonstrations of different worked examples (Renkl et al. 1998) with self-explanation prompts (Chi et al. 1989). The source recording for both the 360º and the alternative experience was the same and came from a 360º camera (Garmin VIRB 360, Garmin Ltd., Kansas City, MO, United States) held by the physician. The 360° VR video provided a 360° view (Fig. 2a) and the 2D VR video (Fig. 2b) provided a 120° view from the camera’s perspective. One physician (filmmaker, located in the middle of the gray character group; Fig. 2c and d) introduced procedures of classical H&P and performed a physical examination on a virtual patient (the colored character). The immersive 360° VR video (Fig. 2a) was created to play through a VR head-mounted display so that the participants could explore the video at 360° arbitrarily (via a VR headset and controllers), with a first-person perspective but de-coupled from that of the physician (Fig. 2c). In contrast, the participants could watch the 2D VR video controlled by the filmmaker in a 120° focused field of view (Fig. 2b) through the same head-mounted display (Fig. 2d). Therefore, the 2D VR video participants observed the scene "from the eyes of the physician." From the software platform, the 360° VR video provided a direct first-person perspective (Fig. 2e), while the 2D VR video was displayed in a theater environment to provide a third-person perspective (Fig. 2f). Two experienced instructors (Chuang HH & Kang CJ) verified that the 360° and 2D VR videos contained similar learning materials. The main differences between both modules included visual angle (360° versus 120°), immersion (immersive versus non-immersive), and perspective (first person versus third person) (Chao et al. 2021). Courseware with the same user interface of the 360°and 2D VR video modules was then developed using Unity 2017.3.1 Editor (Unity Technologies, San Francisco, CA, USA) by two investigators (Chao YP & Lee LA).
2.5 Intervention
After randomization, all participants were given 10 min to watch their allocated VR video (either 360° or 2D) through a VR headset (Vive VR headset, HTC Corp., New Taipei, Taiwan).
In the 360° VR video group, the learners could watch and listen to the instructor’s demonstrations of H&P skills and the responses of a simulated patient and other medical staff from a first-person perspective at a time of their choosing (Fig. 2e).
In the 2D VR video group, the learners simply watched and listened to the instructor’s demonstrations and the responses of a simulated patient and other medical staff from a third-person perspective (Fig. 2f).
2.6 Methods of measurement
Within 60 min after the intervention, each learner conducted a focused H&P with a real outpatient in a teaching clinic for 10 to 20 min. We used the Direct Observation of Procedural Skills (DOPS) to measure physical examination skills, the Mini-CEX to measure generic H&P skills, and the Cognitive Load Component (CLC) questionnaire to measure cognitive load.
2.6.1 Direct observation of procedural skills
Each participant performed a real-patient H&P for a maximum of 20 min. Two separate investigators (Fang TJ & Lee LA), blinded to video group allocation, used a DOPS (J. Norcini & Burch 2007; Yang et al. 2011) rating form to assess the procedural skills of the participant for ORL-HNS physical examinations. In this study, we used a ten-item teacher-scored DOPS form to rate ten behaviors, using a ten-point rating scale (below expectations = 1–2; borderline for completion = 3–5; meets expectations = 6–8; and above expectations = 9–10; not observed = blank) (Table 1). The DOPS has been shown to have acceptable reliability (Cronbach's alpha = 0.70–0.80; intraclass correlation coefficient = 0.25–0.85) and validity (predictive validity coefficient = 0.38–0.51) across different standardized measures (Erfani Khanghahi & Ebadi Fard Azar 2018; Kara et al. 2018).
2.6.2 Mini-clinical evaluation exercise
When each participant performed a real-patient H&P, the same investigators (Fang TJ & Lee LA) assessed the participants’ competencies using a Mini-CEX rating form (Table 2) (Chang et al. 2013; J. J. Norcini 1995). The Mini-CEX has been shown to have good reliability (Cronbach's alpha = 0.75; intraclass correlation coefficient = 0.78) (Eggleton et al. 2016) and a predictive validity coefficient ranging from 0.26 to 0.86 across different standardized measures (Al Ansari, Ali, & Donnon 2013). The rating form contained seven clinical competencies using a nine-point rating scale (1 = unsatisfactory and 9 = superior; blank = not observed). The results were also assessed for the satisfaction of both the assessor and learner using a nine-point rating scale (1 = unsatisfactory and 9 = superior) (Chang, et al. 2013; Kogan et al. 2002).
2.6.3 The cognitive load component questionnaire
The CLC questionnaire includes six items that are used to measure intrinsic (task difficulty and complexity), extraneous (instructional clarity and relevance), and germane (practical focus and amount of learning) cognitive loads (Naismith et al. 2015a, b). The participants rated the level of each item using a five-point Likert scale (1 = not at all and 5 = extremely) (Table 3). Therefore, the score of each type of cognitive load was calculated as the sum of the two specific scales (range = 2–10), and the total cognitive load score was calculated as the sum of all six scales (range = 6–30). The CLC questionnaire has been shown to have moderate correlations with different standardized measures such as the Paas Cognitive Load Scale (Paas F. G. 1992) and National Aeronautics and Space Administration Task Load Index (Hart and Staveland 1988) (Pearson correlation coefficient = 0.40–0.62) (Naismith et al. 2015a, b) and acceptable reliability (Cronbach's alpha = 0.71) (Toy et al. 2020). The validity of CLC questionnaire scores as a psychometric measure has also been shown in workshop designs and evaluations (Naismith et al. 2015a, b).
2.7 Outcome measures
The primary outcome measures of this study were the DOPS-total score and subscale scores after the intervention. The secondary outcomes included the Mini-CEX-total and subscale scores and CLC-total and subscale scores.
2.8 Data analysis
Continuous data were reported as means (standard deviations [SD]) because all variables were normally distributed. Categorical variables were presented as numbers (percentages). A sample size of 32 participants per group was estimated using primary outcome effects (DOPS-total score) based on a priori study (360° VR video group = 90.2 [5.6] and 2D VR video group = 84.9 [5.1]) and an effect size of 0.99, type I error of 0.05, a power of 0.95, and 10–15% dropout rate. Sample size calculations were performed using G*Power 3.1.9.2 software (Heinrich-Heine University, Dusseldorf, Germany). Differences between groups were analyzed using the unpaired Student’s t-test or Fisher’s exact test as appropriate. All P-values were two‐sided, and statistical significance was accepted at P < 0.05. Statistical analyses were performed using GraphPad Prism for Windows version 9.0 (GraphPad Software Inc., San Diego, CA, USA).
3 Results
3.1 Participant characteristics
Seventy volunteers expressed interest, among whom six were excluded (Fig. 1). All of the 64 volunteers (mean age = 24.2 [0.9] years; 44 [69%] males and 20 [31%] females) who were enrolled (100%) completed the study. Baseline characteristics were comparable between the two groups. The mean (SD) age was 24.1 (0.8) years in the 360° VR video group and 24.4 (1.1) years in the 2D VR video group. There were 22 (69%) male students in the 360° VR video group and 22 (69%) in the 2D VR video group. Thirty (94%) students in the 360° VR video group and 30 (94%) in the 2D VR video group were classified as being field-independent.
3.2 Primary outcomes
The mean DOPS-total score (88.4 [4.0]) in the 360° VR video group was significantly higher than that (85.8 [3.2]) in the 2D VR video group (effect size = 0.72) (Table 1; Fig. 3a). For the optimal assessment of physical examination skills, we further used three representative items (DOPS-3 [demonstrating appropriate preparation pre-procedure], DOPS-4 [proper determination of the examination areas], and DOPS-5 [technical ability to perform the skills safely]) which best reflected the observed student-patient interaction within this category. The mean skill DOPS-3 (Fig. 3b), DOPS-4 (Fig. 3c), and DOPS-5 (Fig. 3d) scores in the 360° VR video group were significantly higher than those in the 2D VR video group (effect sizes = 0.57, 0.71, and 0.80, respectively).
3.3 Secondary outcomes
There was no significant difference in the mean Mini CEX-total score between the two groups (Table 2). Among the three most commonly encountered clinical competencies of H&P in ORL-HNS teaching clinics (Mini-CEX-1 [medical interview], Mini-CEX-2 [physical examination], Mini-CEX-5 [counseling skills]), only the mean Mini-CEX-2 score (5.3 [0.8]) in the 360° VR video group was significantly higher than that (4.8 [0.8]) in the 2D VR video group (effect size = 0.63) (Fig. 4a). Furthermore, the 360° VR video group had a significantly higher learner satisfaction score (8.9 [0.3]) than the 2D VR video group (8.6 [0.7]) (effect size = 0.56) (Fig. 4b).
The total (Fig. 4c) and intrinsic (Fig. 4d) cognitive load scores (20.1 [2.0] and 4.7 [1.6], respectively) in the 360° VR video group were significantly higher than those (18.9 [2.5] and 3.7 [1.4], respectively) in the 2D VR video group (effect sizes = 0.53 and 0.67, respectively). In contrast, differences in extraneous and germane cognitive load scores were not statistically significant.
4 Discussion
To our knowledge, this is the first clinical trial to report the outcomes of using VR technology to educate novice UME students on performing fundamental H&P skills. Although the 360° VR video triggered higher total and intrinsic cognitive loads than the 2D VR video, it helped the learners perform physical examinations more readily, appropriately, and safely with higher self-satisfaction. In addition, the 360° VR video review was as effective as the 2D VR video review in learning how to perform a generic H&P skill as evaluated by the Mini-CEX assessment, even though these two models differed in content and perspective.
These findings support the use of 360° VR video to teach novice UME students essential H&P skills. Therefore, immersive 360° VR video with a higher field of view and degree of freedom is a cost-effective pedagogical tool by increasing learner attention, presence, skill enhancement, confident usability, performance, satisfaction, motivation, and engagement (Blair et al. 2021; Buttussi & Chittaro 2018). Notably, although the 360° VR video (i.e., immersive VR) was more effective in teaching clinical skills such as patient communication (Sultan et al. 2019), knot tying (Yoganathan et al. 2018), dental anesthesia (Collaço et al. 2021), procedure safety (Buttussi & Chittaro 2021), and fundamental H&P skills (Chao et al. 2021) than the 2D VR video (i.e., non-immersive VR), not all VR participants preferred immersive VR-based learning activity (Chao et al. 2021). In contrast, users of non-immersive VR (such as desktop-VR) preferred desktop-VR more for mine rescuing (Pedram et al. 2021) or had better knowledge gain for virtual biology learning (Makransky et al. 2019) than immersive 360° VR. Thereby, the suitability of VR technology to the learning task is more important than its advancement (Pedram et al. 2021).
Furthermore, the 360º VR learners may decide to look at something which is not relevant (whatever is happening behind the physician), while the 2D VR video learners can quickly focus on the relevant part of the scenario that the physician controls. In our preliminary study, the 360º VR learners watched the instructional scenes of physical examination with higher interest and engagement and longer secondary-task reaction time than the 2D VR video learners (Chao et al 2021). On the other hand, the 2D VR video learners emphasized that they found the 2D VR video module was easy to follow and highly efficient because the video was directed through the physician’s view. Therefore, each of the two VR video modules had its own advantages and disadvantages in H&P learning.
Moreover, creating 360° VR videos from a close to real-life perspective is crucial to engage the learner's interest without incurring high costs and computer programming obstacles. To reduce stimulator sickness's intensity, we developed a high-quality 360° VR application by using high-resolution (4 K) 360° videos, ergonomic VR software, and a high-end head-mounted display. After improving the quality of the 360° VR video, none of the pilot study subjects reported intolerable simulator sickness after experiencing a 10-min 360º VR video (Chao, et al. 2021). Furthermore, a pre-briefing session of the teaching clinic was usually less than 15 min; thereby, we considered a 10-min 360° VR video review was suitable for the H&P skill learning. In this study, the actual costs (work hours of computer programming) associated with the development of 360º VR video and 2D VR video instructional materials were approximate 25,000 USD (81 h) and 10,000 USD (54 h), respectively. However, the 360° VR video in this study did not increase the skill levels of history taking, counseling, and communication. We supposed that the self-explanation prompts (Chi et al. 1989; Hansen & Richland 2020) could elicit sophisticated self-explanations from the learners to boost deep learning and solve problems regardless of the video module. Nevertheless, relatively high medical interview and counseling skills (mean scores > 5.5) indicated that our participants had sufficient medical knowledge and communication skills regardless of which type of video learning.
Sultan et al. demonstrated that a pre-briefing session and a debriefing session between 360° VR video sessions could allow learners to reflect, give feedback to fill gaps, and enhance medical knowledge and communication skills (Sultan et al. 2019). Therefore, blended learning, including 360° VR video and face-to-face learning, may be able to enhance the competency of history taking to the next level of proficiency. Nevertheless, future studies are warranted to confirm the benefits of blended learning to educate UME students on performing H&P skills.
According to the cognitive load theory framework (Sweller 1988), an increase in intrinsic and extraneous cognitive loads in parallel with a decrease in germane cognitive load may explain the lower performance in video learning as reported with other multimedia. In laparoscopic training, immersive VR simulation training has been shown to trigger a higher cognitive load and contribute to worse performance than conventional VR simulation training. Therefore, we used worked examples (Lange et al. 2021; Renkl et al. 1998) to help the learners process the examples more deeply when learning with the videos. However, even though both video modules contained similar instructional content and used the same device to learn the H&P skills, the amount of mental effort involved in the 360° VR video was significantly higher than that involved in the 2D VR video (Frederiksen et al. 2020). Our preliminary study indicated that worked examples helped keep similar intrinsic cognitive load (assessed by the Paas Cognitive Load Scale and National Aeronautics and Space Administration Task Load Index) and mental efforts (evaluated by the National Aeronautics and Space Administration Task Load Index), but 360° VR video still triggered a higher physical demand than 2D VR video (Chao et al. 2021). However, we found the intrinsic cognitive load (assessed by the CLC questionnaire) of the 360° VR video group was significantly higher than that of the 2D VR video group in this study. Furthermore, we also found a higher physical demand subscale related to a higher level of simulator sickness while using the 360° VR application in the post-hoc study (Hsin LJ et al. 2022).
The increase in intrinsic cognitive load may be due to the complex virtual environment and additional medical staff responses. Hence, the high cognitive load may have hampered the learners’ performance (Sewell et al. 2019). However, the intrinsic cognitive load was relatively lower than the extraneous and germane cognitive loads in this study. According to the feedback we got from the participants with CLC questionnaires, the contents of both the 360° VR video and 2D VR video modules were a little bit more difficult and complex than expected, yet quite clear, relevant, and focused. Participants from both groups reported that they’d learned a lot from the program. Furthermore, the similar learning outcomes in both video groups suggest that the learners effectively increased cognitive function to process excessive information during the 360° VR video learning.
However, this randomized controlled trial indicated that a 10-min 360° VR video learning did not help the UME students perform history taking better and more satisfactorily than a 10-min 2D VR video learning. Notably, novices have been shown to use distributed VR practice better than massed VR practice to improve the learning curve (Andersen et al. 2015) and cognitive load (Andersen, Mikkelsen, Konge, Caye-Thomasen, & Sorensen, 2016a). Therefore, medical teachers should consider including 360° VR video in a competency-based learning curriculum to acquire H&P skills, eventually leading to improved quality of care.
Finally, age, sex, and cognitive style have been covariates of VR (Plancher et al. 2010; Tippett et al. 2009) and mobile e-learning (Lee et al. 2018a, b). For example, we found that both cognitive styles and learning modules were significantly associated with knowledge gain and satisfaction of mobile e-learning for emergent ORL-HNS disorders (Lee et al. 2018a, b). Therefore, we controlled the proportion of cognitive styles in the randomized allocation of participants. In the post-hoc analysis, age was significantly associated with Mini-CEX-total score and Mini-CEX-5 score (Pearson correlation test; r = 0.34 and 0.27, respectively; P = 0.01 and 0.03, respectively), whereas sex and cognitive style were not related to the DOPS-, Mini-CEX-, and CLC-total scores and subscales (Point-Biserial correlation test; all P > 0.05). Therefore, the older age of the UME students may have the more consulting experience to perform consulting skills better.
4.1 Limitations
There are several limitations to this study. First, we did not investigate the participants’ previous experience of H&P and VR. Although we invited final-year UME students who were novices in ORL-HNS to participate, the similar Mini-CEX scores in both groups may be evidence of plateauing. Improvements in H&P skills may reach a plateau due to previously repeated training at other teaching clinics. Furthermore, the 360° VR video learning model differed from the 2D VR video learning module in visual angle, immersion, and perspective; VR beginners may need more practice to engage in 360° VR video learning proficiently. Second, we evaluated H&P skills by directly observing student-patient interactions. However, every real patient has different medical problems and needs special H&P skills and care. Although we selected patients with ORL-HNS disorders before DOPS/Mini-CEX assessments and determined the consensus of all ratings, we could not assess all domains of clinical competencies at the same time. The heterogeneities of the patients’ needs and medical information in real outpatient scenarios may have resulted in similar total scores of the Mini-CEX. However, our study reflects real-world evidence of 360° VR video learning. Third, the validated CLC questionnaire has only been verified in three studies (Naismith et al. 2015a, b, 2015a; Toy et al. 2020). Therefore, this cognitive load assessment may limit the robustness of this study. Accordingly, future research may need a more powerful tool to investigate cognitive load to optimize the instructional content and improve learning outcomes. Finally, these results were based on a short 10-min video. More follow-up studies are required to study the long-term use of 360° VR videos.
5 Conclusions
360° VR video learning helped UME students to perform fundamental H&P skills as effectively as 2D VR video learning. As the 360° VR video learners acquired history taking skills as well as the 2D VR video learners during the same period, the novices who received the 360° VR video learning could perform examinations of the patient’s body more efficiently. By incorporating worked examples and self-explanation prompts, the 360° VR and 2D VR video learners experienced acceptable total cognitive loads. These findings may inspire the design of 360° VR video-based learning protocols to increase the interactive content and control intrinsic cognitive load, thereby enhancing the procedural skills of physical examination.
Availability of data and materials
The datasets generated and analyzed during the current study are not publicly available due to the privacy of the individuals who participated in the study but are available from the corresponding author on reasonable request.
References
Abich J IV, Parker J, Murphy JS, Eudy M (2021) A review of the evidence for training effectiveness with virtual reality technology. Virtual Real 25:919–933. https://doi.org/10.1007/s10055-020-00498-8
Al Ansari A, Ali SK, Donnon T (2013) The construct and criterion validity of the mini-CEX: a meta-analysis of the published research. Acad Med 88:413–420
Alaker M, Wynn GR, Arulampalam T (2016) Virtual reality training in laparoscopic surgery: A systematic review & meta-analysis. Int J Surg 29:85–94. https://doi.org/10.1016/j.ijsu.2016.03.034
Andersen SA, Konge L, Caye-Thomasen P, Sorensen MS (2015) Learning curves of virtual mastoidectomy in distributed and massed practice. JAMA Otolaryngol Head Neck Surg 141:913–918. https://doi.org/10.1001/jamaoto.2015.1563
Andersen SA, Mikkelsen PT, Konge L, Caye-Thomasen P, Sorensen MS (2016a) Cognitive load in distributed and massed practice in virtual reality mastoidectomy simulation. Laryngoscope 126:E74–E79
Andersen SA, Mikkelsen PT, Konge L, Caye-Thomasen P, Sorensen MS (2016b) Cognitive load in mastoidectomy skills training: Virtual reality simulation and traditional dissection compared. J Surg Educ 73:45–50. https://doi.org/10.1016/j.jsurg.2015.09.010
Blair C, Walsh C, Best P (2021) Immersive 360 videos in health and social care education: a scoping review. BMC Med Educ 21:590. https://doi.org/10.1186/s12909-021-03013-y
Buttussi F, Chittaro L (2018) Effects of different types of virtual reality display on presence and learning in a safety training scenario. IEEE Trans vis Comput Graph 24:1063–1076. https://doi.org/10.1109/TVCG.2017.2653117
Buttussi F, Chittaro L (2021) A comparison of procedural safety training in three conditions: virtual reality headset, smartphone, and printed materials. IEEE Trans Learn Technol 14(1–15):21. https://doi.org/10.1109/TLT.2020.3033766
Chang YC, Chen CK, Chen JC, Liao CH, Lee CH, Chen YC, Ng CJ, Huang JL, Lee ST (2013) Implementation of the mini-clinical evaluation exercise in postgraduate year 1 residency training in emergency medicine: Clinical experience at Chang Gung Memorial Hospital. J Acute Med 3:110–115. https://doi.org/10.1016/j.jacme.2013.06.004
Chao YP, Chuang HH, Hsin LJ, Kang CJ, Fang TJ, Li HY, Huang CG, Kuo TBJ, Yang CCH, Shyu HY, Wang SL, Shyu LY, Lee LA (2021) Using a 360 virtual reality or 2D video to learn history taking and physical examination skills for undergraduate medical students: Pilot randomized controlled trial. JMIR Serious Games 9:e13124. https://doi.org/10.2196/13124
Checa D, Miguel-Alonso I, Bustillo A (2021) Immersive virtual-reality computer-assembly serious game to enhance autonomous learning. Virtual Real:In Process. https://doi.org/10.1007/s10055-021-00607-1
Chen G, Zhao Y, Xie F, Shi W, Yang Y, Yang A, Wu D (2021) Educating outpatients for bowel preparation before colonoscopy using conventional methods vs virtual reality videos plus conventional methods: A randomized clinical trial. JAMA Netw Open 4:e2135576. https://doi.org/10.1001/jamanetworkopen.2021.35576
Chi MTH, Bassok M, Lewis MW, Reimann P, Glaser R (1989) Self-explanations: how students study and use examples in learning to solve problems. Cogn Sci 13:145–182. https://doi.org/10.1207/s15516709cog1302_1
Danielson AR, Venugopal S, Mefford JM, Clarke SO (2019) How do novices learn physical examination skills? a systematic review of the literature. Med Educ Online 24:1608142. https://doi.org/10.1080/10872981.2019.1608142
Eggleton K, Goodyear-Smith F, Paton L, Falloon K, Wong C, Lack L, Kennelly J, Fishman T, Moyes SA (2016) Reliability of Mini-CEX assessment of medical students in general practice clinical attachments. Fam Med 48:624–630
Erfani Khanghahi M, Ebadi Fard Azar F (2018) Direct observation of procedural skills evaluation method systematic review of evidence. Med J Islam Republic Iran. https://doi.org/10.14196/mjiri.32.45
Fiorella L, van Gog T, Hoogerheide V, Mayer RE (2017) It’s all a matter of perspective: Viewing first-person video modeling examples promotes learning of an assembly task. J Educ Psychol 109(5):653–665. https://doi.org/10.1037/edu0000161
Frederiksen JG, Sorensen SMD, Konge L, Svendsen MBS, Nobel-Jorgensen M, Bjerrum F, Andersen SAW (2020) Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial. Surg Endosc 34(3):1244–1252
Gutiérrez F, Pierce J, Vergara VM, Coulter R, Saland L, Caudell TP, Goldsmith TE, Alverson DC (2007) The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. Stud Health Technol Inform 125:155–160
Hansen J, Richland LE (2020) Teaching and learning science through multiple representations: Intuitions and executive functions. CBE Life Sci Educ 19:ar61. https://doi.org/10.1187/cbe.19-11-0253
Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Adv Psychol 52:139–183. https://doi.org/10.1016/S0166-4115(08)62386-9
Hsin LJ, Chao YP, Chuang HH, Kuo TBJ, Yang CCH, Huang CG, Kang CJ, Lin WN, Fang TJ, Li HY, Lee LA (Minor revision) Mild simulator sickness can alter heart rate variability, mental workload, and learning outcomes in a 360° virtual reality application for medical education: A post-hoc analysis of a randomized controlled trial. Virtual Real (Minor revision)
Izard SG, Juanes JA, Garcia Penalvo FJ, Estella JMG, Ledesma MJS, Ruisoto P (2018) Virtual reality as an educational and training tool for medicine. J Med Syst 42:50. https://doi.org/10.1007/s10916-018-0900-2
Kara CO, Mengi E, Tumkaya F, Topuz B, Ardic FN (2018) Direct observation of procedural skills in otorhinolaryngology training. Turk Arch Otorhinolaryngol 56:7–14. https://doi.org/10.1186/s12909-015-0443-x
Keifenheim KE, Teufel M, Ip J, Speiser N, Leehr EJ, Zipfel S, Herrmann-Werner A (2015) Teaching history taking to medical students: a systematic review. BMC Med Educ 15:159. https://doi.org/10.1186/s12909-015-0443-x
Khan R, Plahouras J, Johnston BC, Scaffidi MA, Grover SC, Walsh CM (2018) Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev 8:CD008237. https://doi.org/10.1002/14651858.CD008237.pub3
Kogan JR, Bellini LM, Shea JA (2002) Implementation of the mini-CEX to evaluate medical students’ clinical skills. Acad Med 77:1156–1157. https://doi.org/10.1097/00001888-200211000-00021
Kyaw BM, Saxena N, Posadzki P, Vseteckova J, Nikolaou CK, George PP, Divakar U, Masiello I, Kononowicz AA, Zary N, Tudor Car L (2019) Virtual reality for health professions education: Systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res 21:e12959. https://doi.org/10.2196/12959
Lange C, Almusharraf N, Koreshnikova Y, Costley J (2021) The effects of example-free instruction and worked examples on problem-solving. Heliyon 7:e07785. https://doi.org/10.1016/j.heliyon.2021.e07785
Lee LA, Chao YP, Huang CG, Fang JT, Wang SL, Chuang CK, Kang CJ, Hsin LJ, Lin WN, Fang TJ, Li HY (2018a) Cognitive style and mobile e-learning in emergent otorhinolaryngology-head and neck surgery disorders for millennial undergraduate medical students: Randomized controlled trial. J Med Internet Res 20:e56. https://doi.org/10.2196/jmir.8987
Lee LA, Chen HC, Fang TJ, Li HY (2018b) IBVR: Image-based virtual reality for innovative teaching and learning in ORL-HNS teaching clinics. J Laryngol Otol 07:69. https://doi.org/10.4172/2324-8785-c3-014
Lee HS, Lim JH, Jeon BH, Song CS (2020) Non-immersive virtual reality rehabilitation applied to a task-oriented approach for stroke patients: A randomized controlled trial. Restor Neurol Neurosci 38:165–172. https://doi.org/10.3233/RNN-190975
Letterie GS (2002) How virtual reality may enhance training in obstetrics and gynecology. Am J Obstet Gynecol 187:S37-40. https://doi.org/10.1067/mob.2002.127361
Lohre R, Bois AJ, Pollock JW, Lapner P, McIlquham K, Athwal GS, Goel DP (2020) Effectiveness of immersive virtual reality on orthopedic surgical skills and knowledge acquisition among senior surgical residents: a randomized clinical trial. JAMA Netw Open 3:e2031217. https://doi.org/10.1001/jamanetworkopen.2020.31217
Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X (2021) A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev Med Devices 18:47–62. https://doi.org/10.1080/17434440.2021.1860750
Makransky G, Terkildsen TS, Mayer RE (2019) Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn Instruct 60:225–236. https://doi.org/10.1016/j.learninstruc.2017.12.007
Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG (2010) CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 340:c869. https://doi.org/10.1136/bmj.c869
Morrison GR, Ross SM, Kalman HK, Kemp JE (2013) Designing Effective Instruction (7th, Edition. Wiley Inc., Hoboken, NJ
Naismith LM, Cheung JJ, Ringsted C, Cavalcanti RB (2015a) Limitations of subjective cognitive load measures in simulation-based procedural training. Med Educ 49:805–814. https://doi.org/10.1111/medu.12732
Naismith LM, Haji FA, Sibbald M, Cheung JJ, Tavares W, Cavalcanti RB (2015b) Practising what we preach: using cognitive load theory for workshop design and evaluation. Perspect Med Educ 4:344–348. https://doi.org/10.1007/s40037-015-0221-9
Nas J, Thannhauser J, Vart P, van Geuns RJ, Muijsers HEC, Mol JQ, Aarts GWA, Konijnenberg LSF, Gommans DHF, Ahoud-Schoenmakers SGAM, Vos JL, van Royen N, Bonnes JL, Brouwer MA (2020) Effect of face-to-face vs virtual reality training on cardiopulmonary resuscitation quality: a randomized clinical trial. JAMA Cardiol 5:328–335. https://doi.org/10.1001/jamacardio.2019.4992
Noble LM, Scott-Smith W, O’Neill B, Salisbury H (2018) UK Council of Clinical Communication in Undergraduate Medical Education Consensus statement on an updated core communication curriculum for UK undergraduate medical education. Patient Educ Couns 101:1712–1719. https://doi.org/10.1016/j.pec.2018.04.013
Norcini JJ (1995) The Mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med 123:795–799. https://doi.org/10.7326/0003-4819-123-10-199511150-00008
Norcini J, Burch V (2007) Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 29:855–871. https://doi.org/10.1080/01421590701775453
Paas FG (1992) Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive-load approach. J Educ Psychol 84:429–434. https://doi.org/10.1037/0022-0663.84.4.429
Pedram S, Skarbez R, Palmisano S, Farrelly M, Perez P (2021) Lessons learned from immersive and desktop VR training of mines rescuers. Front Virtual Real 2:627333. https://doi.org/10.3389/frvir.2021.627333
Plancher G, Gyselinck V, Nicolas S, Piolino P (2010) Age effect on components of episodic memory and feature binding: a virtual reality study. Neuropsychology 24:379–390. https://doi.org/10.1037/a0018680
Pulijala Y, Ma M, Pears M, Peebles D, Ayoub A (2018) An innovative virtual reality training tool for orthognathic surgery. Int J Oral Maxillofac Surg 47:1199–1205. https://doi.org/10.1016/j.ijom.2018.01.005
Renkl A, Stark R, Gruber H, Mandl H (1998) Learning from worked-out examples: The effects of example variability and elicited self-explanations. Contemp Educ Psychol 23:90–108. https://doi.org/10.1006/ceps.1997.0959
Riva G, Mantovani F, Capideville CS, Preziosa A, Morganti F, Villani D, Gaggioli A, Botella C, Alcaniz M (2007) Affective interactions using virtual reality: the link between presence and emotions. Cyberpsychol Behav 10:45–56. https://doi.org/10.1089/cpb.2006.9993
Sankaranarayanan G, Odlozil CA, Wells KO, Leeds SG, Chauhan S, Fleshman JW, Jones DB, De S (2020) Training with cognitive load improves performance under similar conditions in a real surgical task. Am J Surg 220:620–629. https://doi.org/10.1016/j.amjsurg.2020.02.002
Scott N, Smith DU, Rosenberg IK (1981) Cognitive style and instructional materials for medical students. J Med Educ 56:565–571. https://doi.org/10.1097/00001888-198107000-00004
Sewell JL, Maggio LA, Ten Cate O, van Gog T, Young JQ, O’Sullivan PS (2019) Cognitive load theory for training health professionals in the workplace: A BEME review of studies among diverse professions: BEME Guide No. 53. Med Teach 41:256–270. https://doi.org/10.1080/0142159X.2018.1505034
Sukumar S, Zakaria A, Lai CJ, Sakumoto M, Khanna R, Choi N (2021) Designing and implementing a novel virtual rounds curriculum for medical student’s internal medicine clerkship during the pandemic. MedEdPortal. https://doi.org/10.15766/mep_2374-8265.11106
Sultan L, Abuznadah W, Al-Jifree H, Khan MA, Alsaywid B, Ashour F (2019) An experimental study on usefulness of virtual reality 360 degrees in undergraduate medical education. Adv Med Educ Pract 10:907–916. https://doi.org/10.2147/AMEP.S219344
Suresh KP (2011) An overview of randomization techniques: an unbiased assessment of outcome in clinical research. J Hum Reprod Sci 4:8–11. https://doi.org/10.4103/0974-1208.82352
Sweller J (1988) Cognitive load during problem solving: effects on learning. Cognitive Sci 12:29. https://doi.org/10.1207/s15516709cog1202_4
Tariq EF, Sah PK, Malik A (2020) The plight of COVID-19 pandemic on medical students and residency applicants. Ann Med Surg (lond) 60:1–4. https://doi.org/10.1016/j.amsu.2020.10.010
Tippett WJ, Lee JH, Mraz R, Zakzanis KK, Snyder PJ, Black SE, Graham SJ (2009) Convergent validity and sex differences in healthy elderly adults for performance on 3D virtual reality navigation learning and 2D hidden maze tasks. Cyberpsychol Behav 12:169–174. https://doi.org/10.1089/cpb.2008.0218
Toy S, Miller CR, Daly Guris RJ, Duarte SS, Koessel S, Schiavi A (2020) Evaluation of 3 cognitive load measures during repeated simulation exercises for novice anesthesiology residents. Simul Healthc 15:388–396. https://doi.org/10.1097/SIH.0000000000000458
Tsue TT (2014) Developing the otolaryngology milestones. J Grad Med Educ 6:162–165. https://doi.org/10.4300/JGME-06-01s1-21
Vasile C, Marhan AM, Singer FM, Stoicescu D (2011) Academic self-efficacy and cognitive load in students. Procedia Soc Behav Sci 12:478–482. https://doi.org/10.1016/j.sbspro.2011.02.059
von Fragstein M, Silverman J, Cushing A, Quilligan S, Salisbury H, Wiskin C (2008) UK council for clinical communication skills teaching in undergraduate medical education UK consensus statement on the content of communication curricula in undergraduate medical education. Med Educ 42:1100–1107. https://doi.org/10.1111/j.1365-2923.2008.03137.x
Witkin HA, Oltman PK, Raskin E, Karp SA (1971) A manual for the embedded figures tests. Consulting Psychologists Press, Palo Alto
Wu J, Sun Y, Zhang G, Zhou Z, Ren Z (2021) Virtual reality-assisted cognitive behavioral therapy for anxiety disorders: a systematic review and meta-analysis. Front Psychiatry 12:575094. https://doi.org/10.3389/fpsyt.2021.575094
Yang YY, Lee FY, Hsu HC, Huang CC, Chen JW, Cheng HM, Lee WS, Chuang CL, Chang CC, Huang CC (2011) Assessment of first-year post-graduate residents: usefulness of multiple tools. J Chin Med Assoc 74:531–538. https://doi.org/10.1016/j.jcma.2011.10.002
Yoganathan S, Finch DA, Parkin E, Pollard J (2018) 360 degrees virtual reality video for the acquisition of knot tying skills: a randomised controlled trial. Int J Surg 54:24–27. https://doi.org/10.1016/j.ijsu.2018.04.002
Acknowledgements
We are indebted to our trained study staff, especially Ping-Yun Lin, Yun-Kai Chang, Ming-Yen Chung, and Chung-Fang Hsiao (Department of Otorhinolaryngology, Head and Neck Surgery, Chang Gung Memorial Hospital, Linkou Main Branch, Taoyuan City, Taiwan, ROC) for their technical assistance.
Funding
This work was supported by the Ministry of Science and Technology, Taiwan, R.O.C. (106-2511-S-182A-003-MY2 and 108-2511-H-182A-001-) and a grant from the Chang Gung Medical Foundation, Taiwan, R.O.C. (CMRPG3G1381-3) awarded to L.-A. Lee.
Author information
Authors and Affiliations
Contributions
All authors contributed to the design of the studies and preparation of the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare.
Ethical approval
The Institutional Review Board of Chang Gung Medical Foundation approved this study (No: 201601821B0C501), and all procedures were conducted in compliance with the Declaration of Helsinki 1975. Written consent was given by all participants.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Chao, YP., Kang, CJ., Chuang, HH. et al. Comparison of the effect of 360° versus two-dimensional virtual reality video on history taking and physical examination skills learning among undergraduate medical students: a randomized controlled trial. Virtual Reality 27, 637–650 (2023). https://doi.org/10.1007/s10055-022-00664-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-022-00664-0