1 Introduction

In many curricula around the world, there is a quite sudden transition in the training of medical students from the preclinical, university-based phase to the clinical, hospital-based phase. This transition often results in increased levels of stress and anxiety (Atherley et al. 2019; Moss and McManus 1992; Radcliff and Lester, 2003). Levels of stress and anxiety relate to student’s perceptions of preparedness for the clerkships, and, therefore, improving the perception of preparedness might be an effective way to improve the transition to the clinical phase (Surmon et al. 2016). Several potentially modifiable themes relating to the level of preparedness have already been identified, including competence, curriculum, learning, disconnection, uncertainty, and feeling part of the team (Surmon et al. 2016). Commonly used strategies to ease the important transition to the clinical workplace include ‘transition to clerkship courses’, problem-based learning, early patient contacts, portfolios and reflective interviews (Atherley 2019). Many medical schools have already developed a ‘transition to clerkship course’ or ‘transitional clerkship’ (Chittenden et al. 2009; O’Brien and Poncelet 2010; Atherley 2019). At Leiden University Medical Center (LUMC), there has been such a ‘General clerkship’ course since 1980 (Hoeks and van Rossum 1988). The course has been last updated in 2012 when the bachelor's master's doctorate system was introduced in the Medicine program, and is since then called the Introductory Clerkship. In our experience, students positively evaluate this clerkship, but still believe the gap between the preclinical and clinical phase could be further reduced. In order to improve the preparation of medical students for their clerkships, we explored the addition of 360° degree video-based virtual reality (VR) to the existing program (Pieterse et al. 2018)

360° degree video-based VR is a digital technology in which learners are able to look around in a digital world based on 360° video using VR equipment. For medical education, this technique provides opportunities to expose learners to authentic clinical situations before entering them in real life. In order to create a 360° video, a special camera is used that films in all directions simultaneously. The video images are stitched together in order to create one spherical 360° video. This video can be watched on a smartphone, tablet, personal computer or by using a VR headset, which offers more immersion. While 360° video and 360° video-based VR are increasingly used in education, the amount of published research regarding this topic in Internal Medicine is still scarce (Snelson and Hsu 2020; Blair et al. 2021). The possible educational benefits of 360° videos include reported high levels of interest, engagement, enjoyment and perception that the experience was useful to the students (Lee et al. 2017; Rupp et al. 2019; Johnson et al. 2018; Harrington et al. 2017; Chan et al. 2021). The sense of presence can contribute positively to the learning process (Huber et al. 2017). A small randomized controlled trial comparing 360° video to conventional 2D video for learning surgical knot-tying skills showed significant better learning outcomes in the group using 360° video (Yoganathan et al. 2018). Although the majority of published studies on 360° video VR for medical education is related to learning surgical or other practical procedures (Blair et al. 2021), a few studies explore the use in other domains of medical education. One study showed better learning outcomes of medical students after watching 360° video’s on the topic of communication and collaboration skills, in comparison to an interactive lecture (Sultan et al. 2019). In a pilot study, a group of health care students were provided with 360° videos from a patients point-of-view, in which they experience several health concerns such as visual and hearing impairment to improve understanding of patients (Buchman 2018). However, more research is needed in order to understand how, and under which conditions, 360° video supports the learning process (Snelson and Hsu 2020). Although there are also inspiring examples of computer generated VR for medical education (Pottle 2019), in this paper, we focus on 360° video based VR.

We presumed that the immersive environment of 360° video based VR could create authentic learning experiences for medical students in addition to possibilities of repeated exposure and learning from mistakes, which are all elements that can improve the perception of preparedness. We developed two 360° video-based VR applications for medical students transitioning to the clerkships and piloted these in small group learning sessions. In this study, both VR applications are described, and the first user experiences with these applications were evaluated by an explorative anonymous questionnaire.

2 Material and methods

2.1 Design process of the 360° video-based VR applications

In 2017, a team was formed to create several 360° video-based VR applications as an integral part of a blended learning program. This team was a collaboration between LUMC and Leiden University's Center for Innovation (CFI), and consisted of various stakeholders: clinical teachers, medical students, a VR filmmaker, a VR developer, learning experience designers and project managers. In this study, two VR applications are described and evaluated: ‘Virtual Ward’ and ‘(Un)professional Behavior’. We chose 360° video based VR instead of regular video based learning mainly because of the increased level of immersion. Potential disadvantages of choosing this technique are the costs of developing applications, purchasing and maintaining VR headsets and preparing teachers to use this technology. We chose the technique of 360° video-based VR instead of (computer generated) VR because of the higher level of authenticity (by filming on an actual hospital ward with real-life health care professionals and simulated patients), previous positive experience (Pieterse et al. 2018), and the lower production costs compared to computer generated VR.

The total design process of these applications took around 6 months and started with a ‘Learning Experience Design (LED) session’ during which the target group and learning goals were defined. Next, storyboards for both applications were created. In order to create authentic applications, it was decided to film on a real hospital ward, and the actors were real healthcare professionals and instructed patient actors. All healthcare professionals and patient actors signed a video consent form before filming. After filming the 360° video footage was stitched, spatial audio was added and in case of the first application multiple elements of interaction were added. The applications were developed for use with the Oculus Go®, a type of standalone VR headset. A small group learning session was designed in which students use both applications.

2.2 Technological development of the applications

The ‘Virtual Ward’ application can be used as long as the user needs to achieve the learning goals which is on average 10 to 15 min. The ‘Unprofessional Behavior’ experience is 6:30 min in duration. The two VR applications are described in more detail in the results (see Sects. 3.1.1 and 3.1.2).

For the creation of the VR experiences, a mix of technologies was chosen. The basis of the ‘Virtual Ward’ experience consists of several 3D 360° videos and photos, which are contained in a custom-developed software application. The software application allows for user interactivity, such as moving from location to location and activating information hotspots. The ‘(Un)professional Behavior’ experience consists solely of 360° video footage, combined with spatial audio. For the 360° video, a special camera was used: the Insta360 Pro®. This is a camera with six lenses, each with a 200° angle. These lenses create overlap between the images, which allow stitching them together to form a 360° video. The large angle of the lenses creates sufficient parallax for the Insta360 Pro® to record 3D 360° video, i.e. video with depth experience, simulating the way humans see depth in the real world. Being able to offer video with depth adds an extra sense of reality to the VR experiences.

Both experiences contain spatial audio (‘360° audio’). Spatial audio adds another dimension to the experience, since natural audio is also directional. Sound coming from the left will be heard more clearly in the left ear of the user, sound coming from the right in the right ear, et cetera. To achieve this effect, a spatial microphone was used (the Zoom H2n®) along with several lapel microphones on the actors and several microphones hidden in the set. Next, these audio sources were combined in a spatial audio software suite to simulate the correct location of the sound sources. The detail of the spatial audio was such that users could distinguish sound coming from complex directions, such as back-left or front-right.

Several software applications were used during the post-production phase. The six different video angles had to be stitched together in order to create one seamless video file. This was achieved using the Insta360 Pro Stitcher® application, version 1.2.2 for the Virtual Ward videos and version 1.7.0 for the (Un)professional Behavior video. Editing the 360° video footage was done with Adobe® Premiere Pro® CC 2017, which has native support for this type of editing. After editing, the video was exported to a MOV file, encoded in DNxHR HQ 8-bit, at a resolution of 6400 × 6400@30fps, without audio. The video was then transcoded to a format suitable for playback, using the iFFMPEG® 6 application and the FFMPEG® 3.1.11 encoding library. The format chosen was 4096 × 2160 H.264 MP4, at the time the maximum resolution the Oculus® Go® supported.

Spatial audio was created separately, using Reaper ® (v5.70 for Virtual Ward and v5.91 for (Un)professional Behavior), an advanced audio editing application, along with a plugin called Spatial Workstation® (v2.1), which specifically features complex algorithms to simulate audio sources in 360° space. The plugin was used to correctly position the audio sources of the different actors in space, and track their movements if necessary. The audio was then exported to a lossless WAV file, with 8 channels.

As it did not contain interactivity, the ‘(Un)professional Behavior’ video and audio were muxed together using the FB360 Encoder® (v2.1) to a format suitable for 360° VR video and spatial audio. The output format used for the audio was B-format, 1st order AmbiX, 4 channels. The end-result after muxing was a 360° video with spatial audio encoded, which could playback natively on the Oculus Go® head-mounted display.

The process for the ‘Virtual Ward’ application was more elaborate, as it is an application with some interactive elements. A video shot was exported for every ‘location’ a user could visit in the application. The video was exported and transcoded as described earlier in this section. Spatial audio was exported separately as well, again using the FB360 Encoder®, to a suitable audio format (1st order AmbiX, 4 channel WAV). Furthermore, graphical assets were created. All these media were then imported into Unity® (version 2017.3.1.p4), a software development suite originating in the game development industry. The Oculus® Mobile® SDK (v1.24.0) was added to Unity® in order to be able to develop the application for Oculus® devices. Using all these elements, a custom-developed application was created to enable the functionality needed for the VR-experience. Finally, the application was built and deployed on the Oculus Go® devices using a private release channel in the Oculus® store, from which the app could be downloaded and installed.

Note: Oculus has been rebranded to Meta in 2022. The VR headset used for the described applications (Oculus Go®) is no longer for sale, and, therefore, we have continued using the name Oculus Go® throughout this article.

2.3 Evaluation of the 360° video-based VR applications

The user experiences of medical students with the VR applications were evaluated using an anonymous explorative questionnaire. A small group learning session with the applications is an obligatory part of the ‘Introductory clerkship’. This clerkship is a preparation for the sequential clinical clerkships, and marks the start of the Master’s program of Medicine at our medical school. The questionnaire, as provided as Supplementary material, contained both open and closed questions (with Likert scales) and covered the following topics:

  • General information

  • Use of the 360° video-based VR applications

  • Content of the 360° video-based VR applications

  • Physical discomfort

Medical students at LUMC participating in the obligatory small group learning sessions were included in the study. Immediately following the small group learning session, all the students were invited to fill out an anonymous online questionnaire. It was also possible for the students to start the questionnaire at a later moment. As part of the questionnaire, they provided informed consent for the use of their data for research purposes. Whether students participated or not had no consequences for their clerkship assessments.

Answers of participants to the evaluative questions with a 4 or 5-point Likert scale and to questions regarding experienced physical discomfort were descriptively analyzed. Answers to the open ended questions were qualitatively analyzed by a thematic analysis. In this analysis, the principal investigator (AP) coded the answers in vivo, selected overarching themes in a inductive approach, and assigned quotations to overarching themes (Miles et al. 2018).

3 Results

3.1 Description of a small group learning session with two 360° video-based VR applications

The design process resulted in a small group learning session for groups of 10–12 students, with learning goals related to increasing preparedness for the clerkships by virtually exploring an Internal Medicine ward (see Sect. 3.3.1) and observing and discussing professional behavior (see Sect. 3.1.2). It starts with introducing the learning goals, explaining how to use the VR headset and how to start the applications. Students use the applications in a sitting position and start the VR application simultaneously. We have chosen a sitting position over a standing position for the small group learning sessions, because in our experience, this prevents students from feeling imbalanced and/or unexpectedly walking into each other causing anxiety. Before using the first application (‘Virtual Ward’), students are instructed to explore the hospital ward and answer a set of questions regarding the work environment and health care professionals (e.g. ‘where would you discuss a patient case?’ or ‘who would you ask for help if you would like to know what time your patient will undergo a radiology examination?’). After using the first application, students share whether they have achieved their learning goals and have the possibility to reflect on their experience. Next, the students are provided with the learning goals and assignment for the second application (‘(Un)professional behavior’). After using the VR headset, they have a group discussion about professional and unprofessional behavior in the clinical workplace, under the guidance of a clinical teacher. The total duration of the small group learning session is 45 min. In June 2018, the session was introduced in our medical curriculum. Since then, all students attend this obligatory small group learning session in the second week of the Introductory Clerkship.

3.1.1 Description of the application ‘Virtual Ward’

The first application is an interactive virtual tour of a hospital ward, in which learners explore the Internal Medicine ward of LUMC and learn about the roles of different health care professionals and their mutual interactions. The user can interact with the virtual world, moving from one ‘place’ (i.e. video shot) to another. Users control their cursor, needed for navigating or activating hotspots, by gazing. To navigate, they look at certain hotspots in front of them. For example, a hotspot placed at the entrance to a room will move the user into that room. As the user gazes, the cursor ‘fills up’. When it’s full, they are transitioned to that room. The rooms or spaces that can be visited are: front desk, patient room, nursing station, doctors office, break room and family room. Users must navigate the space as if they were really there. For instance, they cannot jump from the doctor’s office to the patient room. Instead, they must ‘walk’ into the hallway first, after which they can move into the patient room. This heightens the realism of the experience of exploring the ward. Moving into a new room will trigger a voice-over that explains the function of the room and other relevant details. Once the voice-over has been played, it will not trigger automatically again when a user moves into that room a second time. However, it can be activated manually if the user wishes to hear it again, by focusing the cursor on the appropriate hotspot (similar to the navigation mechanism explained above). The learners receive tips how to familiarize themselves with the actual ward where they will soon start their first clerkship, and how to integrate into the team. Furthermore, information hotspots are available in several rooms. If the user focuses the cursor on them by looking at the hotspot, it immediately activates and displays relevant bits of information. This is a useful mechanism to show small details, which could otherwise be lost to the viewer. For example, some hotspots are placed on medical equipment. In Fig. 1, two stills of the application are shown. This application can be used as long as it takes the learner to achieve his/her learning goals, this takes on average 10–15 min.

Fig. 1
figure 1

Stills of the VR application ‘Virtual Ward’ A Nurse working in the nursing station and using the pneumatic tube system; a hotspot provides the additional textual information “Buizenpost” (translation from Dutch: pneumatic tube system) B A patient in a patient room; a hotspot shows the text “Temperatuurmeter” (translation from Dutch: thermometer) and another hotspot (“gang”, translation: corridor) can be used to navigate to different areas

Full size image

3.1.2 Description of the application ‘(Un)professional Behavior’

The second application has been developed to train medical students in observing (un)professional behavior of healthcare providers and preparing to deal with potential unprofessional behavior in the clinical workplace. It is a simulated ward round on an Internal Medicine ward. The students observe a team consisting of a resident, medical student and nurse during their daily ward round. They are confronted with multiple forms of unprofessional behavior, such as a lack of hand hygiene and disrespectful treatment of a medical student and patient. Students get the assignment to observe and memorize all the forms of unprofessional behavior, and afterwards to discuss strategies how to deal with this behavior if they would encounter this during the clerkships. See Fig. 2 for an impression of this experience which is 6:30 min in duration.

Fig. 2
figure 2

Still of the VR application ‘(Un)professional behavior’, showing a ward round by a team of a nurse, resident and medical student

Full size image

3.2 User experience evaluation

From June 2018 until July 2019, 26 groups of students (280 students in total) were asked to fill out the online questionnaire. A total of 171 completed questionnaires were collected: a response rate of 61%. Students who had been involved in the design team were not allowed to participate in this study. All students provided consent for use of their anonymized data for research purposes. The general characteristics of the study participants is shown in Table 1.

Table 1 General information of participants
Full size table

A majority of students evaluated VR as a good (59%) or excellent (26%) tool for learning, while 13% of the students evaluated it as an average tool for learning, and only 2 students rated is as poor. Most participants reported that they enjoyed the VR experience (average score 4,5 on a 5-point Likert scale; range 2–5). The majority of the students felt it was important for the university to pursue VR more intensively relative to other educational innovations (moderately important 45%, very important 36%, extremely important 8%). The participants rated the likelihood that they would recommend the VR experience to a friend or colleague on average 7.6 on a scale from 0 to 10.

For 63% of the respondents, this was their first experience with a VR headset. 35% of the students wore either glasses (n = 30) or contact lenses (n = 30) during the experience, which are both compatible with wearing the VR headset that was provided. One student mentioned that “it was a little uncomfortable with glasses on”. Most participants indicated that the instructions received prior to using the VR applications were clear (70%) or mostly clear (23%). Approximately half of the students (55%) needed additional instructions of the teacher during the VR session. Most students understood how to navigate the VR applications, only 2% of students answered neutral to this question. One student suggested to change the navigation in the ‘Virtual Ward’ application from ‘gazing’ to using a controller. Several students reported flickering video images in the ‘Virtual Ward’ application. All students except one reported that they were able to fully concentrate themselves on the VR experience. The student that was not able to concentrate indicated that this was due to hearing other students talk. Students indicated that they felt they looked around a lot (71%) or somewhat (23%) during the experience.

Qualitative analysis of the open-ended questions showed that students provided positive comments related to a variety of themes: level of realism, experience, interaction, 360 degrees, learning, interaction, enjoyment and novelty. A lot of students provided similar statements or very brief comments. Here, a relevant selection of answers is presented. A student commented that “It felt more real than a normal video, easier to focus”. Another student stated that “It was another way of learning. It gave a lot of insight in the situation, because it feels like you are really there”. Multiple students reported that it felt very realistic (e.g. “It’s very real, impressive that this is possible” and “Unique experience, successful in transporting you there”), and appreciated the fact that they could look around without getting in someone’s way. In answer to the question “What did you dislike?”, 42% of the participants did not report anything they disliked. The other students reported factors related to the following themes: physical discomfort, display resolution, ambient noise, disconnection, white flashes (e.g. “the white flashing in the LUMC ward video in between the locations”), technological issues and not being able to walk. Some students reported that the quality of the experience would be better if the video would have been displayed in a higher resolution.

Forty-five percent of the students experienced physical discomfort, such as nausea, dizziness, headache and disorientation. The extent of the experienced types of discomfort is shown in Fig. 3. In most cases these complaints were mild, although some experienced severe nausea (n = 6), severe headache (n = 2) or severe disorientation (n = 1). There were no participants that reported multiple severe complaints. Students reported various degrees of experiencing a disconnect between their own body and their virtual experience (very little 7%; a little 19%, somewhat 45%, very 14%). Examples of the experienced disconnection included “trying to lean against a virtual table”, “it sometimes felt like I was flying”, “I wanted to touch something in VR which wasn’t there”, “the height of the camera was a bit higher than my own length”, “unable to move from one fixed point”, “could not see my own body”, “I didn’t see my hands when I looked downwards”, and “wanted to touch something”.

Fig. 3
figure 3

Experience of different types of physical discomfort during the use of the VR applications (4-point Likert scale; n = 171)

Full size image

Most suggestions for other types of 360° VR applications for medical education regarded the fields of surgery (25%), anatomy (22%), and emergency care (8%).

4 Discussion

In this study, we have described the result of developing a small group learning session with two 360° video-based VR applications, aimed at improving medical students’ preparedness for the clinical clerkships. As a first step, we have evaluated the user experiences of medical students with the applications. This showed that most students enjoyed the 360° VR applications and valued it as an educational innovation, despite the fact that nearly half of the students reported some level of physical discomfort.

Comments of students pointed in the direction that the use of 360° VR can be more useful than a normal video, to increase their sense of preparedness. Quotations like “It was another way of learning. It gave a lot of insight in the situation, because it feels like you are really there” suggest that the small group learning session helped in increasing (self-perceived) preparedness for the clerkships. This study, however, was a user experience study and not yet focused on testing the effects on reported preparedness.

The most mentioned negative factors were the experience of physical discomfort or disconnection. Cybersickness has been described as a form of motion sickness that is caused by motions in a virtual environment (McCauley and Sharkey 1992). It is a complex phenomenon: many different theories and over 40 different factors have been described that could contribute to the occurrence (Rebenitsch and Owen 2020). These factors are either user, software or hardware related. Some examples are: previous experience with simulator, history of motion sickness, screen luminance, scene complexity, lag variance and tracking method. The incidence of cybersickness has been reported to range from 25 up to 80% (Rebenitsch and Owen 2016). Despite efforts to reduce cybersickness by improving hardware and software, reported cybersickness rates remain substantial. In a more recently published study with the Oculus Rift head-mounted display, the incidence in two experiments ranged from 25 to 56% (Munafo et al. 2016). The incidence of self-reported physical discomfort in our study is thus in line with previous literature. At this moment, the final resolution of our 360° videos is restricted due to the used VR headset (Oculus Go®). It is quite conceivable that the rate of cybersickness could be reduced by using a different VR headset that supports a higher resolution than the Oculus Go®. This has not yet been tested for our applications.

The fact that most participants did not have any experience with a VR headset at all will have influenced our results in multiple ways. On one hand, students reported that the VR experience was new, innovative and fun to experience for the first time. It remains to be studied how they will enjoy and evaluate VR as a tool for learning if they have been repeatedly exposed to VR experiences throughout their study. On the other hand, repeated exposure to VR sessions can help to build tolerance to cybersickness (Rebenitsch and Owen 2020).

This paper contributes to the exploration of 360° video-based virtual reality for medical education beyond the fields such as surgery and anatomy education, and teaching practical skills. It demonstrates that the technique can be used to prepare medical students for their future learning environment of the hospital wards by an immersive experience and learn about professional behavior.

4.1 Limitations

This study was performed in a group of medical students of LUMC in The Netherlands. It cannot be assumed that all findings can be extrapolated to medical students at other institutions, especially if there is a large difference in hospital culture. As the applications contain audio in the Dutch language, future (repeated) experiments with these applications can only be performed in a setting with Dutch-speaking students. Another limitation of this study is that we only used a questionnaire and did not perform in-depth interviews or focus group sessions which could have provided richer data.

During development of the applications, several technical issues were identified and had to be solved. A limitation is that one issue was solved two months after the start of this study, so this could have influenced the results. This resolved issue involved flickering video images: an almost imperceptible flickering of video images caused nausea in the participants. This flickering only occurred in the ‘Virtual Ward’ application, not in the ‘(Un)professional Behavior’ application. The problem was caused by a feature in the SDK of Oculus®. A workaround was created, through which the device was forced to overrule the flickering setting. This was updated in the application in August 2018.

4.2 Lessons learned and future directions

Following this user experience evaluation, some changes were already made to the small group learning session and to one of the VR applications. For example, students are now asked to bring their own earphones to use with the VR headset to minimize distraction by ambient noise.

In addition, we would like to summarize two ‘lessons learned’ from designing and developing the described 360° VR experiences:

  1. 1.

    In the design process, it is of importance to involve a team of relevant stakeholders with different perspectives, including the targeted learners, and use the format of LED sessions.

  2. 2.

    In the technological development, it is needed to carefully consider the desired minimal resolution for the application and explore the resolution of different head-mounted displays in an early phase, before deciding which hard-ware to select.

We believe that the implementation of learning sessions with 360° video based VR in the medical curriculum offers an immersive and realistic learning environment, and is, therefore, suitable for preparing medical students for the transition to the clerkships.

The next step would be to conduct experiments in order to test the effectiveness of the applications in comparison to regular teaching approaches. Also, it will be of interest to study how students evaluate 360° VR once it is not a novel experience anymore.