1 Introduction

Healthcare-associated infections are among the most frequent adverse events affecting patients worldwide (Harbarth et al. 2003; Sax et al. 2020). A significant proportion of these infections may be avoided through the use of preventative measures, such as hand hygiene with alcohol-based hand rub (Allegranzi and Pittet 2009; Harbarth et al. 2002; Rupp et al. 2008). Indeed, the hands of healthcare providers may play a role in the direct and indirect transmission of microorganisms between objects and patients (Clack et al. 2014). When hand hygiene is omitted, contact between hands and surfaces may result in bi-directional exchange of potential pathogens, which may ultimately result in patient contamination or infection. Nevertheless, consistent and reliable performance of hand hygiene by healthcare providers remains a challenge. Given that microorganisms are invisible and infections manifest with latency, healthcare providers lack feedback on missed hand hygiene. One solution, developed by Sax and colleagues (2007), is to support the construction of a mental model of pathogens transmission and hand hygiene with the concept of “five moments for hand hygiene.” This concept has been used in different forms of training, such as scenario-based videos (Longtin et al. 2011; World Health Organisation 2017), dance videos (Hopitaux Universitaires de Genève 2016), and computer-based serious games (Vázquez-Vázquez et al. 2011). Independent of the concept, another hand hygiene training modality is the use of UV-light activated surrogate tracers in role play settings (Alhmidi et al. 2016), which comes with practical challenges (e.g., set up and clean up time). Thus, there are still limitations inherent to currently available training modalities, including restrictive viewpoints (Boucheix et al. 2018) or a low degree of immersion. Decreasing cost and improved performance of virtual reality (VR) technology in the last years, however, have created the unique opportunity to make pathogens visible, and thus, this technology can offer experiential learning for hand hygiene behavior (Nicholson et al. 2006).

Research on VR applications in higher education has shown that the level of representational fidelity and the immersion provided by these technologies are associated with higher student engagement and with better learning outcomes (Luo et al. 2021; Pellas et al. 2021; Wu et al. 2020). This seems especially to be true for VR applications that allow for interaction, experimentation, and exploration within the virtual environment. Medicine has been one of the pioneering disciplines in this regard (Rosen et al. 1996; Satava and Jones 1998). Since the 1990s, VR has been used in the training of healthcare professionals as well as in the education and treatment of patients. Although meta-analyses on the use of VR in medical education have shown promising results, there are substantial variations between the effect sizes that are present in the studies (Kyaw et al. 2019; Zhao et al. 2021). The differences might be explained by the variance of antecedents of participants, design features of VR training applications, properties of the training process, and the choice of outcome measures. In particular, user acceptance, motivation to use, and engagement have been shown to be important factors that need to be considered when explaining the effectiveness of VR technologies in medical education (Aggelidis and Chatzoglou 2009; Barteit et al. 2021; Beke Hen 2019; Holden and Karsh 2010). As only a minority of today’s students are familiar with these technologies, they need to familiarize themselves with the virtual experience before effective performance. This can have an impact on technology acceptance of VR tools, particularly behavioral intention, which might in turn affect engagement and learning (Sagnier et al. 2019; Shen et al. 2019). The interplay of these factors has seldom been analyzed in medical education (Haque and Srinivasan 2006; Piot et al. 2021; Rourke 2020). Further, there seems to be substantial variations in the results among areas of application within medical education. For example, although VR has been proven to be highly effective in fostering conceptual knowledge of anatomy (Zhao et al. 2020) and practical skills in surgery (Alaker et al. 2016; Haque and Srinivasan 2006), this was less observable in nursing education, where VR was used to teach a broader range of conceptual knowledge and practical skills (Chen et al. 2020). The effectiveness of VR applications seems even less clear for non-technical medical skills, such as teamwork and communication (Bracq et al. 2019). The differences between these findings might be attributed to the differences in the type of learning outcomes in these different fields of application. It seems that clearly defined and visually represented types of knowledge and skills are better suited to be taught by VR applications than loosely defined and less visual-focused types. Summing up these findings, the literature seems to indicate that VR is especially suited to teach vision-oriented knowledge and skills to students with adequate technology acceptance. These students are likely to show high levels of engagement that in turn will positively affect learning gains. This is, however, still an open question.

Therefore, this study investigated the following two research questions:

(RQ1) How do preexisting technology acceptance and in-training VR engagement predict hand hygiene performance in a VR environment?

We expect that (H1) learners with higher prior technology acceptance, higher self-reported in-training engagement, and higher initial performance in VR will display a better VR hand hygiene performance at the end of the training session.

(RQ2) Do preexisting technology acceptance, in-training engagement, and performance during VR training affect the development of technology acceptance?

Based on the background research presented, (H2) after the training session, learners with higher prior behavioral intention, higher self-reported in-training VR engagement, and greater performance should report a higher intention to further use VR technologies for learning.

2 Methods

2.1 Study design

We used a single group design for our study. The outcome measures included scores in the first and third levels of the hand hygiene VR trainer (see Materials section for more details) as well as scores on a technology acceptance scale (three components). All data from the questionnaires were collected using Lime Survey (hosted on first author’s University server). This research project involved human beings, but following the authors’ University Ethics Policy, no ethical examination and approval was necessary. A formal waiver was nevertheless obtained. The study was conducted with adults on the basis of informed consent.

2.2 Participants

Following a-prior analysis with G*Power (f2 = 0.2, α = 0.05, power = 0.8, tails: 2), we recruited 43 participants from a medical school class at the University Hospital of Zürich (USZ), Switzerland (21 females, Mage = 23 years and 6 months, SD = 3 years and 9 months). Participants had a high school degree (n = 18), a bachelor (n = 22), or a master (n = 3) and were in their first, second, third, or fifth year of medical school. For 10 participants, the study was part of a patient safety course, and no compensation was offered for completion of the study. The other 33 participants were volunteers who were compensated with CHF 20 vouchers. All participants provided electronic informed consent.

2.3 Materials

2.3.1 Hand hygiene virtual reality trainer

The VR hand hygiene trainer used in this study was developed by Clack and colleagues (2019, 2021) and the setup is a reproduction of a USZ-room hospital environment (Figs. 1 and 2). The physical setup match that of an actual room and trainees can thus navigate within a 4.5 by 10 meters area.

Fig. 1
figure 1

Main: player views of grabbing the stethoscope without (left) and with (right) flora visible; Center: actual setting with the virtual reality equipment, at the same time

Full size image
Fig. 2
figure 2

Main: player view of hands disinfection without (left) and with (right) flora visible; Center: actual setting with the virtual reality equipment, at the same time

Full size image

Before starting the training, the trainees become acquainted with the interactive VR functionalities and equipment (HTC Vive pro helmet and Valve Index controllers) in an abstract VR tutorial environment. The users can practice using the VR headset and handles by performing basic actions (put cubes and balls in a box) and then actions that will later be required in the game (e.g., grasping a stethoscope and listening to a patient’s heart). The three types of pathogens later used in the trainer are also introduced in this tutorial and are distinguished by form and color: flora of the room door handle and one flora per patient. Contact with contaminated surfaces by hands or objects results in their contamination. These hands or objects can then transmit these additional microorganisms to further surfaces. Hands can be decontaminated using one of the hand rub dispensers, but objects cannot be decontaminated.

During the actual training, based on the Four-Component Instructional Design model (Frerejean et al. 2019; van Merriënboer and Kester 2014), trainees are asked to repeat the same four patient care tasks in three levels of increasing difficulty. In each of the training levels, learners are asked to greet the patients, perform cardiac auscultation with a stethoscope, take their body temperature, and write down values and heart condition on a clipboard. Trainees have one trail per level. Check marks appear on a white board as each task is completed. In the easiest first level (level-1), potential pathogens are visible; in the second level (level-2), they are invisible until the level is completed; and in the third (level-3), most challenging level, they remain invisible and the available time is limited to 180 s (Appendix 5.1). If a patient is contaminated with virtual pathogens, he turns pale and starts coughing (“infection event”). Immersion in the virtual environment also includes background noise from a hospital setting, a beep when using a thermometer, listening to a heartbeat, etc.

2.3.2 In-trainer performance

For each level, trainees’ score starts with 100 points and decreases with the following mistakes: missing a hand hygiene opportunity (− 10 points), “infection event” (− 20). Time to complete each level was recorded but only overtime affected performance score, and only on level-3.

, or, in level-3 only, over-running the maximal allowed time of 180 s (− 20). Scores are recorded in all levels, but trainees only see their performance score for the third level after they complete it.

2.3.3 Technology acceptance

Technology acceptance for VR was measured with an adapted version of items from the unified theory of acceptance and use of technology for VR scale developed by Shen and colleagues (2019). Main changes in the wording of the items consisted of replacing “learning” with “education.” This study also focused on the three core components of the original technology acceptance model (Davis 1989): performance expectancy (four items), effort expectancy (four items), and behavioral intention (three items).

2.3.4 User engagement

For user engagement, the short form scale by O’Brien et al. (2018) was used. In this 12-item scale, respondents had to position themselves with regard to: attention focus, perceived usability, aesthetic appeal, and reward factor. According to the original authors, the scale can also be used as an overall index without distinguishing between the sub-factors, which was done in this study.

2.3.5 Introductory video

A video was shot to (1) provide trainees with common background knowledge on hand hygiene in healthcare and (2) give them an overview of how the VR trainer works. The first part of the video on hand hygiene last 6 min and include the presentation of general concepts of hospital hygiene and infection prevention. The second part of the video presents the VR trainer functions and shows someone being equipped with the VR head mounted device and controllers and gives a first glimpse of the virtual environment. It is one minute long.

2.4 Procedure

Before using the VR hand hygiene trainer, participants filled out the questionnaire on technology acceptance. Then, after viewing the introductory videos, medical students were asked to complete the three levels of the trainer. Afterward, participants completed questionnaires on in-training engagement, technology acceptance, and demographics. A visual representation of the procedure, independent and dependent variables is presented in Fig. 3 for hypothesis 1 and Fig. 4 for hypothesis 2. Participants completed the study individually.

Fig. 3
figure 3

Procedure of the study with independent variables (IV) and dependent variables (DV) indication for hypothesis 1

Full size image
Fig. 4
figure 4

Procedure of the study with independent variables (IV) and dependent variables (DV) indication for hypothesis 2

Full size image

2.5 Analyses

The primary outcome measures of our study were the scores on the third level of the trainer, as well as participants’ responses to the post-intervention questionnaire on technology acceptance, specifically the behavioral intention component. After controlling for the questionnaire’s validity and reliability, we determined predictors of performance to level-3 of the trainer with a linear regression analysis. We also used a linear regression analysis to predict intention to use VR for learning in future. All analyses were computed with the statistical software Jamovi for Windows (version 2.0.0) and the underlying R packages.

3 Results

3.1 Measures

3.1.1 User engagement scale

Items for perceived utility were reversed for this analysis.

The user engagement scale short form consisted of 12 items and was found to have a good reliability (α = 0.83, ω = 0.89). The confirmatory factor analysis for the one-dimensional model reached an adequate fit, with five modifications using residual covariances between single items χ2(49) = 58.6, p = 0.163, TLI = 0.951, CFI = 0.963, RMSEA = 0.068, SRMR = 0.077). Modifications concerned items RF1 and RF2, FA1 and RF2, AA2 and RF3, PU2 and PU3, FA1 and AA2 (see appendix for items wording). Two factors (FA1 and PU3) were not significant (p = 0.559 and p = 0.322, respectively).

3.1.2 Technology acceptance sub-scales reliability and validity

Technology acceptance of VR for medical education was measured before and after the VR training session. Performance expectancy consisted of a four-item sub-scale and was found to have good reliability both in the pre- and post-tests (Pre: α = 0.89, ω = 0.89; Post: α = 0.91, ω = 0.91). Effort expectancy entailed a four-item sub-scale that was equally reliable (Pre: α = 0.84, ω = 0.85; Post: α = 0.89, ω = 0.90). Behavioral intention corresponded to a three-item sub-scale and was found to have a good reliability (Pre: α = 0.87, ω = 0.87; Post: α = 0.86, ω = 0.86). Confirmatory factor analysis of the three aspects showed a good overall fit in the pre-test, with five modifications using residual covariances between single items within each sub-scale (χ2(36) = 40.7, p = 0.270, TLI = 0.977, CFI = 0.985, RMSEA = 0.055, SRMR = 0.077). Modifications concerned items BI1 and BI3, BI1 and BI2, PE1 and PE3, PE2 and PE3, PE3 and PE4 (see appendix for items wording). All factors were significant predictors in the model. In the post-test, the confirmatory factor analysis showed an equally good fit with the same modifications (χ2(36) = 47.1, p = 0.102, TLI = 0.954, CFI = 0.970, RMSEA = 0.085, SRMR = 0.061).

3.1.3 Training scores

Scores on level-1 for hand hygiene in the VR trainer (M = 27.1, SD = 28.3) and level-3 (M = 78.3, SD = 19.5) indicated that the VR training resulted in an improvement in virtual hand hygiene procedure (t (41) = − 9.19, p < 0.001).

3.2 Effects of technology acceptance and user in-training engagement on performance score (H1)

To test our first hypothesis, we computed a correlation analysis followed by a linear regression with components of technology acceptance, in-training engagement, and scores in level-1 as predictors of scores in level-3. The correlation analysis is reported in Table 1.

Table 1 Correlations between level-1 performance, prior technology acceptance (three components), user engagement, and level-3 performance
Full size table

The five-item model explained 36.3% of the variance, F (5, 35) = 3.99, p = 0.006 (Fig. 5, Table 2). Although scores in level-1 were not a predictor of scores in level-3, higher engagement during the training session was a highly significant positive predictor. Regarding technology acceptance, effort expectancy was especially a significant negative predictor, suggesting that trainees who expected the VR training to be difficult and cumbersome performed lower on level-3. Prior performance expectancy and prior behavioral intention were not direct predictors. Standardized coefficients showed a medium effect for both significant predictors. Regression diagnostics showed no deviations from the statistical assumptions necessary to perform the analysis (e.g., normal distribution of residuals, avoidance of multicollinearity, homoscedascity, and autocorrelation).

Fig. 5
figure 5

Predictor variables of level-3 performance

Full size image
Table 2 Linear regression: influence of level-1 performance, in-training engagement, and technology acceptance (three components) on level-3 performance
Full size table

3.3 Effects of prior behavioral intention, user in-training engagement, and performance on behavioral intention (H2)

To test our second hypothesis, we ran a correlation analysis followed by a linear regression with prior behavioral intention, user in-training engagement, and scores in level-3 as predictors of post-training behavioral intention. The correlation analysis is reported in Table 3.

Table 3 Correlations between post-training behavioral intention and prior behavioral intention, in-training engagement, and performance in level-3
Full size table

The regression showed that the three-items model explained 74% of the variance, F (3, 38) = 36.2, p < 0.001 (Fig. 6, Table 4). Although scores in level-3 were not a predictor, in-training user engagement and prior behavioral intention were significant positive predictors of post-training behavioral intention. Further, we found no deviations from the statistical prerequisites for linear regressions. (Fig. 6)

Table 4 Linear regression: influence of prior behavioral intention, in-training engagement, and level-3 scores on post-training behavioral intention
Full size table
Fig. 6
figure 6

Predictor variables of post-training behavioral intention

Full size image

4 Discussion

In this exploratory study, we investigated the role of technology acceptance and user engagement when using VR as an alternative approach for training hand hygiene. Our results showed a significant increase in in-trainer hand hygiene performance between a first level in which potential pathogens were made visible and a more challenging third level that had a time pressure and in which microorganisms were invisible. In line with previous studies (Longtin et al. 2011; Sax et al. 2007; World Health Organisation 2017), engagement was high and performance increased when learning with the VR trainer. In particular, we found that successful hand hygiene performance in VR was predicted by trainee’s prior acceptance of this technology as well as their engagement during training. Further, our results showed that intention to use a VR in future was positively influenced by original behavioral intention and in-training engagement. Our first hypothesis (H1) was partially confirmed. Effort expectancy was the only component of technology acceptance predicting performance in VR together with in-training engagement. Lower expected effort and higher in-training engagement were associated with higher performance scores. This result contributes to previous research (Aggelidis and Chatzoglou 2009; Barteit et al. 2021; Beke Hen 2019; Holden and Karsh 2010) as it supports the conclusion that engagement during VR training matters more than the perceived required effort to use VR or trainees’ expected performance in such an environment.

Numerous studies have measured technology acceptance after using a digital learning environment (Cheung and Vogel 2013; Dobricki et al. 2021; Plotzky et al. 2020). The particular contribution of this study is a focus on how learning in a digital environment can impact technology acceptance through a pre-post intervention measure (R2). Our second hypothesis (H2) was that positive changes would be observed in behavioral intention after training (Luo et al. 2021) when in-training engagement and scores in level-3 were high. Our results confirmed this hypothesis, which could indicate that one way to increase technology acceptance and further intention to use a technology may be to actually use the technology.

Our study thus provides novel insights with regard to the effect of technology acceptance and in-training engagement when considering medical training with VR. As a practical implication, our findings suggest that VR training simulations may be effective even if trainees do not have high levels of technology acceptance beforehand.

Nonetheless, several limitations of the study should be addressed. First, our outcome measure focused on performance within the VR trainer. Actual mastery of the hand hygiene procedure would need to be evaluated in a 2- to 3-week delayed post-test in a natural patient care setting. Second, as this is indeed a trainer, a longitudinal study would be needed to determine whether trainees can indeed learn and improve with the use of the VR trainer over time and the number of occurrences needed for effective learning. In this case, more complex clinical scenarios could be included in supplementary levels, along with the time limit in level-3, such as having to use a single stethoscope for both patients, introducing an empty alcohol-based hand rub dispenser, or adding a medical diagnosis or therapeutic challenges. Given that trainees first used the VR environment with a tutorial, we cannot exclude that higher scores in levels may be moderated by increased familiarity with VR. This limitation could be addressed with additional training sessions with the first level prior to presenting trainees with level-2 and level-3. Another question that has not been addressed in this study is that of the effect of the virtual environment on trainees’ cognitive load. Indeed, previous studies showed that design elements in VR could affect germane cognitive load (e.g., Albus et al. 2021), but other found no difference of VR compared to computer-screen training on cognitive load (Wenk et al. 2021). Finally, it is important to underline that the use of VR also comes with practical challenges. In the case of the trainer used in this study, each training field has to be large. While this may decrease risks inherent to current, more stationary VR environments (e.g., smooth movement, teleportation) such as feelings of nausea or disorientation (Zhao et al. 2020), it also requires a lot of available floor space.

Future research could also attempt to replicate our findings with a different VR trainer, such as one targeting training of other medical procedures. Another perspective is the comparison of learning outcomes between training in an actual setting versus in a VR environment, similar to the research by Nicholson and colleagues (2006). In this case, not only learning outcomes but technical and practical considerations could be included, such as the time required to set up the learning environment overall and in between participants. Finally, as observing an expert or a peer performing a medical intervention is common practice (Watling et al. 2012), the effectiveness of avatars as role models in VR could be another question to investigate.

To conclude, the rapid development of VR in the latest decade now allows the development of trainers with high visual immersion (Davis 1989). In this study, the results showed that performance in level-3 is predicted both by in-training engagement and the prior effort expectancy component of technology acceptance. Intention to use VR for learning again in future was also predicted by in-training engagement, together with the prior behavioral intention component of technology acceptance. Thus, while engagement affects both learning and attitude toward VR training, the role of technology acceptance is less clear cut. Longitudinal studies are needed to ascertain the long-term effect of hand hygiene VR training in actual patient care.