Keywords

1 Introduction

It is critical to promptly address issues contributing to the attrition of healthcare students and staff members, as we cannot afford to lose additional health professionals. WHO [1] and UN [2] predict international shortages will increase to 9.9 million healthcare workers by 2030. Higher education in healthcare enhances professionals’ competence, which can positively impact staff retention and the overall quality of healthcare services [3]. For example, according to Directive 2013/55/EU [4], total nurses’ education includes up to 50% of clinical practice, which is dysfunctional and ineffective because of the lack of healthcare staff to supervise students [5, 6]. As students cannot access necessary training because of a lack of healthcare supervisors at clinical placements, the pipeline of qualified healthcare professionals is compromised, exacerbating the broader healthcare workforce shortage. Addressing these challenges requires a comprehensive approach that involves increasing the healthcare workforce, enhancing support for educators and mentors, and finding innovative solutions to provide students with the necessary clinical experiences.

Healthcare education needs to be reformed to sustain quality, faster response to crises and ensure a rapid and efficient graduation path for future healthcare professionals. According to previous evidence, simulation-based extended reality (XR) education can partly replace healthcare clinical practice by supporting students’ learning experiences in safe learning environments [7, 8]. Hybrid intelligence (including extended reality (XR) and artificial intelligence (AI)) is seen to be ‘an effective way to augment traditional forms of pedagogy’ [9] and offer new pedagogical methods for training opportunities.

2 Aim of the Study

Human-centred XR will empower end users and continuously involve healthcare students and educators in co-design and decision-making by aiming to respond to the objectives. This will be achieved by developing a) an intuitive extended reality (XR) virtual simulation environment with realistic scenarios, b) user experience analysis and real-time adaptation through a combination of neurophysiological and behavioural data collected by wearable sensors, and c) the creation of convincing digital humans (patients and mentors).

RQ1.: Does a human-centred XR support healthcare students’ learning in clinical conditions?

RQ2.: How do students’ cognitive and neural responses, as measured by fNIRS and EEG, differ during learning tasks in XR simulations compared to traditional learning?

H1: Human-centred XR will support students’ learning in clinical conditions statistically significantly (p ≤ 0.05).

3 Methodology

Two human-centred XR cases have been developed by integrating self-regulated learning [10], immersive technology principles [11] and interprofessional mentor competence framework [12]. The cases contained scenarios with an XR environment: a) discharging a patient by providing instructions and a home care plan, b) assessing a patient in anaphylactic reaction after administering intervenors antibiotics. In the XR technology, we have used game engine integration, e.g., rendering the full-body avatar of patients and nurses, implementing the single-user interaction design, integrating physical props, and the graphical content and scenario-specific sequences to comprise the whole simulation. The cases contained natural communication between real humans and avatar meta-human patients, making interaction seamless (see Picture 1). The development of XR technologies involved user-centred design and co-design by involving students, educators, and health professionals from the beginning until the testing phase. Expert panels have been used to assess the feasibility of the intervention, using design-thinking workshops to understand human needs, re-frame problems, and create ideas through brainstorming sessions and iterative low-cost rapid prototyping (Fig. 1).

Fig. 1.
figure 1

Human-centred XR development

Full size image

3.1 Data Collection and Analysis

The inclusion criteria for the pilot study were i) nursing students participating in English language degree programs at universities of applied sciences, ii) first-year students for cases of discharging a patient, and second/third-year students for cases of assessing patients in anaphylactic reactions. The effect of the human-centred XR environment was assessed using complex multimodal observational, behavioural, and neurophysiological outcome measures. The observational data included questions from validated scales of Nurse Clinical Reasoning (15 items,1–5 Likert scale) [13], Student Satisfaction and Self-Confidence in Learning Scale (13 items, 1–5 Likert scale) [14], and Simulation Design Scale (20 items, 1–5 Likert scale) [15]. Questionnaires were used before the XR and straight after and were analysed using the χ2 test and t-test.

Wearable sensors (functional near-infrared spectroscopy (fNIRS) brain monitoring device and Shimmer’s wristband) were used to collect three types of neurophysiological signals: brain activity (fNIRS), heart activity (ECG), and electrodermal response from the skin (EDA) [16, 17]. A paired t-test examined any significant changes between the baseline and post-test scores.

Initially, students were introduced to the study and asked to respond to the first questionnaire at T0 measurement. After the questionnaire, wearable sensors were attached, and students had to sit calmly without any action for 5 min to take baseline measurements. After the baseline measurements, students were asked to read the Finnish terms included in the XR case for 10 min after completing the XR case, which lasted between 20 and 30 min. After the XR case, students complete a second questionnaire.

3.2 Ethical Issues

Ethical permission has been granted for the pilot study by the Ethics Committee of Human Sciences at the University of Oulu on 25.09.2023. In the study, we ensure data security, accessibility, and validation of results. Data generated includes large datasets of sensitive personal data (biometric data) and user-environment interactions. The privacy of individuals is protected according to GDPR [18] through monitoring and assessment of data processing activities. Participants received clear and comprehensive informed consent forms and could withdraw their consent at any point.

4 Results

In total, 30 subjects participated in the pilot study representing two universities of applied sciences in the Nordic part of Finland. 61% were first-year students, 36%- second year and 3% - third year. Preliminary results have shown that students’ clinical reasoning improved statistically significantly (varying between p = 0.028 to p < 0.001) in all areas but two, focusing on collecting patient information and evaluating patient condition improvement. The highest p-value area reached (p < 0.001) was in identifying and communicating vital information clearly to the doctors based on the patient's current situation. Students evaluated that XR learning has increased their confidence in learning in all areas, mostly reporting enjoying how the instructor taught the simulation (mean 4.5, SD 0.73) and teaching methods used in the simulation were helpful and effective (mean 4.4., SD 0.62). Students evaluated simulation pedagogical design as having the highest instructor’s support in their learning (mean 4.83, SD 0.46) and a clear understanding of the purpose and objectives of their learning (mean 4.50, SD 0.68). Students evaluated the lowest in the simulation, allowing them to analyse their behaviour and actions (mean 3.87, SD 0.86) and the opportunity to reflect with teachers to build their knowledge to another level (mean 3.93, SD 0.69).

Spectral entropy measures the complexity of the signal in the frequency domain; the higher the value, the more complex the signal. Figures 2, 3, and 4 show the response of oxy-, deoxy-, and total haemoglobin also with cerebrospinal fluid (CSF) in three different bands, very low frequency (0.008–0.1 Hz), respiratory (0.1–0.6 Hz), and cardiac (0.6–5 Hz) bands. P1 represents phase 1 and acts as a baseline for the measurement. P2 and P3 indicate reading and XR tasks. As the subjects only had a little knowledge of Finnish, they looked like they were struggling, and the spectral entropy scores during P2 were relatively high. Although the XR task seemed quite demanding in specific scenarios, the overall experience did not induce complexity in the signal.

Fig. 2.
figure 2

Hemodynamic response in the very low-frequency band (0.008–0.1 Hz)

Full size image
Fig. 3.
figure 3

Hemodynamic response in the very respiratory band (0.1–0.6 Hz)

Full size image
Fig. 4.
figure 4

Hemodynamic response in cardiac band (0.6–5 Hz)

Full size image

5 Discussion and Conclusions

In our study, we aimed to develop an intuitive XR virtual simulation environment with realistic scenarios and metahuman avatars, enabling team interaction to test and analyse participants’ real-time adaptation through a combination of neuro-physiological and behavioural data collected by wearable sensors. Until nowadays, formal healthcare training has focused predominantly on developing discipline-based knowledge, clinical expertise, and technical skills in non-adaptive designs [19]. Contemporary healthcare education needs to develop further solutions to enhance students’ and future professionals’ team-oriented competencies, such as non-technical skills, defined as situation awareness, decision-making, communication and teamwork.

Our study has shown that a human-centred XR learning environment already in short usage increased students’ clinical reasoning statistically significantly. Students also evaluated XR and simulation design to support their learning positively by increasing their confidence in competence development. In this paper, we present preliminary data analysis prior to the synchronisation of the data into multimodal data analysis, which will be further used to develop adaptive learning to support students’ self-regulatory learning mechanisms to master their clinical competencies with the support of XR. By developing human-centred XR with adaptive learning and mentoring, the new knowledge can bring breakthrough evidence of novel learning/teaching methods to enhance clinical training in healthcare. Students’ clinical competence can be developed by complementing existing teaching methods, reducing the number of training hours with actual patients while increasing patient safety. The human-centred XR will allow students to practice clinical skills alone and/or with peers from a distance and in different locations, widening opportunities for clinical and interprofessional training in healthcare education.

5.1 Limitations

The study faced several challenges. Firstly, the XR learning environment caused motion sickness for few participants. It has been observed to happen to participants who participated in the study while hungry. Researchers were trained to react sensitively to the physical discomfort of the participants by providing drinks and snacks to the participants in that case and allowing to discontinue the testing if symptoms did not get better. Secondly, XR technology sometimes failed due to a slow internet connection, which disrupted participants’ learning experience. Researchers assisted participants in that case by manually speeding up avatars’ communication. Thirdly, the fNIRS sensors caused some participants discomfort, slightly pressing their forehead due to the XR headsets, which may affect the analysis results.