The Future of Immersive Technology in Global Surgery Education

Abstract

The second wave of immersive reality technology is required that enhances and exploits current applications, empirical evidence and worldwide interest. If this is successful, low- and middle-income countries will have improved access, less costs and reduced practical limitations. Affordability, availability, accessibility and appropriateness are determinates, and help from several innovative areas can achieve these targets. Artificial intelligence will allow autonomous support of trainees to accelerate their skills when interacting on mobile applications, as deep learning algorithms will generate models that identify data and patterns within them and provide feedback much like a human educator. Future immersive content needs to be high quality, tailored to the learners’ needs and created with minimal time and expenses. The co-creation process involves the integration of learners into the entire development process and a single learning goal can be identified that will have high reusability to surgical students. Sustainability of the material is ensured in the design stage leading to increased cost-effectiveness benefits. One framework has a proven high impact on the co-design of healthcare resources and is discussed. The connectivity of future immersive technology resources has been a major obstacle between regions in their uptake. A handful of collaboration platforms have been created that can deliver immersive content and experiences; the spearhead in this area will be from augmented reality and telesurgery. Opportunity for powerful, large-scale data culture via blockchain collaboration will be an emerging theme that will also drive towards affordability, availability, accessibility and appropriateness in the future global landscape of immersive technology in surgical education.

During the adaption to COVID-19-related restrictions, healthcare professionals worldwide were able to receive educational information through smartphones, computers, tablets, mobiles or computer-powered virtual reality (VR) and augmented reality (AR) headsets. Virtual lectures were rapidly implemented by institutions where possible when physical lectures were cancelled, along with many other uses of technology to minimise educational disruption. Convenience, fidelity, usability and interaction are benefits of technology in pedagogy. Additionally, computer-generated applications were used that hold high fidelity and can be paired with other equipment to include more complex features and realistic interactions. Some examples of products with appropriate empirical evidence are found here [1,2,3,4]. Due to long periods of use, many staff and students in institutions from high-income countries (HIC) have implemented these methods and practices for future consumption regardless of any social restrictions. Low- and middle-income countries (LMIC) may not be able to access similar equipment and online resources due to costs, demand and practical limitations. With recognition from experiences during the COVID-19 pandemic, this paper asserts the future role of immersive technology and suggests efficient methods in design, implementation and development, with a focus on surgical education worldwide.

The dimension of global health is usually labelled as a conceptual framing, a distinct field of practice, and still an evolving science in pursuit of equity. This paradigm is embodied in epidemiological health issues that are generally greater than the capacities of one region. The subcategory of global surgery aims to improve equity for all who need surgical attention, meaning to reduce the systematic disparity between social groups who have different underlying social access levels. Global surgery includes all perioperative services, and its role has been drastically underappreciated in global health [5,6,7]. The issue of technology utilisation in global surgery encompasses the reduced resources available for students. In societies that can accommodate high-level development of resources, the benefits can be absorbed by healthcare and education. However, the affordability, availability, accessibility and appropriateness of technology are determining barriers for regions that cannot facilitate such rapid changes. LIMCs have a large deficit in access to surgical care and educational opportunities [8]. They lack access and resources to maintain innovative and research-driven technology-based interventions for pedagogical use. For immersive technologies, computer-powered VR headsets are high financial burdens due to the memory and graphical processing hardware required, with a worldwide shortage of graphical processing units due to a trend in cryptocurrency mining [9, 10]. Augmented reality headsets will likely continue to be unaffordable, and both LMICs and HICs will likely not use the equipment for mass training.

Therefore, the second wave of immersive reality technology is required that exploits and enhances the current knowledge, stability, empirical evidence and worldwide interest in immersive learning objects. The World Health Organization created the ‘4 As’ of affordability, availability, accessibility and appropriateness in an attempt to improve access, preservation and modernising of healthcare equipment [11], and the same principles are applied for access to educational resources. The upcoming years need to have a focus on delivering innovative tools which meet these criteria. This will be driven by directing the spotlight towards leading institutions to bring about change with LMICs at the forefront of resource development. In the UK, The Royal College of Surgeons investigated which tools could be utilised for surgical services for the next 20 years. Their Future of Surgery [12] commission predicted that inequalities will be reduced through channels such as cheaper robotics, remote support, telesurgery and better online resources for both staff and patients. They further stated the largest immediate impact that can be gained by LMICs is from smartphone usage, as they can fulfil the desired characteristics of affordability, availability, accessibility and appropriateness.

This prediction of uptake is reinforced when considering how prevalent mobile devices will become in LMICs in the coming years. In India, there was a 90% increase in digital adoption levels between 2014 and 2017. India has added more than 240 million smartphone users since 2014 and had a rate of 22.2 phones per 100 individuals in 2017 [13]. The smartphone market in India grew by 26% in the first 4 months of 2021, therefore, increasing access, and the most popular device was the Xiaomi Redmi Note 7 Pro, which supports most virtual reality applications [14]. The rate is predicted to continue until 2025, as internet access grows in parallel. Due to this, the government of India has established supportive legislation for patients’ digital health data [15] as they will progressively use mobile devices to access healthcare services such as telemedicine. Certainly, 7 surgical specialties (orthopaedics, general surgery, otolaryngology, urology, neurosurgery, thoracic surgery and plastic surgery) had published more than double the number of telemedicine research articles in the first 6 months of 2020 than in all of 2019 [16], indicating revamped interest. Therefore, mobile devices will increase in dominance to be an everyday supplementary resource which runs VR and AR applications. They are relatively cheap and low fidelity, but key features such as realistic anatomy, bleeding and real-life videos/images can create immersion and detail, and content can be optimised to support different devices. Google Cardboard and other low-cost devices [17] are well-established and will be the prevailing equipment for VR application usage.

Future immersive technology will continue to improve learning outcomes for surgical education by such areas as integrating areas of artificial intelligence (AI) and deep learning (DL), systematic framework for content creation and better best practice and collaboration. Although there are many current high- and low-fidelity surgical and medical applications [18], they will have stronger capabilities due to this novel synergy. Consequently, we discuss these key factors in the future success in immersive reality to close the gaps in global surgery education. Although it is impossible to foretell their complete benefits, they will allow immersive technology to be widely available, low cost, highly personalised and other desired features. Readers are encouraged to implement these areas when producing the next wave of affordable, available, accessible and appropriate immersive materials for surgical education.

Artificial Intelligence

Although surgical practice with experienced educators facilitates excellent feedback to students, educators are limited in time, and the student-to-teacher ratio will remain low. Simulation tools are used as a substitute, and it is well documented that equipment, such as laparoscopic trainers, offer standardised tasks along with feedback to improve skills if they are measured as suboptimal. Yet VR can replicate these processes without the need for physical equipment and simplify the requirements for students to practice their skills. Furthermore, the synergy of AI can replicate the teacher/educator. Any part of a surgical procedure can be replicated, and the integration of AI can create personalised and high-quality training that can be viewed as a virtual teacher. The advantage of AI with immersive technology is the ability to mimic feedback given by educators. This area is in its infancy, but the authors predict an exponential increase in research for several years.

Such features are possible with the use of deep learning algorithms, such as deep convolutional neural networks. They can be implemented to generate models that identify data and patterns within them. Once a Deep Learning (DL) network is trained, they are highly accurate and such predictive models can be deployed into mobile immersive applications for AR and VR [19]. The features of an application using DL depend upon the data used, therefore, an array of potential features can be made. Recent successes in DL applications using images from radiographs [20], rhinoplasty [21, 22] and photographs [23] suggest future utilisation of AI will include training for artefacts in images and 3D model assessments. This is useful to train students in surgical preparations or problem recognition. DL and computer vision will also be able to highlight differences between the AI and a student and explain decision making that is very similar to their teachers’ explanations (see Borsting et al. [21] for an example). Applications will be analytical tools for performance and track individual progression on a variety of levels [24]. Overall, this precision from AI will autonomously support trainees to improve their skills when interacting on mobile applications. Students will accept an imitation of educators if they are visually represented as a human with positive social characteristics. This is based upon similar research with chatbots which suggests that with the correct design, the same quality of delivery can be achieved as a human agent [25].

Design Learning Goals with a Best Practice Framework

Future immersive resource content needs to be high quality, tailored to the learners’ needs and created with minimal time and expenses. Effective education will fail if development structure, user needs and research-driven processes are not considered. There are insufficient best practice guidelines for immersive reality creation, and no standardised framework exists. Companies who develop learning materials have provided general rules of development but most lack details for developers to follow and few follow a structured development method such as the ASPIRE model [26, 27]. The solution is the use of a methodical co-creation process. A co-creation process involves the integration of learners into the entire development process and encouragement to integrate their knowledge. Learners become the centre in the design of educational resources and experiences, fostering a shift from user-centred design to co-design. In doing this, a single learning goal can be identified that will have high reusability, therefore, availability and accessibility to future surgical students. Sustainability of the immersive reality material is ensured in the design stage leading to increased cost-effectiveness benefits. Different frameworks have been proposed for the co-creation of digital resources, but the ASPIRE framework has a proven high impact on the co-design of web-based healthcare resources addressing a single learning objective [28, 29]. It has the flexibility to be adapted for immersive technology content creation for both LMIC and HIC institutions. ASPIRE stands for Aims, Storyboard, Production, Integration, Release and Evaluate, and it is matched with concrete steps in healthcare resource development [30].

A workshop brings together the stakeholders (learners, educators and technologies) to define the specific aim of the resource based on their needs by identifying their common knowledge on the topic. This can be over the phone, via online group meetings or in-person if possible. This is followed by the design of a resource in a storyboard format. Detailed specifications are created to describe the resource, such as texts, video scripts, and activity interactions. This iterative process encourages review by a content expert, such as a consultant surgeon, in order to ensure the high quality of the content [31]. This streamline process allows the digital resource to be developed without the need for multiple development rounds or multiple inputs from experts. The final resource is reviewed again to ensure its high quality and after piloting it with the learners, it can be released through many distribution channels. ASPIRE has been adjusted to fit resources without compromises on reusability and immersive learning resources, such as with a VR AI patient chatbot [32, 33] and interactive 360 video reusable resources on clinical skills [34]. Effective high-quality resources can be created that can be used on mobile phones following these methods: CoViRR is an ERASMUS + project funded by the European Union currently developing best practices for the creation of low-cost VR/360 interactive videos together with a list of pedagogical recommendations, available in 2022. They are for the use of virtual reality reusable e-resources in educational practice and training educators [35], and anyone can access and follow these recommendations and achieve benefits as previously described.

Capability Through Collaboration

This leads to collaboration; particular consideration should be given to how future applications not only provide one-off benefits to a set of users but also continue to circulate to other institutions for continued usage over several years. Connectivity of resources is a major obstacle between regions in the uptake of immersive technologies and only recently have a handful of collaboration platforms been created that can deliver immersive content and experiences to many users from many locations on different supported devices [36, 37]. These platforms will become ubiquitous, but there is a risk of independence of platforms that reduce access, and an open, connected approach should be adopted. The spearhead in using online platforms will be from AR and telesurgery. They operate well together and have a capacity to transfer expertise across the globe in real time. Surgeons with greater expertise in distant countries can provide ad hoc advice to local teams during specific operations, sharing of knowledge and best practice. Students around the world will be able to view and interact with the surgical team in real time, permitting potentially millions of students per day to have exposure to most surgeries. Live-stream demonstrations will maintain the popularity and use of VR mobile devices. Although students may not be able to see as much detail as they would in-person, 360-degree and 3D videos can immerse the view in the operating theatre and have been shown to improve understanding of events towards error prevention. Indeed, cadaver streaming sessions are available in a growing list of procedures such as otolaryngology dissection webinars [8]. These video and AR hosting platforms will be embedded into operating theatres for the institutions that can accommodate the technology, and it is vital that resources are shared. There are benefits for both the distant learners and the hosts of the material. The authors believe that content platforms will automatically process content, embed data and create learning scenarios for mobile. Powered by AI, it is possible to repurpose recordings of all surgeries into accessible content with minimal costs. The projected uses of the many immersive applications discussed have much data to harness. This lends itself well to powerful, large-scale outputs that can be driven by AI and machine learning to provide more insight to the landscape of global surgery education; and with more collaboration, the greater the research impact. To support this, decentralised storage and applications on the Ethereum blockchain will allow fast and simple connection for patients, data providers and healthcare institutions. Following protection regulations for each region, data can be anonymised and added to the data pool. Start-ups with this premise look promising [38, 39], and the authors implore further investigation on the benefits of a big data culture via a future immersive technology blockchain collaboration.

Conclusions

Clinical practice and intuition are almost impossible to replace. Yet immersive reality has shown its resilience and benefits over the years and more so during the effects of the COVID-19 pandemic. However, caveats have been created that prevent affordable, available, accessible and appropriate immersive materials being created for surgical education around the world. The next wave of development has the opportunity to include exciting and evidence-based components that do not have high resource requirements. The next challenge is how to best incorporate the most advantageous elements of AI techniques, proven co-creation methods and collaborative structures in healthcare curricula. Although well-designed immersive reality and AI will not fulfil training of the wide array of complexities in surgical care that are required for effective outcomes, the impact around the world will change the landscape of surgery and will revitalise interest and aspiration for the next generation of students no matter where they are in the world.