Abstract
At a time of significant global unrest and uncertainty surrounding how the delivery of clinical training will unfold over the coming years, we offer a systematic review, meta-analysis, and bibliometric analysis of global studies showing the crucial role simulation will play in training. Our aim was to determine the types of simulators in use, their effectiveness in improving clinical skills, and whether we have reached a point of global acceptance. A PRISMA-guided global systematic review of the neurosurgical simulators available, a meta-analysis of their effectiveness, and an extended analysis of their progressive scholarly acceptance on studies meeting our inclusion criteria of simulation in neurosurgical education were performed. Improvement in procedural knowledge and technical skills was evaluated. Of the identified 7405 studies, 56 studies met the inclusion criteria, collectively reporting 50 simulator types ranging from cadaveric, low-fidelity, and part-task to virtual reality (VR) simulators. In all, 32 studies were included in the meta-analysis, including 7 randomised controlled trials. A random effects, ratio of means effects measure quantified statistically significant improvement in procedural knowledge by 50.2% (ES 0.502; CI 0.355; 0.649, p < 0.001), technical skill including accuracy by 32.5% (ES 0.325; CI − 0.482; − 0.167, p < 0.001), and speed by 25% (ES − 0.25, CI − 0.399; − 0.107, p < 0.001). The initial number of VR studies (n = 91) was approximately double the number of refining studies (n = 45) indicating it is yet to reach progressive scholarly acceptance. There is strong evidence for a beneficial impact of adopting simulation in the improvement of procedural knowledge and technical skill. We show a growing trend towards the adoption of neurosurgical simulators, although we have not fully gained progressive scholarly acceptance for VR-based simulation technologies in neurosurgical education.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Introduction
Rapid technological advancements have created opportunities to overcome global challenges in neurosurgical training delivery in the twenty-first century [19, 29,30,31, 39, 50, 51, 60, 62, 81,82,83, 97, 98, 101, 106, 110, 111]. Mounting pressures on training time due to consensus-driven working time directives in the USA and Europe have been highlighted as responsible for skills acquisition deficits [21, 22, 26, 41, 56, 57, 65, 71, 73, 82, 91, 109, 126].
Neurosurgery is a high-risk, high-stakes specialty with little margin for error. Standardising the expertise and training of neurosurgeons to ensure the highest quality of care and to minimise patient safety concerns is vital for this growing global specialty [23]. Up to 22.6 million patients suffer from neurological pathologies that warrant the expertise of a neurosurgeon with 13.8 million requiring surgery, but 5 million are unable to undergo the required surgical intervention [23]. There is therefore an argument for a much needed and skilled global neurosurgical workforce. The traditional route to managing this need for education in neurosurgery has been typically craft-based and ad hoc, but more recently, there have been some efforts to derive training through modern educational approaches of simulation [126].
High-fidelity physically immersive simulations have gained widespread adoption in neurosurgical education in recent decades [14, 18, 33, 40, 46, 85, 86]. The use of realistic models designed to closely mimic the clinical situation under scrutiny is gradually supplanting cadaveric methods [6, 37, 38, 123]. In concert, virtual, computer engineered photorealistic and 3D-printed technology for simulation have also seen an accelerated growth in adoption for subspecialty areas of neurosurgical education such as neurovascular aneurysmal surgery—also with increasing levels of fidelity [98].
A growing variety of simulators such as the ImmersiveTouch, VIST, ANGIO Mentor, ROBOSIM, SIMONT, NeuroSIM, 3D printed models as well as mobile, augmented reality (AR), virtual reality (VR), and mixed reality simulator platforms are now available for different neurosurgical subspecialties [11] [76] [84, 117,118,119]. Most have been previously appraised for validity and newer types continue to appear on the market [76] [84, 117,118,119]. Cumulative evidence also supports the development and use of virtual simulators with haptic feedback in neurosurgical training to offer a safe and realistic tactile learning approach [6].
Here, we wanted to identify currently available simulators, evaluate the evidence of their effectiveness, and assess their adoption within the neurosurgical community [102, 103]. By doing so, we review the nature of available simulator varieties with the aim of supporting neurosurgical educators and decision-makers in selecting the best simulation approach for their trainees.
Methods
Search strategy
The objective was to characterise and appraise the literature for outcomes associated with neurosurgical simulation education. The study was registered on PROSPERO (number CRD42019144840). A multiplatform database search was conducted with the terms “Neurosurgery, Simulation and Education” on the OVID platform including the following databases: Books@Ovid (July 19, 2019), Journals@Ovid Full Text (July 19, 2019), CAB Abstracts (1910 to 2019 week 28), Embase (1974 to present), GeoRef (1666 to July week 03 2019), Medline (Ovid MEDLINE Epub Ahead of Print, In-Process and Other Non-Indexed Citations, Ovid MEDLINE Daily and Ovid MEDLINE) (1946 to present), PsycINFO (1806 to present), Zoological Record Archive (1864 to 1977), Zoological Record (2009) (Supplementary Table 1). Additionally, the PubMed platform was searched. An extended literature search for progressive scholarly acceptance used keywords “virtual reality/augmented reality and neurosurgery” in all databases including PUBMED, OVID-MEDLINE, HDAS, and SCOPUS. A PRISMA- and PICOS-guided selection of the imported results onto the Rayyan web platform was screened by two blinded independent resident researchers with expertise in neurosurgical education (JD, SM) (see PRISMA flowchart in Fig. 1) [80]. Included articles were imported into endnote reference manager (Clarivate Analytics Version X.9.3.2). HJM was the tie breaker resolving conflicts that arose during article selection post-screening.
Eligibility criteria
Selected articles satisfying our inclusion criteria reported on primary simulation research, digital simulations including AR, VR, and mobile phone platform-based simulation. Articles were included if they were published in the English language and described simulation-based neurosurgical intervention used in a training setting for the acquisition of procedural knowledge or technical skills. We also included articles that presented patient outcome data following simulation training in neurosurgery, articles looking at cadaveric simulation models in neurosurgery, those describing microsurgical skills, and articles describing machine learning modelling methods in simulating neurosurgery with an educational component. We excluded publications discussing simulation with little or no reference to neurosurgery and education, or that focused exclusively on non-technical skills.
Validity and bias assessment
The Medical Education Research Study Quality Instrument (MERSQI) checklist was used to quantify the validity of studies that reported on neurosurgical simulation education [20, 90]. MERSQI was also used to evaluate study quality and bias. The Cochrane risk of bias tool was additionally applied to RCTs with an assessment of its seven key components [47]. Disagreements regarding quality or bias assessments were resolved through discussion with a senior author (HJM).
Meta-analysis
A meta-analysis was performed for cohort studies and randomized trials that identified improvement in procedural knowledge and technical skills as outcomes achieved using neurosurgical simulation. We used STATA (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.) for random effects modelling. Outcome measures for procedural knowledge were scores on assessments. Outcome measures for technical skills included accuracy; speed (time to task completion and speed of task completion); and other metrics (error, comfort, number of fluoroscopy shots used). For each outcome measure, meta-analyses were performed using all relevant data sources regardless of simulation protocol. A normalised ratio of means (R = [\( \overline{X} \)E-\( \overline{X} \)C]/\( \overline{X} \)C) analysis was adopted because of different outcome scales, where a mean difference or standardised mean difference effect estimate would have been inappropriate. A random effects model using an inverse variance DerSimonian Laird estimator was used for between-study variance with confidence intervals. Study heterogeneity was appraised through the I2 statistic. Significance was set at p < 0.05. The meta-analysis was reported using the QUORUM guidelines [74]. Authors were contacted for missing data. Incomplete sets of parameters were automatically excluded from the models without imputation of missing values.
Progressive scholarly acceptance analysis
The progressive scholarly acceptance (PSA) metric was applied to assess the current stage of acceptance of VR simulation as an educational tool within the global neurosurgery community [102, 103]. In accordance with the original intention of the PSA metric, we defined ‘initial/ refining’ studies as follows: (i) initial studies generally only described simulation with no evidence of its use in training and demonstrated the development of VR simulation models for neurosurgical procedural planning or illustration of neurosurgical anatomy, but no direct evidence of their use for neurosurgical training, (ii) refining studies demonstrate the use of VR simulation models for the training of users in any aspect of neurosurgical practice with an objective or subjective assessment of skill acquisition. PSA is defined as the point in time at which the total number of refining studies exceeds the total number of initial studies, indicating community acceptance of the chosen intervention. The initial time point was defined as the year of publication of the first initial study identified.
Results
Search strategy
We screened 7405 article titles and abstracts, removed 647 duplicates, and excluded 6758. Of the 488 full-texts, 56 studies were included in the final review of which 32 studies including 7 randomised clinical studies were also included in the meta-analysis. The PRISMA flow chart summarises the review parameters and results (Fig. 1). Tables 1, 2, and 3 summarise search methodology and the included studies. Additional detailed data tables are summarised in Supplementary Tables 1and 2.
Systematic review
The selected studies performed from 2011 to 2019 that discuss educational simulation in neurosurgery are summarised in Supplementary Tables 1and 2. Fifty simulator types were identified. Since 2011, simulation studies reported from the USA accounted for 69.6% (n = 39), 7.6 times greater than China and Europe with each accounting for 8.9% (n = 5) of the total selected. Of the percentage of studies selected, 64.3% (n = 36) had a documented focus on cranial neurosurgical simulation methods compared with 25% (n = 14) being focused on spinal procedures. Twenty-five percent of the total selection (n = 56) accounting for a 38.9% (n = 14) of the cranial neurosurgical simulation studies were neurovascularly focused, 3.6% were related to skull-base neurosurgical procedures (n = 2), significantly lower than 16.1% (n = 9) that focused on hydrocephalus-related disorder management. However, pituitary (1.8%, n = 1), functional (1.8%, n = 1), and paediatric spinal surgical (1.8%, n = 1) simulations had the lowest documented representations. For our timeframe of focus (early 2011 till present), we identified a peak in neurosurgical simulation innovation in 2013 with 25% (n = 14) compared with 17.9% in 2018 discussing new devices for various outcome measures in simulation.
Meta-analysis
Our meta-analysis of 32 authored studies followed assumptions considering whether trainees benefitted through simulated skills improvement for training under standardised conditions. Accordingly, we sought to determine whether studies supported bench-to-bedside translation for simulation in clinical neurosurgical settings by augmenting trainee experience which in turn suggests improved patient outcomes. Normalised outlier means exceeding a value of 1 were assumed to have a value close to one to avoid omission and selection bias against these studies.
Procedural Knowledge
Twenty-one studies were included (N = 55 sub-studies). A significant 82.7% improvement in knowledge in all outcome domains was demonstrated (ES 0.827, CI, 0.820–0.833, p = 0.0001). Additionally, a significant improvement of 50% (ES 0.502, CI 0.355; 0.649, p < 0.001) was seen using simulation in the context of objective structured assessments to facilitate procedural knowledge acquisition. The highest effect size estimate with an improvement of up to 99% (ES 0.999, CI 0.997–1.001, p < .0.001) was seen for objective score-based simulation methods for knowledge acquisition (see Fig. 2a–c).
Technical skill
Accuracy
Of the 6 included studies (n = 13 sub-studies), statistical heterogeneity in the ratio of means outcome measure of accuracy was noted (I2, 97.3%). However, a significant improvement of 32.5% (ES 0.325, CI, − 0.482; − 0.167, p < 0.001) accuracy in all learners’ acquisition of procedural skill using simulation was noted (see Fig. 3).
Speed
Ten studies were included in the final meta-analysis (N = 21 sub-studies). A significant effect size was demonstrated for pooled ratio of means assessment of speed (ES 0.427, CI − 0.403; 0.450, p = 0.0006 (see Fig. 4). Additionally, procedural-specific speed improvements measured as a ratio of time to task completion across multiple domains was a factor of 3.95 times faster using various forms of simulation. An absolute effect size estimate of 0.253 (ES − 0.253, CI − 0.399; − 0.107, p < 0.001) is seen.
Other metrics
More objective procedural-related safety parameters included three studies (N = 6 sub-studies) related to measuring the depth of perception, mean distance from target tissue, and consistency of forces applied showing a significant 21.2% improvement (ES − 0.212, CI − 0.307; − 0.116; p < 0.001) in these areas (Supplementary Figure 1).
Seven studies (n = 18 sub-studies) were included in the meta-analysis and showed no significant improvement across the domains of procedure-related comfort (ES − 0.013, CI − 0.411; − 0.386, p = 0.951) and administered fluoroscopy shots (ES − 0.082, CI − 0.179; 0.016, p = 0.102) (Supplementary Figure 2).
The average MERSQI was 11.5 ± 2.2 (SD, 95% confidence level of 0.626). The Cochrane risk of bias assessment is summarised in (Supplementary Figure 3).
Progressive scholarly acceptance
The PSA metric indicates that the use of simulation technologies like VR for neurosurgical simulation has not reached ‘progressive scholarly acceptance’. The initial study by Auer and Auer identified in the literature search was published in 1998, which was set as the initial time point [3]. Over a 20-year period (1998–2018), the number of initial studies (n = 91) was approximately double the number of refining studies (n = 45) (Supplementary Figure 4b). However, the latter decade (2008–2018) demonstrates a surge in the number of refining studies, with consistently increased numbers of studies compared with initial studies between 2013 and 2017 (28 vs 22 studies) (Supplementary Figure 4a).
Discussion
Principal findings
Technological developments have stimulated a growing interest in using simulation for neurosurgical training over the past decade. In this review, 50 simulator subtypes ranging from cadaveric, low-fidelity, and part-task to VR simulators were identified. Collectively, the use of these simulators was associated with a significant 82.7% (ES 0.827, CI, 0.820–0.833, p = 0.0001) improvement in procedural knowledge in all outcome domains. The measurement of technical skills was based on procedure-specific speed improvement as a ratio of time to task completion across multiple domains. This was 3.95 times faster using various forms of simulation (ES − 0.253, CI − 0.399; − 0.107, p < 0.001). Additionally, there was a significant improvement of 32.5% accuracy in all learners’ acquisition of procedural skills using simulation (ES 0.325, CI, − 0.482; − 0.167, p < 0.001). The use of VR studies has increased in recent years. However, our bibliometric analysis suggested we are yet to reach widespread acceptance of VR simulation as an integral part of neurosurgical training. The number of initial studies (n = 91) was approximately double the number of refining studies (n = 45) over a 20-year period, although 2008–2018 showed a surge in the number of refining studies compared with initial studies between 2013 and 2017 (28 vs 22 studies).
Comparison with other studies
Although cadaveric methods of neurosurgical simulation education had long been the gold standard method for neurosurgical education, there has been a slow shift towards digital methods in recent decades [4, 9, 13, 16, 27, 29,30,31, 39, 44, 51, 54, 58, 63, 65,66,67,68, 72, 81, 98, 99, 104, 111, 114, 117, 122]. Our review identified that 5.36% of cadaveric studies were used in association with digital technologies like VR to augment neurosurgical learning [51, 115, 121]. A disadvantage of cadaveric approaches alone is the inability to simulate dynamic pathology or complications such as bleeding or CSF leak. Other concerns regarding cadaveric use include unknown disease vector transfer to trainees, ethico-legal uncertainties, and infrastructural expense for cadaver storage [105]. By contrast, current achievements in graphics technologies have enabled virtual/augmented, mixed reality, and 3D printed technologies for better simulation learning experiences with improved photorealistic fidelity [1, 4, 5, 7,8,9, 12, 13, 15,16,17, 19, 24, 25, 27, 29, 30, 32, 34, 35, 39, 44, 48, 50, 51, 54, 58, 60, 62, 63, 65, 68, 70, 72, 81, 82, 88, 97,98,99,100,101, 104, 106, 111, 114, 116, 117, 122, 124, 125].
From our meta-analysis, we hypothesised whether trainees benefitted through simulated skills improvement for training under standardised conditions. Accordingly, we sought to determine whether studies supported bench-to-bedside translation for simulation in clinical neurosurgical settings by augmenting trainee experience, which in turn improves patient outcomes.
A non-significant improvement in safety through minimising errors associated with procedural-related discomfort was identified, whereas objective procedural-related safety parameters such as depth perception and minimisation of tissue injury showed significant improvement and thus warrant further study. The high degree of heterogeneity is linked to inadequate standardisation and culturally distinct methods of international surgical practices. This in turn influences procedural simulation study design. Moreover, whilst certain institutions may be early adopters with exposure to digital neurosurgical simulation technologies, others may be unable to gain the required exposure due to organisational financial constraints. In fact, various teams have highlighted that the costs of neurosurgical simulation can be high and hence have developed further methods like app-based communication platforms to reduce costs [26, 28, 31, 61, 75, 79, 113]. Study designs for centres with established track records and budgets for neurosurgical simulation differ from centres with financial constraints undertaking an initial study into its usefulness for their trainees. This temporal experiential discrepancy between early and late adopters of digital neurosurgical simulation technologies could limit study designs and bring about design-related heterogeneity.
There is also clinical baseline heterogeneity where within-study participants vary. Some studies had neurosurgical simulation implemented using both students (novices) and graduate doctors (experts and intermediates) at varying levels of their practice, hence different baseline characteristics, some in an assessment capacity [8, 27, 35, 48, 51, 58, 72, 82, 101, 110, 124]. Similar statistical heterogeneity occurred in the ratio of means outcome measure of accuracy.
Evidently, as newer simulation technologies appear and gain traction for use in various subspecialties, results may not generalise across all domains or translate to assessing impacts on most patient clinical outcomes [5]. As most studies have been based in the USA to date, it remains to be seen whether the results are skewed by the cultural norms of a country. It also remains to be assessed whether confounders such as cultural pre-framing of the participants in the simulation process, the problem to be solved, and the length of time required to solve it all influence the cross-border standardisation of neurosurgical simulations. This becomes worthy of exploring in future studies in a standardised environment for policymakers. Moreover, the risk of publication bias is associated with the challenges in blinding, which may also have contributed to sample heterogeneity (see Supplementary Figure 3). Rhodes and colleagues report that up to 37% (95% interval: 0–71%) heterogeneity variance could be explained by trials at high/unclear risk of bias [96]. The average MERSQI was 11.52 ± 2.20 (SD, 95% confidence level of 0.626) suggesting a 25.0% improvement from a decade ago reported by Kirkman and colleagues 9.21 ± 1.95 (SD; range 6–12.5) [56]. The majority of the selected prospective randomised controlled trials were single-blinded trials, as double-blinding to reduce the risk of bias appears technically challenging in simulation trials with only one achieving this [122].
One randomised controlled trial (RCT) looked at VR simulation on patient-reported outcomes of efficacy that also offered a patient-related educational slant on perioperative care delivery and its effect on patient outcome [5]. Considering the paucity of quality improvement initiatives, Bekelis and colleagues performed a randomised clinical trial of patients undergoing cranial and spinal surgical procedures evaluating the use of an immersive pre-operative VR set-up compared with operation-type stratified standard pre-operative experience. Outcomes measured included the Evaluation du Vecu de l'Anesthesie Générale (EVANG) and Amsterdam Preoperative Anxiety and Information scoring systems (APAIS) gauging patient perioperative satisfaction. They reported an improved EVANG and high APAIS score (difference, 29.9; 95% CI, 24.5–35.2) together with lower patient stress scores (VAS; difference of − 41.7; 95% CI, − 33.1 to − 50.2) with patients feeling better prepared (difference, 32.4; 95% CI, 24.9–39.8) for their procedures in the pre-operative period and no association of VR simulation with VAS stress score.
Progressive scholarly acceptance (PSA) which demonstrates an appreciation for how the scientific community accepts emerging technologies in VR-based neurosurgical simulation has not yet been undertaken to the authors’ knowledge [102, 103]. We performed a progressive scholarly acceptance review on VR models in neurosurgery. The PSA results support that compounded initial studies using VR are still exponentially increasing as new types of simulators are frequently being introduced to facilitate and augment neurosurgical education [102, 103]. Evidently, there is a clear divergence between compounded initial studies and refining studies for scholarly acceptance to be reached. On the contrary, the individual publications each year seem to suggest that there is a dual crossover of the initial and refining studies in 2013 and 2017. The year 2013 was identified in our analysis as the modal year for published studies on neurosurgical simulation (Supplementary Figure 4A and B).
PSA analysis suggests that we are yet to reach the point of widespread acceptance of VR simulation as an integral part of neurosurgical training. However, a sustained increase in the annual number of refining studies over the last decade suggests that we will soon see ‘progressive scholarly acceptance’. Whilst the PSA provides an innovative attempt at capturing the difficult concept of community acceptance for a given simulation intervention, it is not without limitations as outlined in its seminal publication [102, 103].
Limitations
Our meta-analysis was conducted on a heterogeneous dataset. Nonetheless, it must be appreciated that studies will have been conducted at different institutions without an internationally standardised methodology, because of various neurosurgical simulations being such early-stage technologies. As this is a rapidly evolving field, extrapolating significant results to clinical practice should be considered with caution. We are at a stage where a global multi-centre randomised controlled crossover study for a single improvement domain for neurosurgical education would be warranted in effectively guiding clinical practice.
The main limitation of our PSA analysis lies with the definition of initial and refining studies, which may be difficult to distinguish at times. We defined initial studies as demonstrating the use of VR simulation models for neurosurgical procedural planning or illustration of neurosurgical anatomy, but not direct evidence of use in neurosurgical training. However, one may argue that this definition is not specific to neurosurgical training given that these VR models could be used for pre-procedural planning alone for fully trained neurosurgeons. This reduced stringency for the inclusion of initial studies may have resulted in a disproportionately greater number of initial studies compared to refining studies. Consequently, this would reduce the likelihood of the PSA metric indicating widespread acceptance of VR simulation for education by the neurosurgical community. Nonetheless, we agreed that this was the optimal definition for initial studies as even the use of VR simulation for pre-procedural planning alone should ultimately culminate in improved performance of fully trained neurosurgeons as well, which is a key element of education and training. Essentially, whilst VR simulation is rapidly gaining attention for its potential role in neurosurgical education, we are in dire need of further studies illustrating objective improvement to further establish its role within the neurosurgical curriculum.
Future directions
An important area that was not fully appraised or discussed by the selected studies involved the psychological aspects of education training delivery using a debriefing process. As the learner reflects during the debrief, it is considered the most important period where enhanced learning experience is achieved. To our knowledge, only a handful of studies in our series made subtle attempts at reporting on the debriefing process, when it comes to neurosurgical simulation education. There is no established consensus on whether the debrief period for digital technological-based simulations designed for neurosurgical training purposes should differ from current methods. Currently, the usual duration for debriefing outside the virtual reality environment (exo-virtual debrief) is 2 to 3x, where x is the duration of time for the simulation activity. With training time constraints and the need for accelerated service delivery, extended debriefing may not be feasible.
Moreover, in attempting to circumvent these potentially negative impacts of the process, will an exo-virtual debrief be a requisite component of the total debrief period in order to manage subjective detachments from reality noted to be linked to post-VR autonomic dysregulation that occurs with inter-individual variability [10, 78]? Consequently, such novel methods could facilitate in-VR-to-reality reorganisation therapy of the senses after simulation experiences and may be effective either conducted together with traditional debriefing methods or alone. It is reasonable to consider if this may also be like the traditional duration (amounting to the 2–3x period) or whether a shortened debrief process should be determined.
Digitisation and automated pooled video analytics of procedures from laypeople are starting to gain recognition and have an advantage of being fast, objective, and cheaper than an expert [49, 59]. Currently, global educators are leveraging internet platform-based technologies to deliver neurosurgical operative education across continents that have little access to technical expertise thereby rapidly bridging the knowledge gap [36, 55, 77, 87].
When it comes to cognitive and social congruences, evidence is linked to distance along the near-peer simulation training spectrum [42]. One study in the neuroanatomical educational environment showed that cognitive and social congruence is influenced by distance along the near-peer teaching spectrum [42], although such phenomena are yet to be fully appraised in VR- and AR-based educational simulation environments.
The impact of artificial intelligence especially neural networks in enabling future objective procedural knowledge and skills analysis as well as for tele-neurosurgery requires further mention [52, 53, 89, 107]. Techniques such as Hidden Markov Models, Support Vector Machines, and other deep learning methods like convolutional neural networks offer tremendous potential for automated feedback directed learning [45, 92,93,94,95, 112]. Further clinical studies will be necessary for face, content and construct validity of these techniques for automated feedback-driven advanced neurosurgical procedural knowledge and skills training. Telesurgery in robotic endonasal surgery extends concepts of procedural practice and training over long distances. Wirz et al. demonstrated that phantom pituitary tumour removal can be performed by a surgeon controlling a robot located approximately 800 km away [108, 120]. In combination with platform procedural tele-mentoring, newer avenues of enhanced procedural feedback training could be delivered.
Conclusions
Operative neurosurgery will continue to benefit from the currently evolving simulation technological revolution for education. Accordingly, there is strong evidence for a beneficial effect of simulation in the improvement of accuracy, time to completion of procedural tasks, and knowledge; however, the size of the effect is yet unclear. We show that areas such as virtual reality in neurosurgical educational practice may not have yet or only partially gained progressive scholarly acceptance. Nonetheless, an understanding of whether other simulation technologies will become completely accepted in practice within the surgical community remains to be fully appreciated as further time-dependent evaluative studies become necessary to reach full progressive scholarly acceptance. Cumulative work will allow the occurrence of progressive scholarly acceptance soon, but robust study designs with consensus standardised metrics will become imperative in order to achieve this.
References
Alaraj A, Luciano CJ, Bailey DP, Elsenousi A, Roitberg BZ, Bernardo A, Banerjee PP, Charbel FT (2015) Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback. Operative. Neurosurgery 11:52–58. https://doi.org/10.1227/NEU.0000000000000583
American Psychological testing (2013) Standards for educational and psychological testing.
Auer LM, Auer DP (1998) Virtual endoscopy for planning and simulation of minimally invasive neurosurgery. Neurosurgery 43:529–537. https://doi.org/10.1097/00006123-199809000-00072
Bairamian D, Liu S, Eftekhar B (2019) Virtual reality angiogram vs 3-dimensional printed angiogram as an educational tool-a comparative study. Neurosurgery 85:E343–E349
Bekelis K, Calnan D, Simmons N, MacKenzie TA, Kakoulides G (2017) Effect of an immersive preoperative virtual reality experience on patient reported outcomes: a randomized controlled trial. Ann Surg 265:1068–1073
Bernardo A (2017) Establishment of next-generation neurosurgery research and training laboratory with integrated human performance monitoring. World Neurosurg 106:991–1000
Boody BS, Rosenthal BD, Jenkins TJ, Patel AA, Savage JW, Hsu WK (2017) The effectiveness of bioskills training for simulated open lumbar laminectomy. Global Spine J 7:794–800
Breimer GE, Haji FA, Bodani V, Cunningham MS, Lopez-Rios A-L, Okrainec A, Drake JM (2017) Simulation-based education for endoscopic third ventriculostomy: a comparison between virtual and physical training models. Oper Neurosurg 13:89–95
Breimer GE, Haji FA, Cinalli G, Hoving EW, Drake JM (2017) Validity evidence for the neuro-endoscopic ventriculostomy assessment tool (NEVAT). Operative. Neurosurgery 13:60–65. https://doi.org/10.1227/NEU.0000000000001158
Bruck S, Watters PA (2011) The factor structure of cybersickness. Displays 32:153–158. https://doi.org/10.1016/j.displa.2011.07.002
Buchanan IA, Min E, Pham MH, et al (2019) Simulation of dural repair in minimally invasive spine surgery with the use of a perfusion-based cadaveric model. Oper Neurosurg (Hagerstown). 17(6):616–621. https://doi.org/10.1093/ons/opz041
Bugdadi A, Sawaya R, Olwi D, Al-Zhrani G, Azarnoush H, Sabbagh AJ, Alsideiri G, Bajunaid K, Alotaibi FE, Winkler-Schwartz A, Del Maestro R (2018) Automaticity of force application during simulated brain tumor resection: testing the fitts and posner model, vol 75. vol 1
Chen S, Pan Z, Wu Y, Gu Z, Li M, Liang Z, Zhu H, Yao Y, Shui W, Shen Z, Zhao J, Pan H (2017) The role of three-dimensional printed models of skull in anatomy education: a randomized controlled trail, vol 7. vol 1
Cheng H, Podolsky DJ, Fisher DM, Wong KW, Lorenz HP, Khosla RK, Drake JM, Forrest CR (2018) Teaching palatoplasty using a high-fidelity cleft palate simulator. Plast Reconstr Surg 141:91e–98e
Chitale R, Ghobrial GM, Lobel D, Harrop J (2013) Simulated lumbar minimally invasive surgery educational model with didactic and technical components. Neurosurgery 73:107–110
Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R (2013) Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 80:e9–e19
Chugh AJ, Pace JR, Singer J, Tatsuoka C, Hoffer A, Selman WR, Bambakidis NC (2017) Use of a surgical rehearsal platform and improvement in aneurysm clipping measures: results of a prospective, randomized trial. J Neurosurg 126:838–844
Cikla U, Sahin B, Hanalioglu S, Ahmed AS, Niemann D, Baskaya MK (2018) A novel, low-cost, reusable, high-fidelity neurosurgical training simulator for cerebrovascular bypass surgery. J Neurosurg 1-9
Clarke DB, Kureshi N, Hong M, Sadeghi M, D'Arcy RCN (2016) Simulation-based training for burr hole surgery instrument recognition. BMC Med Educ 16:153
Cook DA, Reed DA (2015) Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale-Education. Acad Med 90:1067–1076. https://doi.org/10.1097/acm.0000000000000786
Craven C, Baxter D, Cooke M, Carline L, Alberti SJMM, Beard J, Murphy M (2014) Development of a modelled anatomical replica for training young neurosurgeons. Br J Neurosurg 28:707–712
Das S, Mitchell P (2013) Comparison of three aids for teaching lumbar surgical anatomy, vol 27. vol 4
Dewan MC, Rattani A, Fieggen G, Arraez MA, Servadei F, Boop FA, Johnson WD, Warf BC, Park KB, We would like to thank the following individuals for their d, contribution to identifying the global neurosurgical deficit. Collaborators are listed in alphabetical o (2018) Global neurosurgery: the current capacity and deficit in the provision of essential neurosurgical care. Executive Summary of the Global Neurosurgery Initiative at the Program in Global Surgery and Social Change. J Neurosurg 130:1–10. https://doi.org/10.3171/2017.11.JNS171500
El Ahmadieh TY, Aoun SG, El Tecle NE, Nanney AD, Daou MR, Harrop J, Batjer HH, Bendok BR (2013) A didactic and hands-on module enhances resident microsurgical knowledge and technical skill. Neurosurgery 73:51–56
Fargen KM, Siddiqui AH, Veznedaroglu E, Turner RD, Ringer AJ, Mocco J (2012) Simulator based angiography education in neurosurgery: results of a pilot educational program. J Neurointervent Surg 4:438–441
Ferguson D, Agyemang K, Barrett C, Mathieson C (2019) A low cost dural closure simulation model for tomorrow's spinal neurosurgeons. Br J Neurosurg 33:337–340
Ganju A, Aoun SG, Daou MR, El Ahmadieh TY, Chang A, Wang L, Batjer HH, Bendok BR (2013) The role of simulation in neurosurgical education: a survey of 99 United States neurosurgery program directors. World Neurosurg 80:e1–e8
Gasco J, Holbrook TJ, Patel A, Smith A, Paulson D, Muns A, Desai S, Moisi M, Kuo Y-F, Macdonald B, Ortega-Barnett J, Patterson JT (2013) Neurosurgery simulation in residency training: feasibility, cost, and educational benefit, vol 73
Gasco J, Patel A, Luciano C, Holbrook T, Ortega-Barnett J, Kuo Y-F, Rizzi S, Kania P, Banerjee P, Roitberg BZ (2013) A novel virtual reality simulation for hemostasis in a brain surgical cavity: perceived utility for visuomotor skills in current and aspiring neurosurgery residents. World Neurosurg 80:732–737
Gasco J, Patel A, Ortega-Barnett J, Branch D, Desai S, Kuo YF, Luciano C, Rizzi S, Kania P, Matuyauskas M, Banerjee P, Roitberg BZ (2014) Virtual reality spine surgery simulation: an empirical study of its usefulness. Neurol Res 36:968–973
Gasco JH, Thomas; Patel, Achal; , Smith A, Paulson D, Muns A, Desai S, Moisi M, Kuo Y-F, Macdonald BO-B, Juan; , Patterson J (2013) Neurosurgery simulation in residency training: Feasibility, cost, and educational benefit. Neurosurgery 73:S39-S45
Gelinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery: integrating theory into practice. Neurosurgery 73:S30–S38. https://doi.org/10.1227/NEU.0000000000000115
Ghanem A, Podolsky DJ, Fisher DM, Wong Riff KW, Myers S, Drake JM, Forrest CR (2019) Economy of hand motion during cleft palate surgery using a high-fidelity cleft palate simulator. Cleft Palate-Craniofac J 56:432–437
Ghobrial GM, Anderson PA, Chitale R, Campbell PG, Lobel DA, Harrop J (2013) Simulated spinal cerebrospinal fluid leak repair: an educational model with didactic and technical components. Neurosurgery 73:111–115
Ghobrial GM, Balsara K, Maulucci CM, Resnick DK, Selden NR, Sharan AD, Harrop JS (2015) Simulation training curricula for neurosurgical residents: cervical foraminotomy and durotomy repair modules. World Neurosurg 84:751–755.e751-757
Giacobbi P, Cushing P, Popa A, Haggerty T, Hansell A, Sedney C (2018) Mobile Health (mHealth) Use or Non-Use by Residents of West Virginia. South Med J 111:625–627
Giovani A, Sandu AM, Petrescu G, Gorgan RM, Goel A (2019) Application of microanastomosis techniques in vascular neurosurgery training and innovation of future surgical strategies for giant aneurysms. World Neurosurg 122:e1120–e1127
Gmeiner M, Dirnberger J, Fenz W, Gollwitzer M, Wurm G, Trenkler J, Gruber A (2018) Virtual cerebral aneurysm clipping with real-time haptic force feedback in neurosurgical education. World Neurosurg 112:e313–e323
Grillo FW, Souza VH, Matsuda RH, Rondinoni C, Pavan TZ, Baffa O, Machado HR, Carneiro AAO (2018) Patient-specific neurosurgical phantom: assessment of visual quality, accuracy, and scaling effects. 3D Printing Med 4:3
Hajat Z, Dinsmore M, Venkatraghavan L (2019) High-fidelity training model for measurement of dynamic optic nerve sheath diameter using transorbital ultrasonography. J Neurosurg Anesthesiol
Haji F, Dubrowski A, Drake J, De Ribaupierre S (2012) Reducing surgical errors in endoscopic third ventriculostomy through simulation training: a Canadian perspective. Child’s Nervous System Conference:40th Annual Meeting of the International Society for Pediatric Neurosurgery. Sydney, NSW Australia. Conference Publication: (var.pagings). 28 (49) (pp 1623). doi:https://doi.org/10.1007/s00381-012-1849-y
Hall S, Stephens J, Andrade T, Davids J, Powell M, Border S (2014) Perceptions of junior doctors and undergraduate medical students as anatomy teachers: investigating distance along the near-peer teaching spectrum. Anat Sci Educ 7:242–247. https://doi.org/10.1002/ase.1419
Hanrahan J, Sideris M, Tsitsopoulos PP, Bimpis A, Pasha T, Whitfield PC, Papalois AE (2018) Increasing motivation and engagement in neurosurgery for medical students through practical simulation-based learning. Ann Med Surg 34:75–79
Harrop J, Rezai AR, Hoh DJ, Ghobrial GM, Sharan A (2013) Neurosurgical training with a novel cervical spine simulator: posterior foraminotomy and laminectomy. Neurosurgery 73:94–99
He K, Gkioxari G, Dollár P, Girshick R Mask R-CNN. In, 2017. pp 2961-2969
Hermiz J, Rogers N, Kaestner E, Ganji M, Cleary DR, Carter BS, Barba D, Dayeh SA, Halgren E, Gilja V (2018) Sub-millimeter ECoG pitch in human enables higher fidelity cognitive neural state estimation. NeuroImage 176:454–464
Higgins JPET, Green, S. (eds)., (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. .
Holloway T, Lorsch ZS, Chary MA, Sobotka S, Moore MM, Costa AB, Del Maestro RF, Bederson J (2015) Operator experience determines performance in a simulated computer-based brain tumor resection task. Int J Comput Assist Radiol Surg 10:1853–1862. https://doi.org/10.1007/s11548-015-1160-y
Holst D, Kowalewski TM, White LW, Brand TC, Harper JD, Sorensen MD, Truong M, Simpson K, Tanaka A, Smith R, Lendvay TS (2015) Crowd-sourced assessment of technical skills: differentiating animate surgical skill through the wisdom of crowds. J Endourol 29:1183–1188. https://doi.org/10.1089/end.2015.0104
Hooten KG, Lister JR, Lombard G, Lizdas DE, Lampotang S, Rajon DA, Bova F, Murad GJA (2014) Mixed reality ventriculostomy simulation: experience in neurosurgical residency. Neurosurgery 10:576–581 discussion 581
Hou Y, Lin Y, Shi J, Chen H, Yuan W (2018) Effectiveness of the Thoracic Pedicle Screw Placement Using the Virtual Surgical Training System: A Cadaver Study. Oper Neurosurg 15:677–685
Implementing Precision Medicine and Artificial Intelligence in Plastic Surgery: Concepts and Future Prospects (2019). Plastic and Reconstructive Surgery Global Open March 7
An introduction and overview of machine learning in neurosurgical care (2018). Acta Neurochirurgica 160:29-38. doi:https://doi.org/10.1007/s00701-017-3385-8
Jabbour P, Chalouhi N (2013) Simulation-based neurosurgical training for the presigmoid approach with a physical model. Neurosurgery 73:81–84
Kessel KA, Vogel MM, Kessel C, Bier H, Biedermann T, Friess H, Herschbach P, von Eisenhart-Rothe R, Meyer B, Kiechle M, Keller U, Peschel C, Schmid RM, Combs SE (2017) Mobile health in oncology: a patient survey about app-assisted cancer care. JMIR mHealth and uHealth 5:e81
Kirkman MA, Ahmed M, Albert AF, Wilson MH, Nandi D, Sevdalis N (2014) The use of simulation in neurosurgical education and training. A systematic review. J Neurosurg 121:228–246. https://doi.org/10.3171/2014.5.JNS131766
Kirkman MA, Muirhead W, Sevdalis N, Nandi D (2015) Simulated ventriculostomy training with conventional neuronavigational equipment used clinically in the operating room: prospective validation study. J Surg Educ 72:704–716
Kshettry VR, Mullin JP, Schlenk R, Recinos PF, Benzel EC (2014) The role of laboratory dissection training in neurosurgical residency: results of a national survey. World Neurosurg 82:554–559
Lau CY, Greysen SR, Mistry RI, Han SJ, Mummaneni PV, Berger MS (2012) Creating a culture of safety within operative neurosurgery: the design and implementation of a perioperative safety video. Neurosurg Focus 33:E3. https://doi.org/10.3171/2012.9.FOCUS12244
Li Z, Li Z, Xu R, Li M, Li J, Liu Y, Sui D, Zhang W, Chen Z (2015) Three-dimensional printing models improve understanding of spinal fracture--a randomized controlled study in China. Sci Rep 5:11570
Lorias-Espinoza D, Carranza VG, de León FC-P, Escamirosa FP, Martinez AM (2016) A low-cost, passive navigation training system for image-guided spinal intervention. World Neurosurg 95:322–328
Luciano CJ, Banerjee PP, Sorenson JM, Foley KT, Ansari SA, Rizzi S, Germanwala AV, Kranzler L, Chittiboina P, Roitberg BZ (2013) Percutaneous spinal fixation simulation with virtual reality and haptics. Neurosurgery 72:A89–A96. https://doi.org/10.1227/NEU.0b013e3182750a8d
Luciano CJ, Banerjee, Pat P, Bellotte B, Lemole MG Jr, Oh M, Charbel FT, Roitberg B (2011) Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery 69. https://doi.org/10.1227/NEU
Marcus H, Hughes-Hallett A, Pratt P, Yang G, Darzi A, Nandi D (2014) Validation of martyn to simulate the keyhole supraorbital subfrontal approach. RCS Bull 96:120–121. https://doi.org/10.1308/003588414X1381402167869535327
Marcus H, Vakharia V, Kirkman MA, Murphy M, Nandi D (2013) Practice makes perfect? The role of simulation-based deliberate practice and script-based mental rehearsal in the acquisition and maintenance of operative neurosurgical skills. Neurosurgery 72:124–130
Marcus HJ, Darzi A, Nandi D (2013) Surgical simulation to evaluate surgical innovation: preclinical studies with MARTYN. Bull Royal College Surg England 95:299–299. https://doi.org/10.1308/147363513x13690603817986
Marcus HJ, Payne CJ, Kailaya-Vasa A, Griffiths S, Clark J, Yang GZ, Darzi A, Nandi D (2016) A “smart” force-limiting instrument for microsurgery: laboratory and in vivo validation. PLoS One 11:e0162232. https://doi.org/10.1371/journal.pone.0162232
Marcus HJ, Pratt P, Hughes-Hallett A, Cundy TP, Marcus AP, Yang GZ, Darzi A, Nandi D (2015) Comparative effectiveness and safety of image guidance systems in neurosurgery: a preclinical randomized study. J Neurosurg 123:307–313. https://doi.org/10.3171/2014.10.JNS141662
Marcus HJ, Seneci CA, Payne CJ, Nandi D, Darzi A, Yang G-Z (2014) Robotics in keyhole transcranial endoscope-assisted microsurgery: a critical review of existing systems and proposed specifications for new robotic platforms. Neurosurgery 10:84–95. https://doi.org/10.1227/NEU.0000000000000123
Mashiko T, Oguma H, Konno T, Gomi A, Yamaguchi T, Nagayama R, Sato M, Iwase R, Kawai K (2018) Training of intra-axial brain tumor resection using a self-made simple device with agar and gelatin. World Neurosurg 109:e298–e304
Mason KA, Theodorakopoulou E, Pafitanis G, Ghanem AM, Myers SR (2016) Twelve tips for postgraduate or undergraduate medics building a basic microsurgery simulation training course. Med Teach 38:872–878
Mattei TA, Frank C, Bailey J, Lesle E, Macuk A, Lesniak M, Patel A, Morris MJ, Nair K, Lin JJ (2013) Design of a synthetic simulator for pediatric lumbar spine pathologies. J Neurosurg Pediatr 12:192–201
Micko A, Knopp K, Knosp E, Wolfsberger S (2017) Microsurgical performance after sleep interruption: a neurotouch simulator study. World Neurosurg 106:92–101
Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 354:1896–1900. https://doi.org/10.1016/s0140-6736(99)04149-5
Moran D, Shrime MG, Nang S, Vycheth I, Vuthy D, Hong R, Padula WV, Park KB (2017) Cost-Effectiveness of Craniotomy for Epidural Hematomas at a Major Government Hospital in Cambodia. World J Surg 41:2215–2223. https://doi.org/10.1007/s00268-017-4022-7
Morone PJ, Bekelis K, Root BK, Singer RJ (2017) Development and validation of a mobile device-based external ventricular drain simulator. Oper Neurosurg 13:603–608
Nakhla J, Kobets A, De la Garza Ramos R, Haranhalli N, Gelfand Y, Ammar A, Echt M, Scoco A, Kinon M, Yassari R (2017) Use of Google Glass to enhance surgical education of neurosurgery residents: “proof-of-concept” study. vol 98
Ohyama S, Nishiike S, Watanabe H, Matsuoka K, Akizuki H, Takeda N, Harada T (2007) Autonomic responses during motion sickness induced by virtual reality. Auris Nasus Larynx 34:303–306. https://doi.org/10.1016/j.anl.2007.01.002
Olijnyk LD, Patel K, Brandão MR, de Morais ANL, de Carvalho RF, Severino AG, da Silva CE, Stefani MA (2019) The role of low-cost microsurgical training models and the experience with exercises based on a bovine heart. World Neurosurg
Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A (2016) Rayyan — a web and mobile app for systematic reviews. System Rev 5:210. https://doi.org/10.1186/s13643-016-0384-4
Patel A, Koshy N, Ortega-Barnett J, Chan HC, Kuo Y-F, Luciano C, Rizzi S, Matulyauskas M, Kania P, Banerjee P, Gasco J (2014) Neurosurgical tactile discrimination training with haptic-based virtual reality simulation. Neurol Res 36:1035–1039
Perin A, Galbiati TF, Gambatesa E, Ayadi R, Orena EF, Cuomo V, Riker NI, Falsitta LV, Schembari S, Rizzo S, European Neurosurgery Simulation Study Group, Luciano C, Cappabianca P, Meling TR, Schaller K, Dimeco F (2018) Filling the gap between the OR and virtual simulation: a European study on a basic neurosurgical procedure. Acta Neurochir 160:2087–2097
Perin A, Prada FU, Moraldo M, Schiappacasse A, Galbiati TF, Gambatesa E, d'Orio P, Riker NI, Basso C, Santoro M, Meling TR, Schaller K, DiMeco F (2018) USim: a new device and app for case-specific, intraoperative ultrasound simulation and rehearsal in neurosurgery. A Preliminary Study. Oper Neurosurg 14:572–578
Ploch CC, Mansi CSSA, Jayamohan J, Kuhl E (2016) Using 3D printing to create personalized brain models for neurosurgical training and preoperative planning, vol 90
Podolsky DJ, Fisher DM, Wong KW, Looi T, Drake JM, Forrest CR (2017) Evaluation and implementation of a high-fidelity cleft palate simulator, vol 139. vol 1
Podolsky DJ, Wong Riff KW, Drake JM, Forrest CR, Fisher DM (2018) A high fidelity cleft lip simulator. Plastic Reconstruct Surg Global Open 6:e1871
Qureshi MM, Piquer J, Young PH (2013) Mobile endoscopy: a treatment and training model for childhood hydrocephalus. World Neurosurg 79:S24.e21-24
Ray WZ, Ganju A, Harrop JS, Hoh DJ (2013) Developing an anterior cervical diskectomy and fusion simulator for neurosurgical resident training. Neurosurgery 73:100–106
Ready or Not, Here We Go: Decision-making strategies from artificial intelligence based on deep neural networks (2016). Neurosurgery June 78:N11-N12
Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM (2007) Association between funding and quality of published medical education research. Jama 298:1002–1009. https://doi.org/10.1001/jama.298.9.1002
Regelsberger J, Eicker S, Siasios I, Hänggi D, Kirsch M, Horn P, Winkler P, Signoretti S, Fountas K, Dufour H, Barcia JA, Sakowitz O, Westermaier T, Sabel M, Heese O (2015) In vivo porcine training model for cranial neurosurgery. Neurosurg Rev 38:157–163 discussion 163
Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Studies Health Technol Inform 132:396–401
Reiley CE, Lin HC, Yuh DD, Hager GD (2011) Review of methods for objective surgical skill evaluation. Surg Endosc 25:356–366. https://doi.org/10.1007/s00464-010-1190-z
Reiley CE, Plaku E, Hager GD (2010) Motion generation of robotic surgical tasks: Learning from expert demonstrations. 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC'10:967-970. doi:https://doi.org/10.1109/IEMBS.2010.5627594
Ren S, He K, Girshick R, Sun J Faster r-cnn: towards real-time object detection with region proposal networks. In, 2015. pp 91-99
Rhodes KM, Turner RM, Savović J, Jones HE, Mawdsley D, Higgins JPT (2018) Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics. J Clin Epidemiol 95:45–54. https://doi.org/10.1016/j.jclinepi.2017.11.025
Rooney DM, Tai BL, Sagher O, Shih AJ, Wilkinson DA, Savastano LE (2016) Simulator and 2 tools: Validation of performance measures from a novel neurosurgery simulation model using the current Standards framework. Surgery 160:571–579
Ryan JR, Almefty KK, Nakaji P, Frakes DH (2016) Cerebral aneurysm clipping surgery simulation using patient-specific 3D printing and silicone casting. World Neurosurg 88:175–181
Santangelo G, Mix D, Ghazi A, Stoner M, Vates GE, Stone JJ (2018) Development of a whole-task simulator for carotid endarterectomy. Oper Neurosurg (Hagerstown) 14:697–704. https://doi.org/10.1093/ons/opx209
Sawaya R, Bugdadi A, Azarnoush H, Winkler-Schwartz A, Alotaibi FE, Bajunaid K, AlZhrani GA, Alsideiri G, Sabbagh AJ, Del Maestro RF (2018) Virtual reality tumor resection: the force pyramid approach. Oper Neurosurg 14:686–696
Schirmer CM, Elder JB, Roitberg B, Lobel DA (2013) Virtual reality-based simulation training for ventriculostomy: an evidence-based approach. Neurosurgery 73:66–73
Schnurman Z, Kondziolka D (2016) Evaluating innovation. Part 1: The concept of progressive scholarly acceptance. J Neurosurg 124:207–211. https://doi.org/10.3171/2015.1.JNS142661
Schnurman Z, Kondziolka D (2016) Evaluating innovation. Part 2: development in neurosurgery. J Neurosurg 124:212–223. https://doi.org/10.3171/2015.1.JNS142664
Shah KJ, Peterson JC, Beahm DD, Camarata PJ, Chamoun RB (2016) Three-dimensional printed model used to teach skull base anatomy through a transsphenoidal approach for neurosurgery residents. Oper Neurosurg 12:326–329
Shaikh ST (2015) Cadaver dissection in anatomy: the ethical aspect. Anat Physiol: CurrRes 2015 5:S5. https://doi.org/10.4172/2161-0940.S5-007
Shakur SF, Luciano CJ, Kania P, Roitberg BZ, Banerjee PP, Slavin KV, Sorenson J, Charbel FT, Alaraj A (2015) Usefulness of a virtual reality percutaneous trigeminal rhizotomy simulator in neurosurgical training. Neurosurgery 11:420–425 discussion 425
Silva MA, Patel J, Kavouridis V, Gallerani T, Beers A, Chang K, Hoebel KV, Brown J, See AP, Gormley WB, Aziz-Sultan MA, Kalpathy-Cramer J, Arnaout O, Patel NJ (2019) Machine learning models can detect aneurysm rupture and identify clinical features associated with rupture. World Neurosurg
Snyderman CH, Gardner PA, Lanisnik B, Ravnik J (2016) Surgical telementoring: a new model for surgical training, vol 126. vol 6.
Stienen MN, Netuka D, Demetriades AK, Ringel F, Gautschi OP, Gempt J, Kuhlen D, Schaller K (2016) Neurosurgical resident education in Europe-results of a multinational survey. Acta Neurochir 158:3–15. https://doi.org/10.1007/s00701-015-2632-0
Sun C, Qi X (2018) Evaluation of problem- and simulator-based learning in lumbar puncture in adult neurology residency training, vol 109
Sundar SJ, Healy AT, Kshettry VR, Mroz TE, Schlenk R, Benzel EC (2016) A pilot study of the utility of a laboratory-based spinal fixation training program for neurosurgical residents. J Neurosurg Spine 24:850–856
Tao L, Elhamifar E, Khudanpur S, Hager GD, Vidal R (2012) Sparse hidden Markov models for surgical gesture classification and skill evaluation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 7330 LNCS:167-177. doi:https://doi.org/10.1007/978-3-642-30618-1_17
Thapa A, Kc B, Shakya B (2016) Cost Effective Use of Free-to-Use Apps in Neurosurgery (FAN) in developing countries: from clinical decision making to educational courses, strengthening health care delivery. World Neurosurg 95:270–275
Thawani JP, Ramayya AG, Abdullah KG, Hudgins E, Vaughan K, Piazza M, Madsen PJ, Buch V, Sean Grady M (2016) Resident simulation training in endoscopic endonasal surgery utilizing haptic feedback technology, vol 34
Tomlinson JE, Yiasemidou M, Watts AL, Roberts DJH, Timothy J (2016) Cadaveric spinal surgery simulation: a comparison of cadaver types. Global Spine J 6:357–361
Volovici V, Dammers R, Lawton MT, Dirven CMF, Ketelaar T, Lanzino G, Zamfirescu DG (2019) The flower petal training system in microsurgery: validation of a training model using a randomized controlled trial. Ann Plast Surg 83:697–701
Wang J-L, Yuan Z-G, Qian G-L, Bao W-Q, Jin G-L (2018) 3D printing of intracranial aneurysm based on intracranial digital subtraction angiography and its clinical application. Medicine 97:e11103
Waran V, Menon R, Pancharatnam D, Rathinam AK, Balakrishnan YK, Tung TS, Raman R, Prepageran N, Chandran H, Rahman ZAA (2012) The creation and verification of cranial models using three-dimensional rapid prototyping technology in field of transnasal sphenoid endoscopy. Am J Rhinol Allergy 26:e132–e136
Wen G, Cong Z, Liu K, Tang C, Zhong C, Li L, Dai X, Ma C (2016) A practical 3D printed simulator for endoscopic endonasal transsphenoidal surgery to improve basic operational skills. Childs Nerv Syst 32:1109–1116
Wirz R, Torres LG, Swaney PJ, Gilbert HB, Alterovitz R, Webster RJI, Weaver KD, Russell PTI (2015) An experimental feasibility study on robotic endonasal telesurgery. Neurosurgery 76:479–484
Wong D, Unger B, Kraut J, Pisa J, Rhodes C, Hochman JB (2014) Comparison of cadaveric and isomorphic virtual haptic simulation in temporal bone training, vol 43
Xin B, Chen G, Wang Y, Bai G, Gao X, Chu J, Xiao J, Liu T (2018) The efficacy of immersive virtual reality surgical simulator training for pedicle screw placement: a randomized double-blind controlled trial. World Neurosurg
Yi Z, He B, Liu Y, Huang S, Hong W (2019) Development and evaluation of a craniocerebral model with tactile-realistic feature and intracranial pressure for neurosurgical training. J Neurointervent Surg
Yudkowsky R, Luciano C, Banerjee P, Schwartz A, Alaraj A, Lemole GM Jr, Charbel F, Smith K, Rizzi S, Byrne R, Bendok B, Frim D (2013) Practice on an augmented reality/haptic simulator and library of virtual brains improves residents’ ability to perform a ventriculostomy. Simul Healthc 8:25–31. https://doi.org/10.1097/SIH.0b013e3182662c69
Zammar SG, El Tecle NE, El Ahmadieh TY, Adelson PD, Veznedaroglu E, Surdell DL, Harrop JS, Benes V, Rezai AR, Resnick DK, Bendok BR (2015) Impact of a vascular neurosurgery simulation-based course on cognitive knowledge and technical skills in European neurosurgical trainees. World Neurosurg 84:197–201
Zhang L, Kamaly I, Luthra P, Whitfield P (2016) Simulation in neurosurgical training: a blueprint and national approach to implementation for initial years trainees. Br J Neurosurg 30:577–581
Authorship
All authors made substantial contributions to all of the following: (1) the conception and design of the study, acquisition of data, analysis and interpretation of data, (2) drafting the article and revising it critically for important intellectual content, and (3) final approval of the version to be submitted.
JD was responsible for conception and design of the study, acquisition of data, meta-analysis with R and STATA and results interpretation, and drafting and revision of the manuscript.
SM was responsible for study design, data collection and analysis and manuscript writing, and final approval of the version to be submitted.
SG assisted with data collection, analysis and manuscript writing, and final approval of the version to be submitted.
HA was responsible for study design, data collection and meta-analysis with R and STATA and manuscript writing, and final approval of the version to be submitted.
AD was responsible for study design, overseeing data collection and meta-analysis, and final approval of the version to be submitted.
HJM was responsible for conception, study design, data collection, analysis of data, and final approval of the version to be submitted
Funding
Open access funding provided by Imperial College London. HJM is supported by the Wellcome / EPSRC Centre for Interventional and Surgical Sciences (WEISS) and the National Institute of Health Research (NIHR) University College London (UCL) Biomedical Research Centre (BRC).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical declaration
This meta-analysis did not require ethical approval.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
Supplementary Table 1 Studies published between 2010 till present since the last published review by Kirkman et al.[56] Studies classed as RCT = Randomised Controlled Trials had evidence of blinding and randomisation, authors specifically referring to their studies as such. Studies that did not describe blinding but only randomisation were classified as Randomised studies. Average calculated MERSQI was 11.52 ± 2.20 (Mean, SD), Range 5-15. [1, 2, 4, 5, 7,8,9, 12, 13, 15,16,17, 19, 23,24,25, 27, 29,30,31,32, 34, 35, 39, 43, 44, 48, 50, 51, 54, 58, 60, 62,63,64,65,66,67,68,69,70, 72, 81,82,83, 88, 97,98,99,100,101, 104, 106, 110, 111, 114, 116, 117, 122, 124, 125]. Supplementary Table 2 Clinical trials included in the final quantitative meta-analysis were analysed based on improvement in a particular domain. Ne is the sample number of the intervention/event outcome measure, Nc is the sample number of the control measure or comparator, Se is the standard deviation of the sample outcome mean, Sc is the standard deviation of the comparator mean. Me is the mean of the sample outcome, Mc is the mean of the comparator. P-value is set at 0.05. (DOCX 194 kb)
ESM 2
(PDF 294 kb)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Davids, J., Manivannan, S., Darzi, A. et al. Simulation for skills training in neurosurgery: a systematic review, meta-analysis, and analysis of progressive scholarly acceptance. Neurosurg Rev 44, 1853–1867 (2021). https://doi.org/10.1007/s10143-020-01378-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10143-020-01378-0