Introduction

Simulation-based training is a method of experiential teaching that reproduces a real-world scenario in a controlled setting. Conventionally used to replicate high-risk tasks in areas such as military training, aviation, and aerospace professions [1], simulation-based training has become a prominent complement within medical education [2, 3]. Simulations can be performed outside of working hours, and certain procedural skills can be customized or repeated for a trainee’s needs, without compromising patient safety. Radiology residents and medical students are generally supportive of simulation use, as both groups have reported higher procedural confidence after completing a session on a number of diagnostic or interventional simulators [4,5,6,7,8,9,10].

Simulation training is especially relevant for procedural specialties like IR, where learners rely on haptic cues for skill acquisition [11]. There is an extensive amount of simulation options currently integrated into the medical field such as video games for inferior vena cava filter placement or percutaneous image-guided interventions [12, 13], phantom simulators for Computed Tomography (CT) biopsies [10, 14,15,16], and animal or cadaver models to practice endovascular access or interventions [17, 18]. Several studies have used simulators to demonstrate improved procedural technique, either in device manipulation [19,20,21], successful vessel cannulation [22], or reduced procedural time and radiation use [23,24,25]. There is burgeoning evidence demonstrating skill retention after the simulation [7], which has translated to improved patient outcomes. Such an example was documented by Andretta et al., who noted increased procedural proficiency and successful PICC line placement among interns who completed a simulation rotation on ultrasound-guided access [26].

Despite their advantages, simulations have several operational limitations and a correlative cost-benefit ratio. Simulation can be categorized by degree of situational realism, anatomic accuracy, or physiologic replication [27]. Although high-fidelity models have been shown to significantly increase technical performance in a simulated environment [28], they are often of higher cost, may have limited availability, or, if using animal or cadaver sources, can raise ethical concerns [18, 27, 29]. The reference standard in simulation medicine is the ability to develop a reproducible, realistic, and inexpensive product, which may be refined via three-dimensional (3D) printing. 3D printing has an emerging role within medicine and one of its primary benefits for the clinical realm includes the ability to produce patient-specific modeling from de-identified segmented stereolithography (STL) files, as this serves to augment several clinical applications, including trainee simulation [30, 31].

3D-printed models for simulation learning have been tested and positively received by trainees [32,33,34,35,36]. Some studies have found that 3D printed simulations increased student test scores when studying physiologic [34, 37] and pathologic [38,39,40,41] anatomy. 3D printed models have also facilitated groups of dental and surgical trainees in developing better preoperative plans [42, 43]. They have been shown to improve simulator procedural performance among anesthesia residents [44] and reduce fluoroscopy and simulator procedural times on endovascular aortic procedures with vascular surgery residents [45,46,47].

Radiology is well-poised to integrate 3D printing within simulation training. Prototypes, ranging from the lumbar spine to complex vasculature [48,49,50,51], have successfully been printed and can be modified for simulation purposes. While the role of three-dimensional printing for trainee education has been documented in other procedural specialties, there is a lack of knowledge about the role of 3D printing for trainee education within IR. Thus, the purpose of this review was to analyze the current articles that discuss the role of 3D printing for IR trainee education and emphasize the need for future experimentation.

Methods

A literature query was conducted through April 2020 for articles discussing the use of 3D printing in IR simulation experiences via PubMed, Embase, CINAHL, Web of Science, and the Cochrane Library databases. Key search terms and boolean operators can be found in Table 1.

Table 1 Key search terms and boolean operators
Full size table

After removing duplicate records, non-English articles, and abstract-only articles, the remaining articles were reviewed. Articles were excluded if they did not feature the use of a 3D printed simulation models for interventional procedures by trainees, defined as either medical students, diagnostic radiology or interventional radiology residents, or interventional radiology fellows. For combined groups, more than 50% of the participants needed to identify as either medical students, diagnostic radiology or interventional radiology residents, or interventional radiology fellows. Figure 1 outlines the systematic review.

Fig. 1
figure 1

Systematic review flowchart

Full size image

To assess data quality, studies were evaluated by two reviewers using the Quality Assessment of Diagnostic Accuracy Studies - 2 (QUADAS-2) tool (Fig. 2) [52].. The index test was the evaluated 3D printed simulation in each study. The reference standard included a comparison to evaluate the 3D-printed simulation, which could include a pre- and post- assessment, control simulation, or outcomes from a control group. Two independent reviewers appraised these articles.

Fig. 2
figure 2

Summary of Quality Assessment of Diagnostic Accuracy Studies - 2 (QUADAS-2) ratings. The index test was the evaluation of the 3D-printed simulator. Applicability refers to appropriateness of the simulation to measure trainee ability. The reference standard was a comparison to evaluate the 3D-printed simulation. Flow and timing refers to equal treatment among participants and appropriate study follow-up, if applicable. There was a low to moderate degree of bias established due to the variability between study designs in the reviewed articles

Full size image

Results

As noted in Fig. 1, the initial search yielded 843 records and was reduced to a total of 466 after removal of 377 duplicates. Of these, 51 of 466 records were further excluded due to a lack of full-text and availability of the record in English. The majority of excluded articles were original research articles that were not related to 3D printing. Of the 131 articles that discussed 3D printing, 44 of 141 (31.2%) of those articles featured trainee involvement, and 23 of the 44 (52.7%) of these articles featured a 3D printing procedure-oriented simulation. Among the remaining 23 articles, four articles featured our trainee population of interest (i.e. radiology or interventional radiology residents/fellows or medical students) and were included in the primary review.

The 19 remaining articles were excluded because they did not feature the target trainee population or the target trainee population was not represented by at least 50% of trainees in the study. The excluded articles implemented the use of 3D printed models for medical trainees in a variety of specialties. The predominant specialty that integrated 3D printing into its simulations was neurosurgery with seven articles [53,54,55,56,57,58,59]. These studies used 3D printed models of the cerebral vascular network to train neurosurgery residents and fellows on procedures such as cerebral aneurysm clipping and endoscopic ventriculostomy. Vascular surgery and anesthesiology were mentioned in three articles each [46, 47, 60,61,62,63]. The vascular surgery articles focused on endovascular repair of aneurysms and needle puncture of the gluteal artery using 3D printed vascular models while the anesthesiology articles used 3D printed models to simulate fiberoptic bronchoscopy procedures, thoracic spinal needle insertion, and central venous access [46, 47, 60,61,62,63]. Urology had a total of two articles where one of the articles featured interventional radiology trainees; however, this study did not meet the criteria of at least 50% IR trainee involvement [32, 64]. A 3D printed bladder was used to simulate urethrovesical anastomosis repair in one of the articles while the other article had a 3D printed human pelvicalyceal system, kidney, and adjacent structures to simulate percutaneous nephrolithotomy [32, 64]. The remaining articles had only one study associated with its specialty: cardiology (3D printed heart for electrophysiology training) [65], otolaryngology (3D printed model of the sinuses and skull base for anatomical education) [66], oral and maxillofacial surgery (3D printed model of the orbit for orbital surgery training) [67], and dentistry (3D printed model of pediatric dentition for pediatric dental education) [68].

All of the 19 excluded articles showed a general positive attitude toward 3D printed simulations where these models could be used for medical anatomy education as well as development of medical trainees’ procedural skills [32, 46, 47, 53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68]. The majority of these articles, 17 articles, tested the 3D simulations with trainees further in their medical career such as residents, fellows, or attendings [32, 46, 47, 53, 55, 57,58,59,60,61,62,63,64,65,66,67,68]. The remaining two articles consisted of medical students and residents [54, 56]. The higher level of trainees provides a more accurate assessment of particular values such as face, content and constructive validity due to the increased exposure to the real life procedures.

Out of the 19 omitted articles, two of the articles analyzed the face, content and construct validity of 3D printed simulators and revealed a positive trend in both studies [32, 58]. Weinstock et al. had a high face validity mean score at 4.69/5.00, a high content validity at 4.88/5.00, and a construct validity score that indicated that the 3D model provided a significant means to distinguish between novices and expert surgical skills when performing endoscopic ventriculostomy [58]. Ghazi et al. had similar results compared to Weinstock et al. with a high face validity at 4.5/5.0, a high content validity at 4.6/5.0, and a construct validity score that supports the theory that a 3D constructed percutaneous nephrolithotomy simulation provides a comprehensive tool for surgical skills development and evaluation before hand-on experience [32]. The high rated face, content, and construct validity ultimately emphasizes the benefits of 3D printed simulations and its practicality compared to real world procedures.

Although numerous articles reported a decreased cost of their 3D printed simulations [53, 54, 56], only two of the articles (anesthesia and cardiology) directly compared their 3D simulation to commercially available models [62, 65]. Pederson et al. noted a $110 3D printed bronchoscopic simulator compared to a commercial model priced greater than $2590 [62]. Seslar et al. featured a $2000 3D printed electrophysiology simulator compared to commercially available models valued at over $100,000 [65]. These two examples give a limited view of the possible cost benefits of 3D printing with medical training.

All of the criteria fulfilling articles reviewed were single institution studies with small sample sizes. The simulated procedures in the aforementioned articles included endoscopic biliary drainage [69], minimally-invasive CT-guided spine procedures [70], and ultrasound guided femoral artery access [71, 72]. Two of the studies analyzed the performance of IR procedures using medical students as the sole test population. Sheu et al. included a randomized assortment of medical students ranging from all years of training (n = 49) while Li et al. recruited first- and second-year medical students (n = 13). O’Reilly et al. invited a cohort of first year radiology physicians in training, (n = 19) for their study. Bundy et al. recruited the most diverse test population with varying backgrounds in medicine, including: technologists (n = 2), medical students (n = 1), residents (n = 2), fellows (n = 4) with consulting attending physicians (n = 2). To evaluate the effectiveness and provide a statistical method of comparison of simulation training using 3D printed materials, each study implemented a distinct approach. The heterogeneity in reported results was the main issue identified by QUADAS-2 assessment, which the reviewers assessed as a low to moderate degree of bias.

In addition to the similarities in the test populations, all four of the studies implemented a pre- and post-test questionnaire that utilized a Likert scale to quantify subjective opinions on 3D printing in IR training. Bundy et al. and Li et al. used a 10-point likert scale to assess the comfort with endoscopic techniques and lumbar punctures respectively. O’Reilly et al. used a 6-point Likert scale to assess the preference between a 3D printed and a commercially available femoral access model of the lower limb in teaching vascular access. Sheu et al. used a five-point likert scale to compare the ease of use, usefulness in practice and student confidence when using a 3D printed and commercially available phantom for femoral artery access.

All four of the studies reported positive feedback regarding the use of 3D printed models for trainee simulation. The comfortability and preference to use 3D printed models for endoscopic training increased in four different procedures as noted in Bundy et al.; however, the only statistically significant results included a 39.7% (p < 0.05) increase in comfortability performing endoscopic cholecystostomy and a 39.7% (p < 0.05) increase in the trainee’s likelihood to use endoscopy for percutaneous gastrostomy placement. Li et al. showed a significant increase in the confidence in needle stick placement for a control group, which was trained on a 3D model once, and a training group, which was trained on a 3D model twice. The confidence for the control group changed from 1.83 ± 2.0 to 5.8 ± 1.6 with p < 0.004 while the training group had a change in confidence from 1 ± 0 to 6.1 ± 1.1 with p < 0.00001. Additionally, there was a significant reduction in the number of needle stick placements for the training group when comparing the initial session, 8.0 ± 1.3 and the final session, 5.4 ± 1.5, with p < 0.005. The study further associated a 4.5 mGy-cm reduction of dose with a reduction of 3 needle readjustments, which correlates to a decreased reduction of radiation dose for the training group.

The preference to use the 3D printed model over a commercially available model for femoral artery access training was noted in O’Reilly et al. with an average rating of 5.1 out of 6. Sheu et al.’s results demonstrated that the majority of students in both the 3D printed ultrasound compatible vascular access model (3DPVAM) and commercial model (CM) groups agreed or strongly agreed that the models were useful for practice (96.2% and 95.7%, respectively; p < 0.87). Student confidence in performing femoral artery access increased by 2 Likert points in both trainee groups. Additionally, there was no difference in the average confidence change when comparing the training experience using the 3D printed model and the commercially produced model (p < 0.001).

The majority of the test populations in the excluded articles, 17 out of the 19 articles, consisted of higher-level medical professionals including residents and attendings [32, 46, 47, 53, 55, 57,58,59,60,61,62,63,64,65,66,67,68]. This contrasts with the 4 included articles where 2 out of the 4 studies used medical students as their test subjects [70, 72]. Although this provides a unique, unbiased perspective in the evaluation of the ability of the simulators to teach particular procedures, the medical students’ analysis may not accurately reflect the simulators’ realism and overall efficacy compared to the clinical procedures and its real-life application. Further research using residents, fellows and attendings would greatly benefit future studies on 3D printed simulations in IR. Future articles analyzing the benefits of 3D printed simulators would benefit from more stringent statistical analysis by including face, content, and construct validity. As noted in two of the articles, there was a positive trend in all three of the specified validity analyses [55, 65]. A common theme in the 4 included articles and the 19 excluded articles was the positive feedback in favor of the integration of 3D printed models in trainee simulations [32, 46, 47, 53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72]. The feedback was attained through post-simulator testing questionnaires. These evaluations had a diverse array of subjective questions ranging from how the subjects’ confidence changed in performing the procedure after training with the 3D model to the simulations’ ability to effectively mimic the real-life procedures.

Discussion

A recent needs-assessment survey prioritized the development of ultrasound-guided or interventional procedures for simulation-based training [73]. Contrary to aforementioned evidence of existing IR simulators [10, 12,13,14,15,16,17,18], interventional-based simulations remain one of the least common training opportunities available [6]. A survey conducted by Matalon et al. found that 40% of radiology residents do not use simulations as part of their procedural training [6]. The limited number of available articles that features three-dimensional printing in IR simulation training highlights this area of unmet need within radiology training. While it is possible that we missed a relevant study for our review, this limitation was minimized by using free text and MeSH terms across multiple databases. The only articles we did not review were ones that either had no full-text article or were a non-English article, which comprised a minority (51/466 = 10.9%) of our excluded articles.

There is a lack of quantitative or validated simulation measures. Simulators may not always be able to decipher between experienced or novice operators [74, 75] and must be educationally validated to be useful. Mechanisms for validation include: face validity, content validity, concurrent validity, discriminant validity, and predictive validity, defined in Table 2 [76]. While each of our studies fulfilled face validity by demonstrating how the simulations appeared under imaging, only Li et al. evaluated content validity in evaluating the students’ abilities to complete the intervention (i.e. CT-guided facet block) [70]. Sheu et al. tested concurrent validity by comparing their 3D printed model to the FemoraLineMan CM (Simulab Corp, Seattle, Washington), a commercially available femoral access simulator [72]. This degree of variation in objective feedback raised concerns for a low to moderate degree of bias during QUADAS-2 evaluation.

Table 2 Measures of validity and their definitions for simulation-models
Full size table

A potential reason for a limited number of validation measures was that the study population was primarily medical students, as opposed to residents or fellows. Medical students’ lack of clinical experience compared to residents, fellows, or attendings minimizes confounders when evaluating procedural performance [17, 72]. While each study could circumvent its study population limitations by establishing face validity on appropriate imaging modalities, future studies evaluating 3D printed simulation will require more robust testing with residents and attendings, especially in regard to predictive validity. Current literature regarding predictive validity for preoperative planning is encouraging, as investigators have associated the use of 3D printed simulations or guides with reduced operative times [77, 78]. Within simulation literature, this appears to have only been evaluated in audiology trainees for proper hearing-aid placement [79], where trainees who practiced on a 3D printed simulator achieved a higher percentage of proper hearing-aid placement compared to those who practiced on the control model.

Given the strengths and limitations of the available literature, future studies focusing on 3D printed procedural simulation models for interventional radiology trainees could benefit from the inclusion of an adequate number of trainees particularly from senior years of training. Additionally, robust pre study baseline assessment using both subjective Likert scale questionnaires as well as objective measures of skill such as time to complete the procedure or number of attempts needed before success is crucial for post study result analysis. Ensuring the study can evaluate face, content and construct validity is valuable in generalizing the results to a larger population. Studies should have redundancy of raters if subjective observations are being made. Comparison of the 3D printed simulator with commercially available models including the cost analysis would be of great help to those trying to justify incorporating 3D printed simulation in their training program. Although many simulators do exist, the number of applications in interventional radiology remains limited. Table 3 outlines possible procedures that may benefit from 3D printed simulation training.

Table 3 Interventional Radiology procedures amenable to possible 3D printed simulation
Full size table

A common theme noted between these studies was the reduced production cost of these models compared to commercially available alternatives. The biliary simulation was one of the cheapest models to produce, costing approximately $170 for materials, while the lumbar spinal simulation, which required staged printing of bony structures, nerves, and vessels before assembly, was estimated to cost $5400. Sheu et al.’s femoral access model, with an estimated lifetime of approximately a dozen punctures, cost around $647 for the initial model and $100 to replace the 3D-printed vessels. This vastly differs from the competitive alternatives, which may cost between $2000–$4000, with replacement parts costing between $600 and $1600 [72]. This, however, does not factor the printer and software costs, which, again, widely varied between studies. Printed models first require segmentation, which can be completed on either free, open-source software or require purchasing high-end modeling software [30], while actual printers can range from $6000–$750,000 [80]. The financial cost and operational resources associated with 3D printing requires funding and institutional support, which may be another factor in the limited number of simulation articles for IR-related procedures. However, despite upfront investment, these costs can be shared across a hospital, and its resources, from preoperative modeling, simulations, and patient education, are applicable within multiple specialties [30, 31]. Although the costs of owning a printer for a particular lab can require a significant financial upfront investment, there are many various options to circumvent this issue. Many of the larger academic universities have their own 3D printing workshops, often housed in the central library and accessible to any member of campus. With this option in mind, a lab can send their designs into these workshops to help put together their particular devices. If the researchers’ organization does not have any in-house printers or specific materials, the product can be outsourced to a 3D printing service. Although owning a printer provides benefits such as on demand production and eventual lower costs, outsourcing provides its own unique benefits including no upfront investment, a wider array of 3D printing technologies and materials that can fit a certain criteria, and the availability of design expertise and recommendations when determining a design [81]. With these options in mind, it is also reasonable to consider using an in-house as well as an outsourced 3D printer based on the particular material needs as well as specific printer requirements of the project [81].

Conclusion

Available literature, while supportive of the use of 3D-printing within vascular surgery, cardiology and for procedures performed by non-radiological specialties, is sorely lacking for interventional radiology. The paucity of literature regarding 3D-printing for simulation learning within IR reveals an essential area of improvement in medical education curricula. A lack of objective outcomes hinders advancements in simulation learning, to the disadvantage of trainees. Subsequent research and standardization of 3D-printed simulated experiences can offer more uniform and controlled learning opportunities for trainees to safely refine procedural techniques in the unique field of IR.

Summary of points

  • 3D printing has been utilized as a means of producing high-quality simulation models for trainees in procedure-intensive or surgical subspecialties.

  • Less is known about its role for trainee education within interventional radiology.

  • Our search identified a scarcity of published sources that appraised the use of three-dimensional printing for simulation training in IR where a total of four articles were found.

  • Although presently available literature endorses the use of three-dimensional printing within interventional radiology as a teaching tool, there is a lack of studies that evaluates the predictive validity of 3D-printed simulation-based training in IR.

  • Further research and integration of 3D-printed simulations into IR medical training can help develop and expand this field to a great extent.