Background

Since three-dimensional (3D) anatomical models were first created from two-dimensional (2D) computational tomography (CT) images in 1979, 3D reconstruction has become increasingly common in medicine due to rapid technological advances [1,2,3]. 3D reconstructions have various applications, including the manufacture of 3D printed (3DP) models and Virtual Reality (VR) simulators [4, 5].

In surgery, 3D reconstruction has been used for operative planning, surgical training, and patient engagement (Fig. 1). Using 3D reconstructions, complex anatomical relationships can be clearly visualised to help guide surgical decision making [6]. Shen et al. demonstrated that pre-operative 3D reconstructions can improve surgical outcomes and reduce complication rates [7]. As the traditional apprenticeship model of surgical training becomes incompatible with modern practice and working patterns, 3D reconstructive techniques may play an increasing role in surgical training. Creating VR simulators with realistic haptic and stereoscopic feedback has been shown to help junior surgeons develop their skills in a safe environment [8, 9]. 3D reconstructions, whether virtual or 3DP, enhance patient education when compared to standard imaging across a range of specialities [10, 11]. Early studies show that personalised 3D models help individuals gain greater insight into their disease and this improves shared decision-making [12]. It is likely 3D reconstruction will become commonplace in surgical practice and benefit both patients and clinicians.

Fig. 1
figure 1

Creation of a virtual reality 3D model and its application

As a subspecialty, general surgery has been slow in the adoption of 3D reconstruction [13, 14]. This may reflect both limited resource availability as well as image-related, organ-specific complexities. It is generally easier to reconstruct defined structures such as bone or vasculature compared to distensible hollow viscera like the stomach [15, 16]. However, as technological capabilities improve and costs decline [17], 3D reconstruction will become more accessible for use in routine oesophagogastric surgical practices. For the purposes of this review, oesophagogastric surgery is considered to include surgery involving the stomach and oesophagus in benign and malignant states. This will include bariatric surgery.

Herein the authors outline the findings of a scoping review that aims to help clarify current and future roles for 3D modelling in oesophagogastric surgery, highlighting gaps in the literature as well as implications for future practice and research [18, 19].

Methods

Protocol

A scoping review protocol, based on internationally accepted guidelines developed by Arksey and O’Malley and later refined by Levac et al. and the Joanna Briggs Institute [20,21,22], was developed. The authors’ protocol is available via the Open Science Framework (osf.io/ta789) and has been published in a peer-reviewed journal [23]. As the study progressed, minor changes were made to the a priori protocol, as is accepted practice when undertaking scoping reviews [24]. Each amendment was discussed and agreed by the investigatory team prior to its adoption. Since the creation of the scoping review protocol, the title of the study has changed from ‘upper gastrointestinal surgery’ to ‘oesophagogastric surgery’ to clarify the exclusion of hepatobiliary surgery. The review report has been written following the Preferred Reporting Items for Systematic reviews and Meta-analyses extension for Scoping Review (PRISMA-ScR) Checklist [25].

Search strategy

Working with a specialist medical librarian the lead investigator (HR) developed a comprehensive search strategy tailored for each key database using keywords, thesauri terms and Boolean Operators (Appendix 1). MEDLINE, Embase, Elsevier Scopus and ISI Web of Science were searched from their inception to 1/6/2020. Additional grey literature was identified using OpenGrey and Grey Literature Report. The reference lists from selected studies were reviewed by hand using a ‘snowball search’ methodology to identify additional relevant literature.

Study selection

A two-stage screening process using ‘title and abstract screening’ and ‘full text review’ was performed by two independent reviewers (HR and GS). Any disagreement was resolved by discussion and if required, a third reviewer (CK) provided a decisive vote. This process was undertaken using the Covidence systematic review software [26]. Full text articles were included if they reported the use of 3D modelling within the setting of oesophagogastric surgery, specifically focussing on the stomach and oesophagus, in health, benign and malignant states. Only studies published in the English language were included. Studies were excluded if they pertained to 3D modelling outwith oesophagogastric surgery, studied paediatric populations, or were not published in peer-reviewed literature. Additional exclusion criteria used after the publication of the protocol included non-human or animal-based research (Appendix 2). As 3D modelling is an emerging technology it was anticipated selected studies would have a comparatively low evidence grade. A methodological quality analysis for exclusion was therefore not performed, as this may have significantly restricted the number of studies evaluated.

Data extraction

After study selection, two independent reviewers (HR and GS) extracted key study characteristics using a data charting tool to allow both quantitative and narrative assessment. Data extracted included author(s), year of publication, origin of study, study aims, study population, sample size, design, and main findings. To perform a qualitative thematic analysis, selected studies were reviewed and assigned to one of the three pre-determined overarching themes: (i) surgical training and education, (ii) patient education and engagement, and (iii) operative planning and surgical practice. Where applicable, subthemes were generated to enhance thematic assessment. These were then combined into a study summary table which included author(s), title, year of publication, imaging modality, software used, study design, form of 3D modelling and theme explored (Appendix 3).

Results-quantitative assessment

Included studies

The search yielded 4688 results. Snowball searching and grey literature searches later added a further 10 sources. After duplicate removal, 2630 sources underwent text and abstract screening (Fig. 2 [27]). The full text of 292 sources were reviewed and ultimately 56 papers were included for data extraction (Appendix 3 [28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83]). The most frequent reasons for exclusion were conference abstracts and proceedings without publication in a peer-reviewed journal (n = 76), the wrong setting (n = 54) and wrong intervention (n = 44).

Fig. 2
figure 2

PRIMSA flow diagram for study identification, screening and inclusion

Study characteristics

Significant heterogeneity in study design, image processing methods, technologies utilised, statistical analysis and outcomes measured was observed. Evidence reported within the majority of included studies was classified as low-grade, with 88% (n = 49) of publications falling at or below level III evidence [84]. Only 7 studies met level II evidence criteria and no studies met level I evidence criteria. There were no randomised-control trials (RCTs) of 3D modelling in oesophagogastric surgery. Frequently used study designs included Case Reports and Case Series (21%, n = 12), Non-Randomised Controlled Experimental Studies (18%, n = 10) and Basic Research studies (18%, n = 10). Most of the research, 75% (n = 42), originated in East Asia, with South Korea, Japan and China having the highest output. Only 13% (n = 7) of published work originated in Europe. All studies were published during or after 2003 and used either virtual or augmented reality. Only 5% (n = 3) used 3D printing.

The software used, the segmentation methodology and the rendering technique were inconsistently reported throughout the included texts. The 3D reconstruction software used was not stated in 30% (n = 17) of studies. In the remaining literature, 23 different software programmes were described with Zio Software Inc (11%, n = 6) the most frequently used. The rendering technique was not stated in 52% (n = 29) of included studies and the segmentation methodology was not described 73% (n = 41) of studies. However, when stated, volume rendering (41%, n = 23) was most frequently described. Predominantly, automatic (5%, n = 3) or a mix of automatic and manual segmentation (11%, n = 6) methodologies were used.

Thematic analysis of included studies

The majority (86%, n = 48) of studies explored the role of 3D modelling within operative planning and surgical practice. Subtheme analysis showed the most common application of 3D reconstruction was for pre-operative guidance (77%, n = 43) with only 5 papers (9%) investigating the potential impact of 3D modelling on intraoperative guidance during oesophagogastric surgery. A small number of studies (14%, n = 8) reported on surgical training and education, and subtheme analysis showed these were evenly divided between Virtual Reality Simulation and Anatomical Teaching. No studies utilised 3D modelling for patient engagement and education. A summary of key included papers is provided in Table 1.

Table 1 Characteristics and themes of key included studies

Results-narrative summary

Operative planning and surgical practice

Pre-operative guidance

Operative planning and surgical practice is the most populated category in this review, with pre-operative guidance the most researched subtheme. Almost a third of studies in this scoping review looked at virtual oesophagogastric vascular reconstruction (29%, n = 16) and its role in operative planning [36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51]. Although these papers all conclude that there is a benefit to generating a ‘vascular roadmap’ for surgical guidance, particularly for identifying anatomical variants for laparoscopic surgery, few provide quantitative evidence of improved patient outcome. Key exceptions include the work by Wang et al. [49] and Kinoshita et al. [51]. Wang et al. found that pre-operative vascular reconstructions using 3D reconstruction significantly reduced operative time (19.70 ± 5.59 min vs 24.47 ± 9.98 min; p = 0.001) and blood loss at splenic hilum (13.62 ± 4.50 mL vs 17.92 ± 9.08 mL; p = 0.001) compared to no 3D reconstruction during laparoscopic total gastrectomy with spleen-preserving splenic lymph node dissection. However, Kinoshita et al. [51] found no significant difference in operative time, blood loss or complication rate in 3D reconstruction versus no 3D reconstruction but did identify significantly higher lymph node retrieval at station 10 and concluded that the quality of surgery was improved. All these studies offer low-grade evidence, that is often retrospective, and there are no RCTs demonstrating improvements to patient outcomes.

Beyond virtual vascular reconstructions, the review identified several (18%, n = 10) innovative case reports and basic research using 3D reconstruction techniques to model atypical or challenging anatomy for pre-operative planning [53,54,55,56,57,58,59,60,61,62]. For example, Takanami et al. [58] and Kato et al. [59] describe methods to virtually visualise the thoracic duct prior to oesophagectomy. There is also a growing body of work using 3D reconstruction to assess bariatric surgical patients and plan future surgical intervention [60, 61]. Whilst these novel techniques demonstrate various applications for 3D modelling, they are early basic research and do not show quantitative improvements to patient care.

The remaining literature on 3D modelling for pre-operative guidance focused on radiological assessment of malignancy (29%, n = 16) [63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78]. The review identified 10 papers, describing the contribution 3D reconstruction has made to gastric cancer pre-operative staging [64, 66,67,68,69, 71,72,73, 75, 76]. Overall, this work describes how the advancement of multidetector CT (MDCT) in conjunction with 3D modelling techniques has significantly improved the ‘T’ staging of gastric cancer, especially in early gastric cancers. Despite these technological innovations, lymph node staging remains a challenge in gastric malignancy.

For the pre-operative radiological assessment and staging of oesophageal cancer, only 4 papers (7%, n = 4) were identified [63, 65, 70, 74]. Both Onbas et al. and Cai et al. conclude virtual CT 3D reconstruction is an accurate, safe, and effective tool in the staging of oesophageal cancer [63, 65]. Cai et al. also states that 3D reconstruction may have a role in assessing pathological response following chemo- and radiotherapy [63]. Whilst this topic is explored by Alfieri et al. and Mamede et al., both are clear further research is necessary [70, 74].

Intra-operative guidance

The scoping review identified 5 papers exploring 3D reconstruction for intra-operative guidance (9%, n = 5). This includes 2 studies describing the utility of 3D virtual reconstructions as guidance during surgery. The remaining 3 articles, consists of case reports and case series, detailing the application of 3D printed models for operative guidance.

The literature on virtual 3D reconstructions for intra-operative guidance primarily relates to vascular anatomy. In a prospective observational study, vascular reconstructions guided surgeons during robotic gastrectomy with lymphadenectomy in gastric cancer patients [83]. The study demonstrated it was feasible to accurately generate and use individualised vascular roadmaps to prevent vascular injury with minimal (15 minute) increases in operating time. However, this technique required a surgically trained radiologist to orientate the reconstruction throughout the operation. Similarly, Sato et al. used Augmented Reality through the HoloLens system to safely manage aberrant vascular anatomy during a thoracoscopic esophagectomy [82]. Both studies suggest these techniques offer significant advantages to novice surgeons but highlight a need to validate these techniques with respect to surgical outcomes.

The work by Marano et al., Dickenson et al. and Ye et al. describe the subjective clinical benefit 3D printed models have for pre-operative planning and intra-operative guidance [79,80,81]. They encompass its application to robotic, endoscopic, laparoscopic, and open gastro-oesophageal surgery for intricate and complex anatomy. Each paper highlights the improved visual and tactile information provided to the surgical team by 3D printed models in comparison to virtual reconstructions or standard 2D imaging. This information allows the surgeons not only to optimise the surgical approach and rehearse the procedure but then acts as a reference during surgery by showing the relationships of key anatomical structures. Each case was completed without significant complications, which the authors conclude is due to the unique advantages of personalised 3D printed models. However, Marano et al. highlight the considerable production time and high expense of the printed model. No other paper described the cost and time commitments of their technique. No quantitative benefit was demonstrated in these studies.

Although these papers suggest 3D reconstruction and 3D printing could potentially allow for personalised and safer operating, this has not been quantified and questions remain about the high cost and time required.

Surgical education and training

We identified 8 papers (14%, n = 8) under the theme of surgical education and training that use 3D reconstruction for oesophagogastric surgery. These are subcategorised into Virtual Reality (VR) simulation and anatomical teaching.

The review identified 4 papers (7%, n = 4) studying VR simulation for oesophagogastric surgical training. The laboratory research by Choi et al. describes the generation of a graphic and haptic VR model of the oesophagus for simulation training. This technology paper describes the creation of the oesophageal 3D model but does not validate it in the context of surgical training. The remaining research on this topic relates to bariatric surgical training. One paper describes the successful creation and validation of a novel VR laparoscopic adjustable gastric band simulator. Further research by Lewis et al. and Giannotti et al. use a 3D Systems’ ‘Lap Mentor’. Despite their small sample sizes, these papers validate the use of the Lap Mentor in the training and assessment of bariatric surgeons. No other research exploring the application of VR simulators outside bariatric surgery was identified. The use of 3D printing within simulators was not described by any studies.

3D reconstruction techniques have been used to visualise both normal and pathological anatomy for surgeons. Kavic et al. used CT images to create 3D virtual models of hiatus hernias, hoping to aid the understanding of hiatus hernia classifications and perhaps advance existing classification systems. Using publicly available cadaveric datasets, Wu et al. and Shin et al. generated virtual 3D models of the thorax and gastro-intestinal tract, including stomach and oesophagus. Using similar methods, Kwon et al. created virtual endoscopic and laparoscopic simulations of stomach wall anatomy. These papers conclude that virtual 3D reconstructions give junior trainees a deeper understanding of complex anatomical relationships. However, these insights were not supported by quantitative comparisons to traditional educational tools.

In summary, VR simulators have been shown to be an effective tool for the training and certification of bariatric surgeons. However, the application of these simulators to other oesophagogastric surgical procedures is yet to be validated. Virtual 3D reconstructions of oesophagogastric anatomy and pathology have been created, but there is no evidence for their superiority over traditional training methods. Furthermore, physical 3D printed models have yet to be used in the education and training of oesophagogastric surgeons.

Patient engagement and education

The scoping review did not identify any literature that focussed on the application of 3D modelling to patient engagement and education in oesophagogastric surgery. In the conclusion of the case series by Dickenson et al., patient engagement is highlighted as a significant advantage to 3D printed models. Patient engagement is a fundamental application of 3D modelling and disappointingly appears under-researched in oesophagogastric surgery.

Discussion

This scoping review identified and critically reviewed 56 papers in respect to the use of 3D reconstruction and its current and future role within oesophagogastric surgery. The review demonstrates 3D virtual modelling can be used for radiological diagnosis and operative guidance, including vascular reconstruction, in oesophagogastric surgery. The predominant focus on vascular reconstructions likely reflects the relative ease of modelling contrast enhanced vascular structures in comparison to hollow viscera.

Although the subjective benefit of 3D modelling for operative planning is described, few studies demonstrate improved surgical outcomes. For 3D reconstruction to become routine in oesophagogastric surgery, well powered RCTs are essential. Researchers should consider emulating encouraging work in other surgical specialties, such as urology, where a recent RCT demonstrated 3D modelling reduces operative time, blood loss and length of hospital stay [85].

Critics of 3D reconstruction suggest the high cost and time demands of this technique will prevent its wider adoption. Only one study in this review quantified these variables. However, cost-analysis in other surgical specialties has shown the resource-intensive requirements of 3D modelling are matched by downstream savings [17]. Researchers within oesophagogastric surgery should therefore consider a formal cost-analysis to assess the impact of 3D reconstruction on healthcare provision.

Although the segmentation process of 3D reconstruction is highly labour intensive, machine learning methodologies have demonstrated fully automated segmentation is feasible for abdominal viscera and vasculature [86,87,88]. Considering the possible healthcare savings identified in other areas, the development of automated segmentation methods and falling technology costs, it would be prudent to quantify the potential benefits of 3D reconstruction in oesophagogastric surgical practice as the technology becomes easier to adopt.

As surgical training evolves, it moves further from the traditional apprenticeship model [89]. Modern surgical trainees lack the experiential opportunities of their predecessors, and it is hoped surgical simulators could replace this. Training simulators using either virtual or 3D printed reconstructions offer a cheaper and more ethically acceptable alternative to cadaveric or animal-based simulators [90]. With the successful validation of VR simulators for bariatric surgery, it would be sensible to assess their role in bariatric surgical training programmes. This scoping review shows the need to validate VR simulators for oesophagogastric procedures beyond bariatric surgery. Furthermore, there is currently no published research exploring the use of 3D printed surgical simulators in oesophagogastric training.

3D reconstructions, either virtual of physical, can provide significant anatomical educational benefits to medical students and surgeons alike. Innovative applications of 3D reconstructions have been shown to provide an advantage over conventional educational practices in several surgical specialities [6, 91,92,93]. In comparison, this technique appears underutilised within oesophagogastric surgery. This review only identified 4 studies using 3D reconstruction for anatomical education, none of which demonstrated a quantifiable advantage. However, 3D modelling might help junior surgeons to understand the intricacies of complex oesophagogastric procedures and this should be evaluated in future work.

3D reconstruction offers substantial value to patient engagement and education. Disappointingly this is underutilised in oesophagogastric surgery. Many other surgical specialities have explored the application of 3D reconstruction for patient engagement [12, 94,95,96,97,98,99]. Those studies consistently show 3D reconstructions, especially 3D printed models, improve patients’ understanding of their pathology and proposed treatment. It is likely that generic or patient-specific 3D models would be useful tools for securing informed consent for oesophagogastric surgery. Future research should evaluate the application of 3D reconstructions for oesophagogastric surgical patients with particular attention paid to the importance of generic versus patient-specific models.

From a technological perspective, software and reconstruction methodologies were poorly described in the papers reviewed. This is important. Without clear descriptions of the technology and techniques used, a framework for future researchers cannot be constructed or recommended. The predominance of volume rendering over surface rendering likely reflects the comparative simplicity volume rendering provides over surface rendering [45, 100]. A future review detailing the technology and reconstruction techniques best suited to oesophagogastric surgery would be highly valuable to novices in this field.

Only 5% of the studies in this scoping review used 3D printed models and only in the context of operating planning. There are no direct comparisons between virtual and physical 3D models, therefore no conclusions can be drawn about superiority between the two techniques in oesophagogastric surgery. However, it is likely that neither technique is superior overall, instead the choice is dependent on the theme researched. For example, for intra-operative guidance, an augmented reality model is likely to provide greater value than a 3D printed model to the operating and scrubbed surgeon. In contrast, for pre-operative guidance, there is early evidence to suggest 3D printed models provide greater benefit for surgical planning across a range of specialties [101, 102]. Similarly, some work suggests patients gain a greater understanding when shown 3D printed models rather than virtual alternatives [98]. In simulation training, there are considerable benefits and challenges to both virtual and 3D printed simulators [103]. The decision between the two is likely to depend on the skill being taught and the training centre itself. Whilst the findings of this review suggest 3D printing is underutilized in oesophagogastric surgery, both forms of 3D modelling are valuable tools to researchers.

Unfortunately, this review identified little high-quality research into 3D reconstruction. Compared to other surgical specialties (Table 2 [101, 104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166]), oesophagogastric surgery has fallen far behind. Without a concerted effort to correct this, the specialty might miss the opportunity to take advantage of the multi-faceted applications of 3D modelling.

Table 2 Comparison of published randomised control trials and systematic reviews in 3D reconstruction between surgical specialties

Limitations

This scoping review has several acknowledged limitations. Most of the included studies originated from Asia. As this paper was limited to the English language, publications not translated into English may have been excluded, introducing a selection bias. Furthermore, as this scoping review focussed on the application of 3D reconstruction in relation to surgical settings, innovations within other medical fields applicable to surgery may also have been excluded. In the construction of the protocol, it was determined that studies should not be excluded based on a methodological quality analysis. Whilst this allowed a breadth of studies to be included for critical analysis, it may allow studies of poor quality to be overrepresented.

Conclusions

This is the first review, to the authors’ knowledge, summarising the current and future roles of 3D reconstruction within oesophagogastric surgery. Clearly, 3D modelling is in its infancy compared to other surgical specialties. There is early promising evidence suggesting 3D reconstruction could offer significant benefit to oesophagogastric surgery. However, without further high-quality research in this field, the specialty may be left behind. This would be detrimental to all parties including patients, trainees and established oesophagogastric surgeons.