Introduction

As we have learned from surgical and neuroscientific studies, knowledge of white matter anatomy is crucial for the surgical treatment of eloquent gliomas [3, 9, 14]. Preserving these structures is essential during the resection of eloquent brain tumors. Our current knowledge of this complex anatomy is currently based on cadaver dissections, insights through direct electrical stimulation (DES) during awake craniotomy surgery, and white matter tractography on MRI, for example by diffusion tensor imaging fiber tracking (DTI FT) [1, 4, 5, 7, 10, 13, 15]. Thus, learning and understanding white matter pathway anatomy is an important part of the training for neuroscientists and physicians. This is the motivation behind the growing number of white matter dissection courses being developed worldwide. However, these require costly infrastructure and brain anatomical specimens which can make such courses expensive and of only limited availability.

Advances have been made in the interface of digital image display and the usage of augmented reality (AR) devices in daily life. While long employed in gaming, prototype devices for neuroanatomical data visualization are now on the market. Through the use of specialized goggles combined with spatial computation, pre-processed imaging data can be visualized in an office, educational suite or conference room. Both in research and in clinical application, this new development has the potential to improve teaching and understanding of fiber tract anatomy. This system also has a potential advantage of making easier to identify white matter tracts which often remain difficult to identify during dissection of cadavers.

Ultimately, systems such as this could be of use in training neuroscientists and neurosurgeons, as well as for preparation and planning before surgery. While scrolling through the scans and viewing the anatomical objects in a group of clinicians, the surgical approach could be discussed in more detail with the full multidisciplinary team.

To date, the potential benefits of a virtual dissection device have not yet been evaluated reported. The objective of this study was to evaluate AR for the use in virtual fiber dissection, as well as assessing its potential application for virtual fiber dissection courses.

Methods

The European Low-Grade Glioma Network workshop meeting

The European Low-Grade Glioma Network (ELGGN) workshop group, consisting of 15 experts with neurosurgical, neurolinguistic, and neuroscientific experience, met in December 2019. Since the attending ELGGN members were experienced in cadaver and in vivo fiber dissections, we decided to evaluate AR and its potential for virtual fiber dissection courses with this target group.

Prepared cases

The prepared cases consisted of patients who underwent microsurgical glioma resection during awake craniotomy at our department. Table 1 gives an overview on the patients and tumor characteristics as well as the clinical course. The study approved by the local ethics committee and was performed in accordance with the Declaration of Helsinki. All included patients provided written informed consent.

Table 1 Prepared cases
Full size table

Technical setup

Firstly, the cortical surface and tractography of existing fiber bundles, based on commonly used clinical software, and delineation of other relevant regions of interest, such as a tumor to be resected, are required. The MRI sequences which were used for the DTI FT were performed on 3 T magnetic resonance scanners (Philips Medical System, Netherlands B.V.). All patients obtained MRIs according to the standard glioma protocol at our department including a T1-weighted three-dimensional (3-D) gradient echo sequence with intravenous contrast administration for anatomical co-registration, a T2-weighted 3-D FLAIR sequence, and DTI sequences with 32 orthogonal sequences. For the visualization of white matter pathways, we performed DTI FT of eloquent pathways such as the corticospinal tract (CST), the arcuate fasciculus (AF), the inferior and middle fronto-occipital fasciculus (IFOF, MFOF), the frontal aslant tract (FAT), and the superior and inferior longitudinal fasciculus (SLF, ILF). Therefore, we used our standard deterministic algorithm with a fiber assignment by continuous tracking (FACT) (iPlan® Net Cranial 3.0.1 and Brainlab Elements, Brainlab AG, Munich, Germany). The regions of interest (ROI) were chosen based upon anatomy and function. The latter ROIs were based on preoperative navigated transcranial magnetic stimulation mappings (nTMS) [12]. Figure 1 shows the 3-D reconstructions and the visualized steps of the prepared cases.

Fig. 1
figure 1

The figure shows the 3-D reconstructions of the prepared cases 1–5 (rows) as summarized in Table 1. Per row this figure shows the step by step reduction of anatomy until specific function-related fiber tracts are revealed. Case 1: a cerebral cortex and transparent skin including tumor and ventricles; b whole brain tractography; c specific fibers revealed, such as CST (yellow), FAT (blue), IFOF (green), and AF (pink). Case 2: d cerebral cortex and skin including tumor and ventricles; e cerebral cortex and transparent skin; f whole brain tractography; g IFOF (green), tumor, and ventricles; h SLF (pink), tumor, and ventricles. Case 3: i cerebral cortex and transparent skin including tumor and ventricles; j whole brain tractography; k specific fibers revealed for motor and language, such as CST (yellow) and SLF (pink). Case 4: l cerebral cortex and skin including tumor and ventricles; m whole brain tractography; n whole brain tractography without head; o specific language (pink) and motor (yellow)-related fibers revealed. Case 5: p whole brain tractography and skin including tumor and ventricles; q additional cortical location of motor (green) and language (pink) function; r specific fibers revealed, such as CST (yellow), FAT (blue), IFOF (green), SLF (pink), optic radiation (red), and tumor; s the skin, tumor, and ventricles plus MEP-positive sites of cortical motor function (green) with CST (yellow)

Full size image

Here, the special feature for fiber dissection was the opportunity for participants to virtually dissect prepared white matter pathways and objects by the use of a spatial computer and AR goggles (Brainlab AG, Munich, Germany; Magic Leap Inc., Plantation, Florida, USA). In contrast to a virtual reality setup, AR allows to still see the room and other participants while the 3-D objects are projected inside the actual environment.

Augmented reality fiber dissection

A 3-D model of the chosen objects can be added into the AR display (Figs. 2 and 3). The pre-selected cases were presented chronologically (Fig. 1). The participants were divided into subgroups of five. Each participant was equipped with an AR goggle headset. In each subgroup of five, one was designated as “master.” The master was equipped with a controller which allowed them to zoom and rotate the model. They were also able to perform virtual dissection of the fiber tracts by peeling away additional layers. In additional, the master’s view was presented via live screen for review by the remainder of the group who were not wearing goggles for that particular session. Figures 2 and 3 show screenshots of the participant view during the application of AR for fiber dissection. Figure 1 shows the step-by-step neuroanatomy dissection revealing function-related fiber tracts.

Fig. 2
figure 2

The figure shows a screenshot visualizing the participant view during the application of AR for fiber dissection. The 3-D reconstruction is shown from left occipital. The green and red laser pointers can be used to describe structures or answer questions between participants or moderator. Additionally, the patient’s 2-D MRI scan slices can be visualized in the background and can be scrolled

Full size image
Fig. 3
figure 3

The figure shows another screenshot visualizing the participants view during the application of AR for fiber dissection. The different colors show different tracts. The tractography model is projected in the room

Full size image

Questionnaire

To evaluate the application of AR for fiber dissection, participants were asked to complete a questionnaire on their normal use of tractography, as well as this simulation experience of using AR goggles and a spatial computer for fiber dissection. They were also asked about possible future applications and suggestions for improvement (Table 2).

Table 2 Answers on questions 1–14
Full size table

Results

Participant’s experience

Table 2 summarize the overall experience, the usefulness for fiber dissection courses, and the value for educational and clinical purposes of AR for fiber dissection as rated by the participants (Table 2).

Suggestions for future improvements

After the general evaluation of AR fiber dissection by questions 1–14, we also asked for suggestions for future improvements. The participants’ answers on the open questions 15 (“What should be improved before augmented reality for fiber dissection can be used for virtual fiber dissection?”) and 16 (“What should be improved before augmented reality for fiber dissection can be used for clinical purposes?”) are incorporated in the “Discussion” section of this manuscript.

Discussion

Augmented reality for fiber dissection

The described setup in this study currently is the only approach allowing for an augmented reality illustration of real-life cases in an easy and straightforward way not requiring repeated rendering and multistep workflows [11]. While AR was already described extensively in other surgical applications, spatial computing allowing for an accurate and real-time visualization of white matter anatomy was only feasible recently [8], yet as an overlay via microscope integration, neurosurgeons know AR for some years [2, 6].

Overall, the participants in this study of AR fiber tract dissection felt that this was a positive development with the potential for valuable educational and clinical benefits in the future.

As shown by the results of question 1–4, meanwhile, tractography is standardly used for clinical and scientific purposes (Table 2). Moreover, proven by the composition of the participants, various disciplines frequently use this technique for the visualization of white matter pathways, and 92.3% of participants evaluated the presented cases as reflecting their clinical reality. Hence, the present results of the evaluation of AR for fiber dissection as a new technique for visualization are of high relevance for learning, teaching, discussing, and understanding of both the methods for tractography and anatomy.

As was evident from both the individual reactions during the evaluation, as well as the formal questionnaire, the participants were excited and impressed by the technology (questions 5–8, Table 2). Based on the survey responses, the primary applications were felt to be for the educational and training of students, residents, and junior scientists. In addition, the relevance for board-certified neurosurgeons was highly rated, with a median score of 9 of 10 (Table 2).

As expected at the outset of the study, the clinical applicability of AR fiber dissection is restricted at the moment. This was confirmed by the participants. However, 14 of 15 participants said they felt that this AR technique could be developed for clinical settings as a tool for pre-operative preparation of the respective surgeon (Table 2).

Improvement suggestions

Questions 15 and 16 were open questions, allowing participants the opportunity to make suggestions for improvements to the presented technique. Their answers were mainly related to the interaction of the user with the spatial computer, the goggles, and the options of visualization.

Participants recommended that, for future applications, more anatomical structures should be included in the simulation to improve orientation. In fact, this is already available but was not offered as an option for this simulation assessment. The anatomical reconstructions used included pre-selected language pathways, the 3-D reconstruction of the tumor, and the modeling of the patients’ skin (Fig. 1). We also included a reconstruction of the cortex to evaluate this additional request (Fig. 1).

For this study, we used data from patients with brain tumors to demonstrate the virtual dissections in these patients. The participants also suggested that including fiber tracts from healthy subjects would be helpful to facilitate greater understanding of normal anatomy. We feel that a particular strength of the virtual approach is that we are able to use AR to dissect both disease-free normal anatomy and actual tumor cases, improving our experience and understanding of tumor-related changes to normal anatomy in a manner which is not possible in cadaver courses.

The participants also suggested that it would be helpful to develop a more individualized interactivity, which could allow each individual user to control the peeling (virtual dissection), zooming, and rotating of the reconstruction. Likewise, participants asked for development of the ability to separately include and exclude single pathways and to label the fiber tracts with their name and function, either as a default setting or after their identification. During the simulation and assessment session, participants were individually able to walk through the reconstruction. However, functions like zooming, rotating, and the inclusion/exclusion of structures could only be performed by the single master controller, who was the only enabled interactor. We chose this format to facilitate a clearer presentation. However, the currently version of the software already enables all participants to operate individual controllers, so the limitation experienced by the participants was simply due to the setup for the assessment trial.

Another suggestion for future development was to enable the user to interact with the reconstruction with their hands, without needing a controller. Some participants also asked for a wider field-of-view in the goggles, to improve orientation, especially during zooming and rotating of the reconstruction.

In summary, the participants felt that the overall AR system was useful and had good potential for development and a wide range of potential future applications in education and clinical settings. The requested changes are useful for improving the usability of the system in the future.

Limitations and evaluation of tractography

This report only aimed to evaluate a new AR technique for fiber dissection but not the different methods and algorithms for tractography. However, the latter is frequently discussed, so we sought to evaluate this through questions 12–14. As reflected by the answers and through discussions during the meeting, the actual tractography techniques are also an issue when evaluating techniques for fiber tract visualization. The visualized fiber tracts were rated as anatomically correct by 60% and as incorrect by 40%, which must also be seen as a major limitation of tractography. Most of the participants (66.7%) favored the setting of function-based ROIs and 16.7% of participants did not see a difference between anatomy-based and function-based ROIs. Moreover, while 64.3% of participants felt that DTI FT was sufficient for the presented approach of fiber dissection, 35.7% of participants asked for a more sophisticated approach for future applications. The variation of opinion regarding tractography algorithms and methods could be seen as a limitation of this initial study. However, these issues can be addressed with relative ease and speed, so we do not feel that it is a significant limitation for future development of the technique itself.

Apart from these limitations, here, it must also be mentioned that further options might exist for the visualization of fiber tracts in future, for example smartphone-based solutions.

Conclusion

This study evaluating the application of AR for fiber dissection found a generally positive response as rated by different complementary neuroscience specialties. Most participants felt that the greatest initial application would be for educational purposes, with potential clinical uses dependent upon the future developments of the system.