IMHOTEP: virtual reality framework for surgical applications
The data which is available to surgeons before, during and after surgery is steadily increasing in quantity as well as diversity. When planning a patient’s treatment, this large amount of information can be difficult to interpret. To aid in processing the information, new methods need to be found to present multimodal patient data, ideally combining textual, imagery, temporal and 3D data in a holistic and context-aware system.
We present an open-source framework which allows handling of patient data in a virtual reality (VR) environment. By using VR technology, the workspace available to the surgeon is maximized and 3D patient data is rendered in stereo, which increases depth perception. The framework organizes the data into workspaces and contains tools which allow users to control, manipulate and enhance the data. Due to the framework’s modular design, it can easily be adapted and extended for various clinical applications.
The framework was evaluated by clinical personnel (77 participants). The majority of the group stated that a complex surgical situation is easier to comprehend by using the framework, and that it is very well suited for education. Furthermore, the application to various clinical scenarios—including the simulation of excitation propagation in the human atrium—demonstrated the framework’s adaptability. As a feasibility study, the framework was used during the planning phase of the surgical removal of a large central carcinoma from a patient’s liver.
The clinical evaluation showed a large potential and high acceptance for the VR environment in a medical context. The various applications confirmed that the framework is easily extended and can be used in real-time simulation as well as for the manipulation of complex anatomical structures.
KeywordsVirtual reality Surgical planning Advanced medical visualization
We thank the Medien- und Filmgesellschaft Baden-Württemberg for supporting early development of the framework through the Karl-Steinbuch scholarship.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Informed consent was obtained from the study participant.
- 1.Ard T, Krum DM, Phan T, Duncan D, Essex R, Bolas M, Toga A(2017) NIVR: neuro imaging in virtual reality. In: 2017 IEEE virtual reality (VR), pp. 465–466Google Scholar
- 4.Butcher PWS, Ritsos PD (2017) Building immersive data visualizations for the web. In: International conference on cyberworlds (CW), pp 142–145Google Scholar
- 5.Cha YW, Dou M, Chabra R, Menozzi F, State A, Wallen E, Fuchs H (2016) Immersive learning experiences for surgical procedures, vol 220. IOS Press, Amsterdam, pp 55–62Google Scholar
- 6.Cullip TJ, Neumann U(1994) Accelerating volume reconstruction with 3D texture hardware. Technical report, Chapel Hill, NC, USAGoogle Scholar
- 7.Donalek C, Djorgovski SG, Davidoff S, Cioc A, Wang A, Longo G, Norris JS, Zhang J, Lawler E, Yeh S, Mahabal A, Graham MJ, Drake AJ (2014) Immersive and collaborative data visualization using virtual reality platforms. CoRRGoogle Scholar
- 8.Egger J, Gall M, Wallner J, Boechat P, Hann A, Li X, Chen X, Schmalstieg D(2017) HTC Vive MeVisLab integration via OpenVR for medical applications. CoRRGoogle Scholar
- 10.Goldman J, Stebbins G, Fredericks D, Upchurch M (2016) Experiencing Parkinson’s disease psychosis via virtual reality simulation: a novel and effective educational tool. Neurology 86(16 Supplement):P1-011Google Scholar
- 11.He L, Guayaquil-Sosa A, McGraw T (2017) Medical image atlas interaction in virtual reality. In: Immersive analytics workshop. IEEE Vis. http://immersiveanalytics.net/
- 15.Loewe A, Poremba E, Oesterlein TG, Pilia N, Pfeiffer M, Speidel S (2017) An interactive virtual reality environment for analysis of clinical atrial arrhythmias and ablation planning. CinC 44Google Scholar
- 17.Mathur AS (2015) Low cost virtual reality for medical training. In: 2015 IEEE virtual reality (VR), pp 345–346Google Scholar
- 18.Moran A, Gadepally V, Hubbell M, Kepner J (2015) Improving big data visual analytics with interactive virtual reality. In: 2015 IEEE high performance extreme computing conference (HPEC), pp 1–6Google Scholar
- 20.Perhac J, Zeng W, Asada S, Arisona SM, Schubiger S, Burkhard R, Klein B (2017) Urban fusion: visualizing urban data fused with social feeds via a game engine. In: 2017 21st international conference information visualisation (IV), pp 312–317Google Scholar
- 21.Reddivari S, Smith J, Pabalate J (2017) VRvisu: a tool for virtual reality based visualization of medical data. In: 2017 IEEE/ACM international conference on connected health: applications, systems and engineering technologies (CHASE) pp 280–281Google Scholar
- 23.Simpson M, Wallgrn JO, Klippel A, Yang L, Garner G, Keller K, Oprean D, Bansal S (2016) Immersive analytics for multi-objective dynamic integrated climate-economy (DICE) models. In: Proceedings of the 2016 ACM companion on interactive surfaces and spaces, ISS companion 16. ACM, New York, NY, USA, pp 99–105Google Scholar
- 24.Tatzgern M, Kalkofen D, Grasset R, Schmalstieg D (2014) Hedgehog labeling: View management techniques for external labels in 3D space. In: 2014 IEEE virtual reality (VR), pp 27–32Google Scholar
- 26.Vincur J, Navrat P, Polasek I (2017) VR city: software analysis in virtual reality environment. In: 2017 IEEE international conference on software quality, reliability and security companion (QRS-C), pp 509–516Google Scholar
- 27.Zielke MA, Zakhidov D, Hardee G, Evans L, Lenox S, Orr N, Fino D, Mathialagan G (2017) Developing virtual patients with VR/AR for a natural user interface in medical teaching. In: IEEE 5th international conference on serious games and applications for health (SeGAH), vol 5, pp 1–8Google Scholar