Advertisement

IMHOTEP: virtual reality framework for surgical applications

  • Micha PfeifferEmail author
  • Hannes Kenngott
  • Anas Preukschas
  • Matthias Huber
  • Lisa Bettscheider
  • Beat Müller-Stich
  • Stefanie Speidel
Original Article

Abstract

Purpose

The data which is available to surgeons before, during and after surgery is steadily increasing in quantity as well as diversity. When planning a patient’s treatment, this large amount of information can be difficult to interpret. To aid in processing the information, new methods need to be found to present multimodal patient data, ideally combining textual, imagery, temporal and 3D data in a holistic and context-aware system.

Methods

We present an open-source framework which allows handling of patient data in a virtual reality (VR) environment. By using VR technology, the workspace available to the surgeon is maximized and 3D patient data is rendered in stereo, which increases depth perception. The framework organizes the data into workspaces and contains tools which allow users to control, manipulate and enhance the data. Due to the framework’s modular design, it can easily be adapted and extended for various clinical applications.

Results

The framework was evaluated by clinical personnel (77 participants). The majority of the group stated that a complex surgical situation is easier to comprehend by using the framework, and that it is very well suited for education. Furthermore, the application to various clinical scenarios—including the simulation of excitation propagation in the human atrium—demonstrated the framework’s adaptability. As a feasibility study, the framework was used during the planning phase of the surgical removal of a large central carcinoma from a patient’s liver.

Conclusion

The clinical evaluation showed a large potential and high acceptance for the VR environment in a medical context. The various applications confirmed that the framework is easily extended and can be used in real-time simulation as well as for the manipulation of complex anatomical structures.

Keywords

Virtual reality Surgical planning Advanced medical visualization 

Notes

Acknowledgements

We thank the Medien- und Filmgesellschaft Baden-Württemberg for supporting early development of the framework through the Karl-Steinbuch scholarship.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from the study participant.

References

  1. 1.
    Ard T, Krum DM, Phan T, Duncan D, Essex R, Bolas M, Toga A(2017) NIVR: neuro imaging in virtual reality. In: 2017 IEEE virtual reality (VR), pp. 465–466Google Scholar
  2. 2.
    Atherton S, Antley A, Evans N, Cernis E, Lister R, Dunn G, Slater M, Freeman D (2016) Self-confidence and paranoia: an experimental study using an immersive virtual reality social situation. Behav Cognit Psychother 44(1):56–64CrossRefGoogle Scholar
  3. 3.
    Blinn JF (1977) Models of light reflection for computer synthesized pictures. SIGGRAPH Comput Graph 11(2):192–198CrossRefGoogle Scholar
  4. 4.
    Butcher PWS, Ritsos PD (2017) Building immersive data visualizations for the web. In: International conference on cyberworlds (CW), pp 142–145Google Scholar
  5. 5.
    Cha YW, Dou M, Chabra R, Menozzi F, State A, Wallen E, Fuchs H (2016) Immersive learning experiences for surgical procedures, vol 220. IOS Press, Amsterdam, pp 55–62Google Scholar
  6. 6.
    Cullip TJ, Neumann U(1994) Accelerating volume reconstruction with 3D texture hardware. Technical report, Chapel Hill, NC, USAGoogle Scholar
  7. 7.
    Donalek C, Djorgovski SG, Davidoff S, Cioc A, Wang A, Longo G, Norris JS, Zhang J, Lawler E, Yeh S, Mahabal A, Graham MJ, Drake AJ (2014) Immersive and collaborative data visualization using virtual reality platforms. CoRRGoogle Scholar
  8. 8.
    Egger J, Gall M, Wallner J, Boechat P, Hann A, Li X, Chen X, Schmalstieg D(2017) HTC Vive MeVisLab integration via OpenVR for medical applications. CoRRGoogle Scholar
  9. 9.
    Farahani N, Post R, Duboy J, Ahmed I, Kolowitz B, Krinchai T, Monaco S, Fine J, Hartman D, Pantanowitz L (2016) Exploring virtual reality technology and the Oculus Rift for the examination of digital pathology slides. J Pathol Inf 7(1):22CrossRefGoogle Scholar
  10. 10.
    Goldman J, Stebbins G, Fredericks D, Upchurch M (2016) Experiencing Parkinson’s disease psychosis via virtual reality simulation: a novel and effective educational tool. Neurology 86(16 Supplement):P1-011Google Scholar
  11. 11.
    He L, Guayaquil-Sosa A, McGraw T (2017) Medical image atlas interaction in virtual reality. In: Immersive analytics workshop. IEEE Vis. http://immersiveanalytics.net/
  12. 12.
    Herfarth C, Lamad W, Fischer L, Chiu P, Cardenas C, Thorn M, Vetter M, Grenacher L, Meinzer HP (2002) The effect of virtual reality and training on liver operation planning. Swiss Surg 8(2):67–73CrossRefPubMedGoogle Scholar
  13. 13.
    Huber T, Paschold M, Hansen C, Wunderling T, Lang H, Kneist W (2017) New dimensions in surgical training: immersive virtual reality laparoscopic simulation exhilarates surgical staff. Surg Endosc 31:4472CrossRefPubMedGoogle Scholar
  14. 14.
    Khor WS, Baker B, Amin K, Chan A, Patel K, Wong J (2016) Augmented and virtual reality in surgery: the digital surgical environment: applications, limitations and legal pitfalls. Annals Transl Med 4:454CrossRefGoogle Scholar
  15. 15.
    Loewe A, Poremba E, Oesterlein TG, Pilia N, Pfeiffer M, Speidel S (2017) An interactive virtual reality environment for analysis of clinical atrial arrhythmias and ablation planning. CinC 44Google Scholar
  16. 16.
    Maples-Keller JL, Yasinski C, Manjin N, Rothbaum BO (2017) Virtual reality-enhanced extinction of phobias and post-traumatic stress. Neurotherapeutics 14(3):554–563CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Mathur AS (2015) Low cost virtual reality for medical training. In: 2015 IEEE virtual reality (VR), pp 345–346Google Scholar
  18. 18.
    Moran A, Gadepally V, Hubbell M, Kepner J (2015) Improving big data visual analytics with interactive virtual reality. In: 2015 IEEE high performance extreme computing conference (HPEC), pp 1–6Google Scholar
  19. 19.
    Müller-Stich BP, Löb N, Wald D, Bruckner T, Meinzer HP, Kadmon M, Bchler MW, Fischer L (2013) Regular three-dimensional presentations improve in the identification of surgical liver anatomy a randomized study. BMC Med Educ 13(1):131CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Perhac J, Zeng W, Asada S, Arisona SM, Schubiger S, Burkhard R, Klein B (2017) Urban fusion: visualizing urban data fused with social feeds via a game engine. In: 2017 21st international conference information visualisation (IV), pp 312–317Google Scholar
  21. 21.
    Reddivari S, Smith J, Pabalate J (2017) VRvisu: a tool for virtual reality based visualization of medical data. In: 2017 IEEE/ACM international conference on connected health: applications, systems and engineering technologies (CHASE) pp 280–281Google Scholar
  22. 22.
    Schmitt YS, Hoffman HG, Blough DK, Patterson DR, Jensen MP, Soltani M, Carrougher GJ, Nakamura D, Sharar SR (2011) A randomized, controlled trial of immersive virtual reality analgesia, during physical therapy for pediatric burns. Burns 37(1):61–68CrossRefPubMedGoogle Scholar
  23. 23.
    Simpson M, Wallgrn JO, Klippel A, Yang L, Garner G, Keller K, Oprean D, Bansal S (2016) Immersive analytics for multi-objective dynamic integrated climate-economy (DICE) models. In: Proceedings of the 2016 ACM companion on interactive surfaces and spaces, ISS companion 16. ACM, New York, NY, USA, pp 99–105Google Scholar
  24. 24.
    Tatzgern M, Kalkofen D, Grasset R, Schmalstieg D (2014) Hedgehog labeling: View management techniques for external labels in 3D space. In: 2014 IEEE virtual reality (VR), pp 27–32Google Scholar
  25. 25.
    Valmaggia LR, Day F, Rus-Calafell M (2016) Using virtual reality to investigate psychological processes and mechanisms associated with the onset and maintenance of psychosis: a systematic review. Soc Psychiatry Psychiatr Epidemiol 51(7):921–936CrossRefPubMedGoogle Scholar
  26. 26.
    Vincur J, Navrat P, Polasek I (2017) VR city: software analysis in virtual reality environment. In: 2017 IEEE international conference on software quality, reliability and security companion (QRS-C), pp 509–516Google Scholar
  27. 27.
    Zielke MA, Zakhidov D, Hardee G, Evans L, Lenox S, Orr N, Fino D, Mathialagan G (2017) Developing virtual patients with VR/AR for a natural user interface in medical teaching. In: IEEE 5th international conference on serious games and applications for health (SeGAH), vol 5, pp 1–8Google Scholar

Copyright information

© CARS 2018

Authors and Affiliations

  1. 1.National Center for Tumor DiseasesDresdenGermany
  2. 2.Institute for Anthropomatics and RoboticsKarlsruhe Institute of TechnologyKarlsruheGermany
  3. 3.Department of General, Visceral and Transplant SurgeryHeidelberg University HospitalHeidelbergGermany

Personalised recommendations