Abstract
Medical volume data is usually explored on monoscopic monitors. Displaying this data in three-dimensional space facilitates the development of mental maps and the identification of anatomical structures and their spatial relations. Using augmented reality (AR) may further enhance these effects by spatially aligning the volume data with the patient. However, conventional interaction methods, e.g. mouse and keyboard, may not be applicable in this environment. Appropriate interaction techniques are needed to naturally and intuitively manipulate the image data. To this end, a user study comparing four gestural interaction techniques with respect to both clipping and windowing tasks was conducted. Image data was directly displayed on a phantom using stereoscopic projective AR and direct volume visualization. Participants were able to complete both tasks with all interaction techniques with respectively similar clipping accuracy and windowing efficiency. However, results suggest advantages of gestures based on motion-sensitive devices in terms of reduced task completion time and less subjective workload. This work presents an important first step towards a surgical AR visualization system enabling intuitive exploration of volume data. Yet, more research is required to assess the interaction techniques’ applicability for intraoperative use.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
D’Agostino, J., Diana, M., Vix, M., Soler, L., Marescaux, J.: Three-dimensional virtual neck exploration before parathyroidectomy. New Engl. J. Med. 367(11), 1072–1073 (2012)
Egger, J., et al.: HTC Vive MeVisLab integration via OpenVR for medical applications. PloS One 12(3), e0173972 (2017)
Gallo, L.: A study on the degrees of freedom in touchless interaction. In: SIGGRAPH Asia 2013 Technical Briefs, p. 28. ACM (2013)
Gallo, L., De Pietro, G., Marra, I.: 3D interaction with volumetric medical data: experiencing the wiimote. In: Proceedings of the 1st International Conference on Ambient Media and Systems. Ambi-Sys ’08, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Brussels, BEL (2008)
Gierwiało, R., Witkowski, M., Kosieradzki, M., Lisik, W., Groszkowski, Ł., Sitnik, R.: Medical augmented-reality visualizer for surgical training and education in medicine. Appl. Sci. 9(13), 2732 (2019)
Goth, G.: Brave NUI world. Commun. ACM 54(12), 14–16 (2011)
Hatscher, B., Luz, M., Hansen, C.: Foot interaction concepts to support radiological interventions. i-com 17(1), 3–13 (2018)
Hettig, J., Mewes, A., Riabikin, O., Skalej, M., Preim, B., Hansen, C.: Exploration of 3D medical image data for interventional radiology using myoelectric gesture control. In: Proceedings of the Eurographics Workshop on Visual Computing for Biology and Medicine, pp. 177–185. Eurographics Association (2015)
Hettig, J., Saalfeld, P., Luz, M., Becker, M., Skalej, M., Hansen, C.: Comparison of gesture and conventional interaction techniques for interventional neuroradiology. Int. J. Comput. Assist. Radiol. Surg. 12(9), 1643–1653 (2017)
Jalaliniya, S., Smith, J., Sousa, M., Büthe, L., Pederson, T.: Touch-less interaction with medical images using hand & foot gestures. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 1265–1274 (2013)
Jankowski, J., Hachet, M.: A survey of interaction techniques for interactive 3D environments. In: Eurographics (2013)
Kirmizibayrak, C., Radeva, N., Wakid, M., Philbeck, J., Sibert, J., Hahn, J.: Evaluation of gesture based interfaces for medical volume visualization tasks. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and its Applications in Industry, pp. 69–74. ACM (2011)
Mentis, H.M., et al.: Voice or gesture in the operating room. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 773–780. ACM (2015)
Mewes, A., Hensen, B., Wacker, F., Hansen, C.: Touchless interaction with software in interventional radiology and surgery: a systematic literature review. Int. J. Comput. Assist. Radiol. Surg. 12(2), 291–305 (2017)
Mewes, A., Saalfeld, P., Riabikin, O., Skalej, M., Hansen, C.: A gesture-controlled projection display for CT-guided interventions. Int. J. Comput. Assist. Radiol. Surg. 11(1), 157–164 (2016)
Nicolau, S., Soler, L., Mutter, D., Marescaux, J.: Augmented reality in laparoscopic surgical oncology. Surg. Oncol. 20(3), 189–201 (2011)
de Oliveira, M.E., Debarba, H.G., Lädermann, A., Chagué, S., Charbonnier, C.: A hand-eye calibration method for augmented reality applied to computer-assisted orthopedic surgery. Int. J. Med. Robot. Comput. Assist. Surg. 15, e1969 (2019). https://doi.org/10.1002/rcs.1969
Pfeiffer, M., et al.: IMHOTEP: virtual reality framework for surgical applications. Int. J. Comput. Assist. Radiol. Surg. 13(5), 741–748 (2018)
Silén, C., Wirell, S., Kvist, J., Nylander, E., Smedby, Ö.: Advanced 3D visualization in student-centred medical education. Med. Teach. 30(5), e115–e124 (2008)
Silva, É.S., Rodrigues, M.A.F.: Design and evaluation of a gesture-controlled system for interactive manipulation of medical images and 3D models. SBC J. Interact. Syst. 5(3), 53–65 (2014)
Sugimoto, M., et al.: Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J. Hep.-Biliary-Pancreat. Sci. 17(5), 629–636 (2010)
Thomas, R.G., William John, N., Delieu, J.M.: Augmented reality for anatomical education. J. Vis. Commun. Med. 33(1), 6–15 (2010)
Velloso, E., Schmidt, D., Alexander, J., Gellersen, H., Bulling, A.: The feet in human-computer interaction: a survey of foot-based interaction. ACM Comput. Surv. (CSUR) 48(2), 21 (2015)
Wen, R., Nguyen, B.P., Chng, C.B., Chui, C.K.: In situ spatial AR surgical planning using projector-kinect system. In: Proceedings of the Fourth Symposium on Information and Communication Technology, pp. 164–171 (2013)
Acknowledgments
This work was funded by the German Research Foundation (HA 7819/1-2 & LA 3855/1-2).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Heinrich, F., Bornemann, K., Lawonn, K., Hansen, C. (2020). Interacting with Medical Volume Data in Projective Augmented Reality. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12263. Springer, Cham. https://doi.org/10.1007/978-3-030-59716-0_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-59716-0_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59715-3
Online ISBN: 978-3-030-59716-0
eBook Packages: Computer ScienceComputer Science (R0)
