Abstract
This paper presents a novel mixed reality based navigation system for accurate respiratory liver tumor punctures in radiofrequency ablation (RFA). Our system contains an optical see-through head-mounted display device (OST-HMD), Microsoft HoloLens for perfectly overlaying the virtual information on the patient, and a optical tracking system NDI Polaris for calibrating the surgical utilities in the surgical scene. Compared with traditional navigation method with CT, our system aligns the virtual guidance information and real patient and real-timely updates the view of virtual guidance via a position tracking system. In addition, to alleviate the difficulty during needle placement induced by respiratory motion, we reconstruct the patient-specific respiratory liver motion through statistical motion model to assist doctors precisely puncture liver tumors. The proposed system has been experimentally validated on vivo pigs with an accurate real-time registration approximately 5-mm mean FRE and TRE, which has the potential to be applied in clinical RFA guidance.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Wang, Z.; Aarya, I.; Gueorguieva, M.; Liu, D.; Luo, H.; Manfredi, L.; Wang, L.; McLean, D.; Coleman, S.; Brown, S.; Cuschieri, A. Image-based 3D modeling and validation of radiofrequency interstitial tumor ablation using a tissue-mimicking breast phantom. International Journal of Computer Assisted Radiology and Surgery Vol. 7, No. 6, 941–948, 2012.
Flaherty, D. C.; Bilchik, A. J. Radiofrequency ablation of liver tumors. In: Blumgart’s Surgery of the Liver, Biliary Tract and Pancreas, 2-Volume Set, 6th edn. Belghiti, J.; Jarnagin, W. R. Eds. Elsevier, 1436–1447, 2017.
Cai, K.; Yang, R.; Chen, H.; Ning, H.; Ma, A.; Zhou, J.; Huang, W.; Ou, S. Simulation and visualization of liver cancer ablation focus in optical surgical navigation. Journal of Medical Systems Vol. 40, No. 1, 19, 2016.
Clasen, S.; Pereira, P. L. Magnetic resonance guidance for radiofrequency ablation of liver tumors. Journal of Magnetic Resonance Imaging Vol. 27, No. 2, 421–433, 2008.
Crocetti, L.; Della Pina, M. C.; Cioni, D.; Lencioni, R. Image-guided ablation of hepatocellular carcinoma. In: Interventional Oncology: Principles and Practice of Image-Guided Cancer Therapy. Geschwind, J. F. H.; Soulen, M. C. Eds. Cambridge University Press, 91, 2016.
Cazzato, R. L.; Garnon, J.; Ramamurthy, N.; Tsoumakidou, G.; Imperiale, A.; Namer, I. J.; Bachellier, P.; Caudrelier, J.; Rao, P.; Koch, G.; Gangi, A. 18F-FDOPA PET/CT-guided radiofrequency ablation of liver metastases from neuroendocrine tumours: Technical note on a preliminary experience. CardioVascular and Interventional Radiology Vol. 39, No. 9, 1315–1321, 2016.
Bernhardt, S.; Nicolau, S. A.; Soler, L.; Doignon, C. The status of augmented reality in laparoscopic surgery as of 2016. Medical Image Analysis Vol. 37, 66–90, 2017.
Nicolau, S. A.; Pennec, X.; Soler, L.; Ayache, N. Clinical evaluation of a respiratory gated guidance system for liver punctures. In: Medical Image Computing and Computer-Assisted Intervention — MICCAI 2007. Lecture Notes in Computer Science, Vol. 4792. Ayache, N.; Ourselin, S.; Maeder, A. Eds. Springer Berlin Heidelberg, 77–85, 2007.
Biro, P.; Spahn, D. R.; Pfammatter, T. High-frequency jet ventilation for minimizing breathing-related liver motion during percutaneous radiofrequency ablation of multiple hepatic tumours. British Journal of Anaesthesia Vol. 102, No. 5, 650–653, 2009.
Wunderink, W.; Romero, A. M.; de Kruijf, W.; de Boer, H.; Levendag, P.; Heijmen, B. Reduction of respiratory liver tumor motion by abdominal compression in stereotactic body frame, analyzed by tracking fiducial markers implanted in liver. International Journal of Radiation Oncology Biology Physics Vol. 71, No. 3, 907–915, 2008.
Breen, D. J.; Lencioni, R. Image-guided ablation of primary liver and renal tumours. Nature Reviews Clinical Oncology Vol. 12, No. 3, 175–186, 2015.
Tiong, L.; Maddern, G. J. Systematic review and meta-analysis of survival and disease recurrence after radiofrequency ablation for hepatocellular carcinoma. British Journal of Surgery Vol. 98, No. 9, 1210–1224, 2011.
Ahmed, M.; Brace, C. L.; Lee Jr., F. T.; Goldberg, S. N. Principles of and advances in percutaneous ablation. Radiology Vol. 258, No. 2, 351–369, 2011.
Livraghi, T.; Mäkisalo, H.; Line, P.-D. Treatment options in hepatocellular carcinoma today. Scandinavian Journal of Surgery Vol. 100, No. 1, 22–29, 2011.
Kim, P. N.; Choi, D.; Rhim, H.; Rha, S. E.; Hong, H. P.; Lee, J.; Choi, J.-I.; Kim, J. W.; Seo, J. W.; Lee, E. J.; Lim, H. K. Planning ultrasound for percutaneous radiofrequency ablation to treat small (≦3 cm) hepatocellular carcinomas detected on computed tomography or magnetic resonance imaging: A multicenter prospective study to assess factors affecting ultrasound visibility. Journal of Vascular and Interventional Radiology Vol. 23, No. 5, 627–634, 2012.
Amalou, H.; Wood, B. J. Electromagnetic tracking navigation to guide radiofrequency ablation (RFA) of a lung tumor. Journal of Bronchology & Interventional Pulmonology Vol. 19, No. 4, 323–327, 2012.
Sauer, F.; Schoepf, U. J.; Khamene, A.; Vogt, S.; Das, M.; Silverman, S. G. Augmented reality system for CT-guided interventions: System description and initial phantom trials. In: Proceedings of the SPIE 5029, Medical Imaging 2003: Visualization, Image-Guided Procedures, and Display, 384–395, 2003.
Khan, M. F.; Dogan, S.; Maataoui, A.; Wesarg, S.; Gurung, J.; Ackermann, H.; Schiemann, M.; Wimmer-Greinecker, G.; Vogl, T. J. Navigation-based needle puncture of a cadaver using a hybrid tracking navigational system. Investigative Radiology Vol. 41, No. 10, 713–720, 2006.
Ren, H.; Campos-Nanez, E.; Yaniv, Z.; Banovac, F.; Abeledo, H.; Hata, N.; Cleary, K. Treatment planning and image guidance for radiofrequency ablation of large tumors. IEEE Journal of Biomedical and Health Informatics Vol. 18, No. 3, 920–928, 2014.
Chan, W.-Y.; Heng, P.-A. Visualization of needle access pathway and a five-dof evaluation. IEEE Journal of Biomedical and Health Informatics Vol. 18, No. 2, 643–653, 2014.
Schweikard, A.; Shiomi, H.; Adler, J. Respiration tracking in radiosurgery. Medical Physics Vol. 31, No. 10, 2738–2741, 2004.
Ren, Q.; Nishioka, S.; Shirato, H.; Berbeco, R. I. Adaptive prediction of respiratory motion for motion compensation radiotherapy. Physics in Medicine and Biology Vol. 52, No. 22, 6651, 2007.
Jud, C.; Cattin, P. C.; Preiswerk, F. Chapter 14— Statistical respiratory models for motion estimation. In: Statistical Shape and Deformation Analysis. 379–407, 2017.
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (Nos. U1813204 and 61802385), in part by HK RGC TRS project T42-409/18-R, in part by HK RGC project CUHK14225616, in part by CUHK T Stone Robotics Institute, CUHK, and in part by the Science and Technology Plan Project of Guangzhou (No. 201704020141). The authors would like to thank Yanfang Zhang and Jianxi Guo (Shenzhen People’s Hospital) for providing the medical support, and Rui Zheng for the useful discussions. Special thanks to the reviewers and editors of Computational Visual Media.
Author information
Authors and Affiliations
Corresponding author
Additional information
Ruotong Li is a Ph.D. student in Department of Computer Science II at the University of Bonn, Germany. Her research interests include computer graphics and augmented reality applications in medicine.
Weixin Si is an associate researcher in the Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. His research interests include virtual reality/augmented reality/mixed reality (XR) applications in medicine and physically-based simulation.
Xiangyun Liao is an associate researcher in the Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. His research interests include virtual reality, physically-based simulation, and medical imaging.
Qiong Wang is an associate researcher in the Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences. Her research interests include human-computer interaction and computer graphics.
Reinhard Klein is a professor in Department of Computer Science II at the University of Bonn, Germany. His research interests include computer graphics, human-computer interaction as well as virtual and augmented reality.
Pheng-Ann Heng is a professor in the Department of Computer Science and Engineering at the Chinese University of Hong Kong. His research interests include AI and VR for medical applications, surgical simulation, visualization, graphics, and human-computer interaction.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
About this article
Cite this article
Li, R., Si, W., Liao, X. et al. Mixed reality based respiratory liver tumor puncture navigation. Comp. Visual Media 5, 363–374 (2019). https://doi.org/10.1007/s41095-019-0156-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-019-0156-x