Abstract
Background
Current laparoscopic images are rich in surface detail but lack information on deeper structures. This report presents a novel method for highlighting these structures during laparoscopic surgery using continuous multislice computed tomography (CT). This has resulted in a more accurate augmented reality (AR) approach, termed “live AR,” which merges three-dimensional (3D) anatomy from live low-dose intraoperative CT with live images from the laparoscope.
Methods
A series of procedures with swine was conducted in a CT room with a fully equipped laparoscopic surgical suite. A 64-slice CT scanner was used to image the surgical field approximately once per second. The procedures began with a contrast-enhanced, diagnostic-quality CT scan (initial CT) of the liver followed by continuous intraoperative CT and laparoscopic imaging with an optically tracked laparoscope. Intraoperative anatomic changes included user-applied deformations and those from breathing. Through deformable image registration, an intermediate image processing step, the initial CT was warped to align spatially with the low-dose intraoperative CT scans. The registered initial CT then was rendered and merged with laparoscopic images to create live AR.
Results
Superior compensation for soft tissue deformations using the described method led to more accurate spatial registration between laparoscopic and rendered CT images with live AR than with conventional AR. Moreover, substitution of low-dose CT with registered initial CT helped with continuous visualization of the vasculature and offered the potential of at least an eightfold reduction in intraoperative X-ray dose.
Conclusions
The authors proposed and developed live AR, a new surgical visualization approach that merges rich surface detail from a laparoscope with instantaneous 3D anatomy from continuous CT scanning of the surgical field. Through innovative use of deformable image registration, they also demonstrated the feasibility of continuous visualization of the vasculature and considerable X-ray dose reduction. This study provides motivation for further investigation and development of live AR.
Similar content being viewed by others
References
Himal HS (2002) Minimally invasive (laparoscopic) surgery. Surg Endosc 16:1647–1652
Rosen M, Ponsky J (2001) Minimally invasive surgery. Endoscopy 33:358–366
Osborne DA, Alexander G, Boe B, Zervos EE (2006) Laparoscopic cholecystectomy: past, present, and future. Surg Technol Int 15:81–85
Hanly EJ, Talamini MA (2004) Robotic abdominal surgery. Am J Surg 188:19S–26S
Harrell AG, Heniford BT (2005) Minimally invasive abdominal surgery: lux et veritas past, present, and future. Am J Surg 190:239–243
Fuchs H, Livingston MA, Raskar R, Colucci D, Keller K, State A, Crawford JR, Rademacher P, Drake SH, Meyer AA (1998) Augmented reality visualization for laparoscopic surgery. In: Proceedings of the first international conference on medical image computing and computer-assisted intervention. Lecture Notes in Computer Science, vol 1496. Springer Berlin/Heidelberg Cambridge, pp 934–943
Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented reality-assisted laparoscopic adrenalectomy. JAMA 292:2214–2215
Mutter D, Bouras G, Marescaux J (2005) Digital technologies and quality improvement in cancer surgery. Eur J Surg Oncol 31:689–694
Walimbe V, Shekhar R (2006) Automatic elastic image registration by interpolation of 3D rotations and translations from discrete rigid-body transformations. Med Image Anal 10:899–914
Dandekar O, Shekhar R (2007) FPGA-accelerated deformable image registration for improved target-delineation during CT-guided interventions. IEEE Trans Biomed Circuits Syst 1:116–127
Dandekar O, Castro-Pareja C, Shekhar R (2007) FPGA-based real-time 3D image preprocessing for image-guided medical interventions. J Real-Time Image Proc 1:285–301
Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27:355–369
Shahidi R, Bax MR, Maurer CR Jr, Johnson JA, Wilkinson EP, Wang B, West JB, Citardi MJ, Manwaring KH, Khadem R (2002) Implementation, calibration, and accuracy testing of an image-enhanced endoscopy system. IEEE Trans Med Imaging 21:1524–1535
Bouguet J-Y (2009) Camera calibration toolbox for Matlab. Retrieved May 28, 2009, from http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
Hagiike M, Phillips EH, Berci G (2007) Performance differences in laparoscopic surgical skills between true high-definition and three-chip CCD video systems. Surg Endosc 21:1849–1854
Pierre SA, Ferrandino MN, Simmons WN, Fernandez C, Zhong P, Albala DM, Preminger GM (2009) High-definition laparoscopy: objective assessment of performance characteristics and comparison with standard laparoscopy. J Endourol 23:523–528
Miller A, Allen P, Fowler D (2004) In vivo stereoscopic imaging system with 5 degrees of freedom for minimal access surgery. Stud Health Technol Inform 98:234–240
Cannon JW, Stoll JA, Salgo IS, Knowles HB, Howe RD, Dupont PE, Marx GR, del Nido PJ (2003) Real-time three-dimensional ultrasound for guiding surgical tasks. Comput Aided Surg 8:82–90
Sugeng L, Weinert L, Thiele K, Lang RM (2003) Real-time three-dimensional echocardiography using a novel matrix array transducer. Echocardiography 20:623–635
Kalender WA (2006) X-ray computed tomography. Phys Med Biol 51:R29–R43
Rogalla P, Kloeters C, Hein PA (2009) CT technology overview: 64-slice and beyond. Radiol Clin North Am 47:1–11
Dandekar O, Siddiqui J, Walimbe V, Shekhar R (2006) Image registration accuracy with low-dose CT: how low can we go? In: Proceedings of the third IEEE international symposium on biomedical imaging: nano to macro, Institute of Electrical and Electronics Engineers Computer Society, Arlington, pp 502–505
Crane NJ, McHone B, Hawksworth J, Pearl JP, Denobile J, Tadaki D, Pinto PA, Levin IW, Elster EA (2008) Enhanced surgical imaging: laparoscopic vessel identification and assessment of tissue oxygenation. J Am Coll Surg 206:1159–1166
Acknowledgments
This work was supported by U.S. Department of Defense grants DAMD17-03-2-0001 and W81XWH-06-2-0057. We acknowledge the editorial help of Nancy Knight, PhD, and the assistance of Zhitong Yang, PhD, in radiation exposure measurements. Both individuals are affiliated with the Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, Maryland.
Disclosures
Raj Shekhar has an equity interest in IGI Technologies, Inc., a technology startup he has founded. Omkar Dandekar, Carlos Godinez, Erica Sutton, Steven Kavic, Reuben Mezrich, Adrian Park, Venkatesh Bhat, Mathew Philip, Peng Lei, and Ivan George have no conflicts of interest or financial ties to disclose.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Shekhar, R., Dandekar, O., Bhat, V. et al. Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24, 1976–1985 (2010). https://doi.org/10.1007/s00464-010-0890-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-010-0890-8