Skip to main content
Log in

Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Current laparoscopic images are rich in surface detail but lack information on deeper structures. This report presents a novel method for highlighting these structures during laparoscopic surgery using continuous multislice computed tomography (CT). This has resulted in a more accurate augmented reality (AR) approach, termed “live AR,” which merges three-dimensional (3D) anatomy from live low-dose intraoperative CT with live images from the laparoscope.

Methods

A series of procedures with swine was conducted in a CT room with a fully equipped laparoscopic surgical suite. A 64-slice CT scanner was used to image the surgical field approximately once per second. The procedures began with a contrast-enhanced, diagnostic-quality CT scan (initial CT) of the liver followed by continuous intraoperative CT and laparoscopic imaging with an optically tracked laparoscope. Intraoperative anatomic changes included user-applied deformations and those from breathing. Through deformable image registration, an intermediate image processing step, the initial CT was warped to align spatially with the low-dose intraoperative CT scans. The registered initial CT then was rendered and merged with laparoscopic images to create live AR.

Results

Superior compensation for soft tissue deformations using the described method led to more accurate spatial registration between laparoscopic and rendered CT images with live AR than with conventional AR. Moreover, substitution of low-dose CT with registered initial CT helped with continuous visualization of the vasculature and offered the potential of at least an eightfold reduction in intraoperative X-ray dose.

Conclusions

The authors proposed and developed live AR, a new surgical visualization approach that merges rich surface detail from a laparoscope with instantaneous 3D anatomy from continuous CT scanning of the surgical field. Through innovative use of deformable image registration, they also demonstrated the feasibility of continuous visualization of the vasculature and considerable X-ray dose reduction. This study provides motivation for further investigation and development of live AR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Himal HS (2002) Minimally invasive (laparoscopic) surgery. Surg Endosc 16:1647–1652

    Article  CAS  PubMed  Google Scholar 

  2. Rosen M, Ponsky J (2001) Minimally invasive surgery. Endoscopy 33:358–366

    Article  CAS  PubMed  Google Scholar 

  3. Osborne DA, Alexander G, Boe B, Zervos EE (2006) Laparoscopic cholecystectomy: past, present, and future. Surg Technol Int 15:81–85

    PubMed  Google Scholar 

  4. Hanly EJ, Talamini MA (2004) Robotic abdominal surgery. Am J Surg 188:19S–26S

    Article  PubMed  Google Scholar 

  5. Harrell AG, Heniford BT (2005) Minimally invasive abdominal surgery: lux et veritas past, present, and future. Am J Surg 190:239–243

    Article  PubMed  Google Scholar 

  6. Fuchs H, Livingston MA, Raskar R, Colucci D, Keller K, State A, Crawford JR, Rademacher P, Drake SH, Meyer AA (1998) Augmented reality visualization for laparoscopic surgery. In: Proceedings of the first international conference on medical image computing and computer-assisted intervention. Lecture Notes in Computer Science, vol 1496. Springer Berlin/Heidelberg Cambridge, pp 934–943

  7. Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented reality-assisted laparoscopic adrenalectomy. JAMA 292:2214–2215

    Article  CAS  PubMed  Google Scholar 

  8. Mutter D, Bouras G, Marescaux J (2005) Digital technologies and quality improvement in cancer surgery. Eur J Surg Oncol 31:689–694

    Article  CAS  PubMed  Google Scholar 

  9. Walimbe V, Shekhar R (2006) Automatic elastic image registration by interpolation of 3D rotations and translations from discrete rigid-body transformations. Med Image Anal 10:899–914

    Article  PubMed  Google Scholar 

  10. Dandekar O, Shekhar R (2007) FPGA-accelerated deformable image registration for improved target-delineation during CT-guided interventions. IEEE Trans Biomed Circuits Syst 1:116–127

    Article  Google Scholar 

  11. Dandekar O, Castro-Pareja C, Shekhar R (2007) FPGA-based real-time 3D image preprocessing for image-guided medical interventions. J Real-Time Image Proc 1:285–301

    Article  Google Scholar 

  12. Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27:355–369

    Article  PubMed  Google Scholar 

  13. Shahidi R, Bax MR, Maurer CR Jr, Johnson JA, Wilkinson EP, Wang B, West JB, Citardi MJ, Manwaring KH, Khadem R (2002) Implementation, calibration, and accuracy testing of an image-enhanced endoscopy system. IEEE Trans Med Imaging 21:1524–1535

    Article  PubMed  Google Scholar 

  14. Bouguet J-Y (2009) Camera calibration toolbox for Matlab. Retrieved May 28, 2009, from http://www.vision.caltech.edu/bouguetj/calib_doc/index.html

  15. Hagiike M, Phillips EH, Berci G (2007) Performance differences in laparoscopic surgical skills between true high-definition and three-chip CCD video systems. Surg Endosc 21:1849–1854

    Article  CAS  PubMed  Google Scholar 

  16. Pierre SA, Ferrandino MN, Simmons WN, Fernandez C, Zhong P, Albala DM, Preminger GM (2009) High-definition laparoscopy: objective assessment of performance characteristics and comparison with standard laparoscopy. J Endourol 23:523–528

    Article  PubMed  Google Scholar 

  17. Miller A, Allen P, Fowler D (2004) In vivo stereoscopic imaging system with 5 degrees of freedom for minimal access surgery. Stud Health Technol Inform 98:234–240

    PubMed  Google Scholar 

  18. Cannon JW, Stoll JA, Salgo IS, Knowles HB, Howe RD, Dupont PE, Marx GR, del Nido PJ (2003) Real-time three-dimensional ultrasound for guiding surgical tasks. Comput Aided Surg 8:82–90

    Article  PubMed  Google Scholar 

  19. Sugeng L, Weinert L, Thiele K, Lang RM (2003) Real-time three-dimensional echocardiography using a novel matrix array transducer. Echocardiography 20:623–635

    Article  PubMed  Google Scholar 

  20. Kalender WA (2006) X-ray computed tomography. Phys Med Biol 51:R29–R43

    Article  PubMed  Google Scholar 

  21. Rogalla P, Kloeters C, Hein PA (2009) CT technology overview: 64-slice and beyond. Radiol Clin North Am 47:1–11

    Article  PubMed  Google Scholar 

  22. Dandekar O, Siddiqui J, Walimbe V, Shekhar R (2006) Image registration accuracy with low-dose CT: how low can we go? In: Proceedings of the third IEEE international symposium on biomedical imaging: nano to macro, Institute of Electrical and Electronics Engineers Computer Society, Arlington, pp 502–505

  23. Crane NJ, McHone B, Hawksworth J, Pearl JP, Denobile J, Tadaki D, Pinto PA, Levin IW, Elster EA (2008) Enhanced surgical imaging: laparoscopic vessel identification and assessment of tissue oxygenation. J Am Coll Surg 206:1159–1166

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This work was supported by U.S. Department of Defense grants DAMD17-03-2-0001 and W81XWH-06-2-0057. We acknowledge the editorial help of Nancy Knight, PhD, and the assistance of Zhitong Yang, PhD, in radiation exposure measurements. Both individuals are affiliated with the Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, Maryland.

Disclosures

Raj Shekhar has an equity interest in IGI Technologies, Inc., a technology startup he has founded. Omkar Dandekar, Carlos Godinez, Erica Sutton, Steven Kavic, Reuben Mezrich, Adrian Park, Venkatesh Bhat, Mathew Philip, Peng Lei, and Ivan George have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Raj Shekhar.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (M1V 4383 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shekhar, R., Dandekar, O., Bhat, V. et al. Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24, 1976–1985 (2010). https://doi.org/10.1007/s00464-010-0890-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-010-0890-8

Keywords

Navigation