Advertisement

Augmented reality visualization for laparoscopic surgery

  • Henry Fuchs
  • Mark A. Livingston
  • Ramesh Raskar
  • D’nardo Colucci
  • Kurtis Keller
  • Andrei State
  • Jessica R. Crawford
  • Paul Rademacher
  • Samuel H. Drake
  • Anthony A. Meyer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1496)

Abstract

We present the design and a prototype implementation of a three-dimensional visualization system to assist with laparoscopic surgical procedures. The system uses 3D visualization, depth extraction from laparoscopic images, and six degree-of-freedom head and laparoscope tracking to display a merged real and synthetic image in the surgeon’s video-see-through head-mounted display. We also introduce a custom design for this display. A digital light projector, a camera, and a conventional laparoscope create a prototype 3D laparoscope that can extract depth and video imagery.

Such a system can restore the physician’s natural point of view and head motion parallax that are used to understand the 3D structure during open surgery. These cues are not available in conventional laparoscopic surgery due to the displacement of the laparoscopic camera from the physician’s viewpoint. The system can also display multiple laparoscopic range imaging data sets to widen the effective field of view of the device. These data sets can be displayed in true 3D and registered to the exterior anatomy of the patient. Much work remains to realize a clinically useful system, notably in the acquisition speed, reconstruction, and registration of the 3D imagery.

Keywords

Augmented Reality Motion Parallax Conventional Laparoscopic Surgery Laparoscopic Camera Video Imagery 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. [Bajura92]
    Bajura, M., Fuchs, H., and Ohbuchi, R. (1992). Merging virtual objects with the real world: Seeing ultrasound imagery within the patient. In Computer Graphics (SIGGRAPH ’92 Proceedings), volume 26, pages 203–210.CrossRefGoogle Scholar
  2. [Besl89]
    Besl, P. J. (1989). Active optical range imaging sensors. In Advances in Machine Vision, pages 1–63. Springer-Verlag.Google Scholar
  3. [Daley95]
    Daley, R. C., Hassebrook, L. G., Stanley C. Tungate, J., Jones, J. M., Reisig, H. T., Reed, T. A., Williams, B. K., Daugherty, J. S., and Bond, M. (1995). Topographical analysis with time modulated structured light. SPIE Proceedings, 2488(5):396–407.CrossRefGoogle Scholar
  4. [Durrani95]
    Durrani, A. F. and Preminger, G. M. (1995). Three-dimensional video imaging for endoscopic surgery. Computers in Biological Medicine, 25(2):237–247.CrossRefGoogle Scholar
  5. [Edwards95]
    Edwards, P., Hawkes, D., Hill, D., Jewell, D., Spink, R., Strong, A., and Gleeson, M. (1996). Augmentation of reality in the stereo operating microscope for otolaryngology and neurosurgical guidance. Journal of Image-Guided Surgery, 1(3).Google Scholar
  6. [Faugeras93]
    Faugeras, O. (1993). Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press.Google Scholar
  7. [Fuchs96]
    Fuchs, H., State, A., Pisano MD, E. D., Garrett, W. F., Hirota, G., Livingston, M. A., Whitton, M. C., and Pizer, S. M. (1996). Towards performing ultrasound-guided needle biopsies from within a head-mounted display. In Visualization in Biomedical Computing 1996, pages 591–600.Google Scholar
  8. [Grimson95]
    Grimson, W., Ettinger, G., White, S., Gleason, P., Lozano-Pérez, T., Wells III, W., and Kikinis, R. (1995). Evaluating and validating an automated registration system for enhanced reality visualization in surgery. In Proceedings of Computer Vision, Virtual Reality, and Robotics in Medicine ’95 (CVRMed ’95).Google Scholar
  9. [Hornbeck95]
    Hornbeck, L. J. (1995). Digital light processing and MEMS: Timely convergence for a bright future. In Micromachining and Microfabrication ’95.Google Scholar
  10. [IGT97]
    Image-Guided Technologies, Inc. (1997). FlashPoinℳModel 5000 3D Localizer User’s & Programmer’s Manual. Boulder, CO.Google Scholar
  11. [Kelly86]
    Kelly MD, P. J., Kall, B., and Goerss, S. (1986). Computer-assisted stereotaxic resection of intra-axial brain neoplasms. Journal of Neurosurgery, 64:427–439.CrossRefGoogle Scholar
  12. [Lavoie96]
    Lavoie, P., Ionescu, D., and Petriu, E. M. (1996). 3-D object model recovery from 2-D images using structured light. In IEEE Instrument Measurement Technology Conference, pages 377–382.Google Scholar
  13. [Lorensen93]
    Lorensen, W., Cline, H., Nafis, C., Kikinis, R., Altobelli, D., and Gleason, L. (1993). Enhancing reality in the operating room. In Proceedings of IEEE Visualization ’93.Google Scholar
  14. [State94]
    State, A., Chen, D. T., Tector, C., Brandt, A., Chen, H., Ohbuchi, R., Bajura, M., and Fuchs, H. (1994). Case study: Observing a volume-rendered fetus within a pregnant patient. In Proceedings of IEEE Visualization ’94, pages 364–368.Google Scholar
  15. [State96]
    State, A., Livingston, M. A., Hirota, G., Garrett, W. F., Whitton, M. C., and Fuchs, H. (1996). Technologies for augmented-reality systems: Realizing ultrasoundguided needle biopsies. In SIGGRAPH 96 Conference Proceedings, Annual Conference Series, pages 439‐446. ACM SIGGRAPH, Addison Wesley.Google Scholar
  16. [Turk94]
    Turk, G. and Levoy, M. (1994). Zippered polygon meshes from range images. In Proceedings of SIGGRAPH ’94, Computer Graphics Proceedings, Annual Conference Series, pages 311–318.Google Scholar
  17. [Welch96]
    Welch, G. F. (1996). Single-Constraint-At-A-Time Tracking. Ph.D. Dissertation, University of North Carolina at Chapel Hill.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Henry Fuchs
    • 1
  • Mark A. Livingston
    • 1
  • Ramesh Raskar
    • 1
  • D’nardo Colucci
    • 1
  • Kurtis Keller
    • 1
  • Andrei State
    • 1
  • Jessica R. Crawford
    • 1
  • Paul Rademacher
    • 1
  • Samuel H. Drake
    • 3
  • Anthony A. Meyer
    • 2
  1. 1.Department of Computer ScienceUniversity of North Carolina at Chapel HillUSA
  2. 2.Department of SurgeryUniversity of North Carolina at Chapel HillUSA
  3. 3.Department of Computer ScienceUniversity of UtahUSA

Personalised recommendations