Skip to main content
SpringerLink
Log in

Surgical Vision

  • Published:
Annals of Biomedical Engineering Aims and scope Submit manuscript

Abstract

The emergence of Minimal Access Surgery (MAS) as a paradigm in modern healthcare treatment has created new challenges and opportunities for automated image understanding and computer vision. In MAS, images recovered from inside the body using specialized devices are used to visualize and operate on the surgical site but they can also be used to computationally infer in vivo 3D tissue surface shape, soft-tissue morphology, and surgical instrument motion. This information is important for facilitating in vivo biophotonic imaging modalities where the interaction between light and tissue is used to infer the structural and functional properties of the tissue. This article provides a review of the literature for computer vision and image understanding techniques applied to MAS images. The focus of this article is to elucidate a perspective on how computer vision techniques can be used to support and enhance the capabilities of biophotonic imaging modalities during surgery. Note that while MAS encompasses a variety of surgical specializations this review does not involve procedures performed in the interventional suite. The review has been carried out based on searches in the PubMed and IEEE databases using the article’s keywords.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Similar content being viewed by others

References

  1. Albitar, I. C., P. Graebling, and C. Doignon. Robust Structured Light Coding for 3D Reconstruction. In: IEEE 11th International Conference on Computer Vision, 2007—ICCV 2007, pp. 1–6, 2007.

  2. Allain, B., M. Hu, L. Lovat, R. Cook, T. Vercauteren, S. Ourselin, and D. Hawkes. A System for Biopsy Site Re-targeting with Uncertainty in Gastroenterology and Oropharyngeal Examinations. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2010, pp. 514–521, 2010.

  3. André, B., T. Vercauteren, A. M. Buchner, M. B. Wallace, and N. Ayache. A smart atlas for endomicroscopy using automated video retrieval. Med. Image Anal. 4:460–476, 2011.

    Article  Google Scholar 

  4. Arnold, M., A. Ghosh, S. Ameling, and G. Lacey. Automatic segmentation and inpainting of specular highlights for endoscopic imaging. J. Image Video Process. 1–12, 2010.

  5. Atasoy, S., D. Noonan, S. Benhimane, N. Navab, and G.-Z. Yang. A Global Approach for Automatic Fibroscopic Video Mosaicing in Minimally Invasive Diagnosis. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2008, pp. 850–857, 2008.

  6. Atif, M., H. Ullah, M. Y. Hamza, and M. Ikram. Catheters for optical coherence tomography. Laser Phys. Lett. 9:629–646, 2011.

    Google Scholar 

  7. Bachta, W., P. Renaud, L. Cuvillon, E. Laroche, A. Forgione, and J. Gangloff. Motion prediction for computer-assisted beating heart surgery. IEEE Trans. Biomed. Eng. 11:2551–2563, 2009.

    Article  Google Scholar 

  8. Balicki, M., J.-H. Han, I. Iordachita, P. Gehlbach, J. Handa, R. Taylor, and J. Kang. Single Fiber Optical Coherence Tomography Microsurgical Instruments for Computer and Robot-Assisted Retinal Surgery. In: Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention: Part I, pp. 108–115, 2009.

  9. Baumhauer, M., M. Feuerstein, H. P. Meinzer, and J. Rassweiler. Navigation in endoscopic soft tissue surgery: perspectives and limitations. J. Endourol. 4:751–761, 2008.

    Article  Google Scholar 

  10. Becker, V., T. Vercauteren, C. H. von Weyhern, C. Prinz, R. M. Schmid, and A. Meining. High-resolution miniprobe-based confocal microscopy in combination with video mosaicing (with video). Gastrointest. Endosc. 5:1001–1007, 2007.

    Article  Google Scholar 

  11. Boppart, S. A., B. E. Bouma, C. Pitris, G. J. Tearney, J. G. Fujimoto, and M. E. Brezinski. Forward-imaging instruments for optical coherence tomography. Opt. Lett. 21:1618–1620, 1997.

    Article  Google Scholar 

  12. Bouget, J.-Y. Camera Calibration Toolkit for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/. 2011.

  13. Bouma, B. E., and G. J. Tearney. Power-efficient nonreciprocal interferometer and linear-scanning fiber-optic catheter for optical coherence tomography. Opt. Lett. 8:531–533, 1999.

    Article  Google Scholar 

  14. Burschka, D., M. Li, R. Taylor, and G. D. Hager. Scale-Invariant Registration of Monocular Endoscopic Images to CT-Scans for Sinus Surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 413–421, 2004.

  15. Camarillo, D. B., T. M. Krummel, and J. K. Salisbury, Jr. Robotic technology in surgery: past, present, and future. Am. J. Surg. 4(Suppl 1):2–15, 2004.

    Article  Google Scholar 

  16. Can, A., C. V. Stewart, B. Roysam, and H. L. Tanenbaum. A feature-based technique for joint, linear estimation of high-order image-to-mosaic transformations: application to mosaicing the curved human retina. In: IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 585–591, 2000.

  17. Cano, A., F. Gayá, P. Lamata, P. Sánchez-González, and E. Gómez. Laparoscopic tool tracking method for augmented reality surgical applications biomedical simulation. Lect. Notes Comput. Sci. 5104:191–196, 2008.

    Article  Google Scholar 

  18. Cech, J., J. Sanchez-Riera, and R. Horaud. Scene flow estimation by growing correspondence seeds. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2011, pp. 3129–3136, 2011.

  19. Clancy, N. T., D. Stoyanov, V. Sauvage, D. James, G.-Z. Yang, and D. S. Elson. A Triple Endoscope System for Alignment of Multispectral Images of Moving Tissue. In: BTuD27, 2010.

  20. Clancy, N. T., D. Stoyanov, G.-Z. Yang, and D. S. Elson. An endoscopic structured lighting probe using spectral encoding. In: Proceedings of SPIE, 2011.

  21. Collins, T., B. Compte, and A. Bartoli. Deformable Shape-from-Motion in Laparoscopy using a Rigid Sliding Window. In: Medical Image Understanding and Analysis Conference, 2011.

  22. Darzi, A., and S. Mackay. Recent advances in minimal access surgery. Br. Med. J. 7328:31–34, 2002.

    Article  Google Scholar 

  23. Deguchi, K., and T. Okatani. Shape Reconstruction from an Endoscope Image by Shape- from-Shading Technique for a Point Light Source at the Projection Center. In: IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, pp. 290–298, 1996.

  24. Deligianni, F., A. Chung, and G.-z. Yang. Patient-specific bronchoscope simulation with pq-space-based 2D/3D registration. Comput. Aided Surg. 5:215–226, 2004.

    Google Scholar 

  25. Devernay, F., F. Mourgues, and E. Coste-Maniere. Towards endoscopic augmented reality for robotically assisted minimally invasive cardiac surgery. In: Medical Imaging and Augmented Reality, 2001.

  26. Doignon, C., F. Nageotte, and M. de Mathelin. The Role of Insertion Points in the Detection and Positioning of Instruments in Laparoscopy for Robotic Tasks. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006, pp. 527–534, 2006.

  27. dos Santos, T., A. Seitel, H.-P. Meinzer, and L. Maier-Hein. Correspondences Search for Surface-Based Intra-Operative Registration. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2010, pp. 660–667, 2010.

  28. Fleming, I. N., S. Voros, B. Vagvolgyi, Z. Pezzementi, J. Handa, R. Taylor, and G. D. Hager. Intraoperative Visualization of Anatomical Targets in Retinal Surgery. In: Proceedings of the 2008 IEEE Workshop on Applications of Computer Vision, pp. 1–6, 2008.

  29. Fuchs, H., M. A. Livingston, R. Raskar, D. Colucci, K. Keller, A. State, J. R. Crawford, P. Rademacher, S. H. Drake, and A. A. Meyer. Augmented reality visualization for laparoscopic surgery. In: Medical Image Computing and Computer-Assisted Intervention, pp. 934–943, 1998.

  30. Ginhoux, R., J. Gangloff, M. de Mathelin, L. Soler, M. M. A. Sanchez, J. Marescaux, et al. Active filtering of physiological motion in robotized surgery using predictive control. IEEE Trans. Robot. 1:67–79, 2005.

    Article  Google Scholar 

  31. Gioux, S., A. Mazhar, D. J. Cuccia, A. J. Durkin, B. J. Tromberg, and J. V. Frangioni. Three-dimensional surface profile intensity correction for spatially modulated imaging. J. Biomed. Opt. 14:34–45, 2009.

    Google Scholar 

  32. Grasa, Ó. G., J. Civera, and J. M. M. Montiel. EKF Monocular SLAM with Relocalization for Laparoscopic Sequences. In: IEEE International Conference on Robotics and Automation, 2011.

  33. Gröger, M., T. Ortmaier, W. Sepp, and G. Hirzinger. Reconstruction of Image Structure in Presence of Specular Reflections. In: DAGM-Symposium on Pattern Recognition, pp. 53–60, 2001.

  34. Guo-Qing, W., K. Arbter, and G. Hirzinger. Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation. IEEE Eng. Med. Biol. Mag. 1:40–45, 1997.

    Article  Google Scholar 

  35. Guthart, G. S., and J. K. Salisbury. The Intuitive Telesurgery System: Overview and Application. In: IEEE International Conference on Robotics and Automation, pp. 618–621, 2000.

  36. Hager, G., A. Okamura, P. Kazanzides, L. Whitcomb, G. Fichtinger, and R. Taylor. Surgical and interventional robotics: part III Tutorial. IEEE Robot. Autom. Mag. 4:84–93, 2008.

    Article  Google Scholar 

  37. Hager, G., B. Vagvolgyi, and D. Yuh. Stereoscopic Video Overlay with Deformable Registration. In: Medicine Meets Virtual Reality, 2007.

  38. Hamamoto, Y., T. Endo, K. Nosho, Y. Arimura, M. Sato, and K. Imai. Usefulness of narrow-band imaging endoscopy for diagnosis of Barrett’s esophagus. J. Gastroenterol. 1:14–20, 2004.

    Article  Google Scholar 

  39. Hammer, D. X. Advances in Retinal Imaging. In: Advances in Optical Imaging for Clinical Medicine, pp. 85–161, 2011.

  40. Haogang, Z., D. P. Crabb, P. G. Schlottmann, G. Wollstein, and D. F. Garway-Heath. Aligning scan acquisition circles in optical coherence tomography images of the retinal nerve fibre layer. IEEE Trans. Med. Imaging 6:1228–1238, 2011.

    Google Scholar 

  41. Hartley, R., and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge: Cambridge Press, 2000.

    Google Scholar 

  42. Heikkila, J., and O. Silven. A Four-step Camera Calibration Procedure with Implicit Image Correction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1106–1112, 1997.

  43. Helferty, J. P., and W. E. Higgins. Combined Endoscopic Video Tracking and Virtual 3D CT Registration for Surgical Guidance. In: Proceedings of the 2002 International Conference on Image Processing, Vol. 2, pp. II-961–II-964, 2002.

  44. Hernández-Mier, Y., W. C. P. M. Blondel, C. Daul, D. Wolf, and F. Guillemin. Fast construction of panoramic images for cystoscopic exploration. Comput Med Imaging Graph 7:579–592, 2010.

    Article  Google Scholar 

  45. Höller, K., J. Penne, A. Schneider, J. Jahn, J. Guttiérrez Boronat, T. Wittenberg, H. Feußner, and J. Hornegger. Endoscopic Orientation Correction. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2009, pp. 459–466, 2009.

  46. Hu, M., G. Penney, P. Edwards, M. Figl, and D. Hawkes. 3D reconstruction of internal organ surfaces for minimal invasive surgery. Med. Image Comput. Comput. Assist. Interv. Int. Conf. Med. Image Comput. Comput. Assist. Interv. Pt 1:68–77, 2007.

    Google Scholar 

  47. Hu, M., G. Penney, D. Rueckert, P. Edwards, F. Bello, R. Casula, M. Figl, and D. Hawkes. Non-rigid Reconstruction of the Beating Heart Surface for Minimally Invasive Cardiac Surgery. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2009, pp. 34–42, 2009.

  48. Huang, D., et al. Optical coherence tomography. Science 5035:1178, 1991.

    Article  Google Scholar 

  49. Iftimia, N., W. R. Brugge, and D. X. Hammer. Advances in Optical Imaging for Clinical Medicine. Hoboken, NJ: Wiley, 2011.

    Book  Google Scholar 

  50. Keller, K., and J. D. Ackerman. Real-time structured light depth extraction. In: Proceedings of SPIE, p. 11, 2000.

  51. Krupa, A., J. Gangloff, C. Doignon, M. F. de Mathelin, M. Guillaume Morel, J. Leroy, L. Soler, J. Marescaux, et al. Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE Trans. Robot. Autom. 5:842–853, 2003.

    Article  Google Scholar 

  52. Lamata, P., T. Morvan, M. Reimers, E. Samset and J. Declerck. Addressing Shading-Based Laparoscopic Registration. In: World Congress on Medical Physics and Biomedical Engineering, September 7–12, 2009, Munich, Germany, pp. 189–192, 2009.

  53. Lau, W. W., N. A. Ramey, J. Corso, N. V. Thakor, and G. D. Hager. Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 494–501, 2004.

  54. Le Goualher, G., A. Perchant, M. Genet, C. Cavé, B. Viellerobe, F. Berier, B. Abrat, and N. Ayache. Towards Optical Biopsies with an Integrated Fibered Confocal Fluorescence Microscope. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2004, pp. 761–768, 2004.

  55. Lerotic, M., A. J. Chung, J. Clark, S. Valibeik, and G.-Z. Yang. Dynamic View Expansion for Enhanced Navigation in Natural Orifice Transluminal Endoscopic Surgery. In: Proceedings of the 11th International Conference on Medical Image Computing and Computer-Assisted Intervention, Part II, pp. 467–475, 2008.

  56. Lerotic, M., and G. Yang. Super resolution in robotic-assisted minimally invasive surgery. Comput. Aided Surg. 6:347–356, 2007.

    Google Scholar 

  57. Loewke, K. E., D. B. Camarillo, W. Piyawattanametha, M. J. Mandella, C. H. Contag, S. Thrun, and J. K. Salisbury. In vivo micro-image mosaicing. IEEE Trans. Biomed. Eng. 1:159–171, 2011.

    Article  Google Scholar 

  58. Loewke, K., D. Camarillo, K. Salisbury, and S. Thrun. Deformable Image Mosaicing for Optical Biopsy. In: IEEE 11th International Conference on Computer Vision, 2007 (ICCV 2007), pp. 1–8, 2007.

  59. Lucas, B., and T. Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679, 1981.

  60. Maier-Hein, L., M. Schmidt, A. Franz, T. dos Santos, A. Seitel, B. Jähne, J. Fitzpatrick, and H. Meinzer. Accounting for Anisotropic Noise in Fine Registration of Time-of-Flight Range Data with High-Resolution Surface Data. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2010, pp. 251–258, 2010.

  61. Malti, A., A. Bartoli, and T. Collins. Template-Based Conformal Shape-from-Motion from Registered Laparoscopic Images. In: Medical Image Understanding and Analysis Conference. 2011.

  62. Mikolajczyk, K., and C. Schmid. Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 1:63–86, 2004.

    Article  Google Scholar 

  63. Mirota, D. J., M. Ishii, and G. D. Hager. Vision-based navigation in image-guided interventions. Annu. Rev. Biomed. Eng. 1:297–319, 2011.

    Article  Google Scholar 

  64. Mirota, D., H. Wang, R. Taylor, M. Ishii, and G. Hager. Toward Video-Based Navigation for Endoscopic Endonasal Skull Base Surgery. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2009, pp. 91–99, 2009.

  65. Mitchell, B., J. Koo, M. Iordachita, P. Kazanzides, A. Kapoor, J. Handa, G. Hager, and R. Taylor. Development and Application of a New Steady-Hand Manipulator for Retinal Surgery. In: IEEE International Conference on Robotics and Automation, 2007, pp. 623–629, 2007.

  66. Mori, K., D. Deguchi, J. Sugiyama, Y. Suenagaa, J. Toriwakia, C. R. Maurer, Jr., H. Takabatake, and H. Natori. Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images. Med. Image Anal. 3:321–336, 2002.

    Article  Google Scholar 

  67. Mountney, P., S. Giannarou, D. Elson, and G.-Z. Yang. Optical Biopsy Mapping for Minimally Invasive Cancer Screening. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2009, pp. 483–490, 2009.

  68. Mountney, P., and Y. Guang-Zhong. Dynamic view expansion for minimally invasive surgery using simultaneous localization and mapping. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009 (EMBC 2009), pp. 1184–1187, 2009.

  69. Mountney, P., B. P. L. Lo, S. Thiemjarus, D. Stoyanov, and G.-Z. Yang. A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 34–41, 2007.

  70. Mountney, P., D. Stoyanov, A. Davison, and G.-Z. Yang. Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006, pp. 347–354, 2006.

  71. Mountney, P., D. Stoyanov, and Y. Guang-Zhong. Three-dimensional tissue deformation recovery and tracking. IEEE Signal Process. Mag. 4:14–24, 2010.

    Article  Google Scholar 

  72. Mountney, P., and G.-Z. Yang. Motion Compensated SLAM for Image Guided Surgery. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2010, pp. 496–504, 2010.

  73. Mourgues, F., F. Devernay, G. Malandain, and È. Coste-Manière. 3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: International Symposium on Augmented Reality. 2001.

  74. Namati, E., J. Thiesse, J. de Ryk, and G. McLennan. In vivo assessment of alveolar morphology using a flexible catheter-based confocal microscope. IET Comput. Vis. 4:228–235, 2008.

    Article  Google Scholar 

  75. Nasseri, S. S., J. L. Kasperbauer, S. E. Strome, T. V. McCaffrey, J. L. Atkinson, and F. B. Meyer. Endoscopic transnasal pituitary surgery: report on 180 cases. Am. J. Rhinol. 4:281–287, 2001.

    Google Scholar 

  76. Nguyen, F. T., A. M. Zysk, E. J. Chaney, S. G. Adie, J. G. Kotynek, U. J. Oliphant, F. J. Bellafiore, K. M. Rowland, P. A. Johnson, and S. A. Boppart. Optical coherence tomography: the intraoperative assessment of lymph nodes in breast cancer. IEEE Eng. Med. Biol. Mag. 2:63–70, 2010.

    Article  Google Scholar 

  77. Nighswander-Rempel, S. P., R. A. Shaw, V. V. Kupriyanov, J. Rendell, B. Xiang, and H. H. Mantsch. Mapping tissue oxygenation in the beating heart with near-infrared spectroscopic imaging. Vib. Spectrosc. 1:85–89, 2003.

    Article  Google Scholar 

  78. Nighswander-Rempel, S. P., R. A. Shaw, J. R. Mansfield, M. Hewko, V. V. Kupriyanov, and H. H. Mantsch. Regional variations in myocardial tissue oxygenation mapped by near-infrared spectroscopic imaging. J. Mol. Cell Cardiol. 9:1195–1203, 2002.

    Article  Google Scholar 

  79. Nishioka, N. S., and M.-A. Mycek. Initial experience with a real-time video processor for enhancing endoscopic image contrast. Gastrointest. Endosc. 1:62–66, 1998.

    Article  Google Scholar 

  80. Okatani, T., and K. Deguchi. Shape Reconstruction from an Endoscope Image by Shape from Shading Technique for a Point Light Source at the Projection Centre. Comput. Vis. Image Underst. 2:119–131, 1997.

    Article  Google Scholar 

  81. Ortmaier, T., M. Gröger, D. H. Boehm, V. Falk, and G. Hirzinger. Motion estimation in beating heart surgery. IEEE Trans. Biomed. Eng. 10:1729–1740, 2005.

    Article  Google Scholar 

  82. Ortmaier, T., M. Groger, and G. Hirzinger. Robust motion estimation in robotic surgery on the beating heart. In: Proceedings of Computer Assisted Radiology and Surgery, 2002.

  83. Penne, J., K. Holler, M. Sturmer, T. Schrauder, A. Schneider, R. Engelbrecht, H. Feusner, B. Schmauss, and J. Hornegger. Time-of-Flight 3-D Endoscopy. In: Proceedings of the 12th International Conference on Medical Image Computing and Computer-Assisted Intervention: Part I, pp. 467–474, 2009.

  84. Peters, T. M. Image-guidance for surgical procedures. Phys. Med. Biol. 51:R505–R540, 2006.

    Article  PubMed  Google Scholar 

  85. Pollefeys, M., L. V. Gool, and M. Proesmans. Stratified Self-Calibration from Image Sequences with Variable Focal Lengths. In: European Conference on Computer Vision, pp. 31–42, 1996.

  86. Prados, E., and O. Faugeras. Shape From Shading: A Well Posed Problem? In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 870–877, 2005.

  87. Pratt, P., D. Stoyanov, M. Visentini-Scarzanella, and G.-Z. Yang. Dynamic guidance for robotic surgery using image- constrained biomechanical models. In: Proceedings of the 13th International Conference on Medical image Computing and Computer-Assisted Intervention: Part I, pp. 77–85, 2010.

  88. Qiu, L., D. K. Pleskow, R. Chuttani, E. Vitkin, J. Leyden, N. Ozden, S. Itani, L. Guo, A. Sacks, J. D. Goldsmith, M. D. Modell, E. B. Hanlon, I. Itzkan, and L. T. Perelman. Multispectral scanning during endoscopy guides biopsy of dysplasia in Barrett’s esophagus. Nat. Med. 5:603–606, 2010.

    Article  Google Scholar 

  89. Rashid, H. U., and P. Burger. Differential algorithm for the determination of shape from shading using a point light source. Image Vis. Comput. 2:119–127, 1992.

    Article  Google Scholar 

  90. Rattner, D., and A. Kalloo. ASGE/SAGES Working group on natural orifice translumenal endoscopic surgery. Surg. Endosc. 20:329–333, 2006.

    Article  PubMed  CAS  Google Scholar 

  91. Ren, J., J. Wu, E. J. McDowell, C. Yang, et al. Manual-scanning optical coherence tomography probe based on position tracking. Opt. Lett. 21:3400–3402, 2009.

    Article  Google Scholar 

  92. Richa, R., M. Balicki, E. Meisner, R. Sznitman, R. Taylor, and G. Hager. Visual Tracking of Surgical Tools for Proximity Detection in Retinal Surgery. In: Information Processing in Computer-Assisted Interventions, pp. 55–66, 2011.

  93. Richa, R., A. Bó, and P. Poignet. Robust 3D Visual Tracking for Robotic-Assisted Cardiac Interventions. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2010, pp. 267–274, 2010.

  94. Richa, R., A. P. L. Bó, and P. Poignet. Towards robust 3D visual tracking for motion compensation in beating heart surgery. Med. Image Anal. 3:302–315, 2011.

    Article  Google Scholar 

  95. Richa, R., P. Poignet, and C. Liu. Efficient 3D Tracking for Motion Compensation in Beating Heart Surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention, 2008.

  96. Röhl, S., S. Bodenstedt, S. Suwelack, H. Kenngott, B. Mueller-Stich, R. Dillmanna, and S. Speidel. Real-Time Surface Reconstruction from Stereo Endoscopic Images for Intraoperative Registration. In: SPIE Medical Imaging, 2011.

  97. Saint-Pierre, C.-A., J. Boisvert, G. Grimard, and F. Cheriet. Detection and correction of specular reflections for automatic surgical tool segmentation in thoracoscopic images. Mach. Vis. Appl. 1:171–180, 2011.

    Article  Google Scholar 

  98. Scharstein, D., and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 1(2/3):7–42, 2002.

    Article  Google Scholar 

  99. Seshamani, S. Direct global adjustment methods for endoscopic mosaicking. Proc. SPIE 1:72611D, 2009.

    Article  Google Scholar 

  100. Seshamani, S., W. Lau, and G. Hager. Real-Time Endoscopic Mosaicking. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006, pp. 355–363, 2006.

  101. Sorg, B. S., B. J. Moeller, Y. Cao, and M. W. Dewhirst. Hyperspectral imaging of hemoglobin saturation in tumor microvasculature and tumor hypoxia development. J. Biomed. Opt. 10:044004, 2005.

    Article  Google Scholar 

  102. Srivastava, S., J. J. Rodriguez, A. R. Rouse, M. A. Brewer and A. F. Gmitro. Analysis of confocal microendoscope images for automatic detection of ovarian cancer. In: IEEE International Conference on Image Processing, 2005 (ICIP 2005), pp. I-1113–I-1116, 2005.

  103. Stoyanov, D. Camera Calibration Tools. http://www.cs.ucl.ac.uk/staff/Dan.Stoyanov/calib/. 2011.

  104. Stoyanov, D., A. Darzi, and G.-Z. Yang. Dense 3D Depth Recovery for Soft Tissue Deformation During Robotically Assisted Laparoscopic Surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 41–48, 2004.

  105. Stoyanov, D., A. Darzi, and G.-Z. Yang. Laparoscope Self-calibration for Robotic Assisted Minimally Invasive Surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 114–121, 2005.

  106. Stoyanov, D., D. Elson, and G.-Z. Yang. Illumination position estimation for 3D soft-tissue reconstruction in robotic minimally invasive surgery. Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2628–2633, 2009.

  107. Stoyanov, D., M. Lerotic, G. Mylonas, A. J. Chun, and G.-Z. Yang. Intra-operative Visualizations: Perceptual Fidelity and Human Factors. IEEE/OSA J. Display Technol. 4:491–501, 2008.

    Article  Google Scholar 

  108. Stoyanov, D., G. P. Mylonas, F. Deligianni, A. Darzi, and G.-Z. Yang. Soft-tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures. In: International Conference on Medical Image Computing and Computer Assisted Intervention, pp. 139–146, 2005.

  109. Stoyanov, D., M. Scarzanella, P. Pratt, and G.-Z. Yang. Real-Time Stereo Reconstruction in Robotically Assisted Minimally Invasive Surgery. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2010, pp. 275–282, 2010.

  110. Stoyanov, D., and G.-Z. Yang. Removing specular reflection components for robotic assisted laparoscopic surgery. In: IEEE International Conference on Image Processing, 2005 (ICIP 2005), pp. III-632–III-635, 2005.

  111. Stoyanov, D., and G.-Z. Yang. Soft tissue deformation tracking for robotic assisted minimally invasive surgery. In: 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2009), pp. 254–257, 2009.

  112. Strobl, K. H., W. Sepp, S. Fuchs, C. Paredes, and K. Arbter. DLR CalLab CalDe: The DLR Camera Calibration Toolbox. http://www.robotic.dlr.de/callab/. 2011.

  113. Suter, M. J., B. E. Bouma, and G. J. Tearney. High-Resolution Optical Coherence Tomography Imaging in Gastroenterology. In: Advances in Optical Imaging for Clinical Medicine, pp. 187–204, 2011.

  114. Sznitman, R., S. Billings, D. Rother, D. Mirota, Y. Yang, J. Handa, P. Gehlbach, J. U. Kang, G. D. Hager, and R. Taylor. Active Multispectral Illumination and Image Fusion for Retinal Microsurgery. In: Proceedings of the First international Conference on Information Processing in Computer-Assisted Interventions, pp. 12–22, 2010.

  115. Talamini, M. A., S. Chapman, S. Horgan, and W. S. Melvin. A prospective analysis of 211 robotic-assisted surgical procedures. Surg. Endosc. 10:1521–1524, 2003.

    Article  Google Scholar 

  116. Tankus, A., N. Sochen, and Y. Yeshurun. Perspective Shape-from-Shading via Fast Marching. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004.

  117. Taylor, R. H., and D. Stoianovici. Medical robotics in computer-integrated surgery. IEEE Trans. Robot. Autom. 5:765–781, 2003.

    Article  Google Scholar 

  118. Tearney, G. J., S. A. Boppart, B. E. Bouma, M. E. Brezinski, N. J. Weissman, J. F. Southern, and J. G. Fujimoto. Scanning single-mode fiber optic catheter-endoscope for optical coherence tomography. Opt. Lett. 7:543–545, 1996.

    Article  Google Scholar 

  119. Tearney, G. J., M. E. Brezinski, B. E. Bouma, S. A. Boppart, C. Pitris, J. F. Southern, and J. G. Fujimoto. In vivo endoscopic optical biopsy with optical coherence tomography. Science 5321:2037–2039, 1997.

    Article  Google Scholar 

  120. Tonet, O., R. U. Thoranaghatte, G. Megali, and P. Dario. Tracking endoscopic instruments without a localizer: A shape-analysis-based approach. Comput. Aided Surg. 1:35–42, 2007.

    Google Scholar 

  121. Tsai, R. Y. An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 364–374, 1986.

  122. Uecker, D. R., Y. F. Wang, C. Lee, and Y. Wang. Laboratory investigation: automated instrument tracking in robotically assisted laparoscopic surgery. Comput. Aided Surg. 6:308–325, 1995.

    Article  Google Scholar 

  123. Vercauteren, T., A. Perchant, X. Pennec, and N. Ayache. Mosaicing of Confocal Microscopic In Vivo Soft Tissue Video Sequences. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2005, pp. 753–760, 2005.

  124. Voros, S., G. P. Haber, J. F. Menudet, J. A. Long, and P. Cinquin. ViKY Robotic scope holder: initial clinical experience and preliminary results using instrument tracking. IEEE/ASME Trans. Mechatron. 6:879–886, 2010.

    Google Scholar 

  125. Wang, P., S. M. Krishnan, Y. Huang, and N. Srinivasan. An adaptive segmentation technique for clinical endoscopic image processing. In: Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society EMBS/BMES Conference, Engineering in Medicine and Biology, Vol. 2, pp. 1084–1085, 2002.

  126. Warren, A., P. Mountney, D. Noonan, and G.-Z. Yang. Horizon Stabilized—Dynamic View Expansion for Robotic Assisted Surgery (HS-DVE). In: International Journal of Computer Assisted Radiology and Surgery, pp. 1–8, 2011.

  127. West, J. B., and C. R. Maurer, Jr. Designing optically tracked instruments for image-guided surgery. IEEE Trans. Med. Imaging 5:533–545, 2004.

    Article  Google Scholar 

  128. Wu, J., M. Conry, C. Gu, F. Wang, Z. Yaqoob, C. Yang, et al. Paired-angle-rotation scanning optical coherence tomography forward-imaging probe. Opt. Lett. 9:1265–1267, 2006.

    Article  Google Scholar 

  129. Wu, C., B. Jaramaz, and S. G. Narasimhan. A full geometric and photometric calibration method for oblique-viewing endoscopes. Comput. Aided Surg. 1–3:19–31, 2010.

    Article  Google Scholar 

  130. Wu, C., S. Narasimhan, and B. Jaramaz. A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes. Int. J. Comput. Vis. 2:211–228, 2010.

    Article  Google Scholar 

  131. Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 11:1330–1334, 2000.

    Article  Google Scholar 

  132. Zhang, G., J. He, and X. Li. 3D vision inspection for internal surface based on circle structured light. Sens. Actuators A Phys. 1:68–75, 2005.

    Article  CAS  Google Scholar 

  133. Zitová, B., and J. Flusser. Image registration methods: a survey. Image Vis. Comput. 11:977–1000, 2003.

    Article  Google Scholar 

Download references

Acknowledgments

The insightful and constructive comments of the anonymous reviewers were very helpful for improving this article. The work was carried out with the financial support of a Royal Academy of Engineering/EPSRC Research Fellowship.

Author information

Authors and Affiliations

  1. Center for Medical Image Computing, University College London, London, WC1 2BT, UK

    Danail Stoyanov

  2. Hamlyn Centre for Robotic Surgery, Imperial College London, London, SW7 2AZ, UK

    Danail Stoyanov

Authors
  1. Danail Stoyanov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Danail Stoyanov.

Additional information

Associate Editor Daniel Elson oversaw the review of this article.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Stoyanov, D. Surgical Vision. Ann Biomed Eng 40, 332–345 (2012). https://doi.org/10.1007/s10439-011-0441-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-011-0441-z

Keywords

Search

Navigation