2D-3D Pose Tracking of Rigid Instruments in Minimally Invasive Surgery

  • Max Allan
  • Steve Thompson
  • Matthew J. Clarkson
  • Sébastien Ourselin
  • David J. Hawkes
  • John Kelly
  • Danail Stoyanov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8498)

Abstract

Instrument localization and tracking is an important challenge for advanced computer assisted techniques in minimally invasive surgery and image-based solutions to instrument localization can provide a non-invasive, low cost solution. In this study, we present a novel algorithm capable of recovering the 3D pose of laparoscopic surgical instruments combining constraints from a classification algorithm, multiple point features, stereo views (when available) and a linear motion model to robustly track the tool in surgical videos. We demonstrate the improved robustness and performance of our algorithm with optically tracked ground truth and additionally qualitatively demonstrate its performance on in vivo images.

Keywords

Manifold Dinate Kelly 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Azimian, H., Patel, R., Naish, M.: On constrained manipulation in robotics-assisted minimally invasive surgery. In: 3rd IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, pp. 650–655 (2010)Google Scholar
  2. 2.
    van der Putten Westebring, E.P., Goossens, R.H.M., Jakimowicz, J.J., Dankelman, J.: Haptics in minimally invasive surgery a review. Minimally Invasive Therapy & Allied Technologies 17(1), 3–16 (2008)CrossRefGoogle Scholar
  3. 3.
    Speidel, S., Sudra, G., Senemaud, J., Drentschew, M., Müller-Stich, B.P., Gutt, C., Dillmann, R.: Recognition of risk situations based on endoscopic instrument tracking and knowledge based situation modeling. In: Medical Imaging 2008: Visualization, Image-Guided Procedures, and Modeling, vol. 6918 (2008)Google Scholar
  4. 4.
    Chmarra, M.K., Grimbergen, C.A., Dankelman, J.: Systems for tracking minimally invasive surgical instruments. Minimally Invasive Therapy & Allied Technologies 16(6), 328–340 (2007)CrossRefGoogle Scholar
  5. 5.
    Mirota, D.J., Ishii, M., Hager, G.D.: Vision-based navigation in image-guided interventions. Annual Review of Biomedical Engineering 13, 297–319 (2011)CrossRefGoogle Scholar
  6. 6.
    Stoyanov, D.: Surgical vision. Annals of Biomedical Engineering 40(2), 332–345 (2012)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Uecker, D.R., Lee, C., Wang, Y.F., Wang, Y.: Automated instrument tracking in robotically assisted laparoscopic surgery. Journal of Image Guided Surgery 1(6), 308–325 (1995) PMID: 9080352Google Scholar
  8. 8.
    Sznitman, R., Ali, K., Richa, R., Taylor, R., Hager, G., Fua, P.: Data-driven visual tracking in retinal microsurgery. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part II. LNCS, vol. 7511, pp. 568–575. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  9. 9.
    Richa, R., Balicki, M., Meisner, E., Sznitman, R., Taylor, R., Hager, G.: Visual tracking of surgical tools for proximity detection in retinal surgery. In: Taylor, R.H., Yang, G.-Z. (eds.) IPCAI 2011. LNCS, vol. 6689, pp. 55–66. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  10. 10.
    Zhao, T., Zhao, W., Halabe, D.J., Hoffman, B.D., Nowlin, W.C.: Fiducial marker design and detection for locating surgical instrument in images. Patent US 068 395, 07 08 (2010)Google Scholar
  11. 11.
    Voros, S., Long, J., Cinquin, P.: Automatic detection of instruments in laparoscopic images: A first step towards high-level command of robotic endoscopic holders. The International Journal of Robotics Research 26(11-12), 1173–1190 (2007)CrossRefGoogle Scholar
  12. 12.
    Reiter, A., Allen, P.K., Zhao, T.: Appearance learning for 3d tracking of robotic surgical tools. The International Journal of Robotics Research (2013)Google Scholar
  13. 13.
    Austin, R., Allen, P.K., Tao, Z.: Articulated surgical tool detection using virtually-rendered templates. Computer Assisted Radiology and Surgery (2012)Google Scholar
  14. 14.
    Allan, M., Ourselin, S., Thompson, S., Hawkes, D.J., Kelly, J., Stoyanov, D.: Toward detection and localization of instruments in minimally invasive surgery. IEEE Transactions on Biomedical Engineering 60(4), 1050–1058 (2013)CrossRefGoogle Scholar
  15. 15.
    Prisacariu, V.A., Reid, I.D.: PWP3D: Real-Time segmentation and tracking of 3D objects. Int. J. Computer Vision 98(3), 335–354 (2012)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Maier-Hein, L., Mountney, P., Bartoli, A., Elhawary, H., Elson, D., Groch, A., Kolb, A., Rodrigues, M., Sorger, J., Speidel, S., Stoyanov, D.: Optical techniques for 3d surface reconstruction in computer-assisted laparoscopic surgery. Medical Image Analysis 17(8), 974–996 (2013)CrossRefGoogle Scholar
  17. 17.
    Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: Integrating color, texture, motion and shape. Int. J. Comput. Vision 72(2), 195–215 (2007)CrossRefGoogle Scholar
  18. 18.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  19. 19.
    Prince, S.: Computer Vision: Models Learning and Inference. Cambridge University Press (2012)Google Scholar
  20. 20.
    Tsai, R., Lenz, R.: A new technique for fully autonomous and efficient 3d robotics hand/eye calibration. IEEE Transactions on Robotics and Automation 5(3), 345–358 (1989)CrossRefGoogle Scholar
  21. 21.
    Pezzementi, Z., Voros, S., Hager, G.D.: Articulated object tracking by rendering consistent appearance parts. In: IEEE International Conference on Robotics and Automation, ICRA 2009, pp. 3940–3947 (May 2009)Google Scholar
  22. 22.
    Prisacariu, V.A., Reid, I.: Nonlinear shape manifolds as shape priors in level set segmentation and tracking. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, pp. 2185–2192. IEEE Computer Society, Washington, DC (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Max Allan
    • 1
    • 2
  • Steve Thompson
    • 1
    • 3
  • Matthew J. Clarkson
    • 1
    • 3
  • Sébastien Ourselin
    • 1
    • 3
  • David J. Hawkes
    • 1
    • 3
  • John Kelly
    • 4
  • Danail Stoyanov
    • 1
    • 2
  1. 1.Centre for Medical Image ComputingUCLLondonUK
  2. 2.Department of Computer ScienceUCLLondonUK
  3. 3.Department of Medical Physics and BioengineeringUCLLondonUK
  4. 4.Division of Surgery and Interventional Science, Medical SchoolUCLLondonUK

Personalised recommendations