2D/3D Real-Time Tracking of Surgical Instruments Based on Endoscopic Image Processing

  • Anthony AgustinosEmail author
  • Sandrine Voros
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9515)


This paper describes a simple and robust algorithm which permits to track surgical instruments without artificial markers in endoscopic images. Based on image processing, this algorithm can estimate the 2D/3D pose of all the instruments visible in the image, in real-time (30 Hz). The originality of the approach is based on the use of a Frangi filter for detecting edges and the tip of instruments. The accuracy of the instruments’ location in the image is evaluated using an extensive dataset (1500 images, 3 laparoscopic surgeries). Pose estimation of instruments in space is quantitatively evaluated on a test bench through comparison with the ground truth positioning provided by a calibrated robotic instrument holder. This method opens perspectives in the real-time control of surgical robots and the intra-operative recognition of surgical gestures.


Laparoscopy Image processing Surgical instruments Real-time tracking 

Supplementary material

418107_1_En_9_MOESM1_ESM.avi (612 kb)
Supplementary material 1 (avi 612 KB)
418107_1_En_9_MOESM2_ESM.avi (955 kb)
Supplementary material 2 (avi 955 KB)
418107_1_En_9_MOESM3_ESM.avi (870 kb)
Supplementary material 3 (avi 870 KB)
418107_1_En_9_MOESM4_ESM.avi (1.1 mb)
Supplementary material 4 (avi 1164 KB) (31 kb)
Supplementary material 5 (zip 31 KB)


  1. 1.
    Cano, A.M., Gayá, F., Lamata, P., Sánchez-González, P., Gómez, E.J.: Laparoscopic tool tracking method for augmented reality surgical applications. In: Bello, F., Edwards, E. (eds.) ISBMS 2008. LNCS, vol. 5104, pp. 191–196. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Zhou, J., Payandeh, S.: Visual tracking of laparoscopic instruments. J. Autom. Control Eng. 2(3), 234–241 (2014)CrossRefGoogle Scholar
  3. 3.
    Wolf, R., Duchateau, J., Cinquin, P., Voros, S.: 3D tracking of laparoscopic instruments using statistical and geometric modeling. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011, Part I. LNCS, vol. 6891, pp. 203–210. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Allan, M., Ourselin, S., Thompson, S., Hawkes, D.J., Kelly, J., Stoyanov, D.: Toward detection and localization of instruments in minimally invasive surgery. IEEE Trans. Biomed. Eng. 60(4), 1050–1058 (2013)CrossRefGoogle Scholar
  5. 5.
    Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: Wells, W.M., Colchester, A.C.F., Delp, S.L. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 130–137. Springer, Heidelberg (1998)Google Scholar
  6. 6.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)CrossRefGoogle Scholar
  7. 7.
    Doignon, C., Graebling, P., de Mathelin, M.: Real-time segmentation of surgical instruments inside the abdominal cavity using a joint hue saturation color feature. Real-Time Imaging 11(5), 429–442 (2005)CrossRefGoogle Scholar
  8. 8.
    Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11(285–296), 23–27 (1975)Google Scholar
  9. 9.
    Felzenszwalb, P., Huttenlocher, D.: Distance transforms of sampled functions. Cornell University. (2004)Google Scholar
  10. 10.
    Suzuki, S.: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985)CrossRefzbMATHGoogle Scholar
  11. 11.
    Hough, V., Paul, C.: U.S. Patent No. 3,069,654. U.S. Patent and Trademark Office, Washington (1962)Google Scholar
  12. 12.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
  13. 13.
    Sznitman, R., Becker, C., Fua, P.: Fast part-based classification for instrument detection in minimally invasive surgery. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part II. LNCS, vol. 8674, pp. 692–699. Springer, Heidelberg (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.UJF-Grenoble 1, CNRS, TIMC-IMAG UMR 5525GrenobleFrance
  2. 2.UJF-Grenoble 1, INSERM, TIMC-IMAG UMR 5525GrenobleFrance

Personalised recommendations