Real-Time Methods for Long-Term Tissue Feature Tracking in Endoscopic Scenes

  • Michael C. Yip
  • David G. Lowe
  • Septimiu E. Salcudean
  • Robert N. Rohling
  • Christopher Y. Nguan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7330)


Salient feature tracking for endoscopic images has been investigated in the past for 3D reconstruction of endoscopic scenes as well as tracking of tissue through a video sequence. Recent work in the field has shown success in acquiring dense salient feature profiling of the scene. However, there has been relatively little work in performing long-term feature tracking for capturing tissue deformation. In addition, real-time solutions for tracking tissue features result in sparse densities, rely on restrictive scene and camera assumptions, or are limited in feature distinctiveness. In this paper, we develop a novel framework to enable long-term tracking of image features. We implement two fast and robust feature algorithms, STAR and BRIEF, for application to endoscopic images. We show that we are able to acquire dense sets of salient features at real-time speeds, and are able to track their positions for long periods of time.


Laparoscopic Radical Prostatectomy Endoscopic Image Tissue Deformation Subsequent Frame Tracking Framework 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, M., Konolige, K., Blas, M.: CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 102–115. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary Robust Independent Elementary Features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Forward-backward error: Automatic detection of tracking failures. In: IEEE Int. C. Pattern Recognition, pp. 2756–2759 (2010)Google Scholar
  4. 4.
    Lo, B., Chung, A., Stoyanov, D., Mylonas, G., Yang, G.-Z.: Real-time intra-operative 3d tissue deformation recovery. In: I. S. Biomedical Imaging, pp. 1387–1390 (2008)Google Scholar
  5. 5.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Computer Vision 60, 91–110 (2004)CrossRefGoogle Scholar
  6. 6.
    Luó, X., Feuerstein, M., Reichl, T., Kitasaka, T., Mori, K.: An Application Driven Comparison of Several Feature Extraction Algorithms in Bronchoscope Tracking During Navigated Bronchoscopy. In: Liao, H., Edwards, P.J., Pan, X., Fan, Y., Yang, G.-Z. (eds.) MIAR 2010. LNCS, vol. 6326, pp. 475–484. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE T. Pattern Anal. 27, 1615–1630 (2005)CrossRefGoogle Scholar
  8. 8.
    Mountney, P., Stoyanov, D., Davison, A., Yang, G.-Z.: Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 347–354. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Mountney, P., Lo, B., Thiemjarus, S., Stoyanov, D., Zhong-Yang, G.: A Probabilistic Framework for Tracking Deformable Soft Tissue in Minimally Invasive Surgery. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part II. LNCS, vol. 4792, pp. 34–41. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Mountney, P., Yang, G.-Z.: Soft Tissue Tracking for Minimally Invasive Surgery: Learning Local Deformation Online. In: Metaxas, D., Axel, L., Fichtinger, G., Székely, G. (eds.) MICCAI 2008, Part II. LNCS, vol. 5242, pp. 364–372. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Mountney, P., Stoyanov, D., Yang, G.-Z.: Three-dimensional tissue deformation recovery and tracking. IEEE Signal Processing Magazine 27, 14–24 (2010)CrossRefGoogle Scholar
  12. 12.
    Ortmaier, T., Groeger, M., Boehm, D., Falk, V., Hirzinger, G.: Motion estimation in beating heart surgery. IEEE T. Biomed. Eng. 52(10), 1729–1740 (2005)CrossRefGoogle Scholar
  13. 13.
    Richa, R., Bo, A. P., Poignet, P.: Towards robust 3D visual tracking for motion compensation in beating heart surgery. Med. Image Anal. 15(3), 302-315 (2011)Google Scholar
  14. 14.
    Rosten, E., Drummond, T.: Machine Learning for High-Speed Corner Detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 430–443. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  15. 15.
    Sauvee, M., Noce, A., Poignet, P., Triboulet, J., Dombre, E.: Three-dimensional heart motion estimation using endoscopic monocular vision system: From artificial landmarks to texture analysis. Biomed. Signal Process. Control 2(3), 199–207 (2007)CrossRefGoogle Scholar
  16. 16.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE Int. C. Computer Vision and Pattern Recognition, pp. 593–600 (1994)Google Scholar
  17. 17.
    Sinha, S. N., Frahm, J.-M., Pollefeys, M., Genc, Y.: Gpu-based video feature tracking and matching. In: Workshop on Edge Computing Using New Commodity Architectures, Technical Report (2006)Google Scholar
  18. 18.
    Stoyanov, D., Mylonas, G.P., Deligianni, F., Darzi, A., Yang, G.-Z.: Soft-Tissue Motion Tracking and Structure Estimation for Robotic Assisted MIS Procedures. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 139–146. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  19. 19.
    Ukimura, O., Gill, I., Desai, M.O.: Real-time transrectal ultrasonography during laparoscopic radical prostatectomy. Journal of Urology 172(1), 112–118 (2004)CrossRefGoogle Scholar
  20. 20.
    Wang, H., Mirota, D., Ishii, M., Hager, G.: Robust motion estimation and structure recovery from endoscopic image sequences with an adaptive scale kernel consensus estimator. In: Int. C. Computer Vision and Pattern Recognition, pp. 1–7 (2008)Google Scholar
  21. 21.
    Wengert, C., Cattin, P.C., Duff, J.M., Baur, C., Székely, G.: Markerless Endoscopic Registration and Referencing. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4190, pp. 816–823. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  22. 22.
    Willow Garage. Star detector,

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Michael C. Yip
    • 1
  • David G. Lowe
    • 2
  • Septimiu E. Salcudean
    • 1
  • Robert N. Rohling
    • 1
  • Christopher Y. Nguan
    • 3
  1. 1.Electrical and Computer EngineeringUniversity of British ColumbiaCanada
  2. 2.Computer Science DepartmentUniversity of British ColumbiaCanada
  3. 3.Vancouver General HospitalVancouverCanada

Personalised recommendations