Abstract
Detection, tracking, and pose estimation of surgical instruments provide critical information that can be used to correct inaccuracies in kinematic data in robotic-assisted surgery. Such information can be used for various purposes including integration of pre- and intra-operative images into the endoscopic view. In some cases, automatic segmentation of surgical instruments is a crucial step towards full instrument pose estimation but it can also be solely used to improve user interactions with the robotic system. In our work we focus on binary instrument segmentation, where the objective is to label every pixel as instrument or background and instrument part segmentation, where different semantically separate parts of the instrument are labeled. We improve upon previous work by leveraging recent techniques such as deep residual learning and dilated convolutions and advance both binary-segmentation and instrument part segmentation performance on the EndoVis 2017 Robotic Instruments dataset. The source code for the experiments reported in the paper has been made public (https://github.com/warmspringwinds/pytorch-segmentation-detection).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We follow the practice of previous work and use simplified definition without mirroring and centering the filter [5].
References
Allan, M., et al.: Image based surgical instrument pose estimation with multi-class labelling and optical flow. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 331–338. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24553-9_41
Allan, M., et al.: 2017 robotic instrument segmentation challenge. arXiv preprint arXiv:1902.06426 (2019)
Bhayani, S.B., Andriole, G.L.: Three-dimensional (3D) vision: does it improve laparoscopic skills? An assessment of a 3D head-mounted visualization system. Rev. Urol. 7(4), 211 (2005)
Bouget, D., Benenson, R., Omran, M., Riffaud, L., Schiele, B., Jannin, P.: Detecting surgical tools by modelling local appearance and global shape. IEEE Trans. Med. Imaging 34(12), 2603–2617 (2015)
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. arXiv preprint arXiv:1606.00915 (2016)
García-Peraza-Herrera, L.C., et al.: Real-time segmentation of non-rigid surgical tools based on deep learning and tracking. In: Peters, T., et al. (eds.) CARE 2016. LNCS, vol. 10170, pp. 84–95. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54057-3_8
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Laina, I., et al.: Concurrent segmentation and localization for tracking of surgical instruments. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 664–672. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_75
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Okamura, A.M.: Haptic feedback in robot-assisted minimally invasive surgery. Curr. Opin. Urol. 19(1), 102 (2009)
Pezzementi, Z., Voros, S., Hager, G.D.: Articulated object tracking by rendering consistent appearance parts. In: 2009 IEEE International Conference on Robotics and Automation, ICRA 2009, pp. 3940–3947. IEEE (2009)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. CoRR, abs/1505.04597 (2015)
Speidel, S., Delles, M., Gutt, C., Dillmann, R.: Tracking of instruments in minimally invasive surgery for surgical skill analysis. In: Yang, G.-Z., Jiang, T.Z., Shen, D., Gu, L., Yang, J. (eds.) MIAR 2006. LNCS, vol. 4091, pp. 148–155. Springer, Heidelberg (2006). https://doi.org/10.1007/11812715_19
Taylor, R.H., Menciassi, A., Fichtinger, G., Dario, P.: Medical robotics and computer-integrated surgery. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 1199–1222. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-30301-5_53
Tonet, O., Ramesh, T.U., Megali, G., Dario, P.: Tracking endoscopic instruments without localizer: image analysis-based approach. Stud. Health Technol. Inform. 119, 544–549 (2005)
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Pakhomov, D., Premachandran, V., Allan, M., Azizian, M., Navab, N. (2019). Deep Residual Learning for Instrument Segmentation in Robotic Surgery. In: Suk, HI., Liu, M., Yan, P., Lian, C. (eds) Machine Learning in Medical Imaging. MLMI 2019. Lecture Notes in Computer Science(), vol 11861. Springer, Cham. https://doi.org/10.1007/978-3-030-32692-0_65
Download citation
DOI: https://doi.org/10.1007/978-3-030-32692-0_65
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32691-3
Online ISBN: 978-3-030-32692-0
eBook Packages: Computer ScienceComputer Science (R0)
