Abstract
Robot-assisted surgery (RAS) is a type of minimally invasive surgery which is completely different from the traditional surgery. RAS reduces surgeon’s fatigue and the number of doctors participating in surgery. At the same time, it causes less pain and has a faster recovery rate. Real-time surgical tools detection is important for computer-assisted surgery because the prerequisite for controlling surgical tools is to know the location of surgical tools. In order to achieve comparable performance, most Convolutional Neural Network (CNN) employed for detecting surgical tools generate a huge number of feature maps from expensive operation, which results in redundant computation and long inference time. In this paper, we propose an efficient and novel CNN architecture which generate ghost feature maps cheaply based on intrinsic feature maps. The proposed detector is more efficient and simpler than the state-of-the-art detectors. We believe the proposed method is the first to generate ghost feature maps for detecting surgical tools. Experimental results show that the proposed method achieves 91.6% mAP on the Cholec80-locations dataset and 100% mAP on the Endovis Challenge dataset with the detection speed of 38.5 fps, and realizes real-time and accurate surgical tools detection in the Laparoscopic surgery video.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Choi, B., Jo, K., Choi, S., Choi, J.: Surgical-tools detection based on convolutional neural network in laparoscopic robot-assisted surgery. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), vol. 2017, pp. 1756–1759 (2017)
Liu, Y., Zhao, Z., Chang, F., Hu, S.: An anchor-free convolutional neural network for real-time surgical tool detection in robot-assisted surgery. IEEE Access. PP(99), 1 (2020)
Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., De Mathelin, M., Padoy, N.: Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36(1), 86–97 (2017)
Hajj, H.A., Lamard, M., Conze, P.H., Cochener, B., Quellec, G.: Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks. Med. Image Anal. 47, 203–218 (2018)
Sahu, M., Moerman, D., Mewes, P., Mountney, P., Rose, G.: Instrument state recognition and tracking for effective control of robotized laparoscopic systems. Int. J. Mech. Eng. Robot. Res. 5(1), 33–38 (2016)
Du, X., et al.: Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery. Int. J. Comput. Assist. Radiol. Surg. 11(6), 1109–1119 (2016)
Zhao, Z., Voros, S., Chen, Z., Cheng, X.: Surgical tool tracking based on two CNNs: from coarse to fine. The J. Eng. 2019(14), 467–472 (2019)
Nwoye, C.I., Mutter, D., Marescaux, J., Padoy, N.: Weakly supervised convolutional lstm approach for tool tracking in laparoscopic videos. Int. J. Comput. Assist. Radiol. Surg. 14(6), 1059–1067 (2019)
Garc´lła-Peraza-Herrera, L.C., et al.: Real-time segmentation of non-rigid surgical tools based on deep learning and tracking. In: International Workshop on Computer-Assisted and Robotic Endoscopy (2016)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Computer Vision and Pattern Recognition (2016)
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: optimal speed and accuracy of object detection (2020)
Shi, P., Zhao, Z., Hu, S., et al.: Real-time surgical tool detection in minimally invasive surgery based on attention-guided convolutional neural network. IEEE Access PP(99), 1–1 (2020)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional neural networks. In: European Conference on Computer Vision (2013)
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C.: Ghostnet: more features from cheap operations. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Liu, W., et al.: SSD: single shot multibox detector (2016)
Wang, C.Y., Liao, H.Y.M., Wu, Y.H., Chen, P.Y., Yeh, I.H.: CSPNet: a new backbone that can enhance learning capability of CNN. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2020)
Misra, D.: Mish: a self regularized non-monotonic activation function (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2014)
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv e-prints (2018)
Zheng, Z., Wang, P., Liu, W., Li, J., Ren, D.: Distance-IoU loss: faster and better learning for bounding box regression. In: AAAI Conference on Artificial Intelligence (2020)
Du, X., et al.: Articulated multi-instrument 2-d pose estimation using fully convolutional networks. IEEE Trans. Med. Imaging 37(5), 1276–1287 (2018)
Neubeck, A., Gool, L.J.V.: Efficient Non-Maximum Suppression (2006)
Rezatofighi, H., Tsoi, N., Gwak, J.Y., Sadeghian, A., Savarese, S.: Generalized intersection over union: a metric and a loss for bounding box regression. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, Y., Zhao, Z., Shi, P., Hu, S. (2021). An Efficient One-Stage Detector for Real-Time Surgical Tools Detection in Robot-Assisted Surgery. In: Papież, B.W., Yaqub, M., Jiao, J., Namburete, A.I.L., Noble, J.A. (eds) Medical Image Understanding and Analysis. MIUA 2021. Lecture Notes in Computer Science(), vol 12722. Springer, Cham. https://doi.org/10.1007/978-3-030-80432-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-80432-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80431-2
Online ISBN: 978-3-030-80432-9
eBook Packages: Computer ScienceComputer Science (R0)