An Extremely Fast and Precise Convolutional Neural Network for Recognition and Localization of Cataract Surgical Tools
Recognition and localization of surgical tools is a crucial requirement to provide safe tool-tissue interaction in various computer-assisted interventions (CAI). Unfortunately, most state-of-the-art approaches are committed to improving detection precision regardless of the real-time performance, which leads to poor prediction for these methods in intraoperative detection task. In this paper, we propose an extremely fast and precise network (EF-PNet) for tool detection that performs well both in intraoperative tracking and postoperative skill evaluation. The proposed approach takes a single sweep of the single network to achieve rapid tool detection during intraoperative tasks, and also integrates densely connected constraint to guarantee a comparable precision for skill assessment. We demonstrate the superiority of our method on a newly built dataset: cataract surgical tool location (CaSToL). Experimental results with a mean inference time of 3.7 ms per test frame detection (i.e. 270 fps) and a mean average precision (mAP) of 93%, demonstrate the effectiveness of the proposed architecture, and also indicate that our study is far superior to recent region-based methods for tool detection in terms of detection speed, surely with a comparable precision.
This research was supported by the National Key Research and Development Program of China (Grant 2017YFB1302704), the National Natural Science Foundation of China (Grant U1713220) and Youth Innovation Promotion Association of the Chinese Academy of Sciences (Grant 2018165).
Supplementary material 1 (mp4 36465 KB)
- 2.Mishra, K., Sathish, R., Sheet, D.: Tracking of retinal microsurgery tools using late fusion of responses from convolutional neural network over pyramidally decomposed frames. In: Mukherjee, S., et al. (eds.) ICVGIP 2016. LNCS, vol. 10481, pp. 358–366. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68124-5_31CrossRefGoogle Scholar
- 4.Ko, S.Y., Kim, J., Kwon, D.S., Lee, W.J.: Intelligent interaction between surgeon and laparoscopic assistant robot system. In: IEEE International Workshop on Robot & Human Interactive Communication (2005)Google Scholar
- 5.Tonet, O., Thoranaghatte, R.U., Megali, G., Dario, P.: Tracking endoscopic instruments without a localizer: a shape-analysis-based approach. Stud. Health Technol. Inform. 119(1), 544 (2006)Google Scholar
- 6.Raphael, S., Karim, A., Rogério, R., Taylor, R.H., Hager, G.D., Pascal, F.: Data-driven visual tracking in retinal microsurgery. Med. Image Comput. Comput. Assist. Interv. 15(2), 568–575 (2012)Google Scholar
- 7.Ye, M., Zhang, L., Giannarou, S., Yang, G.-Z.: Real-time 3D tracking of articulated tools for robotic surgery. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 386–394. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46720-7_45CrossRefGoogle Scholar
- 8.Li, Y., Chen, C., Huang, X., Huang, J.: Instrument tracking via online learning in retinal microsurgery. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8673, pp. 464–471. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10404-1_58CrossRefGoogle Scholar
- 10.Twinanda, A.P., Mutter, D., Marescaux, J., de Mathelin, M., Padoy, N.: Single-and multi-task architectures for tool presence detection challenge at M2CAI 2016. arXiv preprint arXiv:1610.08851 (2016)
- 11.Jin, A., et al.: Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 691–699. IEEE (2018)Google Scholar
- 13.Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)Google Scholar