Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field

  • Daochang Liu
  • Tingting JiangEmail author
  • Yizhou Wang
  • Rulin Miao
  • Fei Shan
  • Ziyu Li
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11768)


Surgical skill assessment is important for surgery training and quality control. Prior works on this task largely focus on basic surgical tasks such as suturing and knot tying performed in simulation settings. In contrast, surgical skill assessment is studied in this paper on a real clinical dataset, which consists of fifty-seven in-vivo laparoscopic surgeries and corresponding skill scores annotated by six surgeons. From analyses on this dataset, the clearness of operating field (COF) is identified as a good proxy for overall surgical skills, given its strong correlation with overall skills and high inter-annotator consistency. Then an objective and automated framework based on neural network is proposed to predict surgical skills through the proxy of COF. The neural network is jointly trained with a supervised regression loss and an unsupervised rank loss. In experiments, the proposed method achieves 0.55 Spearman’s correlation with the ground truth of overall technical skill, which is even comparable with the human performance of junior surgeons.


Surgical skill assessment Clinical data Neural networks 



This work was partially supported by National Basic Research Program of China (973 Program) under contract 2015CB351803, the Natural Science Foundation of China under contracts 61572042, 61527804 and 61625201. We also acknowledge the Clinical Medicine Plus X-Young Scholars Project and High-Performance Computing Platform of Peking University.

Supplementary material (59.8 mb)
Supplementary material 1 (zip 61186 KB)


  1. 1.
    Ahmidi, N., et al.: String motif-based description of tool motion for detecting skill and gestures in robotic surgery. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 26–33. Springer, Heidelberg (2013). Scholar
  2. 2.
    Ahmidi, N., et al.: Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty. IJCARS 10, 981–991 (2015)Google Scholar
  3. 3.
    Ahmidi, N., et al.: A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. IEEE TBE 64, 2025–2041 (2017)Google Scholar
  4. 4.
    Azari, D.P., et al.: Modeling surgical technical skill using expert assessment for automated computer rating. Ann. Surg. 269, 574–581 (2019)CrossRefGoogle Scholar
  5. 5.
    Birkmeyer, J.D., et al.: Surgical skill and complication rates after bariatric surgery. N. Engl. J. Med. 369, 1434–1442 (2013)CrossRefGoogle Scholar
  6. 6.
    Doughty, H., Damen, D., Mayol-Cuevas, W.: Who’s better? Who’s best? Pairwise deep ranking for skill determination. In: CVPR (2018)Google Scholar
  7. 7.
    Ershad, M., Koesters, Z., Rege, R., Majewicz, A.: Meaningful assessment of surgical expertise: semantic labeling with data and crowds. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 508–515. Springer, Cham (2016). Scholar
  8. 8.
    Fard, M.J., Ameri, S., Darin Ellis, R., Chinnam, R.B., Pandya, A.K., Klein, M.D.: Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int. J. Med. Robot. Comput. Assist. Surg. 14, e1850 (2018)CrossRefGoogle Scholar
  9. 9.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  10. 10.
    Huang, C.M., Zheng, C.H.: Laparoscopic Gastrectomy for Gastric Cancer: Surgical Technique and Lymphadenectomy. Springer, Netherlands (2015). Scholar
  11. 11.
    Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Evaluating surgical skills from kinematic data using convolutional neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 214–221. Springer, Cham (2018). Scholar
  12. 12.
    Jin, A., et al.: Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In: WACV (2018)Google Scholar
  13. 13.
    Martin, J., et al.: Objective structured assessment of technical skill (OSATS) for surgical residents. Br. J. Surg. 84, 273–278 (1997)CrossRefGoogle Scholar
  14. 14.
    Richstone, L., Schwartz, M.J., Seideman, C., Cadeddu, J., Marshall, S., Kavoussi, L.R.: Eye metrics as an objective assessment of surgical skill. Ann. Surg. 252, 177–182 (2010)CrossRefGoogle Scholar
  15. 15.
    Sharma, Y., et al.: Automated surgical OSATS prediction from videos. In: ISBI (2014)Google Scholar
  16. 16.
    Vedula, S.S., Ishii, M., Hager, G.D.: Objective assessment of surgical technical skill and competency in the operating room. Annu. Rev. Biomed. Eng. 19, 301–325 (2017)CrossRefGoogle Scholar
  17. 17.
    Wang, Z., Fey, A.M.: Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. IJCARS 13, 1959–1970 (2018)Google Scholar
  18. 18.
    Zhang, Q., Li, B.: Relative hidden Markov models for video-based evaluation of motion skills in surgical training. TPAMI 37, 1206–1218 (2015)CrossRefGoogle Scholar
  19. 19.
    Zia, A., Essa, I.: Automated surgical skill assessment in RMIS training. IJCARS 13, 731–739 (2018)Google Scholar
  20. 20.
    Zia, A., Sharma, Y., Bettadapura, V., Sarin, E.L., Essa, I.: Video and accelerometer-based motion analysis for automated surgical skills assessment. IJCARS 13, 443–455 (2018)Google Scholar
  21. 21.
    Zia, A., et al.: Automated video-based assessment of surgical skills for training and evaluation in medical schools. IJCARS 11, 1623–1636 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Daochang Liu
    • 1
  • Tingting Jiang
    • 1
    Email author
  • Yizhou Wang
    • 1
    • 3
    • 4
  • Rulin Miao
    • 2
  • Fei Shan
    • 2
  • Ziyu Li
    • 2
  1. 1.NELVT, Department of Computer SciencePeking UniversityBeijingChina
  2. 2.Peking University Cancer HospitalBeijingChina
  3. 3.Peng Cheng LabShenzhenChina
  4. 4.Deepwise AI LabBeijingChina

Personalised recommendations