Ranking Robot-Assisted Surgery Skills Using Kinematic Sensors

  • Burçin Buket OğulEmail author
  • Matthias Felix Gilgien
  • Pınar Duygulu Şahin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11912)


Assessing surgical skills is an essential part of medical performance evaluation and expert training. Since it is typically conducted as a subjective task by individuals, it may lead to misinterpretations of the skill performance and hence lead to suboptimal training and organization of the surgical activities. Therefore, objective assessment of surgical skills using computational intelligence techniques via sensory data has received attention from researchers in recent years. So far, the problem has been approached by employing a classification model where a query action for surgery is assigned to a predefined category that determines the level of expertise. In this study, we consider the skill assessment problem as a pairwise ranking task where we compare two input actions to identify better surgical performance. To this end, we propose a hybrid Siamese network that takes two kinematic motion data acquired from robot-assisted surgery sensors and report the probability of the first sample having a better skill than the second one. Experiments on annotated real surgery data reveals that the proposed framework has high accuracy and seems sufficiently accurate for use in practice. This approach may overcome the limitations of having consistent annotations to define skill levels and provide a more interpretable means for objective skill assessment.


Skill assessment Ambient intelligence in education Ambient intelligence in health Robot-assisted surgery Siamese networks LSTM 



Burçin Buket Oğul was financially supported by the Scientific and Technological Research Council of Turkey (TUBITAK) under 2214-A program.


  1. 1.
    Burges, C.J., Shaked, T., Renshaw, E., et al.: Learning to rank using gradient descent. In: International Conference on Machine Learning, pp. 89–96 (2005)Google Scholar
  2. 2.
    Doughty, H., Damen, D., Mayol-Cuevas, W.: Who’s better? Who’s best? Pairwise deep ranking for skill determination. In: IEEE Conference on Computer Vision and Pattern Recognition (2018)Google Scholar
  3. 3.
    Fard, M.J., Ameri, S., Darin, E.R., et al.: Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int. J. Med. Robot. Comput. Assist. Surg. 14(1), e1850 (2018)CrossRefGoogle Scholar
  4. 4.
    Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Evaluating surgical skills from kinematic data using convolutional neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 214–221. Springer, Cham (2018). Scholar
  5. 5.
    Funke, I., Mees, S.T., Weitz, J., Speidel, S.: Video-based surgical skill assessment using 3D convolutional neural networks. arXiv preprint arXiv:1903.02306 (2019)
  6. 6.
    Gao, Y., Vedula, S.S., Reiley, C.E., et al.: JHU-ISI gesture and skill assessment working set (JIGSAWS): a surgical activity dataset for human motion modelling. In: MICCAI Workshop (2014)Google Scholar
  7. 7.
    Grantcharov, T.P., Bardram, L., Funch-Jensen, P., et al.: Assessment of technical surgical skills. Eur. J. Surg. 168, 139–144 (2002)CrossRefGoogle Scholar
  8. 8.
    Graves, A., Fernández, S., Schmidhuber, J.: Bidirectional LSTM networks for improved phoneme classification and recognition. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 799–804. Springer, Heidelberg (2005). Scholar
  9. 9.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)CrossRefGoogle Scholar
  10. 10.
    Li, Z., Huang, Y., Cai, M., Sato, Y.: Manipulation-skill assessment from videos with spatial attention network. arXiv preprint arXiv:1901.02579 (2019)
  11. 11.
    Martin, J., Regehr, G., Reznick, R., et al.: Objective structured assessment of technical skill (OSATS) for surgical residents. Br. J. Surg. 84, 273–278 (1997)CrossRefGoogle Scholar
  12. 12.
    Peters, B.S., Armijo, P.R., Krause, C., et al.: Review of emerging surgical robotic technology. Surg. Endosc. 32(4), 1636–1655 (2018)CrossRefGoogle Scholar
  13. 13.
    Wang, Z., Fey, A.I.: SATR-DL: improving surgical skill assessment and task recognition in robot-assisted surgery with deep neural networks. In: IEEE Conference of the Engineering in Medicine and Biology Society, pp. 1793–1796 (2018)Google Scholar
  14. 14.
    Wang, Z., Fey, A.M.: Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. Int. J. Comput. Assist. Radiol. Surg. 13, 1959–1970 (2018)CrossRefGoogle Scholar
  15. 15.
    Zia, A., Essa, I.: Automated surgical skill assessment in RMIS training. Int. J. Comput. Assist. Radiol. Surg. 13, 731–739 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Burçin Buket Oğul
    • 1
    • 2
    Email author
  • Matthias Felix Gilgien
    • 2
  • Pınar Duygulu Şahin
    • 1
  1. 1.Department of Computer EngineeringHacettepe UniversityAnkaraTurkey
  2. 2.Department of Physical PerformanceNorwegian School of Sport SciencesOsloNorway

Personalised recommendations