Application of Convolutional Neural Network to Organize the Work of Collaborative Robot as a Surgeon Assistant

  • Shuai Yin
  • Arkady YuschenkoEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11659)


Medicine is a perspective area for collaborative robotics. The paper presents the collaborative robot as a surgeon’s assistant, accompanying the operation, submitting the necessary tools and performing other auxiliary actions. Such a robot must be mobile, have a manipulator, means of visual communication, an autonomous navigation system in the operating room, and an interactive system for interaction with the surgeon. The last task is considered in the paper. At the voice request of the surgeon, the robot have to find the necessary medical tool on the desktop and transmit it to the surgeon. This operation involves three steps: firstly, at the voice request, the robot must determine which tool is required by the surgeon; on the second step- to find the right tool on the desktop and take it; and on the third – to hand the tool to the surgeon. In the paper the neural networks technology is proposed to solve the recognition problems aroused at two first stages.


Dialogue system Image processing system Collaborative robotic Training Robotic system 



This work is financially supported by RFBR, project № 8-07-01313.


  1. 1.
    Bodner, J., et al.: The da Vinci robotic system for general surgical applications: acritical interim appraisal. In: Swiss Med Weekly, pp. 674–679 (2005)Google Scholar
  2. 2.
    Yuschenko, A.S.: Dialog control robots based on fuzzy logic. In: Proceedings of International Scientific and Technical Conference on Extreme Robotics “ER-2012”, pp. 29–36 (2012)Google Scholar
  3. 3.
    Yuschenko, A.S., Morozov, D.N., Zhonin, A.A.: Speech control for mobile Robotic systems. In: Proceedings of 4th International Conference on Mechatronic Systems and Materials “MSM-2008”, pp. 14–17 (2008)Google Scholar
  4. 4.
    Kanis, J., Ryumin, D., Krňoul, Z.: Improvements in 3D hand pose estimation using synthetic data. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds.) ICR 2018. LNCS (LNAI), vol. 11097, pp. 105–115. Springer, Cham (2018). Scholar
  5. 5.
    Gruber, I., Ryumin, D., Hrúz, M., Karpov, A.: Sign language numeral gestures recognition using convolutional neural network. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds.) ICR 2018. LNCS (LNAI), vol. 11097, pp. 70–77. Springer, Cham (2018). Scholar
  6. 6.
  7. 7.
    Sergienko, R.: Text Classification for Spoken Dialogue Systems. Institute of Telecommunications and Institute of Artificial Intelligence, Ulm University, pp. 17–58 (2016)Google Scholar
  8. 8.
    Jurafsky, D., Martin, J.H.: Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, pp. 273–543. Pearson, Upper Saddle River (2014)Google Scholar
  9. 9.
    Mansour, A.H., Salh, G.Z.A., Mohammed, K.A.: Voice recognition using dynamic time warping and mel-frequency cepstral coefficients algorithms. Int. J. Comput. Appl. 116, 34–41 (2015)Google Scholar
  10. 10.
    Meza-Ruiz, I.V., Riedel, S., Lemon, O.: Spoken language understanding in dialogue systems, using a 2-layer Markov logic network. Improving semantic accuracy. In: Semantics and Pragmatics of Dialogue, Londial (2008)Google Scholar
  11. 11.
    Yu, Z.S., Kobayashi, H.: An efficient forward-backward algorithm for an explicit-duration hidden Markov model. IEEE Signal Process. Lett. 10, 11–14 (2003)CrossRefGoogle Scholar
  12. 12.
    Tu, S.: Derivation of Baum-Welch Algorithm for Hidden Markov Models.
  13. 13.
    Tao, C.: A generalization of discrete hidden Markov model and of Viterbi algorithm. Department of Computer Science, pp. 1381–1387 (1992)Google Scholar
  14. 14.
    Pauls, A., Klein, D.: Faster and smaller N-gram Language Models. In: Annual Meeting of the Association for Computation Linguistics. Human Language Technologies, pp. 258–267 (2011)Google Scholar
  15. 15.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  16. 16.
    Cortes, C., Vapnik, V.: Support vector machine. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  17. 17.
    Uijlings, J., Van de Sande, K., Gevers, T., Smeulders, A.: Selective search for object recognition. Int. J. Comput. Vis. (IJCV) 104, 154–171 (2013)CrossRefGoogle Scholar
  18. 18.
    He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 346–361. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Bauman Moscow State Technical UniversityMoscowRussian Federation

Personalised recommendations