On Field Gesture-Based Robot-to-Robot Communication with NAO Soccer Players

  • Valerio Di GiambattistaEmail author
  • Mulham Fawakherji
  • Vincenzo Suriani
  • Domenico D. Bloisi
  • Daniele Nardi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11531)


Gesture-based communication is commonly used by soccer players during matches to exchange information with teammates. Among the possible forms of gesture-based interaction, hand signals are the most used. In this paper, we present a deep learning method for recognizing robot-to-robot hand signals exchanged during a soccer game. A neural network for estimating human body, face, hands, and foot position has been adapted for the application in the robot soccer scenario. Quantitative experiments carried out on NAO V6 robots demonstrate the effectiveness of the proposed approach. Source code and data used in this work are made publicly available for the community.


Communication protocols Team coordination methods Neural systems and deep learning 


  1. 1.
    Albani, D., Youssef, A., Suriani, V., Nardi, D., Bloisi, D.D.: A deep learning approach for object recognition with NAO soccer robots. In: Behnke, S., Sheh, R., Sarıel, S., Lee, D.D. (eds.) RoboCup 2016. LNCS (LNAI), vol. 9776, pp. 392–403. Springer, Cham (2017). Scholar
  2. 2.
    Canal, G., Escalera, S., Angulo, C.: A real-time human-robot interaction system based on gestures for assistive scenarios. Comput. Vis. Image Underst. 149, 65–77 (2016)CrossRefGoogle Scholar
  3. 3.
    Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. arXiv preprint arXiv:1812.08008 (2018)
  4. 4.
    Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: CVPR, pp. 1302–1310 (2017)Google Scholar
  5. 5.
    Fujii, T., Lee, J.H., Okamoto, S.: Gesture recognition system for human-robot interaction and its application to robotic service task. In: Proceedings of the International Multi-Conference of Engineers and Computer Scientists (IMECS), vol. 1 (2014)Google Scholar
  6. 6.
    Genter, K., Laue, T., Stone, P.: The RoboCup 2014 SPL drop-in player competition: encouraging teamwork without pre-coordination. In: AAMAS (2015)Google Scholar
  7. 7.
    Fang, H.-S., Xie, S., Tai, Y.W., Lu, C.: RMPE: regional multi-person pose estimation. In: ICCV (2017)Google Scholar
  8. 8.
    Ju, Z., Ji, X., Li, J., Liu, H.: An integrative framework of human hand gesture segmentation for human-robot interaction. IEEE Syst. J. 11(3), 1326–1336 (2017)CrossRefGoogle Scholar
  9. 9.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: ICCV (2017)Google Scholar
  10. 10.
    Leiva, F., Cruz, N., Bugueño, I., Ruiz-del-Solar, J.: Playing soccer without colors in the SPL: a convolutional neural network approach. CoRR abs/1811.12493 (2018)Google Scholar
  11. 11.
    Mühlenbrock, A., Laue, T.: Vision-based orientation detection of humanoid soccer robots. In: Akiyama, H., Obst, O., Sammut, C., Tonidandel, F. (eds.) RoboCup 2017. LNCS (LNAI), vol. 11175, pp. 204–215. Springer, Cham (2018). Scholar
  12. 12.
    Pennisi, A., Bloisi, D.D., Iocchi, L., Nardi, D.: Ground truth acquisition of humanoid soccer robot behaviour. In: Behnke, S., Veloso, M., Visser, A., Xiong, R. (eds.) RoboCup 2013. LNCS (LNAI), vol. 8371, pp. 560–567. Springer, Heidelberg (2014). Scholar
  13. 13.
    Sigalas, M., Baltzakis, H., Trahanias, P.: Gesture recognition based on arm tracking for human-robot interaction. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5424–5429. IEEE (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Valerio Di Giambattista
    • 1
    Email author
  • Mulham Fawakherji
    • 1
  • Vincenzo Suriani
    • 1
  • Domenico D. Bloisi
    • 2
  • Daniele Nardi
    • 1
  1. 1.Department of Computer, Control and Management EngineeringSapienza University of RomeRomeItaly
  2. 2.Department of Mathematics, Computer Science, and EconomicsUniversity of BasilicataPotenzaItaly

Personalised recommendations