Skip to main content
SpringerLink
Log in

Vision-Based Machine Learning in Robot Soccer

  • Conference paper
  • First Online:
RoboCup 2021: Robot World Cup XXIV (RoboCup 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13132))

Included in the following conference series:

  • 911 Accesses

Abstract

Robots need to perceive their environment in order to properly interact with it. In the RoboCup Soccer Middle Size League (MSL) this happens primarily through cameras mounted on the robots. Machine Learning can be used to extract relevant features from camera imagery. The real-time analysis of camera data is a challenge for both traditional and Machine Learning algorithms, since all computations in the MSL have to be performed on the robot itself.

This contribution shows that it is possible to process camera imagery in real-time using Machine Learning. It does this by presenting the current state of Machine Learning in MSL and providing two examples that won the Scientific and Technical Challenges at RoboCup 2021. Both examples focus on semantic detection of objects and humans in imagery. The Scientific Challenge winner presents how YOLOv5 can be used for object detection in the MSL. The Technical Challenge winner demonstrates how to improve interaction between robots and humans in soccer using OpenPose. This contributes towards the goal of RoboCup to arrive at robots that can beat the human soccer world champion by 2050.

We would like to thank all members of RobotSports, current and past, for their help and support in making this possible. Moreover, additional thanks goes out to the other team members of Tech United Eindhoven: W. H. T. M. Aangenent, P. E. J. van Brakel, L. L. A. van Bree, M. Briegel, D. J. H. Bruijnen, E. Deniz, Y. G. M. Douven, P. H. E. M. van Lith, H. C. T. van de Loo, K. J. Meessen, F. B. F. Schoenmakers, J. Selten, C. A. M. Steijlen, and P. Teurlings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/ultralytics/yolov5.

  2. 2.

    https://github.com/tzutalin/labelImg.

  3. 3.

    The training data is available at https://github.com/Charelvanhoof/robocup_vdl.

  4. 4.

    https://github.com/onnx/onnx.

  5. 5.

    https://github.com/noahmr/yolov5-tensorrt.

  6. 6.

    Hardware designs are available on http://roboticopenplatform.org, open-source source code is available on https://gitlab.tue.nl/tech-united-eindhoven.

  7. 7.

    https://github.com/OpenKinect/libfreenect2.

  8. 8.

    In soccer, to kick the ball between the legs of an opponent. Also known as a panna.

  9. 9.

    https://youtu.be/V2flK6joM4s.

References

  1. Abreu, M., Reis, L.P., Lau, N.: Learning to run faster in a humanoid robot soccer environment through reinforcement learning. In: Chalup, S., Niemueller, T., Suthakorn, J., Williams, M.-A. (eds.) RoboCup 2019. LNCS (LNAI), vol. 11531, pp. 3–15. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35699-6_1

    Chapter  Google Scholar 

  2. Almeida, L., Santos, F., Facchinetti, T., Pedreiras, P., Silva, V., Lopes, L.S.: Coordinating distributed autonomous agents with a real-time database: the CAMBADA project. In: Aykanat, C., Dayar, T., Körpeoğlu, İ (eds.) ISCIS 2004. LNCS, vol. 3280, pp. 876–886. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30182-0_88

    Chapter  Google Scholar 

  3. Andersen, R.S., Madsen, O., Moeslund, T.B., Amor, H.B.: Projecting robot intentions into human environments. In: 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 294–301 (2016). https://doi.org/10.1109/ROMAN.2016.7745145

  4. Asali, E., Valipour, M., Zare, N., Afshar, A., Katebzadeh, M., Dastghaibyfard, G.: Using machine learning approaches to detect opponent formation. In: 2016 Artificial Intelligence and Robotics (IRANOPEN), pp. 140–144 (2016)

    Google Scholar 

  5. van der Burgh, M., et al.: Tech United Eindhoven @home 2020 team description paper (2020). https://www.techunited.nl/uploads/Tech_United_At_Home_TDP_2020.pdf

  6. Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2021). https://doi.org/10.1109/TPAMI.2019.2929257

  7. Di Giambattista, V., Fawakherji, M., Suriani, V., Bloisi, D.D., Nardi, D.: On field gesture-based robot-to-robot communication with NAO soccer players. In: Chalup, S., Niemueller, T., Suthakorn, J., Williams, M.-A. (eds.) RoboCup 2019. LNCS (LNAI), vol. 11531, pp. 367–375. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35699-6_28

    Chapter  Google Scholar 

  8. Eady, T.: Tesla’s deep learning at scale: using billions of miles to train neural networks (2019). https://towardsdatascience.com/teslas-deep-learning-at-scale-7eed85b235d3. Accessed 21 Oct 2021

  9. Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E.: RoboCup: the robot world cup initiative. In: Proceedings of the First International Conference on Autonomous Agents, AGENTS 1997, pp. 340–347. Association for Computing Machinery, New York (1997)

    Google Scholar 

  10. de Koning, L., Mendoza, J.P., Veloso, M., van de Molengraft, R.: Skills, tactics and plays for distributed multi-robot control in adversarial environments. In: Akiyama, H., Obst, O., Sammut, C., Tonidandel, F. (eds.) RoboCup 2017. LNCS (LNAI), vol. 11175, pp. 277–289. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00308-1_23

    Chapter  Google Scholar 

  11. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  12. van Lith, P., van de Molengraft, M., Dubbelman, G., Plantinga, M.: A minimalistic approach to identify and localize robots in RoboCup MSL soccer competitions in real-time (2019). https://www.techunited.nl/uploads/Minimalist%20MSL%20Robot%20Location%205.0.pdf

  13. Lopez Martinez, C., et al.: Tech United Eindhoven team description (2014)

    Google Scholar 

  14. Luo, S., Lu, H., Xiao, J., Yu, Q., Zheng, Z.: Robot detection and localization based on deep learning. In: 2017 Chinese Automation Congress, pp. 7091–7095 (2017). https://doi.org/10.1109/CAC.2017.8244056

  15. Mazhar, O., Ramdani, S., Navarro, B., Passama, R., Cherubini, A.: Towards real-time physical human-robot interaction using skeleton information and hand gestures. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–6 (2018). https://doi.org/10.1109/IROS.2018.8594385

  16. Mizuchi, Y., Inamura, T.: Estimation of subjective evaluation of HRI performance based on objective behaviors of human and robots. In: Chalup, S., Niemueller, T., Suthakorn, J., Williams, M.-A. (eds.) RoboCup 2019. LNCS (LNAI), vol. 11531, pp. 201–212. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35699-6_16

    Chapter  Google Scholar 

  17. MSL Technical Committee 1997–2021: Middle Size Robot League Rules and Regulations for 2021. https://msl.robocup.org/rules

  18. Nakata, T., Sato, T., Mori, T.: Expression of emotion and intention by robot body movement (1998)

    Google Scholar 

  19. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceeding of CVPR, pp. 779–788. IEEE (2016). https://doi.org/10.1109/CVPR.2016.91

  20. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of CVPR, pp. 7263–7271. IEEE (2017)

    Google Scholar 

  21. RoboCup Federation: RoboCup objective. https://www.robocup.org/objective. Accessed 21 Oct 2021

  22. Schneider, P., Memmesheimer, R., Kramer, I., Paulus, D.: Gesture recognition in RGB videos using human body keypoints and dynamic time warping. In: Chalup, S., Niemueller, T., Suthakorn, J., Williams, M.-A. (eds.) RoboCup 2019. LNCS (LNAI), vol. 11531, pp. 281–293. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35699-6_22

    Chapter  Google Scholar 

  23. Schwab, D., Zhu, Y., Veloso, M.: Learning skills for small size league RoboCup. In: Holz, D., Genter, K., Saad, M., von Stryk, O. (eds.) RoboCup 2018. LNCS (LNAI), vol. 11374, pp. 83–95. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27544-0_7

    Chapter  Google Scholar 

  24. Trifan, A., Neves, A.J.R., Cunha, B., Azevedo, J.L.: UAVision: a modular time-constrained vision library for soccer robots. In: Bianchi, R.A.C., Akin, H.L., Ramamoorthy, S., Sugiura, K. (eds.) RoboCup 2014. LNCS (LNAI), vol. 8992, pp. 490–501. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18615-3_40

    Chapter  Google Scholar 

  25. Vanholder, H.: Efficient inference with TensorRT (2016). https://on-demand.gputechconf.com/gtc-eu/2017/presentation/23425-han-vanholder-efficient-inference-with-tensorrt.pdf. Accessed 21 Oct 2021

Download references

Author information

Authors and Affiliations

  1. Tech United Eindhoven, De Rondom 70, P.O. Box 513, 5600 MB, Eindhoven, The Netherlands

    J. J. Olthuis, S. T. Kempers, R. M. Beumer, W. J. P. Kuijpers, A. A. Kokkelmans, W. Houtman, J. J. Kon & M. J. G. van de Molengraft

  2. RobotSports, De Schakel 22, 5651 GH, Eindhoven, The Netherlands

    N. B. van der Meer, C. A. van Hoof, J. J. F. J. van Eijck & A. T. A. Peijnenburg

Authors
  1. J. J. Olthuis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. B. van der Meer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. T. Kempers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. C. A. van Hoof
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. R. M. Beumer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. W. J. P. Kuijpers
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. A. A. Kokkelmans
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. W. Houtman
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. J. J. F. J. van Eijck
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. J. J. Kon
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. A. T. A. Peijnenburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. M. J. G. van de Molengraft
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. J. Olthuis .

Editor information

Editors and Affiliations

  1. LAAS-CNRS/ANITI, Toulouse, France

    Rachid Alami

  2. The University of Texas at Austin, Austin, TX, USA

    Joydeep Biswas

  3. University of Washington, Seattle, WA, USA

    Maya Cakmak

  4. Western Sydney University, Penrith, NSW, Australia

    Oliver Obst

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Olthuis, J.J. et al. (2022). Vision-Based Machine Learning in Robot Soccer. In: Alami, R., Biswas, J., Cakmak, M., Obst, O. (eds) RoboCup 2021: Robot World Cup XXIV. RoboCup 2021. Lecture Notes in Computer Science(), vol 13132. Springer, Cham. https://doi.org/10.1007/978-3-030-98682-7_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-98682-7_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98681-0

  • Online ISBN: 978-3-030-98682-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation