Abstract
FC Portugal, a team from the universities of Porto and Aveiro, won the main competition of the 2022 RoboCup 3D Simulation League, with 17 wins, 1 tie and no losses. During the course of the competition, the team scored 84 goals while conceding only 2. FC Portugal also won the 2022 RoboCup 3D Simulation League Technical Challenge, accumulating the maximum amount of points by ending first in its both events: the Free/Scientific Challenge, and the Fat Proxy Challenge. The team presented in this year’s competition was rebuilt from the ground up since the last RoboCup. No previous code was used or adapted, with the exception of the 6D pose estimation algorithm, and the get-up behaviors, which were re-optimized. This paper describes the team’s new architecture and development approach. Key strategy elements include team coordination, role management, formation, communication, skill management and path planning. New lower-level skills were based on a deterministic analytic model and a shallow neural network that learned residual dynamics through reinforcement learning. This process, together with an overlapped learning approach, improved seamless transitions, learning time, and the behavior in terms of efficiency and stability. In comparison with the previous team, the omnidirectional walk is more stable and went from 0.70 m/s to 0.90 m/s, the long kick from 15 m to 19 m, and the new close-control dribble reaches up to 1.41 m/s.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For previous contributions concerning coaching, visual debugging, team coordination, sim-to-real, optimization algorithms and frameworks please refer to https://tdp.robocup.org/tdp/2022-tdp-fcportugal3d-robocupsoccer-simulation-3d/.
- 2.
The official rules can be found at https://ssim.robocup.org/3d-simulation/3d-rules/.
- 3.
Official results can be found at https://cloud.robocup.org/s/ifX7TDsaHpCFWWH.
References
Abdolmaleki, A., Simões, D., Lau, N., Reis, L.P., Neumann, G.: Learning a humanoid kick with controlled distance. In: Behnke, S., Sheh, R., Sarıel, S., Lee, D.D. (eds.) RoboCup 2016. LNCS (LNAI), vol. 9776, pp. 45–57. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68792-6_4
Abreu, M., Lau, N., Sousa, A., Reis, L.P.: Learning low level skills from scratch for humanoid robot soccer using deep reinforcement learning. In: 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 256–263. IEEE (2019)
Abreu, M., Reis, L.P., Lau, N.: Learning to run faster in a humanoid robot soccer environment through reinforcement learning. In: Chalup, S., Niemueller, T., Suthakorn, J., Williams, M.-A. (eds.) RoboCup 2019. LNCS (LNAI), vol. 11531, pp. 3–15. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35699-6_1
Abreu, M., Silva, T., Teixeira, H., Reis, L.P., Lau, N.: 6D localization and kicking for humanoid robotic soccer. J. Intell. Robot. Syst. 102(2), 1–25 (2021)
Amelia, E., et al.: magmaFatProxy (2022). https://github.com/magmaOffenburg/magmaFatProxy
Hart, P., Nilsson, N., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybern. 4(2), 100–107 (1968)
Kasaei, M., Abreu, M., Lau, N., Pereira, A., Reis, L.P.: Learning hybrid locomotion skills - learn to exploit residual dynamics and modulate model-based gait control. arXiv preprint arXiv:2011.13798 (2020)
Kasaei, M., Abreu, M., Lau, N., Pereira, A., Reis, L.P.: A CPG-based agile and versatile locomotion framework using proximal symmetry loss. arXiv preprint arXiv:2103.00928 (2021)
Kasaei, M., Abreu, M., Lau, N., Pereira, A., Reis, L.P.: Robust biped locomotion using deep reinforcement learning on top of an analytical control approach. Robot. Auton. Syst. 146, 103900 (2021)
Kasaei, S.M., Simões, D., Lau, N., Pereira, A.: A hybrid ZMP-CPG based walk engine for biped robots. In: Ollero, A., Sanfeliu, A., Montano, L., Lau, N., Cardeira, C. (eds.) ROBOT 2017. AISC, vol. 694, pp. 743–755. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-70836-2_61
MacAlpine, P., Liu, B., Macke, W., Wang, C., Stone, P.: UT Austin Villa: RoboCup 2021 3D simulation league competition champions. In: Alami, R., Biswas, J., Cakmak, M., Obst, O. (eds.) RoboCup 2021. LNCS (LNAI), vol. 13132, pp. 314–326. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98682-7_26
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Simões, D., Amaro, P., Silva, T., Lau, N., Reis, L.P.: Learning low-level behaviors and high-level strategies in humanoid soccer. In: Silva, M.F., Luís Lima, J., Reis, L.P., Sanfeliu, A., Tardioli, D. (eds.) ROBOT 2019. AISC, vol. 1093, pp. 537–548. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-36150-1_44
Teixeira, H., Silva, T., Abreu, M., Reis, L.P.: Humanoid robot kick in motion ability for playing robotic soccer. In: 2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 34–39. IEEE (2020)
Xu, Y., Vatankhah, H.: SimSpark: an open source robot simulator developed by the RoboCup community. In: Behnke, S., Veloso, M., Visser, A., Xiong, R. (eds.) RoboCup 2013. LNCS (LNAI), vol. 8371, pp. 632–639. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44468-9_59
Acknowledgment
The first author is supported by FCT—Foundation for Science and Technology under grant SFRH/BD/139926/2018. The work was also partially funded by COMPETE 2020 and FCT, under projects UIDB/00027/2020 (LIACC) and UIDB/00127/2020 (IEETA).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Abreu, M., Kasaei, M., Reis, L.P., Lau, N. (2023). FC Portugal: RoboCup 2022 3D Simulation League and Technical Challenge Champions. In: Eguchi, A., Lau, N., Paetzel-Prüsmann, M., Wanichanon, T. (eds) RoboCup 2022: Robot World Cup XXV. RoboCup 2022. Lecture Notes in Computer Science(), vol 13561. Springer, Cham. https://doi.org/10.1007/978-3-031-28469-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-28469-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28468-7
Online ISBN: 978-3-031-28469-4
eBook Packages: Computer ScienceComputer Science (R0)