Ball Dribbling for Humanoid Biped Robots: A Reinforcement Learning and Fuzzy Control Approach

  • Leonardo Leottau
  • Carlos Celemin
  • Javier Ruiz-del-Solar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8992)

Abstract

In the context of the humanoid robotics soccer, ball dribbling is a complex and challenging behavior that requires a proper interaction of the robot with the ball and the floor. We propose a methodology for modeling this behavior by splitting it in two sub problems: alignment and ball pushing. Alignment is achieved using a fuzzy controller in conjunction with an automatic foot selector. Ball-pushing is achieved using a reinforcement-learning based controller, which learns how to keep the robot near the ball, while controlling its speed when approaching and pushing the ball. Four different models for the reinforcement learning of the ball-pushing behavior are proposed and compared. The entire dribbling engine is tested using a 3D simulator and real NAO robots. Performance indices for evaluating the dribbling speed and ball-control are defined and measured. The obtained results validate the usefulness of the proposed methodology, showing asymptotic convergence in around fifty training episodes, and similar performance between simulated and real robots.

Keywords

Reinforcement learning TSK fuzzy controller Soccer robotics Biped robot NAO Behavior Dribbling 

References

  1. 1.
    Alcaraz, J., Herrero, D., Mart, H.: A closed-loop dribbling gait for the standard platform league. In: Workshop on Humanoid Soccer Robots of the IEEE-RAS International Conference on Humanoid Robots (Humanoids), Bled, Slovenia (2011)Google Scholar
  2. 2.
    Latzke, T., Behnke, S., Bennewitz, M.: Imitative reinforcement learning for soccer playing robots. In: Lakemeyer, G., Sklar, E., Sorrenti, D.G., Takahashi, T. (eds.) RoboCup 2006: Robot Soccer World Cup X. LNCS (LNAI), vol. 4434, pp. 47–58. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Meriçli, Ç., Veloso, M., Akin, H.: Task refinement for autonomous robots using complementary corrective human feedback. Int. J. Adv. Robot. Syst. 8(2), 68–79 (2011)Google Scholar
  4. 4.
    Röfer, T., Laue, T., Müller, J., Bartsch, M., Batram, M.J., Böckmann, A., Lehmann, N., Maa, F., Münder, T., Steinbeck, M., Stolpmann, A., Taddiken, S., Wieschendorf, R., Zitzmann, D.: B-human team report and code release 2012. http://www.b-human.de/wpcontent/ uploads/2012/11/CodeRelease2012.pdf (2012)
  5. 5.
    HTWK-NAO-Team: Team Description Paper 2013. In: RoboCup 2013: Robot Soccer World Cup XVII Preproceedings. Eindhoven, RoboCup Federation, The Netherlands (2013)Google Scholar
  6. 6.
    Carvalho, A., Oliveira, R.. Reinforcement learning for the soccer dribbling task. In: 2011 IEEE Conference on Computational Intelligence and Games (CIG), Seoul, Korea (2011)Google Scholar
  7. 7.
    Riedmiller, M., Hafner, R., Lange, S., Lauer, M.: Learning to dribble on a real robot by success and failure. In: 2008 IEEE International Conference on Robotics and Automation (ICRA). IEEE, Pasadena, California (2008)Google Scholar
  8. 8.
    Ciesielski, V., Lai, S.Y.S.Y.: Developing a dribble-and-score behaviour for robot soccer using neuro evolution. Work. Intell. Evol. Syst. 2001, 70–78 (2013)Google Scholar
  9. 9.
    Nakashima, T., Ishibuchi, H.: Mimicking dribble trajectories by neural networks for RoboCup soccer simulation. In: IEEE 22nd International Symposium on Intelligent Control, ISIC 2007 (2007)Google Scholar
  10. 10.
    Li, X., Wang, M., Zell, A.: Dribbling control of omnidirectional soccer robots. In: Proceedings 2007 IEEE International Conference on Robotics and Automation (2007)Google Scholar
  11. 11.
    Zell, A.: Nonlinear predictive control of an omnidirectional robot dribbling a rolling ball. In: 2008 IEEE International Conference on Robot Automation (2008)Google Scholar
  12. 12.
    Emery, R., Balch, T.: Behavior-based control of a non-holonomic robot in pushing tasks. In: Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164), vol. 3 (2001)Google Scholar
  13. 13.
    Damas, B.D., Lima, P.U., Custódio, L.M.: A modified potential fields method for robot navigation applied to dribbling in robotic soccer. In: Kaminka, G.A., Lima, P.U., Rojas, R. (eds.) RoboCup 2002: Robot Soccer World Cup VI. LNCS, vol. 2752, pp. 65–77. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Tang, L., Liu, Y., Qiu, Y., Gu, G., Feng, X.: The strategy of dribbling based on artificial potential field. In: 2010 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE), vol. 2 (2010)Google Scholar
  15. 15.
    Celemin, C., Leottau, L.: Learning to dribble the ball in humanoid robotics soccer (2014). https://drive.google.com/folderview?id=0B9cesO4NvjiqdUpWaWFyLVQ3anM&usp=sharing
  16. 16.
    Storn, R., Price, K.: Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces (1995)Google Scholar
  17. 17.
    Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  18. 18.
    Leottau, L., Celemin, C.: UCH-Dribbling-Videos. https://www.youtube.com/watch?v=HP8pRh4ic8w. Accessed 28 April 2014

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Leonardo Leottau
    • 1
  • Carlos Celemin
    • 1
  • Javier Ruiz-del-Solar
    • 1
  1. 1.Department of Electrical Engineering and Advanced Mining Technology CenterUniversidad de ChileSantiagoChile

Personalised recommendations