Learning to Approach a Moving Ball with a Simulated Two-Wheeled Robot

  • Felix Flentge
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4020)


We show how a two-wheeled robot can learn to approach a moving ball using Reinforcement Learning. The robot is controlled by setting the velocities of its two wheels. It has to reach the ball under certain conditions to be able to kick it towards a given target. In order to kick, the ball has to be in front of the robot. The robot also has to reach the ball at a certain angle in relation to the target, because the ball is always kicked in the direction from the center of the robot to the ball. The robot learns which velocity differences should be applied to the wheels: one of the wheels is set to the maximum velocity, the other one according to this difference. We apply a REINFORCE algorithm [1] in combination with some kind of extended Growing Neural Gas (GNG) [2] to learn these continuous actions. The resulting algorithm, called ReinforceGNG, is tested in a simulated environment with and without noise.


Quantization Error Radial Basis Function Network Autonomous Underwater Vehicle Average Simulation Time Reinforcement Baseline 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229–256 (1992)MATHGoogle Scholar
  2. 2.
    Fritzke, B.: A growing neural gas network learns topologies. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, vol. 7, Cambridge, MA (1995)Google Scholar
  3. 3.
    Balkcom, D., Mason, M.: Time optimal trajectories for bounded velocity differential drive vehicles. International Journal of Robotics Research 21, 199–217 (2002)CrossRefGoogle Scholar
  4. 4.
    Riedmiller, M., Janusz, B.: Using neural reinforcement controllers in robotics. In: Proc. 8th Australian Conference on Artificial Intelligence, Canberra (1995)Google Scholar
  5. 5.
    Gaskett, C., Wettergreen, D., Zelinsky, A.: Q-learning in continuous state and action spaces. In: Australian Joint Conference on Artificial Intelligence, pp. 417–428 (1999)Google Scholar
  6. 6.
    Sutton, R., Barto, A.: Reinforcement Learning. An Introduction. MIT Press, Cambridge (2000)Google Scholar
  7. 7.
    Martinetz, T.: Competitive hebbian learning rule forms perfectly topology preserving maps. In: Proc. of ICANN 1993, pp. 427–434. Springer, Heidelberg (1993)Google Scholar
  8. 8.
    Fritzke, B.: Fast learning with incremental RBF networks. Neural Processing Letters 1, 2–5 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Felix Flentge
    • 1
  1. 1.Department of Computer ScienceJohannes Gutenberg-University MainzMainzGermany

Personalised recommendations