Abstract
We show how to apply learning methods to two robotics problems, namely the optimization of the on-board controller of an omnidirectional robot, and the derivation of a model of the physical driving behavior for use in a simulator.
We show that optimal control parameters for several PID controllers can be learned adaptively by driving an omni directional robot on a field while evaluating its behavior, using an reinforcement learning algorithm. After training, the robots can follow the desired path faster and more elegantly than with manually adjusted parameters.
Secondly, we show how to learn the physical behavior of a robot. Our system learns to predict the position of the robots in the future according to their reactions to sent commands. We use the learned behavior in the simulation of the robots instead of adjusting the physical simulation model whenever the mechanics of the robot changes. The updated simulation reflects then the modified physics of the robot.
Chapter PDF
Similar content being viewed by others
References
Aeström, K.J., Hägglund, T., Hang, C., Ho, W.: Automatic tuning and adaptation for PID controllers—A survey. In: Dugard, L., M’Saad, M., Landau, I.D. (eds.) Adaptive Systems in Control and Signal Processing, pp. 371–376. Pergamon Press, Oxford (1992)
Aeström, K.J., Hägglund, T.: PID Controllers: Theory, Design, and Tuning, 2nd edn., Research Triangle Park, NC, Instrument Society of America (1995)
Behnke, S., Frötschl, B., Rojas, R., Ackers, P., Lindstrot, W., de Melo, M., Schebesch, A., Simon, M., Sprengel, M., Tenchio, O.: Using Hierarchical Dynamical Systems to Control Reactive Behavior. In: Veloso, M.M., Pagello, E., Kitano, H. (eds.) RoboCup 1999. LNCS (LNAI), vol. 1856, pp. 186–195. Springer, Heidelberg (2000)
Behnke, S., Egorova, A., Gloye, A., Rojas, R., Simon, M.: Predicting away the Delay. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS (LNAI), vol. 3020, pp. 712–719. Springer, Heidelberg (2004)
Brooks, R.A., Mataric, M.J.: Real robots, real learning problems. In: Connell, J.H., Mahadevan, S. (eds.) Robot Learning. Kluwer Academic Publishers, Dordrecht (1993)
Callender, A., Stevenson, A.B.: Automatic Control of Variable Physical Characteristics U.S. patent 2,175,985. Issued October 10, in the United States (1939)
Egorova, A., Gloye, A., Liers, A., Rojas, R., Schreiber, M., Simon, M., Tenchio, O., Wiesel, F.: FU-Fighters 2003 (Global Vision). In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup-2003: Robot Soccer World Cup VII. Springer, Heidelberg (2004)
Gloye, A., Simon, M., Egorova, A., Wiesel, F., Tenchio, O., Schreiber, M., Behnke, S., Rojas, R.: Predicting away robot control latency., Technical Report B08-03, Free University Berlin (2003)
Gloye, A., Simon, M., Egorova, A., Wiesel, F., Rojas, R.: Plug & Play: Fast Automatic Geometry andColor Calibration for a Camera Tracking Mobile Robots. In: Institut für Informatik, February 2004, Freie Universität, Berlin (2004) (paper under review)
Janusz, B., Riedmiller, M.: Self-Learning neural control of a mobile robot. In: Proceedings of the IEEE ICNN 1995, Perth, Australia (1995)
Kleiner, A., Dietl, M., Nebel, B.: Towards a Life-Long Learning Soccer Agent. In: Polani, D., Kaminka, G., Lima, P., Rojas, R. (eds.) RoboCup 2002. LNCS (LNAI), vol. 2752, pp. 119–127. Springer, Heidelberg (2003)
Kohl, N., Stone, P.: Policy Gradient Reinforcement Learning for Fast Quadrupedal Locomotion. Department of Computer Science, November 2003, The University of Texas, Austin (2003) (paper under review)
Koza, J.R., Keane, M.A., Streeter, M.J., Mydlowec, W., Yu, J., Lanza, G.: Genetic Programming IV: Routine Human-Competitive Machine Intelligence. Kluwer Academic Publishers, Dordrecht (2003)
Riedmiller, M., Schoknecht, R.: Einsatzmöglichkeiten selbständig lernender neuronaler Regler im Automobilbereich. In: Proceedings of the VDI-GMA Aussprachetag, Berlin (March 1998)
Simon, M., Behnke, S., Rojas, R.: Robust Real Time Color Tracking. In: Stone, P., Balch, T., Kraetzschmar, G.K. (eds.) RoboCup 2000. LNCS (LNAI), vol. 2019, pp. 239–248. Springer, Heidelberg (2001)
Stone, P., Sutton, R., Singh, S.: Reinforcement Learning for 3 vs. 2 Keepaway. In: Stone, P., Balch, T., Kraetzschmar, G.K. (eds.) RoboCup 2000. LNCS (LNAI), vol. 2019, pp. 249–258. Springer, Heidelberg (2001)
Tan, K.K., Wang, Q.G., Hang, C.C., Hägglund, T.: Advances in PID Control. In: Advances in Industrial Control Series. Springer, London (1999)
Ziegler, J.G., Nichols, N.B.: Optimum settings for automatic controllers. Transactions of ASME 64, 759–768 (1942)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gloye, A., Göktekin, C., Egorova, A., Tenchio, O., Rojas, R. (2005). Learning to Drive and Simulate Autonomous Mobile Robots. In: Nardi, D., Riedmiller, M., Sammut, C., Santos-Victor, J. (eds) RoboCup 2004: Robot Soccer World Cup VIII. RoboCup 2004. Lecture Notes in Computer Science(), vol 3276. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-32256-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-540-32256-6_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25046-3
Online ISBN: 978-3-540-32256-6
eBook Packages: Computer ScienceComputer Science (R0)