Abstract
Learning algorithms for autonomous robots in complex, real-world environments usually have to deal with many degrees of freedom and continuous state spaces. Reinforcement learning is a promising concept for unsupervised learning, but common algorithms suffer from huge storage and calculation requirements if they are used to construct an internal model by estimating a value-function for every action in every possible state. In our attempt to approximate this function at the lowest cost, we introduce a flexible method that focuses on the states of greatest interest, and interpolates between them with a fast and easy-to-implement algorithm. In order to provide the highest accuracy to any predefined limit, we enhanced this algorithm by a fast converging multilayer error approximator.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge, MA, London, UK.
Fischer J (1999) Strategiebildung mit neuronalen Netzen (in German). Diploma thesis, Department of Applied Physics, WWU-Muenster
Breithaupt R (1999) Adaptive und kooperative Automaten in statischen und dynamischen Umgebungen (in German). Diploma thesis, department of Applied Physics, WWU-Muenster
Sutton RS, Santamaria CJ, Ram A (1996) Experiments with reinforcement learning problems with continuous state and action spaces. University of Massachusetts Amherst (UM-CS-1996-088)
Kroese BJA, van Dam JW (1992) Adaptive space quantisation for reinforcement learning of collision-free navigation. Faculty of Mathematics and Computer Science University of Amsterdam
Samejima K, Omori T (1999) Adaptive internal state-space construction method for reinforcement learning of a real-world agent. Neural Networks 12:1143–1156
Okabe A, Boots B, Sugihara K, et al. (2000) Concepts and applications of Voronoi diagrams, 2nd edn. Wiley, Chichester
Kohonen T (1989) Self-organization and associative memory. Springer series in information sciences, 3rd edn. Springer, Berlin
Herz J, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley, Reading
Hertzberg J, Kirchner F (1997) A prototype study of an autonomous robot platform for sewerage system maintainance. Auton Robots J 4:319–331
Tesauro G (1992) Temporal difference learning of backgammon strategy. In: Sleeman D, Edwards P (eds) Machine learning. Morgan Kaufmann, San Mateo, p 451–457
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Fischer, J., Breithaupt, R. & Bode, M. Adaptive and economic data representation in control architectures of autonomous real-world robots. Artif Life Robotics 6, 200–204 (2002). https://doi.org/10.1007/BF02481268
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02481268