Abstract
The general framework tackled in this paper is the automatic generation of intelligent collective behaviors using genetic programming and reinforcement learning. We define a behavior-based system relying on automatic design process using artificial evolution to synthesize high level behaviors for autonomous agents. Behavioral strategies are described by tree-based structures, and manipulated by genetic evolving processes. Each strategy is dynamically evaluated during simulation, and weighted by an adaptative value. This value is a quality factor that reflects the relevance of a strategy as a good solution for the learning task. It is computed using heterogeneous reinforcement techniques associating immediate and delayed reinforcements as dynamic progress estimators. This work has been tested upon a canonical experimentation framework: the foraging robots problem. Simulations have been conducted and have produced some promising results.
This is a preview of subscription content, log in via an institution.
Preview
Unable to display preview. Download preview PDF.
References
J.E. Baker, “Adaptive Selection Methods for Genetic Algorithms”, in Proceedings of the First International Conference on Genetic Algorithm, pp. 101–111, Eds. L. Erlbaum Associates, L. Erlbaum Associates: Hillsdale, Pittsburgh, USA, 1985.
R.A. Brooks, “A Robust Layered Control System for a Mobile Robot: An Approach to Programming Behavior”, in IEEE Journal of Robots and Automation, Vol. RA-2(1), pp. 14–23, 1986.
R.A. Brooks, “Elephants Don’t Play Chess”, in Journal of Robotics and Autonomous Systems, Volume 6, pp. 3–15, 1990.
K.A. De Jong, “An Analysis of the Behavior of a Class of Genetic Adaptive Systems”, PhD thesis, University of Michigan, USA, 1975.
J.L. Deneubourg, S. Aron, S. Goss, J.M. Pasteels and G. Duerinck, “Random Behaviour, Amplification Processes and Number of Participants: How they Contribute to the Foraging Properties of Ants”, in Physica 22D, pp. 176–186, Amsterdam, North Holland, 1986.
M. Dorigo and U. Schnepf, “Genetics-based Machine Learning and Behaviour-based Robotics: A New Synthesis”, in IEEE Transactions on Systems, Man and Cybernetics, 23, 1, pp. 141–154, 1993.
A. Drogoul and J. Ferber, “From Tom Thumb to the Dockers: Some Experiments with Foraging Robots”, in From Animals to Animats II, pp. 451–459, MIT Press, Cambridge, USA, 1993.
J.H. Holland, “Adaptation in Natural and Artificial Systems”, Michigan Press University, Ann Arbor, USA, 1975.
J.H. Holland and J.S. Reitman, “Cognitive System based on Adaptive Algorithms”, in Pattern Directed Inference Systems, Eds. D.A. Waterman and F. Hayes-Roth, Academic Press, New-York, USA, 1978.
K.E. Kinnear, Jr., editor, “Advances in Genetic Programming”, MIT Press, Cambridge, USA, 1994.
J.R. Koza, “Hierarchical Genetic Algorithm Operating on Populations of Computer Programs”, in Proceedings of the 11th International Joint Conference on Artificial Intelligence, Eds. Morgan and Kaufman, Morgan and Kaufman, San Mateo, USA, 1989.
J.R. Koza, “Evolution and Co-Evolution of Computer Programs to Control Independent-Acting Agents”, in From Animals to Animats, MIT Press, Cambridge, USA, 1990.
J.R. Koza, “Genetic Programming II, Automatic Discovery of Reusable Programs”, MIT Press, Cambridge, USA, 1994.
M. Mataric, “Learning in Multi-Robot Systems”, in Adaptation and Learning in Multi-Agent Systems, Eds. G. Wei\ and S. Sen, Lecture Notes in Artificial Intelligence, Vol. 1042, pp. 152–163, Springer-Verlag, 1996.
M. Mataric, “Reinforcement Learning in the Multi-Robot Domain”, in Autonomous Robots, Vol. 4, Nℴ1, pp. 73–83, January, 1997.
M. Mataric, “Learning Social Behaviors”, to be published in Journal of Robotics and Autonomous Systems, 1997.
A. Safiotti, K. Konolige and E.H. Ruspini, “A Multivalued-logic Approach to Integrating Planning and Control”, in Artificial Intelligence, 76, 1-2, pp. 481–526, 1995.
T.W. Sandholm and R.H. Crites, “On Multi-Agent Q-Learning in a Semi-Competitive Domain”, in Proceedings of the 14th International Joint Conference on Artificial Intelligence, Workshop on Adaptation and Learning in Multi-Agent Systems, pp. 191–205, Montreal, Canada, 1995.
L. Steels, “Cooperation between distributed agents through self-organization”, in Journal of Robotics and Autonomous Systems, Amsterdam, North Holland, 1989.
C. Watkins and P. Dayan, “Technical Note on Q-Learning”, in Machine Learning, Vol. 8, pp. 279–292, Kluwer Academic Publishers, Boston, USA, 1992.
G. Wei\, “Adaptation and Learning in Multi-Agent Systems: Some Remarks and a Bibliography”, in Adaptation and Learning in Multi-Agent Systems, Eds. G. Wei\ and S. Sen, Lecture Notes of Artificial Intelligence, Vol. 1042, pp. 1–21, Springer-Verlag, 1996.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Calderoni, S., Marcenac, P. (1998). Genetic programming for automatic design of self-adaptive robots. In: Banzhaf, W., Poli, R., Schoenauer, M., Fogarty, T.C. (eds) Genetic Programming. EuroGP 1998. Lecture Notes in Computer Science, vol 1391. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0055936
Download citation
DOI: https://doi.org/10.1007/BFb0055936
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64360-9
Online ISBN: 978-3-540-69758-9
eBook Packages: Springer Book Archive