Genetic programming for automatic design of self-adaptive robots

  • Stéphane Calderoni
  • Pierre Marcenac
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1391)


The general framework tackled in this paper is the automatic generation of intelligent collective behaviors using genetic programming and reinforcement learning. We define a behavior-based system relying on automatic design process using artificial evolution to synthesize high level behaviors for autonomous agents. Behavioral strategies are described by tree-based structures, and manipulated by genetic evolving processes. Each strategy is dynamically evaluated during simulation, and weighted by an adaptative value. This value is a quality factor that reflects the relevance of a strategy as a good solution for the learning task. It is computed using heterogeneous reinforcement techniques associating immediate and delayed reinforcements as dynamic progress estimators. This work has been tested upon a canonical experimentation framework: the foraging robots problem. Simulations have been conducted and have produced some promising results.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Baker 1985]
    J.E. Baker, “Adaptive Selection Methods for Genetic Algorithms”, in Proceedings of the First International Conference on Genetic Algorithm, pp. 101–111, Eds. L. Erlbaum Associates, L. Erlbaum Associates: Hillsdale, Pittsburgh, USA, 1985.Google Scholar
  2. [Brooks 1986]
    R.A. Brooks, “A Robust Layered Control System for a Mobile Robot: An Approach to Programming Behavior”, in IEEE Journal of Robots and Automation, Vol. RA-2(1), pp. 14–23, 1986.Google Scholar
  3. [Brooks 1990]
    R.A. Brooks, “Elephants Don’t Play Chess”, in Journal of Robotics and Autonomous Systems, Volume 6, pp. 3–15, 1990.CrossRefGoogle Scholar
  4. [De Jong 1975]
    K.A. De Jong, “An Analysis of the Behavior of a Class of Genetic Adaptive Systems”, PhD thesis, University of Michigan, USA, 1975.Google Scholar
  5. [Deneubourg and al. 1986]
    J.L. Deneubourg, S. Aron, S. Goss, J.M. Pasteels and G. Duerinck, “Random Behaviour, Amplification Processes and Number of Participants: How they Contribute to the Foraging Properties of Ants”, in Physica 22D, pp. 176–186, Amsterdam, North Holland, 1986.Google Scholar
  6. [Dorigo and Schnepf 1993]
    M. Dorigo and U. Schnepf, “Genetics-based Machine Learning and Behaviour-based Robotics: A New Synthesis”, in IEEE Transactions on Systems, Man and Cybernetics, 23, 1, pp. 141–154, 1993.CrossRefGoogle Scholar
  7. [Drogoul and Ferber 1993]
    A. Drogoul and J. Ferber, “From Tom Thumb to the Dockers: Some Experiments with Foraging Robots”, in From Animals to Animats II, pp. 451–459, MIT Press, Cambridge, USA, 1993.Google Scholar
  8. [Holland 1975]
    J.H. Holland, “Adaptation in Natural and Artificial Systems”, Michigan Press University, Ann Arbor, USA, 1975.Google Scholar
  9. [Holland and Reitman 1978]
    J.H. Holland and J.S. Reitman, “Cognitive System based on Adaptive Algorithms”, in Pattern Directed Inference Systems, Eds. D.A. Waterman and F. Hayes-Roth, Academic Press, New-York, USA, 1978.Google Scholar
  10. [Kinnear 1994]
    K.E. Kinnear, Jr., editor, “Advances in Genetic Programming”, MIT Press, Cambridge, USA, 1994.Google Scholar
  11. [Koza 1989]
    J.R. Koza, “Hierarchical Genetic Algorithm Operating on Populations of Computer Programs”, in Proceedings of the 11th International Joint Conference on Artificial Intelligence, Eds. Morgan and Kaufman, Morgan and Kaufman, San Mateo, USA, 1989.Google Scholar
  12. [Koza 1990]
    J.R. Koza, “Evolution and Co-Evolution of Computer Programs to Control Independent-Acting Agents”, in From Animals to Animats, MIT Press, Cambridge, USA, 1990.Google Scholar
  13. [Koza 1994]
    J.R. Koza, “Genetic Programming II, Automatic Discovery of Reusable Programs”, MIT Press, Cambridge, USA, 1994.Google Scholar
  14. [Mataric 1996]
    M. Mataric, “Learning in Multi-Robot Systems”, in Adaptation and Learning in Multi-Agent Systems, Eds. G. Wei\ and S. Sen, Lecture Notes in Artificial Intelligence, Vol. 1042, pp. 152–163, Springer-Verlag, 1996.Google Scholar
  15. [Mataric 1997a]
    M. Mataric, “Reinforcement Learning in the Multi-Robot Domain”, in Autonomous Robots, Vol. 4, Nℴ1, pp. 73–83, January, 1997.CrossRefGoogle Scholar
  16. [Mataric 1997b]
    M. Mataric, “Learning Social Behaviors”, to be published in Journal of Robotics and Autonomous Systems, 1997.Google Scholar
  17. [Safiotti and al. 1995]
    A. Safiotti, K. Konolige and E.H. Ruspini, “A Multivalued-logic Approach to Integrating Planning and Control”, in Artificial Intelligence, 76, 1-2, pp. 481–526, 1995.CrossRefGoogle Scholar
  18. [Sandholm and Crites 1995]
    T.W. Sandholm and R.H. Crites, “On Multi-Agent Q-Learning in a Semi-Competitive Domain”, in Proceedings of the 14th International Joint Conference on Artificial Intelligence, Workshop on Adaptation and Learning in Multi-Agent Systems, pp. 191–205, Montreal, Canada, 1995.Google Scholar
  19. [Steels 1989]
    L. Steels, “Cooperation between distributed agents through self-organization”, in Journal of Robotics and Autonomous Systems, Amsterdam, North Holland, 1989.Google Scholar
  20. [Watkins and Dayan 1992]
    C. Watkins and P. Dayan, “Technical Note on Q-Learning”, in Machine Learning, Vol. 8, pp. 279–292, Kluwer Academic Publishers, Boston, USA, 1992.Google Scholar
  21. [Wei\ 1996]
    G. Wei\, “Adaptation and Learning in Multi-Agent Systems: Some Remarks and a Bibliography”, in Adaptation and Learning in Multi-Agent Systems, Eds. G. Wei\ and S. Sen, Lecture Notes of Artificial Intelligence, Vol. 1042, pp. 1–21, Springer-Verlag, 1996.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Stéphane Calderoni
    • 1
  • Pierre Marcenac
    • 1
  1. 1.Iremia - Université de La ReunionSaint-Denis messag cedex 9

Personalised recommendations