Skip to main content
SpringerLink
Log in

Exploratory analysis of an on-line evolutionary algorithm in simulated robots

  • Research Paper
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

In traditional evolutionary robotics, robot controllers are evolved in a separate design phase preceding actual deployment; we call this off-line evolution. Alternatively, robot controllers can evolve while the robots perform their proper tasks, during the actual operational phase; we call this on-line evolution. In this paper we describe three principal categories of on-line evolution for developing robot controllers (encapsulated, distributed, and hybrid), present an evolutionary algorithm belonging to the first category (the (μ + 1) on-line algorithm), and perform an extensive study of its behaviour. In particular, we use the Bonesa parameter tuning method to explore its parameter space. This delivers near-optimal settings for our algorithm in a number of tasks and, even more importantly, it offers profound insights into the impact of our algorithm’s parameters and features. Our experimental analysis of (μ + 1) on-line shows that it seems preferable to try many alternative solutions and spend little effort on refining possibly faulty assessments; that there is no single combination of parameters that performs well on all problem instances and that the most influential parameter of this algorithm—and therefore the prime candidate for a control scheme—is the evaluation length τ.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://www.cyberbotics.com/.

  2. Source code for the algorithm as well as Webots configuration files for the experiments described here is available at http://www.few.vu.nl/~ehaasdi/papers/MuPlusOne-2012.

References

  1. Bäck T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, Oxford, UK

    MATH  Google Scholar 

  2. Baldassarre G, Nolfi S, Parisi D (2002) Evolving mobile robots able to display collective behaviours. Artif Life 9:255–267

    Article  Google Scholar 

  3. Bartz-Beielstein T, Lasarczyk C, Preuss M (2005) Sequential parameter optimization. In: Proceedings of the 2005 IEEE congress on evolutionary computation IEEE congress on evolutionary computation, vol 1. IEEE Press, Edinburgh, UK, pp 773–780. doi:10.1109/CEC.2005.1554761

  4. Beyer HG (2000) Evolutionary algorithms in noisy environments: theoretical issues and guidelines for practice. Comput Methods Appl Mech Eng 186(2–4):239–267. doi:10.1016/S0045-7825(99)00386-2, http://www.sciencedirect.com/science/article/B6V29-40CRYKF-7/2/d4ea05ab01cc73ab36dbdf737c752439

  5. Branke J, Schmidt C, Schmec H (2001) Efficient fitness estimation in noisy environments. In: Proceedings of genetic and evolutionary computation, pp 243–250

  6. Bredeche N, Haasdijk E, Eiben A (2009) On-line, on-board evolution of robot controllers. In: Collet P, Monmarché N, Legrand P, Schoenauer M, Lutton E (eds) Artificial evolution, Springer, Lecture Notes in Computer Science, pp 110–121. http://few.vu.nl/∼ehaasdi/papers/EA2009selfadapt1robot.pdf

  7. Brooks RA (1992) Artificial life and real robots. In: Varela F, Bourgine P (eds) Toward a practice of autonomous systems: proceedings of the 1st European conference on artificial life, MIT Press, Cambridge, MA, pp 3–10. http://people.csail.mit.edu/brooks/papers/real-robots.pdf

  8. Christensen DJ, Spröwitz A, Ijspeert AJ (2010) Distributed online learning of central pattern generators in modular robots. In: Doncieux S, Girard B, Guillot A, Hallam J, Meyer JA, Mouret JB (eds) From animals to animats 11, Springer Berlin, Lecture Notes in Computer Science, vol 6226. pp 402–412

  9. Eiben A, Smith J (2003) Introduction to evolutionary computing. Springer, Berlin

    MATH  Google Scholar 

  10. Eiben AE, Smit SK (2011) Parameter tuning for configuring and analyzing evolutionary algorithms. Swarm Evol Comput 1(1):19–31

    Article  Google Scholar 

  11. El-Beltagy M, Nair P, Keane A (1999) Metamodeling techniques for evolutionary optimization of computationally expensive problems: Promises and limitations. In: Banzhaf W, Daida J, Eiben A, Garzon M, Honavar V, Jakiela M, Smith R (eds). Proceedings of the genetic and evolutionary computation conference (GECCO-1999). Morgan Kaufmann, San Francisco, pp 196–203

  12. Floreano D, Schoeni N, Caprari G, Blynel J (2002) Evolutionary bits’n’spikes. In: Standish RK, Bedau MA, Abbass HA (eds) Artificial Life VIII : Proceedings of the eighth international conference on artificial life. MIT Press, Cambridge, MA, pp 335–344

  13. Haasdijk E, Eiben AE, Karafotias G (2010) On-line evolution of robot controllers by an encapsulated evolution strategy. In: Proceedings of the 2010 IEEE congress on evolutionary computation, IEEE computational intelligence society, IEEE Press, Barcelona. http://www.few.vu.nl/∼ehaasdi/papers/cec-2010-MuPlusOne.pdf

  14. Haasdijk E, ul Qayyum AA, Eiben A (2011) Racing to improve on-line, on-board evolutionary robotics. In: Krasnogor N (ed) Proceedings of the genetic and evolutionary computation conference (GECCO-2011), ACM, Dublin, Ireland, pp 187–194. http://www.cs.vu.nl/ci/pubs/2011/MuPlusOneAndRacing.pdf

  15. Haralick R, Shapiro L (1992) Computer and robot vision, chap 7, vol 1, Addison-Wesley Publishing Company, Reading, MA

  16. Haroun Mahdavi S, Bentley PJ (2006) Innately adaptive robotics through embodied evolution. Auton Robots 20(2):149–163. doi:10.1007/s10514-006-5941-6

    Google Scholar 

  17. Hart W, Belew R (1991) Optimizing an arbitrary function is hard for the genetic algorithm. In: Belew RK, Booker LB (eds) Proceedings of the fourth international conference on genetic algorithms (ICGA ’91), Morgan Kaufmann Publishers, Inc., San Mateo CA, pp 190–195. URL citeseer.ist.psu.edu/hart91optimizing.html

  18. Hoeffding W (1963) Probability inequalities for sums of bounded random variables. J Am Stat Assoc 58(301):13–30

    Article  MathSciNet  MATH  Google Scholar 

  19. Hooker J (1995) Testing heuristics: we have it all wrong. J Heuristics 1:33–42. doi:10.1007/BF02430364

    Google Scholar 

  20. Jelasity M, Montresor A, Babaoglu O (2005) Gossip-based aggregation in large dynamic networks. ACM Trans Comput Syst 23:219–252. doi:10.1145/1082469.1082470

    Google Scholar 

  21. Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput 9(1):3–12

    Article  Google Scholar 

  22. Johnson DS (2002) A theoretician’s guide to the experimental analysis of algorithms. In: Goldwasser MH, Johnson DS, McGeoch CC (eds) Data structures, near neighbor searches, and methodology: fifth and sixth DIMACS implementation challenges. American Mathematical Society, Providence, RI, pp 215–250

  23. Jones T (1995) Crossover, macromutation, and population-based search. In: Eshelman L (ed) Proceedings of the 6th international conference on genetic algorithms. Morgan Kaufmann, San Francisco, pp 73–80

  24. Karafotias G, Haasdijk E, Eiben A (2011) An algorithm for distributed on-line, on-board evolutionary robotics. In: Krasnogor N (ed) pp 171–178. http://www.cs.vu.nl/ci/pubs/2011/Encapsulated-vs-Distributed.pdf

  25. Krasnogor N (ed) (2011) Proceedings of the genetic and evolutionary computation conference (GECCO-2011), ACM, Dublin, Ireland

  26. Martinoli A (1999) Swarm intelligence in autonomous collective robotics from tools to the analysis and synthesis of distributed control strategies. PhD thesis, Ecole Polytechnique Fédérale de Lausanne

  27. Nannen V, Eiben A (2007) Relevance estimation and value calibration of evolutionary algorithm parameters. In: Proceedings of the international joint conference on atrificial intelligence (IJCAI’07), international joint conferences on artificial intelligence. AAAI Press, Menlo Park, CA, pp 975–980

  28. Nehmzow U (2002) Physically embedded genetic algorithm learning in multi-robot scenarios: The pega algorithm. In: Prince C, Demiris Y, Marom Y, Kozima H, Balkenius C (eds) Proceedings of the second international workshop on epigenetic robotics: modeling cognitive development in robotic systems, LUCS, Edinburgh, UK, no. 94 in Lund University Cognitive Studies

  29. Nelson AL, Barlow GJ, Doitsidis L (2009) Fitness functions in evolutionary robotics: a survey and analysis. Robot Auton Syst 57(4):345–370. doi:10.1016/j.robot.2008.09.009, http://www.sciencedirect.com/science/article/B6V16-4TTMJV3-1/2/2549524d8e0f3982730659e49ad3fa75

    Google Scholar 

  30. Nolfi S, Floreano D (2000) Evolutionary robotics: the biology, intelligence, and technology of self-organizing machines. MIT Press, Cambridge, MA

    Google Scholar 

  31. Nordin P, Banzhaf W (1997) An on-line method to evolve behavior and to control a miniature robot in real time with genetic programming. Adapt Behav 5:107–140

    Article  Google Scholar 

  32. Ostermeier A, Gawelczyk A, Hansen N (1994) A derandomized approach to self adaptation of evolution strategies. Evol Comput 2(4):369–380. doi:10.1162/evco.1994.2.4.369

    Google Scholar 

  33. Pedersen MEH et al (2008) Parameter tuning versus adaptation: proof of principle study on differential evolution. Tech. rep., Hvass Laboratories. http://citeseer.ist.psu.edu/viewdoc/download?doi=10.1.1.149.6020&rep=rep1&type=pdf

  34. Schwefel HP (1995) Evolution and optimum seeking. Wiley, New York

    Google Scholar 

  35. Simões EDV, Dimond KR (2001) Embedding a distributed evolutionary system into population of autonomous mobile robots. In: Proceedings of the 2001 IEEE systems, man, and cybernetics conference. http://citeseer.ist.psu.edu/simoes01embedding.html

  36. Smit S, Eiben A (2010) Parameter tuning of evolutionary algorithms: Generalist vs. specialist. In: Chio CD et al. (eds) Applications of evolutionary computation, Springer, Lecture Notes in Computer Science, vol 6024, pp 542–551. http://www.cs.vu.nl/∼gusz/papers/2010-EVOSTAR-GeneralistSpecialist.pdf

  37. Smit S, Eiben A (2011) Multi-problem parameter tuning using bonesa. In: Hao JK, Legrand P, Collet P, Monmarché N, Lutton E, Schoenauer M (eds) Proceedings of artificial evolution, 10th international conference, evolution artificielle (EA 2011), Springer, Lecture Notes in Computer Science, pp 222–233

  38. Tuci E, Quinn M, Harvey I (2002) Evolving fixed-weight networks for learning robots. In: CEC ’02: Proceedings of the evolutionary computation on 2002. CEC ’02. Proceedings of the 2002 congress, IEEE Computer Society, pp 1970–1975

  39. Walker JH, Garrett SM, Wilson MS (2006) The balance between initial training and lifelong adaptation in evolving robot controllers. IEEE Trans Syst Man Cybern Part B 36(2):423–432

    Article  Google Scholar 

  40. Watson RA, Ficici SG, Pollack JB (2002) Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robot Auton Syst 39(1):1–18. http://eprints.ecs.soton.ac.uk/10620/

  41. Wischmann S, Stamm K, Wörgötter F (2007) Embodied evolution and learning: the neglected timing of maturation. In: Almeidae Costa F (ed) Advances in artificial life: 9th European conference on artificial life, lecture notes in artificial intelligence, vol 4648. Springer, Lisbon, Portugal, pp 284–293

  42. Zitzler E, Laumanns M, Thiele L (2001) SPEA2: Improving the strength pareto evolutionary algorithm. Tech. Rep. 103, Computer Engineering and Networks Laboratory (TIK), Swiss Federal Institute of Technology (ETH) Zürich, Gloriastrasse 35, CH-8092 Zürich, Switzerland

Download references

Acknowledgments

The authors gratefully acknowledge D.J. Christensen’s providing the code on which we based our own locomotion experiments. Giorgos Karafotias was very helpful in setting up the other experiments. The authors thank the reviewers for their extensive and insightful comments; this has helped us produce a much better paper.

Author information

Authors and Affiliations

  1. Vrije Universiteit, Amsterdam, The Netherlands

    Evert Haasdijk & A. E. Eiben

  2. TNO, The Hague, The Netherlands

    S. K. Smit

Authors
  1. Evert Haasdijk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. K. Smit
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. E. Eiben
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Evert Haasdijk.

Additional information

Parts of this work were made possible by the European Union FET Proactive Initiative: Pervasive Adaptation funding the symbrion project under grant agreement 216342.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Haasdijk, E., Smit, S.K. & Eiben, A.E. Exploratory analysis of an on-line evolutionary algorithm in simulated robots. Evol. Intel. 5, 213–230 (2012). https://doi.org/10.1007/s12065-012-0083-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-012-0083-6

Keywords

Search

Navigation