Skip to main content
SpringerLink
Log in

Controlling maximum evaluation duration in on-line and on-board evolutionary robotics

On-the-fly control of maximum evaluation duration in on-line and on-board evolutionary robotics in highly dynamic and uncertain environments

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

On-line evolution of robot controllers allows robots to adapt while they perform their proper tasks. In our investigations, robots contain their own self-sufficient evolutionary algorithm (known as the encapsulated approach) where individual solutions are evaluated by means of a time sharing scheme: an individual controller is given the run of the robot for some amount of time and fitness corresponds to the robot’s task performance during that period. In this paper, we propose and provide a detailed analysis of two on-the-fly control schemes to set the evaluation time in highly dynamic scenarios with completely different tasks. One scheme, called the roulette-wheel selection scheme, stochastically selects evaluation time from promising intervals similar to multi-armed bandit schemes. The other scheme, named Heuristic-Rule (H-Rule), tweaks the evaluation time using specific heuristics. Our experiments show that H-Rule gives stable performance in different scenarios and can serve as a viable alternative to pre-selected optimal evaluation time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Angelov P, Filev DP, Kasabov N (2010) Evolving intelligent systems: methodology and applications. Wiley-IEEE Press. ISBN 0470287195, 9780470287194

  • Arif AQ, Nedev D, Haasdijk E (2013) Controlling evaluation duration in on-line and on-board evolutionary robotics. In: Proceedings of the 2013 IEEE symposium series on computational intelligence. IEEE Press

  • Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Mach Learn 47(2):235–256

    Article  MATH  Google Scholar 

  • Berry DA, Fristedt B (1985) Bandit problems: sequential allocation of experiments. Springer, Berlin, October 1985. ISBN 0412248107

  • Bredeche N, Montanier JM, Weel B, Haasdijk E (2013) Roborobo! a fast robot simulator for swarm and collective robotics. CoRR, abs/1304.2888, 2013

  • Davis L (1986) Adapting operator probabilities in genetic algorithms. In third international conference on genetic algorithms, pp 61–69

  • Eiben AE, Smit SK (2011) Parameter tuning for configuring and analyzing evolutionary algorithms. Swarm Evol Comput 1:19–31

    Article  Google Scholar 

  • Eiben AE, Michalewicz Z, Schoenauer M, Smith JE (2007) Parameter control in evolutionary algorithms. In: Lobo FG, Lima CF, Michalewicz Z (eds.) Parameter setting in evolutionary algorithms, volume 54 of studies in computational intelligence, Springer, Berlin, pp 19–46. ISBN 978-3-540-69431-1

  • Eiben AE, Haasdijk E, Bredeche N (2010) Symbiotic multi-robot organisms: reliability, adaptability, evolution. In: Levi P, Kernbach S (eds) Embodied, on-line, on-board evolution for autonomous robotics. Springer, Berlin, pp 361–382

    Google Scholar 

  • Floreano D, Mondada F (1998) Evolutionary neurocontrollers for autonomous mobile robots. Neural Netw, 11(78): 1461–1478. doi:10.1016/S0893-6080(98)00082-3.http://www.sciencedirect.com/science/article/pii/S0893608098000823. ISSN 0893–6080

  • Gittins JC (1989) Multi-armed bandit allocation indices. Wiley-interscience series in systems and optimization. Wiley, Chichester

    Google Scholar 

  • Haasdijk E, Arif A-Q, Eiben AE (2011) Racing to improve on-line, on-board evolutionary robotics. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2011). ACM, 2011

  • Haasdijk E, Smit SK, Eiben AE (2012) Exploratory analysis of an on-line evolutionary algorithm in simulated robots. Evol Intell 5(4):213–230

    Article  Google Scholar 

  • Hart WE, Belew RK (1991) Optimizing an arbitrary function is hard for the genetic algorithm. In: Proceedings of the fourth international conference on genetic algorithms, Morgan Kaufmann, pp 190–195

  • Karafotias G, Smit SK, Eiben AE (2012) A generic approach to parameter control. In: Di Chio C, Agapitos A, Cagnoni S, Cotta C, de Vega FF, Di Caro GA, Drechsler R, Ekárt A, Isabel Esparcia-Alcázar A, Farooq M, Langdon WB, Merelo Guervós JJ, Preuss M, Richter H, Silva S, Simões A, Squillero G, Tarantino E, Tettamanzi A, Togelius J, Urquhart N, Uyar S, Yannakakis GN, (eds.), EvoApplications, volume 7248 of lecture notes in computer science, pp 366–375. Springer, Berlin, ISBN 978-3-642-29177-7

  • Kasabov N (2007) Evolving connectionist systems. Springer, London

    MATH  Google Scholar 

  • Mitobe K, Capi G, Nasu Y (2000) Control of walking robots based on manipulation of the zero moment point. Robotica, 18(6):651–657. http://dblp.uni-trier.de/db/journals/robotica/robotica18.html#MitobeCN00

  • Nolfi Stefano, Floreano Dario (2000) Evolutionary robotics: the biology, intelligence, and technology. MIT Press, Cambridge

    Google Scholar 

  • Nordin P, Banzhaf W (1997) Real time control of a khepera robot using genetic programming. Cybernet Control, 26(3):533–561. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.52.6310

  • Noskov N, Haasdijk E, Weel B, Eiben AE (2013) Monee: using parental investment to combine open-ended and task-driven evolution. In: Isabel Esparcia-Alcázar A, (ed) EvoApplications, volume 7835 of Lecture Notes in Computer Science. Springer, Berlin. pp 569–578. ISBN 978-3-642-37191-2.

  • Pollack JB, Lipson H, Ficici S, Funes P, Hornby G, Watson RA (2000) Evolutionary techniques in physical robotics. In: Proceedings of the third international conference (ICES2000. Springer, Berlin, pp 175–186

  • Robbins Herbert (1952) Some aspects of the sequential design of experiments. Bull Am Math Soc 58(5):527–535

    Article  MATH  Google Scholar 

  • Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments. Springer, New York

    Book  MATH  Google Scholar 

  • Watson RA, Ficici SG, Pollack JB (1999) Embodied evolution: embodying an evolutionary algorithm in a population of robots. In congress on evolutionary computation. IEEE Press, pp 335–342

  • Watson RA, Ficici SG, Pollack JB (2002) Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robot Autonom Syst, 39(1):1–18. ISSN 0921–8890. doi:10.1016/S0921-8890(02)00170-7. http://www.sciencedirect.com/science/article/pii/S0921889002001707

Download references

Author information

Authors and Affiliations

  1. Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

    Atta-ul-Qayyum Arif & Evert Haasdijk

  2. MonetDB Solutions, Amsterdam, Netherlands

    D. G. Nedev

Authors
  1. Atta-ul-Qayyum Arif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. G. Nedev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Evert Haasdijk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Atta-ul-Qayyum Arif.

Additional information

Some of the work for this paper was done while D. G. Nedev was employed by SDL Fredhopper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arif, AuQ., Nedev, D.G. & Haasdijk, E. Controlling maximum evaluation duration in on-line and on-board evolutionary robotics. Evolving Systems 5, 275–286 (2014). https://doi.org/10.1007/s12530-014-9117-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-014-9117-x

Keywords

Search

Navigation