Abstract
On-line evolution of robot controllers allows robots to adapt while they perform their proper tasks. In our investigations, robots contain their own self-sufficient evolutionary algorithm (known as the encapsulated approach) where individual solutions are evaluated by means of a time sharing scheme: an individual controller is given the run of the robot for some amount of time and fitness corresponds to the robot’s task performance during that period. In this paper, we propose and provide a detailed analysis of two on-the-fly control schemes to set the evaluation time in highly dynamic scenarios with completely different tasks. One scheme, called the roulette-wheel selection scheme, stochastically selects evaluation time from promising intervals similar to multi-armed bandit schemes. The other scheme, named Heuristic-Rule (H-Rule), tweaks the evaluation time using specific heuristics. Our experiments show that H-Rule gives stable performance in different scenarios and can serve as a viable alternative to pre-selected optimal evaluation time.
Similar content being viewed by others
References
Angelov P, Filev DP, Kasabov N (2010) Evolving intelligent systems: methodology and applications. Wiley-IEEE Press. ISBN 0470287195, 9780470287194
Arif AQ, Nedev D, Haasdijk E (2013) Controlling evaluation duration in on-line and on-board evolutionary robotics. In: Proceedings of the 2013 IEEE symposium series on computational intelligence. IEEE Press
Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Mach Learn 47(2):235–256
Berry DA, Fristedt B (1985) Bandit problems: sequential allocation of experiments. Springer, Berlin, October 1985. ISBN 0412248107
Bredeche N, Montanier JM, Weel B, Haasdijk E (2013) Roborobo! a fast robot simulator for swarm and collective robotics. CoRR, abs/1304.2888, 2013
Davis L (1986) Adapting operator probabilities in genetic algorithms. In third international conference on genetic algorithms, pp 61–69
Eiben AE, Smit SK (2011) Parameter tuning for configuring and analyzing evolutionary algorithms. Swarm Evol Comput 1:19–31
Eiben AE, Michalewicz Z, Schoenauer M, Smith JE (2007) Parameter control in evolutionary algorithms. In: Lobo FG, Lima CF, Michalewicz Z (eds.) Parameter setting in evolutionary algorithms, volume 54 of studies in computational intelligence, Springer, Berlin, pp 19–46. ISBN 978-3-540-69431-1
Eiben AE, Haasdijk E, Bredeche N (2010) Symbiotic multi-robot organisms: reliability, adaptability, evolution. In: Levi P, Kernbach S (eds) Embodied, on-line, on-board evolution for autonomous robotics. Springer, Berlin, pp 361–382
Floreano D, Mondada F (1998) Evolutionary neurocontrollers for autonomous mobile robots. Neural Netw, 11(78): 1461–1478. doi:10.1016/S0893-6080(98)00082-3.http://www.sciencedirect.com/science/article/pii/S0893608098000823. ISSN 0893–6080
Gittins JC (1989) Multi-armed bandit allocation indices. Wiley-interscience series in systems and optimization. Wiley, Chichester
Haasdijk E, Arif A-Q, Eiben AE (2011) Racing to improve on-line, on-board evolutionary robotics. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2011). ACM, 2011
Haasdijk E, Smit SK, Eiben AE (2012) Exploratory analysis of an on-line evolutionary algorithm in simulated robots. Evol Intell 5(4):213–230
Hart WE, Belew RK (1991) Optimizing an arbitrary function is hard for the genetic algorithm. In: Proceedings of the fourth international conference on genetic algorithms, Morgan Kaufmann, pp 190–195
Karafotias G, Smit SK, Eiben AE (2012) A generic approach to parameter control. In: Di Chio C, Agapitos A, Cagnoni S, Cotta C, de Vega FF, Di Caro GA, Drechsler R, Ekárt A, Isabel Esparcia-Alcázar A, Farooq M, Langdon WB, Merelo Guervós JJ, Preuss M, Richter H, Silva S, Simões A, Squillero G, Tarantino E, Tettamanzi A, Togelius J, Urquhart N, Uyar S, Yannakakis GN, (eds.), EvoApplications, volume 7248 of lecture notes in computer science, pp 366–375. Springer, Berlin, ISBN 978-3-642-29177-7
Kasabov N (2007) Evolving connectionist systems. Springer, London
Mitobe K, Capi G, Nasu Y (2000) Control of walking robots based on manipulation of the zero moment point. Robotica, 18(6):651–657. http://dblp.uni-trier.de/db/journals/robotica/robotica18.html#MitobeCN00
Nolfi Stefano, Floreano Dario (2000) Evolutionary robotics: the biology, intelligence, and technology. MIT Press, Cambridge
Nordin P, Banzhaf W (1997) Real time control of a khepera robot using genetic programming. Cybernet Control, 26(3):533–561. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.52.6310
Noskov N, Haasdijk E, Weel B, Eiben AE (2013) Monee: using parental investment to combine open-ended and task-driven evolution. In: Isabel Esparcia-Alcázar A, (ed) EvoApplications, volume 7835 of Lecture Notes in Computer Science. Springer, Berlin. pp 569–578. ISBN 978-3-642-37191-2.
Pollack JB, Lipson H, Ficici S, Funes P, Hornby G, Watson RA (2000) Evolutionary techniques in physical robotics. In: Proceedings of the third international conference (ICES2000. Springer, Berlin, pp 175–186
Robbins Herbert (1952) Some aspects of the sequential design of experiments. Bull Am Math Soc 58(5):527–535
Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments. Springer, New York
Watson RA, Ficici SG, Pollack JB (1999) Embodied evolution: embodying an evolutionary algorithm in a population of robots. In congress on evolutionary computation. IEEE Press, pp 335–342
Watson RA, Ficici SG, Pollack JB (2002) Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robot Autonom Syst, 39(1):1–18. ISSN 0921–8890. doi:10.1016/S0921-8890(02)00170-7. http://www.sciencedirect.com/science/article/pii/S0921889002001707
Author information
Authors and Affiliations
Corresponding author
Additional information
Some of the work for this paper was done while D. G. Nedev was employed by SDL Fredhopper.
Rights and permissions
About this article
Cite this article
Arif, AuQ., Nedev, D.G. & Haasdijk, E. Controlling maximum evaluation duration in on-line and on-board evolutionary robotics. Evolving Systems 5, 275–286 (2014). https://doi.org/10.1007/s12530-014-9117-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-014-9117-x