Genetic Programming and Evolvable Machines

, Volume 15, Issue 4, pp 433–476 | Cite as

Driving as a human: a track learning based adaptable architecture for a car racing controller

Article

Abstract

We present the evolution and current state of the Mr. Racer car racing controller that excelled at the corresponding TORCS competitions of the last years. Although several heuristics and black-box optimization methods are employed, the basic idea of the controller architecture has been to take over many of the mechanisms human racing drivers apply. They learn the track geometry, plan ahead, and wherever necessary, adapt their plans to the current circumstances quickly. Mr. Racer consists of several modules that have partly been adapted and optimized separately, where the final tuning is usually done with respect to a certain racing track during the warmup phase of the TORCS competitions. We also undertake an experimental evaluation that investigates how the controller profits from adding some of the modules to a basic configuration and which modules are most important for reaching the best possible performance.

Keywords

Car racing Evolutionary computation TORCS Planning controller 

References

  1. 1.
    A. Agapitos, M. O’Neill, A. Brabazon, T. Theodoridis, Learning environment models in car racing using stateful genetic programming, in 2011 IEEE Conference on Computational Intelligence and Games (CIG) (2011) pp. 219–226. doi:10.1109/CIG.2011.6032010
  2. 2.
    A. Auger, N. Hansen, A restart CMA evolution strategy with increasing population size, in The 2005 IEEE International Congress on Evolutionary Computation (CEC’05), ed. by B. McKay, et al., vol. 2 (2005) pp. 1769–1776Google Scholar
  3. 3.
    M. Botta, V. Gautieri, D. Loiacono, P.L. Lanzi, Evolving the optimal racing line in a high-end racing game, in 2012 IEEE Conference on Computational Intelligence and Games (CIG) (2012) pp. 108–115. doi:10.1109/CIG.2012.6374145
  4. 4.
    M. Butz, M. Linhardt, T. Lönneker, Effective racing on partially observable tracks: indirectly coupling anticipatory egocentric sensors with motor commands. IEEE Trans. Comput. Intell. AI Games 3(1), 31–42 (2011)CrossRefGoogle Scholar
  5. 5.
    M. Butz, T. Lönneker, Optimized sensory-motor couplings plus strategy extensions for the torcs car racing challenge, in Proceedings of the IEEE Symposium on Computational Intelligence and Games (2009), pp. 317–324Google Scholar
  6. 6.
    L. Cardamone, A. Caiazzo, D. Loiacono, P.L. Lanzi, Transfer of driving behaviors across different racing games, in 2011 IEEE Conference onComputational Intelligence and Games (CIG) (2011), pp. 227–234. DOI:10.1109/CIG.2011.6032011
  7. 7.
    L. Cardamone, D. Loiacono, P.L. Lanzi, Evolving competitive car controllers for racing games with neuroevolution, in Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, GECCO ’09 (2009), pp. 1179–1186. doi:10.1145/1569901.1570060
  8. 8.
    L. Cardamone, D. Loiacono, P.L. Lanzi, Learning drivers for torcs through imitation using supervised methods, in IEEE Symposium on Computational Intelligence and Games, 2009 (CIG 2009) (2009), pp. 148–155. doi:10.1109/CIG.2009.5286480
  9. 9.
    L. Cardamone, D. Loiacono, P.L. Lanzi, On-line neuroevolution applied to the open racing car simulator, in Proceedings of the IEEE Congress on Evolutionary Computation (2009), pp. 2622–2629Google Scholar
  10. 10.
    L. Cardamone, D. Loiacono, P.L. Lanzi, Applying cooperative coevolution to compete in the 2009 torcs endurance world championship, in 2010 IEEE Congress on Evolutionary Computation (CEC) (2010), pp. 1–8. doi:10.1109/CEC.2010.5586041
  11. 11.
    L. Cardamone, D. Loiacono, P.L. Lanzi, Learning to drive in the open racing car simulator using online neuroevolution. IEEE Trans. Comput. Intell. AI Games 2(3), 176–190 (2010)CrossRefGoogle Scholar
  12. 12.
    L. Cardamone, D. Loiacono, P.L. Lanzi, Interactive evolution for the procedural generation of tracks in a high-end racing game, in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GECCO ’11 (ACM, New York, 2011), pp. 395–402. doi:10.1145/2001576.2001631
  13. 13.
    L. Cardamone, D. Loiacono, P.L. Lanzi, A.P. Bardelli, Searching for the optimal racing line using genetic algorithms, in 2010 IEEE Symposium on Computational Intelligence and Games (CIG) (2010), pp. 388–394. doi:10.1109/ITW.2010.5593330
  14. 14.
    M. Ebner, T. Tiede, Evolving driving controllers using genetic programming, in Proceedings of the IEEE Symposium on Computational Intelligence and Games 2009 (2009), pp. 279–286Google Scholar
  15. 15.
    N. Hansen, The CMA Evolution Strategy: A Tutorial. https://www.lri.fr/hansen/cmatutorial110628.pdf (2011). 28 June 2011
  16. 16.
    M. Kemmerling, M. Preuss, Automatic adaptation to generated content via car setup optimization in torcs, in 2010 IEEE Symposium on Computational Intelligence and Games (CIG) (2010), pp. 131–138. doi:10.1109/ITW.2010.5593361
  17. 17.
    D. Loiacono, L. Cardamone, P.L. Lanzi, Automatic track generation for high-end racing games using evolutionary computation. IEEE Trans. Comput. Intell. AI Games 3(3), 245–259 (2011). doi:10.1109/TCIAIG.2011.2163692 CrossRefGoogle Scholar
  18. 18.
    D. Loiacono, L. Cardamone, P.L. Lanzi, Simulated Car Racing Championship: Competition Software Manual (Politecnico di Milano, Dipartimento di Elettronica, Informazione e Bioingegneria, Italy, 2013). http://arxiv.org/abs/1304.1672v2
  19. 19.
    D. Loiacono, P.L. Lanzi, J. Togelius, E. Onieva, D.A. Pelta, M.V. Butz, T.D. Lönneker, L. Cardamone, D. Perez, Y. Saez, M. Preuss, J. Quadflieg, The 2009 simulated car racing championship. IEEE Trans. Comput. Intell. AI in Games 2(2), 131–147 (2010)CrossRefGoogle Scholar
  20. 20.
    D. Loiacono, A. Prete, P.L. Lanzi, L. Cardamone, Learning to overtake in torcs using simple reinforcement learning, in 2010 IEEE Congress on Evolutionary Computation (CEC) (2010), pp. 1–8. doi:10.1109/CEC.2010.5586191
  21. 21.
    D. Loiacono, J. Togelius, P.L. Lanzi, L. Kinnaird-Heether, S.M. Lucas, M. Simmerson, D. Perez, R.G. Reynolds, Y. Saez, The wcci 2008 simulated car racing competition, in Proceedings of the IEEE Symposium on Computational Intelligence and Games (2008), pp. 119–126Google Scholar
  22. 22.
    J. Muñoz, G. Gutierrez, A. Sanchis, Controller for torcs created by imitation, in Proceedings of the IEEE Symposium on Computational Intelligence and Games (2009), pp. 271–278Google Scholar
  23. 23.
    J. Muñoz, G. Gutierrez, A. Sanchis, A human-like torcs controller for the simulated car racing championship, in Proceedings of the IEEE Conference on Computational Intelligence and Games (2010), pp. 473–480Google Scholar
  24. 24.
    E. Onieva, L. Cardamone, D. Loiacono, P.L. Lanzi, Overtaking opponents with blocking strategies using fuzzy logic, in 2010 IEEE Symposium on Computational Intelligence and Games (CIG) (2010), pp. 123–130. doi:10.1109/ITW.2010.5593364
  25. 25.
    E. Onieva, D. Pelta, V. Milanés, J. Pérez, A fuzzy-rule-based driving architecture for non-player characters in a car racing game. Soft Comput. Fusion Found. Methodol. Appl. 15, 1617–1629 (2011). doi:10.1007/s00500-011-0691-6 Google Scholar
  26. 26.
    E. Onieva, D.A. Pelta, J. Alonso, V. Milanés, J. Pérez, A modular parametric architecture for the torcs racing engine, in Proceedings of the IEEE Symposium on Computational Intelligence and Games (2009), pp. 256–262Google Scholar
  27. 27.
    D. Perez, G. Recio, Y. Saez, P. Isasi, Evolving a fuzzy controller for a car racing competition, in Proceedings of the IEEE Symposium on Computational Intelligence and Games (2009), pp. 263–270Google Scholar
  28. 28.
    D. Perez, Y. Saez, G. Recio, P. Isasi, Evolving a rule system controller for automatic driving in a car racing competition, in Proceedings of the IEEE Symposium on Computational Intelligence and Games (2008), pp. 336–342Google Scholar
  29. 29.
    M. Preuss, J. Quadflieg, G. Rudolph, Torcs sensor noise removal and multi-objective track selection for driving style adaptation, in 2011 IEEE Conference on Computational Intelligence and Games (CIG) (2011), pp. 337–344. doi:10.1109/CIG.2011.6032025
  30. 30.
    J. Quadflieg, M. Preuss, O. Kramer, G. Rudolph, Learning the track and planning ahead in a car racing controller, in 2010 IEEE Symposium on Computational Intelligence and Games (CIG) (2010), pp. 395–402. DOI:10.1109/ITW.2010.5593327
  31. 31.
    J. Quadflieg, M. Preuss, G. Rudolph, Driving faster than a human player, in Applications of Evolutionary Computation, Lecture Notes in Computer Science ed. by C. Di Chio, S. Cagnoni, C. Cotta, M. Ebner, A. Ekrt, A. Esparcia-Alczar, J. Merelo, F. Neri, M. Preuss, H. Richter, J. Togelius, G. Yannakakis, vol 6624. (Springer, Berlin, 2011), pp. 143–152. doi:10.1007/978-3-642-20525-5_15
  32. 32.
    Y. Saez, D. Perez, O. Sanjuan, P. Isasi, Driving cars by means of genetic algorithms, in Proceedings of tenth International Conference on Parallel Problem Solving from Nature (2008), pp. 1101–1110Google Scholar
  33. 33.
    K. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002). doi:10.1162/106365602320169811 CrossRefGoogle Scholar
  34. 34.
    T. Uusitalo, S. Johansson, A reactive multi-agent approach to car driving using artificial potential fields, in 2011 IEEE Conference on Computational Intelligence and Games (CIG) (2011), pp. 203–210. doi:10.1109/CIG.2011.6032008
  35. 35.
    B. Wymann, Torcs Robot Tutorial—Optimal Brake Distance, (2013). http://www.berniw.org/tutorials/robot/torcs/robot/ch3/braking3.html. 25 Oct 2013
  36. 36.
    B. Wymann, The Torcs Racing Board, (2014). http://www.berniw.org/trb/index.php. 26 Mar 2014

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Chair of Algorithm Engineering, Computational Intelligence Group, Department of Computer ScienceTechnische Universität DortmundDortmundGermany
  2. 2.European Research Center for Information Systems (ERCIS)WWU MünsterMünsterGermany

Personalised recommendations