Learning Autonomous Helicopter Flight with Evolutionary Reinforcement Learning

  • José Antonio Martín H.
  • Javier de Lope
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5717)


In this paper we present a method to obtain a near optimal neuro-controller for the autonomous helicopter flight by means of an ad hoc evolutionary reinforcement learning method. The method presented here was developed for the Second Annual Reinforcement Learning Competition (RL2008) held in Helsinki-Finland. The present work uses a Helicopter Hovering simulator created in the Stanford University that simulates a Radio Control XCell Tempest helicopter in the flight regime close to hover. The objective of the controller is to hover the helicopter by manipulating four continuous control actions based on a 12-dimensional state space.


Reinforcement Learning Evolutionary Computation Autonomous Helicopter 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    de Lope Asiaín, J., Martín, J.J.S., José Antonio Martin, H.: Helicopter flight dynamics using soft computing models. In: Moreno Díaz, R., Pichler, F., Quesada-Arencibia, A. (eds.) EUROCAST 2007. LNCS, vol. 4739, pp. 621–628. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Moriarty, D.E., Schultz, A.C., Grefenstette, J.J.: Reinforcement learning through evolutionary computation (1999)Google Scholar
  3. 3.
    Koppejan, R., Whiteson, S.: Neuroevolutionary reinforcement learning for generalized helicopter control. In: GECCO 2009: Proceedings of the Genetic and Evolutionary Computation Conference (to appear, July 2009)Google Scholar
  4. 4.
    Ng, A.Y., Kim, H.J., Jordan, M.I., Sastry, S.: Autonomous helicopter flight via reinforcement learning. In: Thrun, S., Saul, L.K., Schölkopf, B. (eds.) NIPS. MIT Press, Cambridge (2003)Google Scholar
  5. 5.
    Jose Antonio Martin, H., de Lope, J., Santos, M.: Evolution of neuro-controllers for multi-link robots. In: Innovations in Hybrid Intelligent Systems. Advances in Soft Computing, vol. 44, pp. 175–182 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • José Antonio Martín H.
    • 1
  • Javier de Lope
    • 2
  1. 1.Dep. Sistemas Informáticos y ComputaciónUniversidad Complutense de MadridSpain
  2. 2.Dept. Applied Intelligent SystemsUniversidad Politécnica de MadridSpain

Personalised recommendations