A staged approach to evolving real-world UAV controllers

  • Gerard David HowardEmail author
  • Alberto Elfes
Research Paper


A testbed has recently been introduced that evolves controllers for arbitrary hover-capable UAVs, with evaluations occurring directly on the robot. To prepare the testbed for real-world deployment, we investigate the effects of state-space limitations brought about by physical tethering (which prevents damage to the UAV during stochastic tuning), on the generality of the evolved controllers. We identify generalisation issues in some controllers, and propose an improved method that comprises two stages: in the first stage, controllers are evolved as normal using standard tethers, but experiments are terminated when the population displays basic flight competency. Optimisation then continues on a much less restrictive tether, effectively free-flying, and is allowed to explore a larger state-space envelope. We compare the two methods on a hover task using a real UAV, and show that more general solutions are generated in fewer generations using the two-stage approach. A secondary experiment undertakes a sensitivity analysis of the evolved controllers.


Differential evolution Evolutionary robotics Evolutionary hardware UAV control 


Compliance with ethical standards


This research received funding from the CSIRO Office of the Chief Executive for the Postdoctoral position in Evolutionary Aerial Robotics.

Conflicts of interest

The author declares that they have no conflict of interest.


  1. 1.
    Acar E, Zhang Y, Choset H, Schervish M, Costa AG, Melamud R, Lean D, Graveline A (2001) Path planning for robotic demining and development of a test platform. Int Conferen Field Serv Robot 1:161–168Google Scholar
  2. 2.
    Biswas A, Das S, Abraham A, Dasgupta S (2009) Design of fractional-order pi \(\lambda \) d \(\mu \) controllers with an improved differential evolution. Eng Appl Artif Intell 22(2):343–350CrossRefGoogle Scholar
  3. 3.
    Chiha I, Ghabi J, Liouane N (2012) Tuning pid controller with multi-objective differential evolution. In: 2012 5th international symposium on communications control and signal processing (ISCCSP), pp 1–4Google Scholar
  4. 4.
    De Nardi R, Togelius J, Holland O, Lucas S (2006) Evolution of neural networks for helicopter control: why modularity matters. In: IEEE congress on evolutionary computation, 2006—CEC 2006, pp 1799–1806Google Scholar
  5. 5.
    Degrave J, Burm M, Kindermans PJ, Dambre J, wyffels F (2015) Transfer learning of gaits on a quadrupedal robot. Adapt Behav 23(2):69–82CrossRefGoogle Scholar
  6. 6.
    Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, BerlinCrossRefzbMATHGoogle Scholar
  7. 7.
    Faina AJ, Toft L, Risi S (2017) Automating the incremental evolution of controllers for physical robots. Artif Life 23(2):142–168CrossRefGoogle Scholar
  8. 8.
    Floreano D, Zufferey JC, Nicoud JD (2005) From wheels to wings with evolutionary spiking circuits. Artif life 11(1-2):121–138CrossRefGoogle Scholar
  9. 9.
    Ghiglino P, Forshaw JL, Lappas VJ (2015) Online evolutionary swarm algorithm for self-tuning unmanned flight control laws. J. Guid. Control Dyn. 38(4):772–782CrossRefGoogle Scholar
  10. 10.
    Gongora M, Passow B, Hopgood A (2009) Robustness analysis of evolutionary controller tuning using real systems. In: IEEE congress on evolutionary computation, 2009. CEC ’09, pp 606–613Google Scholar
  11. 11.
    Harvey I, Husbands P, Cliff D (1994) Seeing the light: artificial evolution, real vision. School of Cognitive and Computing Sciences, University of Sussex FalmerGoogle Scholar
  12. 12.
    Heijnen H, Howard D, Kottege N (2017) A testbed that evolved hexapod controllers in hardware. In: 2017 IEEE/RSJ international conference on robotics and automation (ICRA), IEEE (in press)Google Scholar
  13. 13.
    Holland OE, Nardi RD (2008) Coevolutionary modelling of a miniature rotorcraft. In: Burgard W, Dillmann R, Plagemann C, Vahrenkamp N (eds) Intelligent autonomous systems 10 (IAS10). Univ Freiburg, Autonomous Intelligent Syst Lab, Baden Baden, Germany (loc)Google Scholar
  14. 14.
    How JP, BEHIHKE B, Frank A, Dale D, Vian J (2008) Real-time indoor autonomous vehicle test environment. IEEE Control Syst 28(2):51–64MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Howard D (2017) A platform that directly evolves multirotor controllers. IEEE Trans Evolut Comput 21(6):943–955CrossRefGoogle Scholar
  16. 16.
    Howard D, Elfes A (2014) Evolving spiking networks for turbulence-tolerant quadrotor control. In: International conference on artificial life (ALIFE14), pp 431–438Google Scholar
  17. 17.
    Howard D, Merz T (2015) A platform for the direct hardware evolution of quadcopter controllers. In: 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 4614–4619Google Scholar
  18. 18.
    Howard GD (2017) On self-adaptive mutation restarts for evolutionary robotics with real rotorcraft. In: Proceedings of the 17th annual conference on genetic and evolutionary computation. ACM (in press)Google Scholar
  19. 19.
    Jakobi N, Husbands P, Harvey I (1995) Noise and the reality gap: The use of simulation in evolutionary robotics. In: Morán F, Moreno A, Merelo JJ, Chacón P (eds) Advances in artificial life. ECAL 1995. Lecture notes in computer science (Lecture notes in artificial intelligence), vol 929. Springer, Berlin, HeidelbergGoogle Scholar
  20. 20.
    Johnson W (2012) Helicopter theory. Courier Corporation, ChelmsfordGoogle Scholar
  21. 21.
    Koos S, Mouret JB, Doncieux S (2010) Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In: Proceedings of the 12th annual conference on genetic and evolutionary computation, GECCO ’10. ACM, New York, pp 119–126Google Scholar
  22. 22.
    Koppejan R, Whiteson S (2009) Neuroevolutionary reinforcement learning for generalized helicopter control. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, GECCO ’09. ACM, New York, NY, pp 145–152Google Scholar
  23. 23.
    Merz T, Rudol P, Wzorek M (2006) Control system framework for autonomous robots based on extended state machines. In: 2006 International conference on autonomic and autonomous systems, 2006. ICAS’06. IEEE, pp 14–14Google Scholar
  24. 24.
    Moravec J, Pošík P (2014) A comparative study: the effect of the perturbation vector type in the differential evolution algorithm on the accuracy of robot pose and heading estimation. Evolut. Intell. 6(3):171–191. CrossRefGoogle Scholar
  25. 25.
    Nishiwaki K, Sugihara T, Kagami S, Kanehiro F, Inaba M, Inoue H (2000) Design and development of research platform for perception-action integration in humanoid robot: H6. In: Proceedings of 2000 IEEE/RSJ international conference on intelligent robots and systems 2000, (IROS 2000). IEEE, vol 3, pp 1559–1564Google Scholar
  26. 26.
    Powers C, Mellinger D, Kushleyev A, Kothmann B, Kumar V (2013) Influence of aerodynamics and proximity effects in quadrotor flight. In: Desai J, Dudek G, Khatib O, Kumar V (eds) Experimental robotics. Springer tracts in advanced robotics, vol 88. Springer, HeidelbergGoogle Scholar
  27. 27.
    Rechenberg I (1973) Evolutionsstrategie: optimierung technischer systeme nach prinzipien der biologischen evolution. Frommann-Holzboog, StuttgartGoogle Scholar
  28. 28.
    Rossi C, Eiben AE (2014) Simultaneous versus incremental learning of multiple skills by modular robots. Evolut. Intell. 7(2):119–131. CrossRefGoogle Scholar
  29. 29.
    Samuele R, Varshneya R, Johnson T, Johnson A, Glassman T (2010) Progress at the starshade testbed at northrop grumman aerospace systems: comparisons with computer simulations. In: Proceedings of SPIE, vol 7731, p 773151Google Scholar
  30. 30.
    Scheper KYW, Tijmons S, de Visser CC, de Croon GCHE (2016) Behavior trees for evolutionary robotics. Artif Life 22(1):23–48CrossRefGoogle Scholar
  31. 31.
    Storn R, Price K (1997) Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgezbMATHGoogle Scholar
  33. 33.
    Yosinski J, Clune J, Hidalgo D, Nguyen S, Zagal J, Lipson H (2011) Evolving robot gaits in hardware: the hyperneat generative encoding vs. parameter optimization. In: Proceedings of the 20th European conference on artificial life, pp 890–897Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.QCATPullenvaleAustralia

Personalised recommendations