A staged approach to evolving real-world UAV controllers

Abstract

A testbed has recently been introduced that evolves controllers for arbitrary hover-capable UAVs, with evaluations occurring directly on the robot. To prepare the testbed for real-world deployment, we investigate the effects of state-space limitations brought about by physical tethering (which prevents damage to the UAV during stochastic tuning), on the generality of the evolved controllers. We identify generalisation issues in some controllers, and propose an improved method that comprises two stages: in the first stage, controllers are evolved as normal using standard tethers, but experiments are terminated when the population displays basic flight competency. Optimisation then continues on a much less restrictive tether, effectively free-flying, and is allowed to explore a larger state-space envelope. We compare the two methods on a hover task using a real UAV, and show that more general solutions are generated in fewer generations using the two-stage approach. A secondary experiment undertakes a sensitivity analysis of the evolved controllers.

Appendix A: Fitness function

$$\begin{aligned}& f_{\mathrm{cycle}}=f_{\mathrm{p}}+f_{\mathrm{h}}+f_\psi + f_{\mathrm{a}}+f_{\mathrm{vh}}+f_{\mathrm{vv}}+f_{{\omega }}+f_{\mathrm{l}} \\&f_{\mathrm{p}}=f\left( \sqrt{(p_{\mathrm{nsp}}-p_{\mathrm{n}})^2+(p_{\mathrm{esp}}-p_{\mathrm{e}})^2},l_{\mathrm{p}},l_{\mathrm{pc}}\right) \\&f_{\mathrm{h}}=f(|h_{\mathrm{sp}}-h|,l_{\mathrm{h}},l_{\mathrm{hc}})\\&f_{{\psi }}=f\left( |{\mathrm{wrap}}(\psi _{\mathrm{sp}}-\psi )|,l_\psi ,l_{\psi {\mathrm{c}}}\right) \\ &f_{\mathrm{a}}=f(|\phi _{\mathrm{sp}}-\phi |,l_{\mathrm{a}},l_{\mathrm{ac}})+f(|\theta _{\mathrm{sp}}-\theta |,l_{\mathrm{a}},l_{\mathrm{ac}})\\&f_{\mathrm{vh}}=f\left( \sqrt{v_{\mathrm{n}}^2+v_{\mathrm{e}}^2},l_{\mathrm{vh}},l_{\mathrm{vhc}}\right) \\&f_{\mathrm{vv}}=f(|v_{\mathrm{v}}|,l_{\mathrm{vv}},l_{\mathrm{vvc}})\\&f_{{\omega }}=f(|p|,l_\omega ,l_{\omega {\mathrm{c}}})+f(|q|,l_\omega ,l_{\omega {\mathrm{c}}})\\&f_{\mathrm{l}}=1-\frac{{\text{number of controllers exceeding }}l_\delta \text{ limits}}{4} \\&f(e,l,l_{\mathrm{c}})= {\left\{ \begin{array}{ll} {\mathrm{max}}\{\frac{l-e}{4(l-l_{\mathrm{c}})},0\}, & \text{if } e>l_{\mathrm{c}} \\ \frac{3(l_{\mathrm{c}}-e)}{4 l_{\mathrm{c}}}+\frac{1}{4}, & \text{otherwise} \end{array}\right. } \\&{\mathrm{wrap}}(\gamma )={\mathrm{atan2}}(\sin (\gamma ),\cos (\gamma )) \end{aligned}$$

1.1 Symbol definitions: fitness function

$$\begin{aligned}& f_{\mathrm{cycle:}} \text{fitness for one control cycle}\\&f_{\mathrm{p}}/f_{\mathrm{h}}: \text{fitness for position/height tracking}\\&f_\psi : \text{fitness for yaw angle tracking}\\&f_{\mathrm{a}}: \text{fitness for pitch and roll angle tracking}\\&f_{\mathrm{vh}}/f_{\mathrm{vv}}: \text{fitness for low horizontal/vertical velocity}\\&f_{{\omega }}: \text{fitness for low pitch and roll rates} \\&f_{\mathrm{l}}: \text{fitness for staying within controller limits}\\&p_{\mathrm{n}}/p_{\mathrm{e}}: \text{north/east position}\\&p_{\mathrm{nsp}}/p_{\mathrm{esp}}: \text{north/east position setpoint}\\&h: \text{height above ground}\\&h_{\mathrm{sp}}: \text{height setpoint}\\&\phi /\theta /\psi : \text{roll/pitch/yaw angle}\\&\phi _{\mathrm{sp}}/\theta _{\mathrm{sp}}/\psi _{\mathrm{sp}}: \text{roll/pitch/yaw setpoint}\\&v_{\mathrm{n}}/v_{\mathrm{e}}/v_{\mathrm{v}}: \text{north/east/vertical velocity}\\&p/q/r: \text{roll/pitch/yaw rate}\\&l_{\mathrm{p}}: \text{position error range (0.2 m)}\\&l_{\mathrm{pc}}: \text{core position error range (0.07 m)}\\&l_{\mathrm{h}}: \text{height error range (0.2 m)}\\&l_{\mathrm{hc}}: \text{core height error range (0.03 m)}\\&l_{\psi }: \text{yaw error range}\, (30^{\circ})\\&l_{\psi c}: \text{core yaw error range}\, (10^{\circ})\\&l_{\mathrm{a}}: \text{attitude angle error range}\, (15^{\circ})\\&l_{\mathrm{ac}}: \text{core attitude angle error range}\, (3^{\circ})\\&l_{\mathrm{vh}}: \text{horizontal velocity error range (1 m/s)}\\&l_{\mathrm{vhc}}: \text{core horizontal velocity error range (0.2 m/s)}\\&l_{\mathrm{vv}}: \text{vertical velocity error range (1 m/s)}\\&l_{\mathrm{vvc}}: \text{core vertical velocity error range (0.1 m/s)}\\&l_{{\omega }}: \text{pitch and roll rate error range}\, (250^{\circ}/\text{s})\\&l_{\omega \mathrm{c}}: \text{core pitch and roll rate error range}\, (30^{\circ}/\text{s)}\\ \end{aligned}$$