Learning motion primitives for planning swift maneuvers of quadrotor

  • Efe Camci
  • Erdal KayacanEmail author


This work proposes a novel, learning-based method to leverage navigation time performance of unmanned aerial vehicles in dense environments by planning swift maneuvers using motion primitives. In the proposed planning framework, desirable motion primitives are explored by reinforcement learning. Two-stage training composed of learning in simulations and real flights is conducted to build up a swift motion primitive library. The library is then referred in real-time and the primitives are utilized by an intelligent control authority switch mechanism when swift maneuvers are needed for particular portions of a trajectory. Since the library is constructed upon realistic Gazebo simulations and real flights together, probable modeling uncertainties which can degrade planning performance are minimal. Moreover, since the library is in the form of motion primitives, it is computationally inexpensive to be retained and used for planning as compared to solving optimal motion planning problem algebraically. Overall, the proposed method allows for exceptional, swift maneuvers and enhances navigation time performance in dense environments up to 20% as being demonstrated by real flights with Diatone FPV250 quadrotor equipped with PX4 FMU.


Reinforcement learning Path planning Quadrotor Agile maneuvers 



This work is financially supported by the Singapore Ministry of Education (RG185/17).

Supplementary material

Supplementary material 1 (mp4 5631 KB)


  1. Abbeel, P., Coates, A., & Ng, A. Y. (2010). Autonomous helicopter aerobatics through apprenticeship learning. The International Journal of Robotics Research, 29(13), 1608–1639.Google Scholar
  2. Bareiss, D., Bourne, J. R., & Leang, K. K. (2017). On-board model-based automatic collision avoidance: Application in remotely-piloted unmanned aerial vehicles. Autonomous Robots, 41(7), 1539–1554.Google Scholar
  3. Camci, E., & Kayacan, E. (2016). Waitress quadcopter explores how to serve drinks by reinforcement learning. In 2016 IEEE region 10 conference (TENCON) (pp. 28–32). IEEE.Google Scholar
  4. De Casteljau, P. (1959). Outillages méthodes calcul. Paris: Andr e Citro en Automobiles SA.Google Scholar
  5. Degiovanni, L., & Bernard, P. (2016). Tp drone diatone 250 fpv. Accessed May 2017.
  6. Deits, R., & Tedrake, R. (2015). Efficient mixed-integer planning for UAVs in cluttered environments. In 2015 IEEE international conference on robotics and automation (ICRA) (pp. 42–49). IEEE.Google Scholar
  7. Dong, Y., Camci, E., & Kayacan, E. (2018). Faster RRT-based nonholonomic path planning in 2D building environments using skeleton-constrained path biasing. Journal of Intelligent & Robotic Systems, 89(3–4), 387–401.Google Scholar
  8. Fuller, S. B., Teoh, Z. E., Chirarattananon, P., Pérez-Arancibia, N. O., Greenberg, J., & Wood, R. J. (2017). Stabilizing air dampers for hovering aerial robotics: Design, insect-scale flight tests, and scaling. Autonomous Robots, 41(8), 1555–1573.Google Scholar
  9. Gillula, J. H., Huang, H., Vitus, M. P., & Tomlin, C. J. (2010). Design of guaranteed safe maneuvers using reachable sets: Autonomous quadrotor aerobatics in theory and practice. In 2010 IEEE international conference on robotics and automation (ICRA) (pp. 1649–1654). IEEE.Google Scholar
  10. Hehn, M., & D’Andrea, R. (2011). Quadrocopter trajectory generation and control. IFAC Proceedings Volumes, 44(1), 1485–1491.Google Scholar
  11. Hehn, M., & D’Andrea, R. (2015). Real-time trajectory generation for quadrocopters. IEEE Transactions on Robotics, 31(4), 877–892.Google Scholar
  12. Hwangbo, J., Sa, I., Siegwart, R., & Hutter, M. (2017). Control of a quadrotor with reinforcement learning. IEEE Robotics and Automation Letters, 2(4), 2096–2103.Google Scholar
  13. Korrapati, H., & Mezouar, Y. (2017). Multi-resolution map building and loop closure with omnidirectional images. Autonomous Robots, 41(4), 967–987.Google Scholar
  14. Landry, B., Deits, R., Florence, P. R., & Tedrake, R. (2016). Aggressive quadrotor flight through cluttered environments using mixed integer programming. In 2016 IEEE international conference on robotics and automation (ICRA) (pp. 1469–1475). IEEE.Google Scholar
  15. LaValle, S. M., & Kuffner, J. J. (2001). Randomized kinodynamic planning. The International Journal of Robotics Research, 20(5), 378–400.Google Scholar
  16. Lee, T., Leoky, M., & McClamroch, N. H. (2010). Geometric tracking control of a quadrotor UAV on se (3). In 2010 49th IEEE conference on decision and control (CDC) (pp. 5420–5425). IEEE.Google Scholar
  17. Ling, Y., Kuse, M., & Shen, S. (2018). Edge alignment-based visual-inertial fusion for tracking of aggressive motions. Autonomous Robots, 42(3), 513–528.Google Scholar
  18. Loianno, G., Brunner, C., McGrath, G., & Kumar, V. (2017). Estimation, control, and planning for aggressive flight with a small quadrotor with a single camera and imu. IEEE Robotics and Automation Letters, 2(2), 404–411.Google Scholar
  19. Lupashin, S., & D’Andrea, R. (2012). Adaptive fast open-loop maneuvers for quadrocopters. Autonomous Robots, 33(1–2), 89–102.Google Scholar
  20. Lupashin, S., Schöllig, A., Sherback, M., & D’Andrea, R. (2010). A simple learning strategy for high-speed quadrocopter multi-flips. In 2010 IEEE international conference on robotics and automation (ICRA) (pp. 1642–1648). IEEE.Google Scholar
  21. Mebarki, R., Lippiello, V., & Siciliano, B. (2017). Vision-based and imu-aided scale factor-free linear velocity estimator. Autonomous Robots, 41(4), 903–917.Google Scholar
  22. Mehndiratta, M., & Kayacan, E. (2017). Receding horizon control of a 3 DOF helicopter using online estimation of aerodynamic parameters. In Proceedings of the institution of mechanical engineers, part G: Journal of aerospace engineering (p. 0954410017703414).Google Scholar
  23. Meier, L., Tanskanen, P., Heng, L., Lee, G. H., Fraundorfer, F., & Pollefeys, M. (2012). Pixhawk: A micro aerial vehicle design for autonomous flight using onboard computer vision. Autonomous Robots, 33(1–2), 21–39.Google Scholar
  24. Mellinger, D., & Kumar, V. (2011). Minimum snap trajectory generation and control for quadrotors. In 2011 IEEE international conference on robotics and automation (ICRA) (pp. 2520–2525). IEEE.Google Scholar
  25. Mellinger, D., Michael, N., & Kumar, V. (2012). Trajectory generation and control for precise aggressive maneuvers with quadrotors. The International Journal of Robotics Research, 31(5), 664–674.Google Scholar
  26. Mueller, M. W., Hehn, M., & D’Andrea, R. (2015). A computationally efficient motion primitive for quadrocopter trajectory generation. IEEE Transactions on Robotics, 31(6), 1294–1310.Google Scholar
  27. Müller, M., Lupashin, S., & D’Andrea, R. (2011). Quadrocopter ball juggling. In 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 5113–5120). IEEE.Google Scholar
  28. Neunert, M., de Crousaz, C., Furrer, F., Kamel, M., Farshidian, F., Siegwart, R., Buchli, J. (2016). Fast nonlinear model predictive control for unified trajectory optimization and tracking. In 2016 IEEE international conference on robotics and automation (ICRA) (pp. 1398–1404). IEEE.Google Scholar
  29. Oosedo, A., Abiko, S., Konno, A., & Uchiyama, M. (2017). Optimal transition from hovering to level-flight of a quadrotor tail-sitter UAV. Autonomous Robots, 41(5), 1143–1159.Google Scholar
  30. Pfeiffer, M., Schaeuble, M., Nieto, J., Siegwart, R., & Cadena, C. (2016). From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots. arXiv preprint arXiv:1609.07910.
  31. Santamaria-Navarro, A., Loianno, G., Solà, J., Kumar, V., & Andrade-Cetto, J. (2018). Autonomous navigation of micro aerial vehicles using high-rate and low-cost sensors. Autonomous Robots, 42(6), 1263–1280.Google Scholar
  32. Sarabakha, A., Fu, C., Kayacan, E., & Kumbasar, T. (2018). Type-2 fuzzy logic controllers made even simpler: From design to deployment for UAVs. IEEE Transactions on Industrial Electronics, 65(6), 5069–5077.Google Scholar
  33. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1). Cambridge: MIT Press.Google Scholar
  34. Tang, J., Singh, A., Goehausen, N., & Abbeel, P. (2010). Parameterized maneuver learning for autonomous helicopter flight. In 2010 IEEE international conference on robotics and automation (ICRA) (pp. 1142–1148). IEEE.Google Scholar
  35. Ure, N. K., & Inalhan, G. (2012). Autonomous control of unmanned combat air vehicles: Design of a multimodal control and flight planning framework for agile maneuvering. IEEE Control Systems, 32(5), 74–95.MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Mechanical and Aerospace EngineeringNanyang Technological UniversitySingaporeSingapore
  2. 2.Department of EngineeringAarhus UniversityAarhus CDenmark

Personalised recommendations