Advertisement

Autonomous Robots

, Volume 33, Issue 1–2, pp 21–39 | Cite as

PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision

  • Lorenz MeierEmail author
  • Petri Tanskanen
  • Lionel Heng
  • Gim Hee Lee
  • Friedrich Fraundorfer
  • Marc Pollefeys
Article

Abstract

We describe a novel quadrotor Micro Air Vehicle (MAV) system that is designed to use computer vision algorithms within the flight control loop. The main contribution is a MAV system that is able to run both the vision-based flight control and stereo-vision-based obstacle detection parallelly on an embedded computer onboard the MAV. The system design features the integration of a powerful onboard computer and the synchronization of IMU-Vision measurements by hardware timestamping which allows tight integration of IMU measurements into the computer vision pipeline. We evaluate the accuracy of marker-based visual pose estimation for flight control and demonstrate marker-based autonomous flight including obstacle detection using stereo vision. We also show the benefits of our IMU-Vision synchronization for egomotion estimation in additional experiments where we use the synchronized measurements for pose estimation using the 2pt+gravity formulation of the PnP problem.

Keywords

Micro aerial vehicles Quadrotor Computer vision Stereo vision 

Notes

Acknowledgements

We would like to thank our students (in alphabetical order) Bastian Bücheler, Andi Cortinovis, Christian Dobler, Dominik Honegger, Fabian Landau, Laurens Mackay, Tobias Nägeli, Philippe Petit, Martin Rutschmann, Amirehsan Sarabadani, Christian Schluchter and Oliver Scheuss for their contributions to the current system and the students of the previous semesters for the foundations they provided. Raffaello d’Andrea and Sergei Lupashin (ETH IDSC) provided valuable feedback.

This work was supported in part by the European Community’s Seventh Framework Programme (FP7/2007-2013) under grant #231855 (sFly) and by the Swiss National Science Foundation (SNF) under grant # 200021-125017.

Supplementary material

<Pixhawk_Alpha_Autonomous.avi: This video shows one of the first autonomous flights using computer vision localization with ARToolkit+ markers. It shows the general flight behaviour of our vision based system. The MAV is not thetered, images are fully processed online. The MAV flies several rounds, defined by pre-set waypoints.> (AVI 11.3 MB)

References

  1. Achtelik, M., Achtelik, M., Weiss, S., & Siegwart, R. (2011). Onboard IMU and monocular vision based control for MAVs in unknown in- and outdoor environments. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 3056–3063). CrossRefGoogle Scholar
  2. Bachrach, A., de Winter, A., He, R., Hemann, G., Prentice, S., & Roy, N. (2010). Range—robust autonomous navigation in GPS-denied environments. In Robotics and automation (ICRA), 2010 IEEE international conference on (pp. 1096–1097). doi: 10.1109/ROBOT.2010.5509990. CrossRefGoogle Scholar
  3. Bills, C., Chen, J., & Saxena, A. (2011). Autonomous MAV flight in indoor environments using single image perspective cues. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 5776–5783). CrossRefGoogle Scholar
  4. Blösch, M., Weiss, S., Scaramuzza, D., & Siegwart, R. (2010). Vision based MAV navigation in unknown and unstructured environments. In Robotics and automation (ICRA), 2010 IEEE international conference on (pp. 21–28). doi: 10.1109/ROBOT.2010.5509920. CrossRefGoogle Scholar
  5. Bosch, S., Lacroix, S., & Caballero, F. (2006). Autonomous detection of safe landing areas for an UAV from monocular images. In Intelligent robots and systems, 2006 IEEE/RSJ international conference on (pp. 5522–5527). doi: 10.1109/IROS.2006.282188. CrossRefGoogle Scholar
  6. Bouabdallah, S., Murrieri, P., & Siegwart, R. (2004). Design and control of an indoor micro quadrotor. In Proceedings of int. conf. on robotics and automation. URL http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.89.842. Google Scholar
  7. Bouabdallah, S., & Siegwart, R. (2007). Full control of a quadrotor. In Intelligent robots and systems, 2007. IROS 2007. IEEE/RSJ international conference on (pp. 153–158). doi: 10.1109/IROS.2007.4399042. CrossRefGoogle Scholar
  8. Conte, G., & Doherty, P. (2008). An integrated UAV navigation system based on aerial image matching. In Proceedings of the IEEE aerospace conference. URL http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.119.3963&rep=rep1&type=pdf. Google Scholar
  9. Dryanovski, I., Morris, W., & Xiao, J. (2011). An open-source pose estimation system for micro-air vehicles. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 4449–4454). CrossRefGoogle Scholar
  10. Ducard, G., & D’Andrea, R. (2009). Autonomous quadrotor flight using a vision system and accommodating frames misalignment. In Industrial embedded systems, 2009. SIES ’09. IEEE international symposium on (pp. 261–264). doi: 10.1109/SIES.2009.5196224. CrossRefGoogle Scholar
  11. Eberli, D., Scaramuzza, D., Weiss, S., & Siegwart, R. (2011). Vision based position control for MAVs using one single circular landmark. Journal of Intelligent and Robotic Systems, 61(1–4), 495–512. CrossRefGoogle Scholar
  12. Fowers, S., Lee, D. J., Tippetts, B., Lillywhite, K., Dennis, A., & Archibald, J. (2007). Vision aided stabilization and the development of a quad-rotor micro UAV. In International symposium on computational intelligence in robotics and automation, 2007. CIRA 2007 (pp. 143–148). doi: 10.1109/CIRA.2007.382886 CrossRefGoogle Scholar
  13. Heng, L., Meier, L., Tanskanen, P., Fraundorfer, F., & Pollefeys, M. (2011). Autonomous obstacle avoidance and maneuvering on a vision-guided MAV using on-board processing. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 2472–2477). CrossRefGoogle Scholar
  14. Hofiann, G., Rajnarqan, D., & Waslander, S. (2004). The Stanford testbed of autonomous rotorcraft for multi agent control (starmac). In Proceedings of digital avionics systems conference (DASC04). URL http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1390847. Google Scholar
  15. Hrabar, S., & Sukhatme, G. (2009). Vision-based navigation through urban canyons. Journal of Field Robotics, 26(5), 431–452. doi: 10.1002/rob.20284. CrossRefGoogle Scholar
  16. Huang, A., Olson, E., & Moore, D. (2010). LCM: Lightweight Communications and Marshalling. In Intelligent robots and systems (IROS), 2010 IEEE/RSJ international conference on (pp. 4057–4062). CrossRefGoogle Scholar
  17. Johnson, A., Montgomery, J., & Matthies, L. (2005). Vision guided landing of an autonomous helicopter in hazardous terrain. In Robotics and automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE international conference on (pp. 3966–3971). doi: 10.1109/ROBOT.2005.1570727. CrossRefGoogle Scholar
  18. Kanade, T., Amidi, O., & Ke, Q. (2004). Real-time and 3d vision for autonomous small and micro air vehicles. In Decision and control, 2004. CDC. 43rd IEEE conference on (vol. 2, pp. 1655–1662). doi: 10.1109/CDC.2004.1430282. Google Scholar
  19. Kemp, C. (2006). Visual control of a miniature quad-rotor helicopter. Ph.D. thesis, Churchill College, University of Cambridge. Google Scholar
  20. Kukelova, Z., Bujnak, M., & Pajdla, T. (2010). Closed-form solutions to the minimal absolute pose problems with known vertical direction. In ACCV. Google Scholar
  21. Kukelova, Z., Bujnak, M., & Pajdla, T. (2011). Closed-form solutions to minimal absolute pose problems with known vertical direction. In Computer vision–ACCV 2010. URL http://www.springerlink.com/index/M012M78244081306.pdf. Google Scholar
  22. Lobo, J., & Dias, J. (2007). Relative pose calibration between visual and inertial sensors. International Journal of Robotics Research, 26(6), 561–575. CrossRefGoogle Scholar
  23. Meier, L., Tanskanen, P., Fraundorfer, F., & Pollefeys, M. (2011). Pixhawk: A system for autonomous flight using onboard computer vision. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 2992–2997). CrossRefGoogle Scholar
  24. Mellinger, D., & Kumar, V. (2011). Minimum snap trajectory generation and control for quadrotors. In Proceedings of the IEEE international conference on robotics and automation (ICRA). Google Scholar
  25. Mellinger, D., Shomin, M., Michael, N., & Kumar, V. (2010). Cooperative grasping and transport using multiple quadrotors. In Proceedings of the international symposium on distributed autonomous robotic systems. Google Scholar
  26. Montemerlo, M., Roy, N., & Thrun, S. (2003). Perspectives on standardization in mobile robot programming: the Carnegie Mellon navigation (Carmen) toolkit. In Intelligent robots and systems, 2003. (IROS 2003). Proceedings. 2003 IEEE/RSJ international conference on (vol. 3, pp. 2436–2441). doi: 10.1109/IROS.2003.1249235. CrossRefGoogle Scholar
  27. Proctor, A. A., Johnson, E. N., & Apker, T. B. (2006). Vision-only control and guidance for aircraft. Journal of Field Robotics, 23(10), 863–890. doi: 10.1002/rob.20155. zbMATHCrossRefGoogle Scholar
  28. Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., & Ng, A. (2009). Ros: An open-source robot operating system. Google Scholar
  29. Roy, N., He, R., Bachrach, A., & Achtelik, M. (2009). On the design and use of a micro air vehicle to track and avoid adversaries. International Journal of Robotics Research. URL http://ijr.sagepub.com/cgi/content/abstract/29/5/529.
  30. Saripalli, S., Montgomery, J., & Sukhatme, G. (2002). Vision-based autonomous landing of an unmanned aerial vehicle. In Robotics and automation, 2002. Proceedings. ICRA ’02. IEEE international conference on (vol. 3, pp. 2799–2804). doi: 10.1109/ROBOT.2002.1013656. Google Scholar
  31. Scherer, S., Singh, S., Chamberlain, L., & Elgersma, M. (2008). Flying fast and low among obstacles: Methodology and experiments. The International Journal of Robotics Research, 27(5), 549–574. doi: 10.1177/0278364908090949. CrossRefGoogle Scholar
  32. Shen, S., Michael, N., & Kumar, V. (2011). Autonomous multi-floor indoor navigation with a computationally constrained MAV. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 20–25). CrossRefGoogle Scholar
  33. Volpe, R., Nesnas, I., Estlin, T., Mutz, D., Petras, R., & Das, H. (2001). The CLARAty architecture for robotic autonomy. In Aerospace conference, 2001, IEEE proceedings (vol. 1, pp. 1/121–1/132). doi: 10.1109/AERO.2001.931701. CrossRefGoogle Scholar
  34. Wagner, D., & Schmalstieg, D. (2007). Artoolkitplus for pose tracking on mobile devices. In Proceedings of 12th computer vision winter workshop. URL http://www.icg.tu-graz.ac.at/Members/daniel/ARToolKitPlusMobilePoseTracking. Google Scholar
  35. Li, W., Zhang, T., & Klihnlenz, K. (2011). A vision-guided autonomous quadrotor in an air-ground multi-robot system. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 2980–2985). CrossRefGoogle Scholar
  36. Wenzel, K., Masselli, A., & Zell, A. (2011). Automatic take off, tracking and landing of a miniature UAV on a moving carrier vehicle. Journal of Intelligent Robotic Systems, 61, 221–238. doi: 10.1007/s10846-010-9473-0. CrossRefGoogle Scholar
  37. Williams, B., Hudson, N., Tweddle, B., Brockers, R., & Matthies, L. (2011). Feature and pose constrained visual aided inertial navigation for computationally constrained aerial vehicles. In Robotics and automation (ICRA), 2011 IEEE international conference on (pp. 431–438). CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Lorenz Meier
    • 1
    Email author
  • Petri Tanskanen
    • 1
  • Lionel Heng
    • 1
  • Gim Hee Lee
    • 1
  • Friedrich Fraundorfer
    • 1
  • Marc Pollefeys
    • 1
  1. 1.ETH ZurichZurichSwitzerland

Personalised recommendations