Skip to main content
SpringerLink
Log in
Book cover

Autonomous Flying Robots pp 267–302Cite as

Vision-Based Navigation and Visual Servoing of Mini Flying Machines

  • Chapter

  • 4978 Accesses

Abstract

The design of reliable navigation and control systems for Unmanned Aerial Vehicles (UAVs) based only on visual cues and inertial data has many unsolved challenging problems, ranging from hardware and software development to pure control-theoretical issues. This chapter addresses these issues by developing and implementing an adaptive vision-based autopilot for navigation and control of small and mini rotorcraft UAVs. The proposed autopilot includes a Visual Odometer (VO) for navigation in GPS-denied environments and a nonlinear control system for flight control and target tracking. The VO estimates the rotorcraft ego-motion by identifying and tracking visual features in the environment, using a single camera mounted on-board the vehicle. The VO has been augmented by an adaptive mechanism that fuses optic flow and inertial measurements to determine the range and to recover the 3D position and velocity of the vehicle. The adaptive VO pose estimates are then exploited by a nonlinear hierarchical controller for achieving various navigational tasks including take-off, landing, hovering, trajectory tracking, target tracking, etc. Furthermore, the asymptotic stability of the entire closed-loop system has been established using systems in cascade and adaptive control theories. Experimental flight test data over various ranges of the flight envelope illustrate that the proposed vision-based autopilot performs well and allows a mini rotorcraft UAV to achieve autonomously advanced flight behaviours by using vision.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Pseudo-motion means here, motion in the image frame which is expressed in pixels.

  2. 2.

    In the presence of external disturbances, \(\tilde{{b}}_{i} = b -\hat{ {b}}_{i}\) converges exponentially to the residual set: \(\{\tilde{{b}}_{i}/\vert \tilde{{b}}_{i}\vert \leq c({\rho }_{0} +\bar{ d})\}\) , where \(\bar{d}\) is the disturbance upper bound, c is a positive constant and ρ 0 characterizes the dead-zone.

  3. 3.

    It is easy to implement and guarantees good flight performance.

  4. 4.

    It considers system nonlinearities/coupling and guarantees the stability of the closed-loop system.

  5. 5.

    In fact, height is reduced to 3 m to avoid damaging the platform in case where the MAV crashes because of empty battery (there was no charged battery for this test).

References

  1. Ahrens S, Levine D, Andrews G, How JP (2009) Vision-based guidance and control of a hovering vehicle in unknown, GPS-denied environments. In: Proceedings of the IEEE international conference on robotics and automation, Kobe, Japan, pp 2643–2648

    Google Scholar 

  2. Altug E, Ostrowski JP, Taylor CJ (2005) Control of a quadrotor helicopter using dual camera visual feedback. Int J Robot Res 24(5):329–341

    Article  Google Scholar 

  3. Amidi O, Kanade T, Fujita K (1999) A visual odometer for autonomous helicopter flight. Rob Auton Syst 28(2–3):185–193

    Article  Google Scholar 

  4. Astrom KJ, Wittenmark B (1989) Adaptive control. Addison-Wesley, Reading, MA

    Google Scholar 

  5. Barrows GL (1999) Mixed-mode VLSI optic flow sensors for micro air vehicles. PhD Dissertation, Department of Electrical Engineering, University of Maryland

    Google Scholar 

  6. Caballero F, Merino L, Ferruz J, Ollero A (2009) Vision-based odometry and slam for medium and high altitude flying UAVs. J Intell Rob Syst 54:137–161

    Article  Google Scholar 

  7. Chahl J, Srinivasan M, Zhang S (2004) Landing strategies in honeybees and applications to uninhabited airborne vehicles. Int J Robot Res 23(2):101–110

    Article  Google Scholar 

  8. Frew E, McGee T, ZuWhan K, Xiao X, Jackson S, Morimoto M, Rathinam S, Padial J, Sengupta R (2004) Vision-based road-following using a small autonomous aircraft. In: Proceedings of the IEEE aerospace conference, vol 5, pp 3006–3015

    Google Scholar 

  9. Ettinger SM, Nechyba MC, Ifju PG, Waszak M (2002) Towards flight autonomy: vision-based horizon detection for micro air vehicles. In: Proceedings of the Florida conference on recent advances in robotics, Miami, FL

    Google Scholar 

  10. Fowers S, Lee DJ, Tippetts B, Lillywhite K, Dennis A, Archibald J (2007) Vision aided stabilization and the development of a quad-rotor micro UAV. In: Proceedings of the IEEE international symposium on computational intelligence in robotics and automation, Florida, USA, pp 143–148

    Google Scholar 

  11. Garratt MA, Chahl JS (2008) Vision-based terrain following for an unmanned rotorcraft. J Field Robot 25(4–5):284–301

    Article  Google Scholar 

  12. Goodwin GC, Sin KC (1984) Adaptive filtering prediction and control. Prentice Hall, Englewood Cliffs, NJ

    MATH  Google Scholar 

  13. Green WE, Oh PY, Sevcik K, Barrows G (2003) Autonomous landing for indoor flying robots using optic flow. In: Proceedings of the 2003 ASME international mechanical engineering congress, Washington, 15–21 November 2003

    Google Scholar 

  14. Guenard N, Hamel T, Mahony R (2008) A practical visual servo control for a unmanned aerial vehicle. IEEE Trans Robot 24(2):331–341

    Article  Google Scholar 

  15. Herisse B, Russotto FX, Hamel T, Mahony R (2008) Hovering flight and vertical landing control of a VTOL unmanned aerial vehicle using optical flow. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, Nice, France, pp 801–806

    Google Scholar 

  16. Hrabar S, Sukhatme GS, Corke P, Usher K, Roberts J (2005) Combined optic-flow and stereo-based navigation of urban canyons for a UAV. In: Proceedings of the IEEE international conference on intelligent robots and systems, Canada, pp 302–309

    Google Scholar 

  17. Ioannou P, Sun J (1996) Robust adaptive control. Prentice Hall, Englewood Cliffs, NJ

    MATH  Google Scholar 

  18. Johnson A, Montgomery J, Matthies L (2005) Vision guided landing of an autonomous helicopter in hazardous terrain. In: Proceedings of the IEEE international conference on robotics and automation, Barcelona, Spain, pp 4470–4475

    Google Scholar 

  19. Johnson EN, Calise AJ, Watanabe Y, Ha J, Neidhoefer JC (2007) Real-time vision-based relative aircraft navigation. AIAA J Aerosp Comput Inform Commun 4:707–738

    Article  Google Scholar 

  20. Kanade T, Amidi O, Ke Q (2004) Real-time and 3D vision for autonomous small and micro air vehicles. In: Proceedings of the 43rd IEEE conference on decision and control, Atlantis, Bahamas, pp 1655–1662

    Google Scholar 

  21. Kendoul F (2007) Modelling and control of unmanned aerial vehicles, and development of a vision-based autopilot for small rotorcraft navigation. PhD Thesis Report, CNRS Heudiasyc Laboratory, University of Technology of Compiegne, France

    Google Scholar 

  22. Kendoul F, Fantoni I, Lozano R (2008) Adaptive vision-based controller for small rotorcraft UAVS control and guidance. In: Proceedings of the 17th IFAC world congress, Seoul, Korea, pp 797–802

    Google Scholar 

  23. Kendoul F, Fantoni I, Nonami K (2009) Optic flow-based vision system for autonomous 3D localization and control of small aerial vehicles. Rob Auton Syst 57:591–602

    Article  Google Scholar 

  24. Kendoul F, Zhenyu Y, Nonami K (2009) Embedded autopilot for accurate waypoint navigation and trajectory tracking: application to miniature rotorcraft UAVS. In: Proceedings of the IEEE international conference on robotics and automation, Kobe, Japan, pp 2884–2890

    Google Scholar 

  25. Kima J, Sukkarieh S (2007) Real-time implementation of airborne inertial-SLAM. Rob Auton Syst 55:62–71

    Article  Google Scholar 

  26. Lacroix S, Jung I, Soueres P, Hygounenc E, Berry J (2002) The autonomous blimp project of LAAS/CNRS: current status and research challenges. In: Proceedings of the workshop WS6 aerial robotics, IEEE/RSJ international conference on intelligent robots and systems, Lausanne, Switzerland, pp 35–42

    Google Scholar 

  27. Landau I, Lozano R, M’Saad M (1998) Adaptive control. Communications and control engineering. Springer, Berlin

    Book  Google Scholar 

  28. Lucas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of the DARPA IU workshop, pp 121–130

    Google Scholar 

  29. Mejias L, Saripalli S, Campoy P, Sukhatme G (2006) Visual servoing of an autonomous helicopter in urban areas using feature tracking. J Field Robot 23(3/4):185–199

    Article  Google Scholar 

  30. Proctor AA, Johnson EN, Apker TB (2006) Vision-only control and guidance for aircraft. J Field Robot 23(10):863–890

    Article  MATH  Google Scholar 

  31. Ruffier F, Franceschini N (2005) Optic flow regulation: the key to aircraft automatic guidance. Rob Auton Syst 50(4):177–194

    Article  Google Scholar 

  32. Saripalli S, Montgomery J, Sukhatme G (2003) Visually-guided landing of an unmanned aerial vehicle. IEEE Trans Robot Autom 19(3):371–381

    Article  Google Scholar 

  33. Scherer S, Singh S, Chamberlain L, Elgersma M (2008) Flying fast and low among obstacles: methodology and experiments. Int J Robot Res 27(5):549–574

    Article  Google Scholar 

  34. Sepulcre R, Jankovic M, Kokotovic P (1997) Constructive nonlinear control. Communications and control engineering series. Springer, Berlin

    Book  Google Scholar 

  35. Serres J, Dray D, Ruffier F, Franceschini N (2008) A vision-based autopilot for a miniature air vehicle: joint speed control and lateral obstacle avoidance. Auton Robots 25:103–122

    Article  Google Scholar 

  36. Shakernia O, Sharp C, Vidal R, Shim D, Ma Y, Sastry S (2002) Multiple view motion estimation and control for landing an unmanned aerial vehicle. In: Proceedings of the IEEE conference on robotics and automation, vol 3, pp 2793–2798

    Google Scholar 

  37. Shi J, Tomasi C (1994) Good features to track. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Seattle, WA, USA, pp 593–600

    Google Scholar 

  38. Sontag E (1988) Smooth stabilization implies coprime factorization. IEEE Trans Autom Contr 34:435–443

    Article  MathSciNet  Google Scholar 

  39. Srinivasan MV, Zhang S, Lehrer M, Collett T (1996) Honeybee navigation en route to the gaoal: visual flight control and odometry. J Exp Biol 199(1):237–244

    Google Scholar 

  40. Srinivasan MV, Zhang S, Bidwell N (1997) Visually mediated odometry in honeybees. J Exp Biol 200:2513–2522

    Google Scholar 

  41. Tammero L, Dickinson M (2002) The influence of visual landscape on the free flight behavior of the fruit fly drosophila melanogaster. J Exp Biol 205:327–343

    Google Scholar 

  42. Yu Z, Nonami K, Shin J, Celestino D (2007) 3D vision based landing control of a small scale autonomous helicopter. Int J Adv Robot Syst 4(1):51–56

    Google Scholar 

  43. Zufferey JC, Floreano D (2006) Fly-inspired visual steering of an ultralight indoor aircraft. IEEE Trans Robot 22(1):137–146

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba, 263-8522, Japan

    Kenzo Nonami Ph.D. (Vice President, Professor)

  2. CSIRO Queensland Centre for Advanced Technologies, Autonomous Systems Laboratory, 1 Technology Court, Pullenvale, QLD, 4069, Australia

    Farid Kendoul Ph.D. (Research Scientist)

  3. International Young Researchers Empowerment Center, Shinshu University, 3-15-1 Tokida, Ueda, Nagano, 386-8567, Japan

    Satoshi Suzuki Ph.D. (Assistant Professor)

  4. College of Information and Control Engineering, Nanjing University of Information Science & Technology, 219 Ning Liu Road, Nanjing, Jiangsu, 210044, P.R. China

    Wei Wang Ph.D. (Professor)

  5. Advanced Technology R&D Center, Mitsubishi Electric Corporation, 8-1-1 Tsukaguchi-honmachi, Amagasaki, Hyogo, 661-8661, Japan

    Daisuke Nakazawa Ph.D. (Engineer)

Authors
  1. Kenzo Nonami Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farid Kendoul Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Satoshi Suzuki Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Wang Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daisuke Nakazawa Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kenzo Nonami Ph.D. .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer

About this chapter

Cite this chapter

Nonami, K., Kendoul, F., Suzuki, S., Wang, W., Nakazawa, D. (2010). Vision-Based Navigation and Visual Servoing of Mini Flying Machines. In: Autonomous Flying Robots. Springer, Tokyo. https://doi.org/10.1007/978-4-431-53856-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-4-431-53856-1_12

  • Publisher Name: Springer, Tokyo

  • Print ISBN: 978-4-431-53855-4

  • Online ISBN: 978-4-431-53856-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation