Skip to main content
SpringerLink
Log in

An adaptive vision-based autopilot for mini flying machines guidance, navigation and control

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

The design of reliable navigation and control systems for Unmanned Aerial Vehicles (UAVs) based only on visual cues and inertial data has many unsolved challenging problems, ranging from hardware and software development to pure control-theoretical issues. This paper addresses these issues by developing and implementing an adaptive vision-based autopilot for navigation and control of small and mini rotorcraft UAVs. The proposed autopilot includes a Visual Odometer (VO) for navigation in GPS-denied environments and a nonlinear control system for flight control and target tracking. The VO estimates the rotorcraft ego-motion by identifying and tracking visual features in the environment, using a single camera mounted on-board the vehicle. The VO has been augmented by an adaptive mechanism that fuses optic flow and inertial measurements to determine the range and to recover the 3D position and velocity of the vehicle. The adaptive VO pose estimates are then exploited by a nonlinear hierarchical controller for achieving various navigational tasks such as take-off, landing, hovering, trajectory tracking, target tracking, etc. Furthermore, the asymptotic stability of the entire closed-loop system has been established using systems in cascade and adaptive control theories. Experimental flight test data over various ranges of the flight envelope illustrate that the proposed vision-based autopilot performs well and allows a mini rotorcraft UAV to achieve autonomously advanced flight behaviours by using vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahrens, S., Levine, D., Andrews, G., & How, J. P. (2009). Vision-based guidance and control of a hovering vehicle in unknown, GPS-denied environments. In Proceedings of the IEEE international conference on robotics and automation (pp. 2643–2648), Kobe, Japan, May 2009.

  • Altug, E., Ostrowski, J. P., & Taylor, C. J. (2005). Control of a quadrotor helicopter using dual camera visual feedback. International Journal of Robotics Research, 24(5), 329–341.

    Article  Google Scholar 

  • Amidi, O., Kanade, T., & Fujita, K. (1999). A visual odometer for autonomous helicopter flight. Robotics and Autonomous Systems, 28(2–3), 185–193.

    Article  Google Scholar 

  • Astrom, K. J., & Wittenmark, B. (1989). Adaptive control. Reading: Addison-Wesley.

    Google Scholar 

  • Barrows, G. L. (1999). Mixed-mode VLSI optic flow sensors for micro air vehicles. Ph.D. Dissertation, Department of Electrical Engineering, University of Maryland.

  • Caballero, F., Merino, L., Ferruz, J., & Ollero, A. (2009). Vision-based odometry and slam for medium and high altitude flying UAVs. Journal of Intelligent Robotic Systems, 54, 137–161.

    Article  Google Scholar 

  • Chahl, J. S., Srinivasan, M. V., & Zhang, S. W. (2004). Landing strategies in honeybees and applications to uninhabited airborne vehicles. International Journal of Robotics Research, 23(2), 101–110.

    Article  Google Scholar 

  • Frew, E., McGee, T., ZuWhan, K., Xiao, X., Jackson, S., Morimoto, M., Rathinam, S., Padial, J., & Sengupta, R. (2004). Vision-based road-following using a small autonomous aircraft. In Proceedings of the IEEE aerospace conference (Vol. 5, pp. 3006–3015), March 2004.

  • Ettinger, S. M., Nechyba, M. C., Ifju, P. G., & Waszak, M. (2002). Towards flight autonomy: Vision-based horizon detection for micro air vehicles. In Proceedings of the Florida conference on recent advances in robotics, Miami, May 2002.

  • Fowers, S. G., Lee, D.-J., Tippetts, B. J., Lillywhite, K. D., Dennis, A. W., & Archibald, J. K. (2007). Vision aided stabilization and the development of a quad-rotor micro uav. In Proceedings of the IEEE international symposium on computational intelligence in robotics and automation (pp. 143–148), Florida, USA, June 2007.

  • Garratt, M. A., & Chahl, J. S. (2008). Vision-based terrain following for an unmanned rotorcraft. Journal of Field Robotics, 25(4–5), 284–301.

    Article  Google Scholar 

  • Goodwin, G. C., & Sin, K. C. (1984). Adaptive filtering prediction and control. Englewood Cliffs: Prentice Hall.

    MATH  Google Scholar 

  • Green, W. E., Oh, P. Y., Sevcik, K., & Barrows, G. (2003). Autonomous landing for indoor flying robots using optic flow. In Proceedings of the 2003 ASME international mechanical engineering congress, Washington, 15–21 November 2003.

  • Guenard, N., Hamel, T., & Mahony, R. (2008). A practical visual servo control for a unmanned aerial vehicle. IEEE Transactions on Robotics, 24(2), 331–341.

    Article  Google Scholar 

  • He, R., Prentice, S., & Roy, N. (2008). Planning in information space for a quadrotor helicopter in a GPS-denied environment. In Proceedings of the IEEE international conference on robotics and automation (pp. 1814–1820), California, USA, May 2008.

  • Herisse, B., Russotto, F.-X., Hamel, T., & Mahony, R. (2008). Hovering flight and vertical landing control of a VTOL unmanned aerial vehicle using optical flow. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 801–806), Nice, France, September 2008.

  • Hrabar, S., Sukhatme, G. S., Corke, P., Usher, K., & Roberts, J. (2005). Combined optic-flow and stereo-based navigation of urban canyons for a UAV. In Proceedings of the IEEE international conference on intelligent robots and systems (pp. 302–309), Canada, 2005.

  • Ioannou, P., & Sun, J., (1996). Robust adaptive control. Englewood Cliffs: Prentice Hall Inc.

    MATH  Google Scholar 

  • Johnson, A., Montgomery, J., & Matthies, L. (2005). Vision guided landing of an autonomous helicopter in hazardous terrain. In Proceedings of the IEEE international conference on robotics and automation (pp. 4470–4475), Barcelona, Spain, April 2005.

  • Calise, A. J., Watanabe, Y., Ha, J., Neidhoefer, J. C., & Johnson, E. N. (2007). Real-time vision-based relative aircraft navigation. AIAA Journal of Aerospace Computing, Information, and Communication, 4(4), 707–738.

    Article  Google Scholar 

  • Kanade, T., Amidi, O., & Ke, Q. (2004). Real-time and 3d vision for autonomous small and micro air vehicles. In Proceedings of the 43rd IEEE conference on decision and control (pp. 1655–1662), Atlantis, Bahamas, December 2004.

  • Kendoul, F. (2007). Modelling and control of unmanned aerial vehicles, and development of a vision-based autopilot for small rotorcraft navigation. Ph.D. Thesis Report, CNRS Heudiasyc Laboratory, University of Technology of Compiegne, France.

  • Kendoul, F., Fantoni, I., & Lozano, R. (2008). Adaptive vision-based controller for small rotorcraft uavs control and guidance. In Proceedings of the 17th IFAC world congress (pp. 797–802), Seoul, Korea, 6–11 July 2008.

  • Kendoul, F., Fantoni, I., & Nonami, K. (2009a). Optic flow-based vision system for autonomous 3D localization and control of small aerial vehicles. Robotics and Autonomous Systems (Elsevier), 57, 591–602.

    Article  Google Scholar 

  • Kendoul, F., Zhenyu, Y., & Nonami, K. (2009b). Embedded autopilot for accurate waypoint navigation and trajectory tracking: Application to miniature rotorcraft uavs. In Proceedings of the IEEE international conference on robotics and automation (pp. 2884–2890), Kobe, Japan, May 2009.

  • Kima, J., & Sukkarieh, S. (2007). Real-time implementation of airborne inertial-SLAM. Robotics and Autonomous Systems, 55, 62–71.

    Article  Google Scholar 

  • Lacroix, S., Jung, I. K., Soueres, P., Hygounenc, E., & Berry, J. P. (2002). The autonomous blimp project of LAAS/CNRS: Current status and research challenges. In Proceedings of the workshop WS6 aerial robotics, IEEE/RSJ international conference on intelligent robots and systems (pp. 35–42), Lausanne, Switzerland, 2002.

  • Landau, I. D., Lozano, R., & M’Saad, M. (1998). Adaptive control. Communications and control engineering. Berlin: Springer.

    Google Scholar 

  • Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of the DARPA IU workshop (pp. 121–130).

  • Mejias, L., Saripalli, S., Campoy, P., & S Sukhatme, G. (2006). Visual servoing of an autonomous helicopter in urban areas using feature tracking. Journal of Field Robotics, 23(3/4), 185–199.

    Article  Google Scholar 

  • Mori, R., Hirata, K., & Kinoshita, T. (2007). Vision-based guidance control of a small-scale unmanned helicopter. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 2648–2653), San Diego, CA, USA, Oct. 29–Nov. 2007.

  • Olfati-Saber, R. (2001). Nonlinear control of underactuated mechanical systems with application to robotics and aerospace vehicles. Ph.D. Thesis Report, Department of Electrical Engineering and Computer Science, MIT, USA, February 2001.

  • Proctor, A. A., Johnson, E. N., & Apker, T. B. (2006). Vision-only control and guidance for aircraft. Journal of Field Robotics, 23(10), 863–890.

    Article  Google Scholar 

  • Roberts, J. F., Stirling, T., Zufferey, J. C., & Floreano, D. (2007). Quadrotor using minimal sensing for autonomous indoor flight. In European micro air vehicle conference and flight competition (EMAV), Toulouse, France, September 2007.

  • Ruffier, F., & Franceschini, N. (2005). Optic flow regulation: the key to aircraft automatic guidance. Robotics and Autonomous Systems, 50(4), 177–194.

    Article  Google Scholar 

  • Saripalli, S., Montgomery, J. F., & Sukhatme, G. S. (2003). Visually-guided landing of an unmanned aerial vehicle. IEEE Transactions on Robotics and Automation, 19(3), 371–381.

    Article  Google Scholar 

  • Scherer, S., Singh, S., Chamberlain, L., & Elgersma, M. (2008). Flying fast and low among obstacles: Methodology and experiments. International Journal of Robotics Research, 27(5), 549–574.

    Article  Google Scholar 

  • Sepulcre, R., Jankovic, M., & Kokotovic, P. (1997). Constructive nonlinear control. Communications and control engineering series. Berlin: Springer.

    Google Scholar 

  • Serres, J., Dray, D., Ruffier, F., & Franceschini, N. (2008). A vision-based autopilot for a miniature air vehicle: joint speed control and lateral obstacle avoidance. Autonomous Robots, 25, 103–122.

    Article  Google Scholar 

  • Shakernia, O., Sharp, C. S., Vidal, R., Shim, D. H., Ma, Y., & Sastry, S. (2002). Multiple view motion estimation and control for landing an unmanned aerial vehicle. In Proceedings of the IEEE conference on robotics and automation (Vol. 3, pp. 2793–2798).

  • Shi, J., & Tomasi, C. (1994). Good features to track. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 593–600), Seattle, WA, USA.

  • Sontag, E. D. (1988). Smooth stabilization implies coprime factorization. IEEE Transactions on Automatic Control, 34, 435–443.

    Article  MathSciNet  Google Scholar 

  • Srinivasan, M. V., Miles, F. A., & Wallman, J. (1993). Visual motion and its role in the stabilisation of gaze. Amsterdam: Elsevier.

    Google Scholar 

  • Srinivasan, M. V., Zhang, S. W., Lehrer, M., & Collett, T. S. (1996). Honeybee navigation en route to the gaoal: Visual flight control and odometry. The Journal of Experimental Biology, 199(1), 237–244.

    Google Scholar 

  • Srinivasan, M. V., Zhang, S. W., & Bidwell, N. J. (1997). Visually mediated odometry in honeybees. The Journal of Experimental Biology, 200, 2513–2522.

    Google Scholar 

  • Tammero, L. F., & Dickinson, M. H. (2002). The influence of visual landscape on the free flight behavior of the fruit fly drosophila melanogaster. Journal of Experimental Biology, 205, 327–343.

    Google Scholar 

  • Wallace, G. K. (1959). Visual scanning in the desert locust schistocerca gregaria. The Journal of Experimental Biology, 36, 512–525.

    Google Scholar 

  • Yamada, H., Ichikawa, M., & Takeuchi, J. (2001). Flying robot with biologically inspired vision. Journal of Robotics and Mechatronics, 13(6), 621–624.

    Google Scholar 

  • Yu, Z., Nonami, K., Shin, J., & Celestino, D. (2007). 3D vision based landing control of a small scale autonomous helicopter. International Journal of Advanced Robotic Systems, 4(1), 51–56.

    Google Scholar 

  • Zufferey, J. C., & Floreano, D. (2006). Fly-inspired visual steering of an ultralight indoor aircraft. IEEE Transactions On Robotics, 22(1), 137–146.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Robotics and Control Lab., Electronics and Mechanical Engineering Dept., Chiba University, 263-8522, Chiba City, Japan

    Farid Kendoul & Kenzo Nonami

  2. Laboratoire Heudiasyc, UMR CNRS 6599, Computer Science Dept., Universite de Technologie de Compiegne, 60200, Compiègne, France

    Isabelle Fantoni & Rogelio Lozano

Authors
  1. Farid Kendoul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenzo Nonami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Isabelle Fantoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rogelio Lozano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farid Kendoul.

Electronic Supplementary Material

Below is the link to the supplementary material. (15.3 MB)

Below is the link to the supplementary material. (8.74 MB)

Below is the link to the supplementary material. (2.83 MB)

Below is the link to the supplementary material. (8.17 MB)

Below is the link to the supplementary material. (6.88 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kendoul, F., Nonami, K., Fantoni, I. et al. An adaptive vision-based autopilot for mini flying machines guidance, navigation and control. Auton Robot 27, 165–188 (2009). https://doi.org/10.1007/s10514-009-9135-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-009-9135-x

Keywords

Search

Navigation