Journal of Intelligent and Robotic Systems

, Volume 55, Issue 4–5, pp 323–343 | Cite as

Unmanned Aerial Vehicle Localization Based on Monocular Vision and Online Mosaicking

A New Mapping Framework
  • Fernando Caballero
  • Luis Merino
  • Joaquín Ferruz
  • Aníbal Ollero
Article

Abstract

This paper presents a new approach for vision-based UAV localization, using mosaics as environment representations. Inter-image motions are used to estimate the motion of the UAV. Online mosaicking is applied to reduce the impact of the accumulative errors in UAV position estimation. A new method to build an stochastic mosaic given the image-to-image homographies is detailed. The mosaic consists of a network of inter-image relations, and is used to create a consistent view of the environment of the UAV and hence, to detect the drift in position estimation by using the mosaic as a resource. The technique could be called simultaneous localization and mosaicking. This technique is specially suitable for monitoring and surveillance tasks in which the UAV will repeatedly cover the same area. The paper also shows experimental results with real UAVs where the benefits of the proposed method are evident.

Keywords

Computer vision Mosaic Localization Unmanned aerial vehicles 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amidi, O., Kanade, T., Fujita, K.: A visual odometer for autonomous helicopter fight. In: Proceedings of the Fifth International Conference on Intelligent Autonomous Systems (IAS-5), Sapporo, June 1998Google Scholar
  2. 2.
    Byrne, J., Cosgrove, M., Mehra, R.: Stereo based obstacle detection for an unmanned air vehicle. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, May, pp. 2830–2835. IEEE, Piscataway (2006)CrossRefGoogle Scholar
  3. 3.
    Caballero, F., Ferruz, J., Ollero, A.: Image motion compensation in aerial image sequences (in Spanish). In: XXV Jornadas de Automtica (2004, September)Google Scholar
  4. 4.
    Caballero, F., Merino, L., Ferruz, J., Ollero, A.: A visual odometer without 3D reconstruction for aerial vehicles. Applications to building inspection. In: Proceedings of the International Conference on Robotics and Automation, April, pp. 4684–4689. IEEE, Piscataway (2005)Google Scholar
  5. 5.
    Caballero, F., Merino, L., Ferruz, J., Ollero, A.: Improving vision-based planar motion estimation for unmanned aerial vehicles through online mosaicing. In: Proceedings of the International Conference on Robotics and Automation, May, pp. 2860–2865. IEEE, Piscataway (2006)Google Scholar
  6. 6.
    Corke, P., Sikka, P., Roberts, J.M.: Height estimation for an autonomous helicopter. In: Proceedings of ISER, pp. 101–110 (2000)Google Scholar
  7. 7.
    Del-Cerro, J., Barrientos, A., Campoy, P., García, P.: An autonomous helicopter guided by computer vision for inspection of overhead power cable. In: Proceedings of the Workshop WS6 Aerial Robotics, IEEE/RSJ Int. Conf. on Intelligent Robots and Systems - IROS 2002, pp. 69–78. IEEE, Piscataway (2002)Google Scholar
  8. 8.
    Demonceaux, C., Vasseur, P., Pegard, C.: Omnidirectional vision on UAV for attitude computation. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, May, pp. 2842–2847. IEEE, Piscataway (2006)CrossRefGoogle Scholar
  9. 9.
    Dickmanns, E.D., Schell, F.R.: Autonomous landing of airplanes using dynamic machine vision. In: Proc. of the IEEE Workshop Applications of Computer Vision, December, pp. 172–179. IEEE, Piscataway (1992)CrossRefGoogle Scholar
  10. 10.
    Ferruz, J., Ollero, A.: Real-time feature matching in image sequences for non-structured environments. applications to vehicle guidance. J. Intell. Robot. Syst. 28, 85–123 (2000)Google Scholar
  11. 11.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Fukao, T., Fujitani, K., Kanade, T.: Image-based tracking control of a blimp. In: Proc. 42nd IEEE Conference on Decision and Control, vol. 5, pp. 5414–5419. IEEE, Piscataway (2003)Google Scholar
  13. 13.
    Garcia, R., Batlle, J., Cuf, X., Amat, J.: Positioning an underwater vehicle through image mosaicking. In: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, vol. 3, pp. 2779–2784, Seoul, Rep. of Korea. IEEE, Piscataway (2001)Google Scholar
  14. 14.
    Garcia, R., Puig, J., Ridao, P., Cufi, X.: Augmented state kalman filtering for auv navigation. In: Proceedings of the International Conference on Robotics and Automation, pp. 4010–4015, Washington, DC, May 2002Google Scholar
  15. 15.
    Garcia-Pardo, P.J., Sukhatme, G.S., Montgomery, J.F.: Towards vision-based safe landing for an autonomous helicopter. Robot. Auton. Syst. 38(1), 19–29 (2001)CrossRefGoogle Scholar
  16. 16.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)MATHGoogle Scholar
  17. 17.
    Hrabar, S., Sukhatme, G.S.: Omnidirectional vision for an autonomous helicopter. In: Proceedings of the International Conference on Robotics and Automation, vol. 1, pp. 558–563. IEEE, Piscataway (2003)Google Scholar
  18. 18.
    Hrabar, S., Sukhatme, G.S., Corke, P., Usher, K., Roberts, J.: Combined optic-flow and stereo-based navigation of urban canyons for a UAV. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3309–3316. IEEE, Piscataway (2005)Google Scholar
  19. 19.
    Hygounenc, E., Jung, I.-K., Soueres, P., Lacroix, S.: The autonomous blimp project of LAAS-CNRS: achievements in flight control and terrain mapping. Int. J. Rob. Res. 23(4–5), 473–511 (2004)CrossRefGoogle Scholar
  20. 20.
    Julier, S., Uhlmann, J.: A new extension of the Kalman filter to nonlinear systems. In: Proceedings of the 11th Int. Symp. on Aerospace/Defence Sensing, Simulation and Controls, Orlando, April 1997Google Scholar
  21. 21.
    Kim, J., Sukkarieh, S.: Autonomous airborne navigation in unknown terrain environments. IEEE Trans. Aerosp. Electron. Syst. 40(3), 1031–1045 (2004)CrossRefGoogle Scholar
  22. 22.
    Lacroix, S., Jung, I.K., Soueres, P., Hygounenc, E., Berry, J.P.: The autonomous blimp project of LAAS/CNRS—current status and research challenges. In: Proceeding of the International Conference on Intelligent Robots and Systems, IROS, Workshop WS6 Aerial Robotics, pp. 35–42. IEEE/RSJ, Piscataway (2002)Google Scholar
  23. 23.
    Lemaire, T., Lacroix, S., Solà, J.: A practical 3D bearing only SLAM algorithm. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2449–2454. IEEE/RSJ, Piscataway (2005)Google Scholar
  24. 24.
    Ling, L., Ridley, M., Kim, J.-H., Nettleton, E., Sukkarieh, S.: Six DoF decentralised SLAM. In: Proceedings of the Australasian Conference on Robotics and Automation, Taipei, September 2003Google Scholar
  25. 25.
    Mahalanobis, P.C.: On the generalized distance in statistics. In: Proceedings of the National Institute of Science of India, vol. 12, pp. 49–55 (1936)Google Scholar
  26. 26.
    Mahony, R., Hamel, T.: Image-based visual servo control of aerial robotic systems using linear image features. IEEE Trans. Robot. 21(2), 227–239 (2005)CrossRefGoogle Scholar
  27. 27.
    Mejías, L., Saripalli, S., Campoy, P., Sukhatme, G.S.: Visual servoing of an autonomous helicopter in urban areas using feature tracking. J. Field Robot. 23(3–4), 185–199 (2006)CrossRefGoogle Scholar
  28. 28.
    Ollero, A., Ferruz, J., Caballero, F., Hurtado, S., Merino, L.: Motion compensation and object detection for autonomous helicopter visual navigation in the COMETS system. In: Proceedings of the International Conference on Robotics and Automation, ICRA, pp. 19–24. IEEE, Piscataway (2004)Google Scholar
  29. 29.
    Ollero, A., Merino, L.: Control and perception techniques for aerial robotics. Ann. Rev. Control 28, 167–178 (2004)CrossRefGoogle Scholar
  30. 30.
    Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley-IEEE, New York (1987)MATHCrossRefGoogle Scholar
  31. 31.
    Saripalli, S., Montgomery, J.F., Sukhatme, G.S.: Visually guided landing of an unmanned aerial vehicle. IEEE Trans. Robot. Autom. 19(3), 371–380 (2003)CrossRefGoogle Scholar
  32. 32.
    Saripalli, S., Sukhatme, G.S.: Landing on a mobile target using an autonomous helicopter. In: Proceedings of the International Conference on Field and Service Robotics, July, FSR (2003)Google Scholar
  33. 33.
    Shakernia, O., Vidal, R., Sharp, C., Ma, Y., Sastry, S.: Multiple view motion estimation and control for landing an aerial vehicle. In: Proceedings of the International Conference on Robotics and Automation, ICRA, May, vol. 3, pp. 2793–2798. IEEE, Piscataway (2002)Google Scholar
  34. 34.
    Tardos, J., Neira, J., Newman, P., Leonard, J.: Robust mapping and localization in indoor environments using sonar data. Int. J. Rob. Res. 24(4), 311–330 (2002)CrossRefGoogle Scholar
  35. 35.
    Triggs, B.: Autocalibration from planar scenes. In: Proceedings of the 5th European Conference on Computer Vision, ECCV, vol. 1, pp. 89–105. Springer, London (1998)Google Scholar
  36. 36.
    Vidal, R., Sastry, S., Kim, J., Shakernia, O., Shim, D.: The Berkeley aerial robot project (BEAR). In: Proceeding of the International Conference on Intelligent Robots and Systems, IROS, pp. 1–10. IEEE/RSJ, Piscataway (2002)Google Scholar
  37. 37.
    Welch, G., Bishop, G.: An introduction to the Kalman Filter. Technical Report 95-041, University of North Carolina at Chapel Hill (1995)Google Scholar
  38. 38.
    Wu, A.D., Johnson, E.N., Proctor, A.A.: Vision-aided inertial navigation for flight control. In: Proc. of AIAA Guidance, Navigation, and Control Conference and Exhibit (2005)Google Scholar
  39. 39.
    Yakimenko, O.A., Kaminer, I.I., Lentz, W.J., Ghyzel, P.A.: Unmanned aircraft navigation for shipboard landing using infrared vision. IEEE Trans. Aerosp. Electron. Syst. 38(4), 1181–1200 (2002)CrossRefGoogle Scholar
  40. 40.
    Zhang, H., Ostrowski, J.P.: Visual servoing with dynamics: control of an unmanned blimp. In: Proceedings of the International Conference on Robotics and Automation, vol. 1, pp. 618–623 (1999)Google Scholar
  41. 41.
    Zhang, Z.: Parameters estimation techniques. a tutorial with application to conic fitting. Technical report, INRIA, France (1995, October)Google Scholar
  42. 42.
    Zhang, Z., Hintz, K.J.: Evolving neural networks for video attitude and hight sensor. In: Proc. of the SPIE International Symposium on Aerospace/Defense Sensing and Control, April, vol. 2484, pp. 383–393 (1995)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  • Fernando Caballero
    • 1
  • Luis Merino
    • 2
  • Joaquín Ferruz
    • 1
  • Aníbal Ollero
    • 1
    • 3
  1. 1.Robotics, Vision and Control GroupUniversity of SevilleSevillaSpain
  2. 2.Pablo de Olavide UniversitySevillaSpain
  3. 3.Centro Avanzado de Tecnologías AeroespacialesAéropolisLa Rinconada, SevillaSpain

Personalised recommendations