Skip to main content
SpringerLink
Log in

Flyphone: Visual Self-Localisation Using a Mobile Phone as Onboard Image Processor on a Quadrocopter

  • Published:
Journal of Intelligent and Robotic Systems Aims and scope Submit manuscript

Abstract

An unmanned aerial vehicle (UAV) needs to orient itself in its operating environment to fly autonomously. Localisation methods based on visual data are independent of erroneous GPS measurements or imprecise inertial sensors. In our approach, a quadrocopter first establishes an image database of the environment. Afterwards, the quadrocopter is able to locate itself by comparing a current image taken of the environment with earlier images in the database. Therefore, characteristic image features are extracted which can be compared efficiently. We analyse three feature extraction methods and five feature similarity measures. The evaluation is based on two datasets recorded under real conditions. The computations are performed on a Nokia N95 mobile phone, which is mounted on the quadrocopter. This lightweight, yet powerful device offers an integrated camera and serves as central processing unit. The mobile phone proved to be a good choice for visual localisation on a quadrocopter.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Artac, M., Leonardis, A.: Outdoor mobile robot localisation using global and local features. In: Proceedings of the 9th Computer Vision Winter Workshop (CVWW), pp. 175–184. Piran, Slovenia (2004)

  2. Bath, W., Paxman, J.: UAV localisation and control through computer vision. In: Proceedings of the Australasian Conference on Robotics and Automation. Sydney, Australia (2004)

  3. Bay, H., Ess, A., Tuytelaars, T., Gool, L.: Surf: speeded up robust features. In: Computer Vision and Image Understanding (CVIU), vol. 110, pp. 346–359 (2008)

  4. Bradley, D., Patel, R., Vandapel, N., Thayer, S.: Real-time image-based topological localization in large outdoor environments. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), pp. 3670–3677. Edmonton, Alberta, Canada (2005)

  5. Castle, R., Gawley, D., Klein, G., Murray, D.: Towards simultaneous recognition, localization and mapping for hand-held and wearable cameras. In: Proceedings of the International Conference on Robotics and Automation (ICRA), pp. 4102–4107. Rome, Italy (2007)

  6. Corke, P., Strelow, D., Singh, S.: Omnidirectional visual odometry for a planetary rover. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS), vol. 4, pp. 4007–4012. Sendai, Japan (2004)

  7. De Wagter, C., Proctor, A., Johnson, E.: Vision-only aircraft flight control. In: The 22nd Digital Avionics Systems Conference (DASC), vol. 2, pp. 8.B.2–81–11 (2003)

  8. Gurdan, D., Stumpf, J., Achtelik, M., Doth, K.M., Hirzinger, G., Rus, D.: Energy-efficient autonomous four-rotor flying robot controlled at 1 khz. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 361–366. Roma, Italy (2007)

  9. Hofmeister, M., Liebsch, M., Zell, A.: Visual self-localization for small mobile robots with weighted gradient orientation histograms. In: 40th International Symposium on Robotics (ISR), pp. 87–91. Barcelona, Spain (2009)

  10. Jogan, M., Leonardis, A., Wildenauer, H., Bischof, H.: Mobile robot localization under varying illumination. In: 16th International Conference on Pattern Recognition (ICPR), vol. II, pp. 741–744. Los Alamitos, CA, USA (2002)

  11. Kim, J., Sukkarieh, S.: Real-time implementation of airborne inertial-SLAM. In: Robotics and Autonomous Systems archieve, vol. 55, pp. 62–71. Amsterdam, The Netherlands (2007)

  12. Lamon, P., Nourbakhsh, I., Jensen, B., Siegwart, R.: Deriving and matching image fingerprint sequences for mobile robot localization. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1609–1614. Seoul, Korea (2001)

  13. Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 1150–1157. Corfu (1999)

  14. Maimone, M., Cheng, Y., Matthies, L.: Two years of visual odometry on the mars exploration rovers: field reports. Journal of Field Robotics 24(3), 169–186 (2007)

    Article  Google Scholar 

  15. Manay, S., Hong, B.-W., Yezzi, A.J., Soatto, S.: Integral invariant signatures. In: Proceedings of the 8th European Conference on Computer Vision (ECCV). LNCS, vol. 3024, pp. 87–99. Prague, Czech Republic (2004)

  16. Mondragon, I., Campoy, P., Correa, J., Mejias, L.: Visual model feature tracking for UAV control. In: IEEE International Symposium on Intelligent Signal Processing (WISP), pp. 1–6. Alcala de Henares (2007)

  17. Musial, M., Brandenburg, U., Hommel, G.: Cooperative autonomous mission planning and execution for the flying robot MARVIN. In: Intelligent Autonomous Systems, vol. 6, pp. 636–643. Amsterdam, The Netherlands (2000)

  18. Pinies, P., Lupton, T., Sukkarieh, S., Tardos, J.D.: Inertial aiding of inverse depth SLAM using a monocular camera. In: Proceedings IEEE International Conference on Robotics and Automation (ICRA), pp. 2797–2802. Rome, Italy (2007)

  19. Saripalli, S., Montgomery, J.F., Sukhatme, G.S.: Vision-based autonomous landing of an unmanned aerial vehicle. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2799–2804. Washington, DC, USA (2002)

  20. Scaramuzza, D., Siegwart, R.: Appearance guided monocular omnidirectional visual odometry for outdoor ground vehicles. In: IEEE Transactions on Robotics, vol. 24, no. 5, pp. 1015–1026 (2008, appearance – guided)

  21. Scaramuzza, D., Siegwart, R.: Monocular omnidirectional visual odometry for outdoor ground vehicles. In: Lecture Notes in Computer Science, chap. Computer Vision Systems, vol. 5008/2008, pp. 206–215. Springer, New York (2008)

    Google Scholar 

  22. Siggelkow, S.: Feature histograms for content-based image retrieval. Ph.D. thesis, Albert-Ludwigs-Universität Freiburg, Fakultät für Angewandte Wissenschaften, Freiburg, Germany (2002)

  23. Sim, R., Dudek, G.: Comparing image-based localization methods. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1560–1562. Apaculco, Mexico (2003)

  24. Sinopoli, B., Micheli, M., Donato, G., Koo, T.J.: Vision based navigation for an unmanned aerial vehicle. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 1757–1765. Seoul, Korea (2001)

  25. Steder, B., Rottmann, A., Grisetti, G., Stachniss, C., Burgard, W.: Autonomous navigation for small flying vehicles. In: Workshop on Micro Aerial Vehicles at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). San Diego, CA, USA (2007)

  26. Takacs, G., Chandrasekhar, V., Gelfand, N., Xiong, Y., Chen, W., Bismpigiannis, T., Grzeszczuk, R., Pulli, K., Girod, B.: Outdoors augmented reality on mobile phone using loxel-based visual feature organization. In: Proceedings of the 1st ACM international conference on Multimedia information retrieval (MIR), pp. 427–434. New York, NY, USA (2008)

  27. Tamimi, H.: Vision-based features for mobile robot localization. Ph.D. thesis, Eberhard-Karls-Universität Tübingen, Tübingen, Germany (2006)

  28. Ulrich, I., Nourbakhsh, I.: Appearance-based place recognition for topological localization. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 1023–1029. San Francisco, CA, USA (2000)

  29. Valgren, C., Lilienthal, A.: SIFT, SURF and seasons: long-term outdoor localization using local features. In: Proceedings of the European Conference on Mobile Robots (ECMR), pp. 253–258. Freiburg, Germany (2007)

  30. Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T., Schmalstieg, D.: Pose tracking from natural features on mobile phones. In: 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR), pp. 125–134. Cambridge, UK (2008)

  31. Wang, J., Zhai, S., Canny, J.: Camera phone based motion sensing: interaction techniques, applications and performance study. In: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST), pp. 101–110. New York, NY, USA (2006)

  32. Weiss, C., Masselli, A., Tamimi, H., Zell, A.: Fast outdoor robot localization using integral invariants. In: Proceedings of the 5th International Conference on Computer Vision Systems (ICVS). Bielefeld, Germany (2007)

  33. Williams, B., Klein, G., Reid, I.: Real-time SLAM relocalisation. In: IEEE 11th International Conference on Computer Vision (ICCV), pp. 1–8. Rio de Janeiro, Brazil (2007)

  34. Zhou, C., Wei, Y., Tan, T.: Mobile robot self-localization based on global visual appearance based features. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 1271–1276. Taipei, Taiwan (2003)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Tübingen, Sand 1, 72076, Tübingen, Germany

    Sara Erhard, Karl E. Wenzel & Andreas Zell

Authors
  1. Sara Erhard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karl E. Wenzel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Zell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sara Erhard.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Erhard, S., Wenzel, K.E. & Zell, A. Flyphone: Visual Self-Localisation Using a Mobile Phone as Onboard Image Processor on a Quadrocopter. J Intell Robot Syst 57, 451–465 (2010). https://doi.org/10.1007/s10846-009-9360-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-009-9360-8

Keywords

Search

Navigation