Landmark-Based Pedestrian Navigation with Enhanced Spatial Reasoning

  • Harlan Hile
  • Radek Grzeszczuk
  • Alan Liu
  • Ramakrishna Vedantham
  • Jana Košecka
  • Gaetano Borriello
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5538)


Computer vision techniques can enhance landmark-based navigation by better utilizing online photo collections. We use spatial reasoning to compute camera poses, which are then registered to the world using GPS information extracted from the image tags. Computed camera pose is used to augment the images with navigational arrows that fit the environment. We develop a system to use high-level reasoning to influence the selection of landmarks along a navigation path, and lower-level reasoning to select appropriate images of those landmarks. We also utilize an image matching pipeline based on robust local descriptors to give users of the system the ability to capture an image and receive navigational instructions overlaid on their current context. These enhancements to our previous navigation system produce a more natural navigation plan and more understandable images in a fully automatic way.


Live Image Spatial Reasoning Navigation Path Computer Vision Technique Camera Orientation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hile, H., Vedantham, R., Liu, A., Gelfand, N., Cuellar, G., Grzeszczuk, R., Borriello, G.: Landmark-Based Pedestrian Navigation from Collections of Geotagged Photos. In: Proceedings of ACM International Conference on Mobile and Ubiquitous Multimedia (MUM 2008). ACM Press, New York (2008)Google Scholar
  2. 2.
    Beeharee, A.K., Steed, A.: A Natural Wayfinding Exploiting Photos in Pedestrian Navigation Systems. In: MobileHCI 2006: Proc. of the 8th Conf. on Human-Computer Interaction with Mobile Devices and Services, pp. 81–88. ACM Press, New York (2006)Google Scholar
  3. 3.
    Millonig, A., Schechtner, K.: Developing Landmark-Based Pedestrian-Navigation Systems. IEEE Transactions on Intelligent Transportation Systems 8, 43–49 (2007)CrossRefGoogle Scholar
  4. 4.
    Kolbe, T.H.: Augmented Videos and Panoramas for Pedestrian Navigation. In: Gartner, G. (ed.) Proceedings of the 2nd Symposium on Location Based Services and TeleCartography (2004)Google Scholar
  5. 5.
    Zhang, W., Košecka, J.: Image Based Localization in Urban Environments. In: International Symposium on 3D Data Processing, Visualization and Transmission, 3DPVT 2006, North Carolina, Chapel Hill (2006)Google Scholar
  6. 6.
    Takacs, G., Chandrasekhar, V., Gelfand, N., Xiong, Y., Chen, W.C., Bismpigiannis, T., Grzeszczuk, R., Pulli, K., Girod, B.: Outdoors Augmented Reality on Mobile Phone using Loxel-Based Visual Feature Organization. In: ACM International Conference on Multimedia Information Retrieval (MIR 2008) (2008)Google Scholar
  7. 7.
    May, A.J., Ross, T., Bayer, S.H., Tarkiainen, M.J.: Pedestrian Navigation Aids: Information Requirements and Design Implications. Personal Ubiquitous Computing 7, 331–338 (2003)CrossRefGoogle Scholar
  8. 8.
    Goodman, J., Gray, P., Khammampad, K.: Using Landmarks to Support Older People in Navigation. In: Brewster, S., Dunlop, M.D. (eds.) Mobile HCI 2004. LNCS, vol. 3160, pp. 38–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Kähäri, M., Murphy, D.: Sensor-fusion Based Augmented Reality with off the Shelf Mobile Phone. In: ISMAR 2008 Demo (2008),
  10. 10.
    Shankland, S.: Google Street View Goes Mobile. CNET Download (2008),
  11. 11.
    Reitmayr, G., Schmalstieg, D.: Scalable Techniques for Collaborative Outdoor Augmented Reality. In: ISMAR 2004: Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2004) (2004)Google Scholar
  12. 12.
    Winter, S.: Route Adaptive Selection of Salient Features. In: Kuhn, W., Worboys, M.F., Timpf, S. (eds.) COSIT 2003. LNCS, vol. 2825, pp. 349–361. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo Tourism: Exploring Photo Collections in 3D. In: SIGGRAPH Conference Proceedings, pp. 835–846. ACM Press, New York (2006)Google Scholar
  14. 14.
    Hile, H., Borriello, G.: Information Overlay for Camera Phones in Indoor Environments. In: Hightower, J., Schiele, B., Strang, T. (eds.) LoCA 2007. LNCS, vol. 4718, pp. 68–84. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Look, G., Kottahachchi, B., Laddaga, R., Shrobe, H.: A Location Representation for Generating Descriptive Walking Directions. In: IUI 2005: Proc of the 10th International Conference on Intelligent User Interfaces, pp. 122–129. ACM Press, New York (2005)Google Scholar
  16. 16.
    Lourakis, M., Argyros, A.: The design and implementation of a generic sparse bundle adjustment software package based on the levenberg-marquardt algorithm. Technical Report 340, Institute of Computer Science - FORTH, Heraklion, Crete, Greece (2004),
  17. 17.
    Lowe, D.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60, 91–110 (2004)CrossRefGoogle Scholar
  18. 18.
    Szeliski, R.: Image Alignment and Stitching: A Tutorial 1. Technical Report MSR-TR-2004-92 (2005)Google Scholar
  19. 19.
    Chen, W.C., Bouguet, J.Y., Chu, M.H., Grzeszczuk, R.: Light Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields. In: SIGGRAPH 2002: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, pp. 447–456. ACM, New York (2002)CrossRefGoogle Scholar
  20. 20.
    Liu, A., Hile, H., Borriello, G., Kautz, H., Brown, P., Harniss, M., Johnson, K.: Informing the design of an automated wayfinding system for individuals with cognitive impairments. In: Third International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2009 (to appear, 2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Harlan Hile
    • 1
    • 2
  • Radek Grzeszczuk
    • 2
  • Alan Liu
    • 1
    • 2
  • Ramakrishna Vedantham
    • 2
  • Jana Košecka
    • 2
    • 3
  • Gaetano Borriello
    • 1
  1. 1.University of WashingtonUSA
  2. 2.Nokia Research CenterUSA
  3. 3.George Mason UniversityUSA

Personalised recommendations