A Memory Efficient Discriminative Approach for Location Aided Recognition

  • Varsha Hedau
  • Sudipta N. Sinha
  • C. Lawrence Zitnick
  • Richard Szeliski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7583)


We propose a visual recognition approach aimed at fast recognition of urban landmarks on a GPS-enabled mobile device. While most existing methods offload their computation to a server, the latency of an image upload over a slow network can be a significant bottleneck. In this paper, we investigate a new approach to mobile visual recognition that would involve uploading only GPS coordinates to a server, following which a compact location specific classifier would be downloaded to the client and recognition would be computed completely on the client. To achieve this goal, we have developed an approach based on supervised learning that involves training very compact random forest classifiers based on labeled geo-tagged images. Our approach selectively chooses highly discriminative yet repeatable visual features in the database images during offline processing. Classification is efficient at query time as we first rectify the image based on vanishing points and then use random binary patterns to densely match a small set of downloaded features with min-hashing used to speedup the search. We evaluate our method on two public benchmarks and on two streetside datasets where we outperform standard bag-of-words retrieval as well as direct feature matching approaches, both of which are infeasible for client-side query processing.


  1. 1.
    Aly, M., Welinder, P., Munich, M., Perona, P.: Towards automated large scale discovery of image families. In: CVPR Workshop on Internet Vision, pp. 9–16 (2009)Google Scholar
  2. 2.
    Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9 (1997)Google Scholar
  3. 3.
    Arth, C., Wagner, D., Klopschitz, M., Irschara, A., Schmalstieg, D.: Wide area localization on mobile phones. In: ISMAR, pp. 73–82 (2009)Google Scholar
  4. 4.
    Baatz, G., Köser, K., Chen, D., Grzeszczuk, R., Pollefeys, M.: Handling Urban Location Recognition as a 2D Homothetic Problem. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part VI. LNCS, vol. 6316, pp. 266–279. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Bansal, M., Sawhney, H.S., Cheng, H., Daniilidis, K.: Geo-localization of street views with aerial image databases. In: MM 2011, pp. 1125–1128 (2011)Google Scholar
  6. 6.
    Breiman, L.: Random forests. Machine Learning 45 (2001)Google Scholar
  7. 7.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary Robust Independent Elementary Features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Chandrasekhar, V., Takacs, G., Chen, D., Tsai, S., Grzeszczuk, R., Girod, B.: CHoG: Compressed histogram of gradients a low bit-rate feature descriptor. In: CVPR, pp. 2504–2511 (2009)Google Scholar
  9. 9.
    Chen, D., Baatz, G., Köser, S.T., Vedantham, R., Pylvanainen, T., Roimela, K., Chen, X., Bach, J., Pollefeys, M., Girod, B., Grzeszczuk, R.: City-scale landmark identification on mobile devices. In: CVPR (2011)Google Scholar
  10. 10.
    Chum, O., Perdoch, M., Matas, J.: Geometric min-hashing: Finding a (thick) needle in a haystack. In: CVPR (2009)Google Scholar
  11. 11.
    Chum, O., Philbin, J., Zisserman, A.: Near duplicate image detection: min-hash and tf-idf weighting. In: BMVC (2008)Google Scholar
  12. 12.
    Hays, J., Efros, A.: Im2gps: estimating geographic information from a single image. In: CVPR (2008)Google Scholar
  13. 13.
    Irschara, A., Zach, C., Frahm, J.-M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: CVPR, pp. 2599–2606 (2009)Google Scholar
  14. 14.
    Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: CVPR, pp. 3304–3311 (2010)Google Scholar
  15. 15.
    Knopp, J., Sivic, J., Pajdla, T.: Avoiding Confusing Features in Place Recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 748–761. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Lepetit, V., Fua, P.: Keypoint recognition using randomized trees. PAMI 28, 1465–1479 (2006)CrossRefGoogle Scholar
  17. 17.
    Li, X., Wu, C., Zach, C., Lazebnik, S., Frahm, J.-M.: Modeling and Recognition of Landmark Image Collections Using Iconic Scene Graphs. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 427–440. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Li, Y., Crandall, D., Huttenlocher, D.: Landmark classification in large-scale image collections. In: ICCV (2009)Google Scholar
  19. 19.
    Li, Y., Snavely, N., Huttenlocher, D.P.: Location Recognition Using Prioritized Feature Matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 791–804. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. 20.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int’l J. of Computer Vision 60 (2004)Google Scholar
  21. 21.
    Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: CVPR, pp. 2161–2168 (2006)Google Scholar
  22. 22.
    Robertson, D., Cipolla, R.: An image based system for urban navigation. In: BMVC, pp. 819–828 (2004)Google Scholar
  23. 23.
    Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In: CVPR (2007)Google Scholar
  24. 24.
    Shao, H., Svoboda, T., Gool, L.V.: Zubud-zurich buildings database for image based recognition. Technical report, No. 260, ETH Zurich (2003)Google Scholar
  25. 25.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: CVPR (2008)Google Scholar
  26. 26.
    Turcot, P., Lowe, D.G.: Better matching with fewer features: The selection of useful features in large database recognition problems. In: ICCV WS-LAVD (2009)Google Scholar
  27. 27.
    Winder, S., Hua, G., Brown, M.: Picking the best DAISY. In: CVPR, pp. 178–185 (2009)Google Scholar
  28. 28.
    Zamir, A.R., Shah, M.: Accurate Image Localization Based on Google Maps Street View. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 255–268. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  29. 29.
    Zhang, W., Kosecka, J.: Hierarchical building recognition. Image Vision Comput 25(5), 704–716 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Varsha Hedau
    • 1
  • Sudipta N. Sinha
    • 2
  • C. Lawrence Zitnick
    • 2
  • Richard Szeliski
    • 2
  1. 1.Nokia ResearchSunnyvaleUSA
  2. 2.Microsoft ResearchRedmondUSA

Personalised recommendations