Skip to main content
SpringerLink
Log in

Multi-support-region image descriptors and its application to street landmark localization

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel local image descriptor that is robust to general image deformations, and its application to street landmark localization. A limitation with traditional image descriptors is that they use a single support region for each interest point. For general image deformations, the amount of deformation for each location varies and is unpredictable such that it is difficult to choose the best scale of the support region. To overcome this difficulty, we propose to use multiple support regions (MSRs) of different sizes surrounding an interest point. A feature vector is computed for each support region, and the concatenation of these feature vectors forms the descriptor for this interest point. Furthermore, we propose a new similarity measure model, a local-to-global similarity (LGS) model, for point matching that takes advantage of the multi-size support regions. Each support region acts as a ‘weak’ classifier and the weights of these classifiers are learned in an unsupervised manner. Based on LGS model, we propose a MSR oriented efficient subimage retrieval (MSR-ESR) for object localization. The proposed approach is evaluated on a number of images with real and synthetic deformations, and also 15 US street landmarks’ images and videos. The experiment results show that our method outperforms existing techniques under different deformations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Winder, S.A., Brown, M.: Learning local image descriptors. In: IEEE CVPR. (2007)

  2. Mikolajczyk K., Schmid C.: A performance evaluation of local descriptors. IEEE Trans. PAMI 27(10), 1615–1630 (2005)

    Article  Google Scholar 

  3. Lowe D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)

    Article  Google Scholar 

  4. Lin, H., Jacobs, D.W.: Deformation invariant image matching. In: IEEE ICCV (2005)

  5. Mikolajczyk K., Schmid C.: Scale and affine invariant interest point detectors. IJCV 60(1), 63–86 (2004)

    Article  Google Scholar 

  6. Ke, Y., Sukthankar, R.: PCA-SHIFT: A more distinctive representation for local image descriptors. In: IEEE CVPR (2004)

  7. Carneiro G., Jepson A.D.: Flexible spatial configuration of local image features. IEEE Trans. PAMI 29(12), 2089–2104 (2007)

    Article  Google Scholar 

  8. Ling, H., Yang, X., Latecki, L.J.: Balancing deformability and discriminability for shape matching. In: ECCV (2010)

  9. Chen, J., Shan, S., He, C., Zhao, G., Pietikainen, M., Chen, X., Gao, W.: WLD: A robust local image descriptor. IEEE Trans. Pattern. Anal. Mach. Intell. (2009)

  10. Chen, J., Shan, S., He, C., Zhao, G., Pietikainen, M., Chen, X., Gao, W.: WLD: A robust local image descriptor. IEEE CVPR (2008)

  11. Sanchez-Riera, J., Ostlund, J., Fua, P., Moreno-Noguer, F.: Simultaneous pose, correspondence, and non-rigid shape. IEEE CVPR (2010)

  12. Mortensen, E.N., Deng, H., Shapiro, L.: A SIFT descriptor with global context. In: IEEE CVPR (2005)

  13. Johnson A.E., Hebert M.: Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes. IEEE Trans. PAMI 21(5), 433–449 (1999)

    Article  Google Scholar 

  14. Belongie S., Jitendra M.: Shape matching and object recognition using shape contexts. IEEE TPAMI 24(4), 509–522 (2002)

    Article  Google Scholar 

  15. Freeman W.T., Adelson E.H.: The desgin and use of steerable filter. IEEE Trans. PAMI 13(9), 891–906 (1991)

    Article  Google Scholar 

  16. Lepetit V., Fua P.: Keypoint recognition using randomized trees. IEEE Trans. PAMI 28(9), 1465–1479 (2006)

    Article  Google Scholar 

  17. Hua, G., Brown, M., Winder, S.: Discriminant embedding for local image descriptors. In: IEEE ICCV (2007)

  18. Babenko, B., Dollar, P., Belongie, S.: Task specific local region matching. In: IEEE ICCV (2007)

  19. Lejsek, H., Asmundsson, F.H., Jonsson, B.T.: Scalability of local image descriptors: a comparative study. In: ACM Multimedia (2006)

  20. Jegou, H., Douze, M., Schmid, C., Perez, P.: Aggregating local descriptors into a compact image representation. In: ECCV (2008)

  21. Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: ECCV (2008)

  22. Yang, L., Meer, P., Foran, D.J.: Multiple class segmentation using a unified framework over mean-shift patches. In: IEEE CVPR (2007)

  23. Opelt, A., Fussenegger, M., Pinz, A., Auer, P.: Weak hypotheses and boosting for generic object detection and recognition. In: ECCV (2004)

  24. Tuytelaars, T., Schmid, C.: Vector quantizing feature space with a regular lattice. In: IEEE ICCV (2007)

  25. Vedaldi, A., Soatto, S.: local features, all grown up. In: IEEE CVPR (2006)

  26. Murphy, K., Torralba, A., Eaton, D., Freeman, W.: Object detection and localization using local and global features. In: Ponce, J., Hebert, M., Schmid, C., Zisserman, A. (eds.) Toward category-level object recognition. Springer LNCS, Berlin (2006)

  27. Wu, W., Yang, J.: Object fingerprints for content analysis with applications to street landmark localization. ACM MM (2008)

  28. Lehmann, A., Leibe, B., Gool, L.V.: Feature-centric efficient subwindow search. In: IEEE ICCV (2009)

  29. Lampert, C.H., Blaschko, M.B., Hofmann, T.: Beyond sliding windows: object localization by efficient subwindow search. In: IEEE CVPR (2008)

  30. Lampert C.H., Blaschko M.B., Hofmann T.: Efficient subwindow search: a branch and bound framework for object localization. IEEE Trans. PAMI 31(12), 2129–2142 (2009)

    Article  Google Scholar 

  31. An, S., Peursum, P., Liu, W., Venkatesh, S.: Efficient algorithms for subwindow search in object detection and localization. In: IEEE CVPR (2009)

  32. Yeh, T., Lee, J., Darrell, T.: Fast concurrent object localization and recognition. In: IEEE CVPR (2009)

  33. Yuan, J., Liu, Z., Wu, Y.: Discriminative search for efficient action detection. In: IEEE CVPR (2009)

  34. Yuan, J., Liu, Z., Wu, Y.: Speeding up spatio-temporal sliding-window search for efficient event detection in crowded videos. In: ACM EiMM (2009)

  35. Nowak, E., Jurie, F., Triggs, B.: Sampling strategies for bag-of-features image classification. In: ECCV (2006)

  36. Cheng, H., Liu, Z., Zheng, N., Yang, J.: An deformable local descriptors. In: IEEE CVPR (2008)

  37. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE CVPR (2007)

  38. Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of The Fourth Alvey Vision Conference (1988)

  39. Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In ECCV (2002)

  40. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE CVPR (2001)

  41. Athitsos, V., Alon, J., Sclaroff, S., Kollios, G.: Boostmap: An embedding method for efficient nearest neighbor retrieval. In: IEEE CVPR (2004)

  42. Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification, IEEE CVPR (2008)

  43. Yang, Y., Liu, X.: A re-examination of text categorization methods. Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’99) (1999)

  44. Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. California Institute of Technology (2007). http://authors.library.caltech.edu/7694

  45. Berg, A.C., Malik, J.: Geometric blur for template matching. In: IEEE CVPR (2001)

Download references

Author information

Authors and Affiliations

  1. University of Electronic Science and Technology of China, Chengdu, China

    Hong Cheng

  2. Microsoft Research Redmond, Redmond, USA

    Zicheng Liu

  3. Carnegie Mellon University, Pittsburgh, USA

    Jie Yang

Authors
  1. Hong Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zicheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong Cheng.

Additional information

The preliminary version of the paper appeared in IEEE Proceedings of CVPR2008.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cheng, H., Liu, Z. & Yang, J. Multi-support-region image descriptors and its application to street landmark localization. Machine Vision and Applications 23, 805–819 (2012). https://doi.org/10.1007/s00138-011-0323-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-011-0323-2

Keywords

Search

Navigation