Machine Vision and Applications

, Volume 23, Issue 4, pp 805–819 | Cite as

Multi-support-region image descriptors and its application to street landmark localization

  • Hong ChengEmail author
  • Zicheng Liu
  • Jie Yang
Original Paper


This paper presents a novel local image descriptor that is robust to general image deformations, and its application to street landmark localization. A limitation with traditional image descriptors is that they use a single support region for each interest point. For general image deformations, the amount of deformation for each location varies and is unpredictable such that it is difficult to choose the best scale of the support region. To overcome this difficulty, we propose to use multiple support regions (MSRs) of different sizes surrounding an interest point. A feature vector is computed for each support region, and the concatenation of these feature vectors forms the descriptor for this interest point. Furthermore, we propose a new similarity measure model, a local-to-global similarity (LGS) model, for point matching that takes advantage of the multi-size support regions. Each support region acts as a ‘weak’ classifier and the weights of these classifiers are learned in an unsupervised manner. Based on LGS model, we propose a MSR oriented efficient subimage retrieval (MSR-ESR) for object localization. The proposed approach is evaluated on a number of images with real and synthetic deformations, and also 15 US street landmarks’ images and videos. The experiment results show that our method outperforms existing techniques under different deformations.


Local image descriptors Deformability Street landmark localization Object detection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Winder, S.A., Brown, M.: Learning local image descriptors. In: IEEE CVPR. (2007)Google Scholar
  2. 2.
    Mikolajczyk K., Schmid C.: A performance evaluation of local descriptors. IEEE Trans. PAMI 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  3. 3.
    Lowe D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRefGoogle Scholar
  4. 4.
    Lin, H., Jacobs, D.W.: Deformation invariant image matching. In: IEEE ICCV (2005)Google Scholar
  5. 5.
    Mikolajczyk K., Schmid C.: Scale and affine invariant interest point detectors. IJCV 60(1), 63–86 (2004)CrossRefGoogle Scholar
  6. 6.
    Ke, Y., Sukthankar, R.: PCA-SHIFT: A more distinctive representation for local image descriptors. In: IEEE CVPR (2004)Google Scholar
  7. 7.
    Carneiro G., Jepson A.D.: Flexible spatial configuration of local image features. IEEE Trans. PAMI 29(12), 2089–2104 (2007)CrossRefGoogle Scholar
  8. 8.
    Ling, H., Yang, X., Latecki, L.J.: Balancing deformability and discriminability for shape matching. In: ECCV (2010)Google Scholar
  9. 9.
    Chen, J., Shan, S., He, C., Zhao, G., Pietikainen, M., Chen, X., Gao, W.: WLD: A robust local image descriptor. IEEE Trans. Pattern. Anal. Mach. Intell. (2009)Google Scholar
  10. 10.
    Chen, J., Shan, S., He, C., Zhao, G., Pietikainen, M., Chen, X., Gao, W.: WLD: A robust local image descriptor. IEEE CVPR (2008)Google Scholar
  11. 11.
    Sanchez-Riera, J., Ostlund, J., Fua, P., Moreno-Noguer, F.: Simultaneous pose, correspondence, and non-rigid shape. IEEE CVPR (2010)Google Scholar
  12. 12.
    Mortensen, E.N., Deng, H., Shapiro, L.: A SIFT descriptor with global context. In: IEEE CVPR (2005)Google Scholar
  13. 13.
    Johnson A.E., Hebert M.: Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes. IEEE Trans. PAMI 21(5), 433–449 (1999)CrossRefGoogle Scholar
  14. 14.
    Belongie S., Jitendra M.: Shape matching and object recognition using shape contexts. IEEE TPAMI 24(4), 509–522 (2002)CrossRefGoogle Scholar
  15. 15.
    Freeman W.T., Adelson E.H.: The desgin and use of steerable filter. IEEE Trans. PAMI 13(9), 891–906 (1991)CrossRefGoogle Scholar
  16. 16.
    Lepetit V., Fua P.: Keypoint recognition using randomized trees. IEEE Trans. PAMI 28(9), 1465–1479 (2006)CrossRefGoogle Scholar
  17. 17.
    Hua, G., Brown, M., Winder, S.: Discriminant embedding for local image descriptors. In: IEEE ICCV (2007)Google Scholar
  18. 18.
    Babenko, B., Dollar, P., Belongie, S.: Task specific local region matching. In: IEEE ICCV (2007)Google Scholar
  19. 19.
    Lejsek, H., Asmundsson, F.H., Jonsson, B.T.: Scalability of local image descriptors: a comparative study. In: ACM Multimedia (2006)Google Scholar
  20. 20.
    Jegou, H., Douze, M., Schmid, C., Perez, P.: Aggregating local descriptors into a compact image representation. In: ECCV (2008)Google Scholar
  21. 21.
    Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: ECCV (2008)Google Scholar
  22. 22.
    Yang, L., Meer, P., Foran, D.J.: Multiple class segmentation using a unified framework over mean-shift patches. In: IEEE CVPR (2007)Google Scholar
  23. 23.
    Opelt, A., Fussenegger, M., Pinz, A., Auer, P.: Weak hypotheses and boosting for generic object detection and recognition. In: ECCV (2004)Google Scholar
  24. 24.
    Tuytelaars, T., Schmid, C.: Vector quantizing feature space with a regular lattice. In: IEEE ICCV (2007)Google Scholar
  25. 25.
    Vedaldi, A., Soatto, S.: local features, all grown up. In: IEEE CVPR (2006)Google Scholar
  26. 26.
    Murphy, K., Torralba, A., Eaton, D., Freeman, W.: Object detection and localization using local and global features. In: Ponce, J., Hebert, M., Schmid, C., Zisserman, A. (eds.) Toward category-level object recognition. Springer LNCS, Berlin (2006)Google Scholar
  27. 27.
    Wu, W., Yang, J.: Object fingerprints for content analysis with applications to street landmark localization. ACM MM (2008)Google Scholar
  28. 28.
    Lehmann, A., Leibe, B., Gool, L.V.: Feature-centric efficient subwindow search. In: IEEE ICCV (2009)Google Scholar
  29. 29.
    Lampert, C.H., Blaschko, M.B., Hofmann, T.: Beyond sliding windows: object localization by efficient subwindow search. In: IEEE CVPR (2008)Google Scholar
  30. 30.
    Lampert C.H., Blaschko M.B., Hofmann T.: Efficient subwindow search: a branch and bound framework for object localization. IEEE Trans. PAMI 31(12), 2129–2142 (2009)CrossRefGoogle Scholar
  31. 31.
    An, S., Peursum, P., Liu, W., Venkatesh, S.: Efficient algorithms for subwindow search in object detection and localization. In: IEEE CVPR (2009)Google Scholar
  32. 32.
    Yeh, T., Lee, J., Darrell, T.: Fast concurrent object localization and recognition. In: IEEE CVPR (2009)Google Scholar
  33. 33.
    Yuan, J., Liu, Z., Wu, Y.: Discriminative search for efficient action detection. In: IEEE CVPR (2009)Google Scholar
  34. 34.
    Yuan, J., Liu, Z., Wu, Y.: Speeding up spatio-temporal sliding-window search for efficient event detection in crowded videos. In: ACM EiMM (2009)Google Scholar
  35. 35.
    Nowak, E., Jurie, F., Triggs, B.: Sampling strategies for bag-of-features image classification. In: ECCV (2006)Google Scholar
  36. 36.
    Cheng, H., Liu, Z., Zheng, N., Yang, J.: An deformable local descriptors. In: IEEE CVPR (2008)Google Scholar
  37. 37.
    Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE CVPR (2007)Google Scholar
  38. 38.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of The Fourth Alvey Vision Conference (1988)Google Scholar
  39. 39.
    Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In ECCV (2002)Google Scholar
  40. 40.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE CVPR (2001)Google Scholar
  41. 41.
    Athitsos, V., Alon, J., Sclaroff, S., Kollios, G.: Boostmap: An embedding method for efficient nearest neighbor retrieval. In: IEEE CVPR (2004)Google Scholar
  42. 42.
    Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification, IEEE CVPR (2008)Google Scholar
  43. 43.
    Yang, Y., Liu, X.: A re-examination of text categorization methods. Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’99) (1999)Google Scholar
  44. 44.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. California Institute of Technology (2007).
  45. 45.
    Berg, A.C., Malik, J.: Geometric blur for template matching. In: IEEE CVPR (2001)Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.University of Electronic Science and Technology of ChinaChengduChina
  2. 2.Microsoft Research RedmondRedmondUSA
  3. 3.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations