Sparse Flexible Models of Local Features

  • Gustavo Carneiro
  • David Lowe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3953)


In recent years there has been growing interest in recognition models using local image features for applications ranging from long range motion matching to object class recognition systems. Currently, many state-of-the-art approaches have models involving very restrictive priors in terms of the number of local features and their spatial relations. The adoption of such priors in those models are necessary for simplifying both the learning and inference tasks. Also, most of the state-of-the-art learning approaches are semi-supervised batch processes, which considerably reduce their suitability in dynamic environments, where unannotated new images are continuously presented to the learning system. In this work we propose: 1) a new model representation that has a less restrictive prior on the geometry and number of local features, where the geometry of each local feature is influenced by its k closest neighbors and models may contain hundreds of features; and 2) a novel unsupervised on-line learning algorithm that is capable of estimating the model parameters efficiently and accurately. We implement a visual class recognition system using the new model and learning method proposed here, and demonstrate that our system produces competitive classification and localization results compared to state-of-the-art methods. Moreover, we show that the learning algorithm is able to model not only classes with consistent texture (e.g., faces), but also classes with shape only (e.g., leaves), classes with a common shape but with a great variability in terms of internal texture (e.g., cups), and classes of flexible objects (e.g., snake).


Local Feature Test Image Model Feature Equal Error Rate Pairwise Variance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Amit, Y., Geman, D.: A computational model for visual selection. Neural Computation 11, 1691–1715 (1999)CrossRefGoogle Scholar
  3. 3.
    Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.Y.: An optimal algorithm for approximate nearest neighbor searching. Journal of the ACM 45, 891–923 (1998)MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Bouchard, G., Triggs, B.: Hierarchical part-based visual object categorization. In: CVPR (2005)Google Scholar
  5. 5.
    Burl, M., Weber, M., Perona, P.: A probabilistic approach to object recognition using local photometry and global geometry. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, p. 628. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  6. 6.
    Carneiro, G., Jepson, A.: Multi-scale local phase features. In: CVPR (2003)Google Scholar
  7. 7.
    Carneiro, G., Jepson, A.: Flexible spatial models for grouping local image features. In: CVPR (2004)Google Scholar
  8. 8.
    Carneiro, G., Jepson, A.: The distinctiveness, detectability, and robustness of local image features. In: CVPR (2005)Google Scholar
  9. 9.
    Crandall, D., Felzenszwalb, P., Huttenlocher, D.: Spatial priors for part-based recognition using statistical models. In: CVPR (2005)Google Scholar
  10. 10.
    Csurka, G., Bray, C., Dance, C., Fan, L.: Visual categorization with bags of keypoints. In: ECCV Workshop on Statistical Learning in Computer Vision (2004)Google Scholar
  11. 11.
    Everingham, M., Van Gool, L., Williams, C., Zisserman, A.: Pascal Visual Object Classes Challenge Results (2005),
  12. 12.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. IJCV 61(1), 55–79 (2005)CrossRefGoogle Scholar
  13. 13.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: CVPR (2003)Google Scholar
  14. 14.
    Fergus, R., Perona, P., Zisserman, A.: A sparse object category model for efficient learning and exhaustive recognition. In: CVPR (2005)Google Scholar
  15. 15.
    Ferrari, V., Tuytelaars, T., Van Gool, L.: Simultaneous object recognition and segmentation by image exploration. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 40–54. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Lazebnik, S., Schmid, C., Ponce, J.: Semi-local affine parts for object recognition. In: BMVC (2004)Google Scholar
  17. 17.
    Liebe, B., Schiele, B.: Interleaved object categorization and segmentation. In: BMVC (2003)Google Scholar
  18. 18.
    Lowe, D.: Object recognition from local scale-invariant features. In: ICCV (1999)Google Scholar
  19. 19.
    Moreels, P., Maire, M., Perona, P.: Recognition by probabilistic hypothesis construction. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 55–68. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  20. 20.
    Ramanan, D., Forsyth, D., Barnard, K.: Detecting, localizing, and recovering kinematics of textured animals. In: CVPR (2004)Google Scholar
  21. 21.
    Vasconcelos, N.: Bayesian models for visual information retrieval. PhD thesis, Massachusetts Institute of Technology (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Gustavo Carneiro
    • 1
  • David Lowe
    • 2
  1. 1.Integrated Data Systems DepartmentSiemens Corporate ResearchPrincetonUSA
  2. 2.Department of Computer ScienceUniversity of British ColumbiaVancouverCanada

Personalised recommendations