Per-patch Descriptor Selection Using Surface and Scene Properties

  • Ivo Everts
  • Jan C. van Gemert
  • Theo Gevers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7577)


Local image descriptors are generally designed for describing all possible image patches. Such patches may be subject to complex variations in appearance due to incidental object, scene and recording conditions. Because of this, a single-best descriptor for accurate image representation under all conditions does not exist. Therefore, we propose to automatically select from a pool of descriptors the one that is best suitable based on object surface and scene properties. These properties are measured on the fly from a single image patch through a set of attributes. Attributes are input to a classifier which selects the best descriptor. Our experiments on a large dataset of colored object patches show that the proposed selection method outperforms the best single descriptor and a-priori combinations of the descriptor pool.


Average Precision Image Patch Image Descriptor Disturbance Level Descriptor Selection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    van de Weijer, J., Gevers, T., Geusebroek, J.M.: Edge and corner detection by photometric quasi-invariants. PAMI (2005)Google Scholar
  2. 2.
    Varma, M., Ray, D.: Learning the discriminative power-invariance trade-off. In: ICCV (2007)Google Scholar
  3. 3.
    van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. PAMI (2010)Google Scholar
  4. 4.
    Brown, M., Hua, G., Winder, S.: Discriminative learning of local image descriptors. PAMI (2011)Google Scholar
  5. 5.
    Adelson, E.H.: On seeing stuff: the perception of materials by humans and machines. In: Society of Photo-Optical Instrumentation Engineers (SPIE) (2001)Google Scholar
  6. 6.
    Liu, C., Sharan, L., Adelson, E.H., Rosenholtz, R.: Exploring features in a bayesian framework for material recognition. In: CVPR (2010)Google Scholar
  7. 7.
    Cai, H., Mikolajczyk, K., Matas, J.: Learning linear discriminant projections for dimensionality reduction of image descriptors. TPAMI (2010)Google Scholar
  8. 8.
    Philbin, J., Isard, M., Sivic, J., Zisserman, A.: Descriptor Learning for Efficient Retrieval. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 677–691. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Dubout, C., Fleuret, F.: Tasting families of features for image classification. In: ICCV (2011)Google Scholar
  10. 10.
    Gehler, P.V., Nowozin, S.: On feature combination for multiclass object classification. In: ICCV (2009)Google Scholar
  11. 11.
    Opelt, A., Pinz, A., Fussenegger, M., Auer, P.: Generic object recognition with boosting. TPAMI 28 (2006)Google Scholar
  12. 12.
    Kavukcuoglu, K., Ranzato, M.A., Fergus, R., LeCun, Y.: Learning invariant features through topographic filter maps. In: CVPR (2009)Google Scholar
  13. 13.
    Hu, D., Bo, L., Ren, X.: Toward robust material recognition for everyday objects. In: BMVC (2011)Google Scholar
  14. 14.
    Guo, Y., Zhao, G., Pietikäinen, M., Xu, Z.: Descriptor Learning Based on Fisher Separation Criterion for Texture Classification. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 185–198. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  15. 15.
    Caputo, B., Hayman, E., Fritz, M., Eklundh, J.O.: Classifying materials in the real world. Image Vision Comput., 150–163 (2010)Google Scholar
  16. 16.
    Dror, R.O., Adelson, E.H., Willsky, A.S.: Recognition of surface reflectance properties from a single image under unknown real-world illumination. In: CVPR Workshop on Identifying Object Across Variations in Lighting (2001)Google Scholar
  17. 17.
    Romeiro, F., Zickler, T.: Blind Reflectometry. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 45–58. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  18. 18.
    Wang, O., Gunawardane, P., Scher, S., Davis, J.: Material classification using brdf slices. In: CVPR (2009)Google Scholar
  19. 19.
    Shafer, S.A.: Using color to separate reflection components. Color Research and Applications 10, 210–218 (1985)CrossRefGoogle Scholar
  20. 20.
    Alexe, B., Deselaers, T., Ferrari, V.: Measuring the objectness of image windows. PAMI 99 (2012)Google Scholar
  21. 21.
    van Gemert, J.C.: Exploiting photographic style for category-level image classification by generalizing the spatial pyramid. In: ICMR (2011)Google Scholar
  22. 22.
    Motoyoshi, I., Nishida, S., Sharan, L., Adelson, E.H.: Image statistics and the perception of surface qualities. Nature 447, 206–209 (2007)CrossRefGoogle Scholar
  23. 23.
    Sharan, L., Li, Y., Motoyoshi, I., Nishida, S., Adelson, E.H.: Image statistics for surface reflectance perception. J. Opt. Soc. Am. A 25, 846–865 (2008)CrossRefGoogle Scholar
  24. 24.
    Gijsenij, A., Gevers, T., van de Weijer, J.: Improving color constancy by photometric edge weighting. PAMI (2011)Google Scholar
  25. 25.
    Yanulevskaya, V., Geusebroek, J.M.: Significance of the weibull distribution and its submodels in natural image statistics. In: VISSAP (2009)Google Scholar
  26. 26.
    Zadrozny, B., Langford, J., Abe, N.: Cost-sensitive learning by cost-proportionate example weighting. In: IEEE International Conference on Data Mining (2003)Google Scholar
  27. 27.
    Barnard, K., Martin, L., Funt, B., Coath, A.: Data for colour research. Color Research and Application (2000)Google Scholar
  28. 28.
    Geusebroek, J.M., Burghouts, G.J., Smeulders, A.W.M.: The amsterdam library of object images. IJCV (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ivo Everts
    • 1
  • Jan C. van Gemert
    • 1
  • Theo Gevers
    • 1
  1. 1.Intelligent Systems Lab Amsterdam (ISLA)University of AmsterdamAmsterdamThe Netherlands

Personalised recommendations