Image-to-Class Distance Metric Learning for Image Classification

  • Zhengxiang Wang
  • Yiqun Hu
  • Liang-Tien Chia
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


Image-To-Class (I2C) distance is first used in Naive-Bayes Nearest-Neighbor (NBNN) classifier for image classification and has successfully handled datasets with large intra-class variances. However, the performance of this distance relies heavily on the large number of local features in the training set and test image, which need heavy computation cost for nearest-neighbor (NN) search in the testing phase. If using small number of local features for accelerating the NN search, the performance will be poor.

In this paper, we propose a large margin framework to improve the discrimination of I2C distance especially for small number of local features by learning Per-Class Mahalanobis metrics. Our I2C distance is adaptive to different class by combining with the learned metric for each class. These multiple Per-Class metrics are learned simultaneously by forming a convex optimization problem with the constraints that the I2C distance from each training image to its belonging class should be less than the distance to other classes by a large margin. A gradient descent method is applied to efficiently solve this optimization problem. For efficiency and performance improved, we also adopt the idea of spatial pyramid restriction and learning I2C distance function to improve this I2C distance. We show in experiments that the proposed method can significantly outperform the original NBNN in several prevalent image datasets, and our best results can achieve state-of-the-art performance on most datasets.


Recognition Accuracy Spatial Restriction Spatial Pyramid Candidate Class Spatial Division 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification. In: CVPR (2008)Google Scholar
  2. 2.
    Bosch, A., Zisserman, A., Munoz, X.: Scene classification using a hybrid generative/discriminative approach. TPAMI (2008)Google Scholar
  3. 3.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. CVPR Workshop on Generative-Model Based Vision (2004)Google Scholar
  4. 4.
    Fei-Fei, L., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: CVPR (2005)Google Scholar
  5. 5.
    Frome, A., Singer, Y., Malik, J.: Image retrieval and classification using local distance functions. In: NIPS (2006)Google Scholar
  6. 6.
    Frome, A., Singer, Y., Sha, F., Malik, J.: Learning globally-consistent local distance functions for shape-based image retrieval and classification. In: ICCV (2007)Google Scholar
  7. 7.
    van Gemert, J.C., Geusebroek, J.M., Veenman, C.J.: Kernel codebooks for scene categorization. In: ECCV (2008)Google Scholar
  8. 8.
    Grauman, K., Darrell, T.: The pyramid match kernel: Discriminative classification with sets of image features. In: ICCV (2005)Google Scholar
  9. 9.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar
  10. 10.
    Li, J., Fei-Fei, L.: What, where and who? Classifying events by scene and object recognition. In: ICCV (2007)Google Scholar
  11. 11.
    Liu, J., Shah, M.: Scene modeling using co-clustering. In: ICCV (2007)Google Scholar
  12. 12.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2) (2004)Google Scholar
  13. 13.
    Lu, Z., Ip, H.H.: Image categorization by learning with context and consistency. In: CVPR (2009)Google Scholar
  14. 14.
    Lu, Z., Ip, H.H.: Image categorization with spatial mismatch kernels. In: CVPR (2009)Google Scholar
  15. 15.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42(3) (2001)Google Scholar
  16. 16.
    Wang, Z., Hu, Y., Chia, L.T.: Learning instance-to-class distance for human action recognition. In: ICIP (2009)Google Scholar
  17. 17.
    Rasiwasia, N., Vasconcelos, N.: Scene classification with low-dimensional semantic spaces and weak supervision. In: CVPR (2008)Google Scholar
  18. 18.
    Weinberger, K.Q., Blitzer, J., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. In: NIPS (2005)Google Scholar
  19. 19.
    Weinberger, K.Q., Saul, L.K.: Fast solvers and efficient implementations for distance metric learning. In: ICML (2008)Google Scholar
  20. 20.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research 10, 207–244 (2009)Google Scholar
  21. 21.
    Wu, J., Rehg, J.M.: Where am I: Place instance and category recognition using spatial PACT. In: CVPR (2008)Google Scholar
  22. 22.
    Wu, J., Rehg, J.M.: Beyond the euclidean distance: Creating effective visual codebooks using the histogram intersection kernel. In: ICCV (2009)Google Scholar
  23. 23.
    Wu, J., Rehg, J.M.: CENTRIST: A visual descriptor for scene categorization. Technical Report GIT-GVU-09-05, GVU Center, Georgia Institute of Technology (2009)Google Scholar
  24. 24.
    Yang, J., Lu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: CVPR (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Zhengxiang Wang
    • 1
  • Yiqun Hu
    • 1
  • Liang-Tien Chia
    • 1
  1. 1.Center for Multimedia and Network Technology, School of Computer EngineeringNanyang Technological UniversitySingapore

Personalised recommendations