Improving Image Distance Metric Learning by Embedding Semantic Relations

  • Fang Wang
  • Shuqiang Jiang
  • Luis Herranz
  • Qingming Huang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7674)

Abstract

Learning a proper distance metric is crucial for many computer vision and image classification applications. Neighborhood Components Analysis (NCA) is an effective distance metric learning method which maximizes the kNN leave-out-one score on the training data by considering visual similarity between images. However, only using visual similarity to learn image distances could not satisfactorily cope with the diversity and complexity of a large number of real images with many concepts. To overcome this problem, integrating concrete semantic relations of images into the distance metric learning procedure can be a useful solution. This can more accurately model the image similarities and better reflect the perception of human in the classification system. In this paper, we propose Semantic NCA (SNCA), a novel approach which integrates semantic similarity into NCA, where neighborhood relations between images in the training dataset are measured by both visual characteristics and their concept relations. We evaluated several semantic similarity measures based on the WordNet tree. Experimental results show that the proposed approach improves the performance compared to the traditional distance metric learning methods.

Keywords

Metric Learning kNN Image Classification NCA Semantic Relations 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13(1), 21–27 (1967)MATHCrossRefGoogle Scholar
  2. 2.
    Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification. In: Proc. of IEEE Conf. Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  3. 3.
    Shen, C., Kim, J., Wang, L.: A scalable dual approach to semidefinite metric learning. In: Proc. of IEEE Computer Vision and Pattern Recognition, pp. 2601–2608 (2011)Google Scholar
  4. 4.
    Singh-Miller, N., Collins, M., Hazen, T.J.: Dimensionality reduction for speech recognition using neighborhood components analysis. In: INTERSPEECH 2007, pp. 1158–1161 (2007)Google Scholar
  5. 5.
    Globerson, A., Roweis, S.: Metric learning by collapsing classes. In: Proc. of the Conference on Advances in Neural Information Processing Systems (2006)Google Scholar
  6. 6.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research 10, 207–244 (2009)MATHGoogle Scholar
  7. 7.
    Wang, Z., Hu, Y., Chia, L.-T.: Image-to-Class Distance Metric Learning for Image Classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 706–719. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Davis, J., Kulis, B., Sra, S., Dhillon, I.: Information-theoretic metric learning. In: Proc. of the International Conference on Machine Learning, pp. 209–216. ACM, New York (2007)Google Scholar
  9. 9.
    Bronstein, M.M., Bronstein, A.M.: Data fusion through cross-modality metric learning using similarity-sensitive hashing. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 3594–3601 (2010)Google Scholar
  10. 10.
    Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Proc. of the Conference on Advances in Neural Information Processing Systems (2005)Google Scholar
  11. 11.
    Sugiyama, M.: Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. Journal of Machine Learning Research 8, 1027–1061Google Scholar
  12. 12.
    Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance metric learning, with application to clustering with side-information. In: Proc. of the Conference on Advances in Neural Information Processing Systems, vol. 40 (2003)Google Scholar
  13. 13.
    Hwang, S.J., Grauman, K., Sha, F.: Learning a tree of metrics with disjoint visual features. In: Proc. of the Conference on Advances in Neural Information Processing Systems (2011)Google Scholar
  14. 14.
    Li, L., Jiang, S., Huang, Q.: Learning hierarchical semantic description via mixed-norm regularization for image understanding. IEEE Transactions on Multimedia 14 (2012)Google Scholar
  15. 15.
    Fellbaum, C.: WordNet: An electronic lexical Database (1998)Google Scholar
  16. 16.
    Resnik, P.: Using information content to evaluate semantic similarity. In: Proc. of the International Joint Conference on Artificial Intelligence, pp. 448–453 (1995)Google Scholar
  17. 17.
    Leacock, C., Chodorow, M.: Combining local context and WordNet similarity for word sense identification. In: WordNet: An Electronic Lexical Database. MIT Press (1998)Google Scholar
  18. 18.
    Fergus, R., Bernal, H., Weiss, Y., Torralba, A.: Semantic Label Sharing for Learning with Many Categories. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 762–775. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  19. 19.
    Hope, D.: Java WordNet similarity (2008), http://www.sussex.ac.uk/Users/drh21
  20. 20.
    Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. Tech. Rep. 7694, California Institute of Technology (2007)Google Scholar
  21. 21.
    Deng, J., Dong, W., Socher, R., Jia Li, L., Li, K., Fei-Fei, L.: ImageNet: a large scale hierarchical image database. In: Proc. of IEEE Computer Vision and Pattern Recognition, pp. 248–255 (2009)Google Scholar
  22. 22.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42, 145–175 (2001)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Fang Wang
    • 1
    • 2
  • Shuqiang Jiang
    • 1
    • 2
  • Luis Herranz
    • 1
    • 2
  • Qingming Huang
    • 1
    • 2
    • 3
  1. 1.Key Lab of Intell. Info. ProcessChinese Academy of SciencesBeijingChina
  2. 2.Institute of Computing TechnologyChinese Academy of SciencesBeijingChina
  3. 3.Graduate University of Chinese Academy of SciencesBeijingChina

Personalised recommendations