Joint Semi-supervised Similarity Learning for Linear Classification

  • Maria-Irina NicolaeEmail author
  • Éric Gaussier
  • Amaury Habrard
  • Marc Sebban
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9284)


The importance of metrics in machine learning has attracted a growing interest for distance and similarity learning. We study here this problem in the situation where few labeled data (and potentially few unlabeled data as well) is available, a situation that arises in several practical contexts. We also provide a complete theoretical analysis of the proposed approach. It is indeed worth noting that the metric learning research field lacks theoretical guarantees that can be expected on the generalization capacity of the classifier associated to a learned metric. The theoretical framework of \((\epsilon , \gamma , \tau )\)-good similarity functions [1] has been one of the first attempts to draw a link between the properties of a similarity function and those of a linear classifier making use of it. In this paper, we extend this theory to a method where the metric and the separator are jointly learned in a semi-supervised way, setting that has not been explored before, and provide a theoretical analysis of this joint learning via Rademacher complexity. Experiments performed on standard datasets show the benefits of our approach over state-of-the-art methods.


Similarity learning (\(\epsilon, \gamma, \tau \))-good similarity Rademacher complexity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Balcan, M.-F., Blum, A., Srebro, N.: Improved guarantees for learning via similarity functions. In: COLT, pp. 287–298. Omnipress (2008)Google Scholar
  2. 2.
    Bao, J.-P., Shen, J.-Y., Liu, X.-D., Liu, H.-Y.: Quick asymmetric text similarity measures. ICMLC 1, 374–379 (2003)Google Scholar
  3. 3.
    Baoli, L., Qin, L., Shiwen, Y.: An adaptive k-nearest neighbor text categorization strategy. ACM TALIP (2004)Google Scholar
  4. 4.
    Bellet, A., Habrard, A., Sebban, M.: Similarity learning for provably accurate sparse linear classification. In: ICML, pp. 1871–1878 (2012)Google Scholar
  5. 5.
    Bellet, A., Habrard, A., Sebban, M.: A survey on metric learning for feature vectors and structured data. arXiv preprint arXiv:1306.6709 (2013)
  6. 6.
    Bellet, A., Habrard, A., Sebban, M.: Metric Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers (2015)Google Scholar
  7. 7.
    Boucheron, S., Bousquet, O., Lugosi, G.: Theory of classification : a survey of some recent advances. ESAIM: Probability and Statistics 9, 323–375 (2005)Google Scholar
  8. 8.
    Bousquet, O., Elisseeff, A.: Stability and generalization. JMLR 2, 499–526 (2002)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-theoretic metric learning. In: ICML, pp. 209–216. ACM, New York (2007)Google Scholar
  10. 10.
    Diligenti, M., Maggini, M., Rigutini, L.: Learning similarities for text documents using neural networks. In: ANNPR (2003)Google Scholar
  11. 11.
    Freund, Y., Schapire, R.E.: Large margin classification using the perceptron algorithm. Machine Learning 37(3), 277–296 (1999)CrossRefzbMATHGoogle Scholar
  12. 12.
    Grabowski, M., Szałas, A.: A Technique for Learning Similarities on Complex Structures with Applications to Extracting Ontologies. In: Szczepaniak, P.S., Kacprzyk, J., Niewiadomski, A. (eds.) AWIC 2005. LNCS (LNAI), vol. 3528, pp. 183–189. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  13. 13.
    Guo, Z.-C., Ying, Y.: Guaranteed classification via regularized similarity learning. CoRR, abs/1306.3108 (2013)Google Scholar
  14. 14.
    Hoi, S.C.H., Liu, W., Chang, S.-F.: Semi-supervised distance metric learning for collaborative image retrieval. In: CVPR (2008)Google Scholar
  15. 15.
    Hoi, S.C.H., Liu, W., Chang, S.-F.: Semi-supervised distance metric learning for collaborative image retrieval and clustering. TOMCCAP 6(3) (2010)Google Scholar
  16. 16.
    Hust, A.: Learning Similarities for Collaborative Information Retrieval. In: Machine Learning and Interaction for Text-Based Information Retrieval Workshop, TIR 2004, pp. 43–54 (2004)Google Scholar
  17. 17.
    Ledoux, M., Talagrand, M.: Probability in Banach Spaces: Isoperimetry and Processes. Springer, New York (1991)Google Scholar
  18. 18.
    Nicolae, M.-I., Sebban, M., Habrard, A., Gaussier, É., Amini, M.: Algorithmic robustness for learning via (\(\epsilon \), \(\gamma \), \(\tau \))-good similarity functions. CoRR, abs/1412.6452 (2014)Google Scholar
  19. 19.
    Niu, G., Dai, B., Yamada, M., Sugiyama, M.: Information-theoretic semi-supervised metric learning via entropy regularization. In: ICML. Omnipress (2012)Google Scholar
  20. 20.
    Qamar, A.M., Gaussier, É.: Online and batch learning of generalized cosine similarities. In: ICDM, pp. 926–931 (2009)Google Scholar
  21. 21.
    Qamar, A.M., Gaussier, É., Chevallet, J., Lim, J.: Similarity learning for nearest neighbor classification. In: ICDM, pp. 983–988 (2008)Google Scholar
  22. 22.
    Shalev-Shwartz, S., Singer, Y., Ng, A.Y.: Online and batch learning of pseudo-metrics. In: ICML. ACM, New York (2004)Google Scholar
  23. 23.
    Weinberger, K., Saul, L.: Fast solvers and efficient implementations for distance metric learning. In: ICML, pp. 1160–1167. ACM (2008)Google Scholar
  24. 24.
    Weinberger, K., Saul, L.: Distance metric learning for large margin nearest neighbor classification. JMLR 10, 207–244 (2009)zbMATHGoogle Scholar
  25. 25.
    Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance metric learning, with application to clustering with side-information. NIPS 15, 505–512 (2002)Google Scholar
  26. 26.
    Xu, H., Mannor, S.: Robustness and generalization. In: COLT, pp. 503–515 (2010)Google Scholar
  27. 27.
    Xu, H., Mannor, S.: Robustness and generalization. Machine Learning 86(3), 391–423 (2012)CrossRefMathSciNetzbMATHGoogle Scholar
  28. 28.
    Zha, Z.-J., Mei, T., Wang, M., Wang, Z., Hua, X.-S.: Robust distance metric learning with auxiliary knowledge. In: IJCAI, pp. 1327–1332 (2009)Google Scholar
  29. 29.
    Zhu, J., Rosset, S., Hastie, T., Tibshirani, R.: 1-norm support vector machines. In: NIPS, page 16. MIT Press (2003)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Maria-Irina Nicolae
    • 1
    • 2
    Email author
  • Éric Gaussier
    • 2
  • Amaury Habrard
    • 1
  • Marc Sebban
    • 1
  1. 1.Université Jean Monnet, Laboratoire Hubert CurienSaint-ÉtienneFrance
  2. 2.Université Grenoble Alpes, CNRS-LIG/AMASaint-Martin-d’HéresFrance

Personalised recommendations