Advertisement

Algorithmic Robustness for Semi-Supervised \((\epsilon , \gamma , \tau )\)-Good Metric Learning

  • Maria-Irina Nicolae
  • Marc Sebban
  • Amaury Habrard
  • Eric Gaussier
  • Massih-Reza Amini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9489)

Abstract

Using the appropriate metric is crucial for the performance of most of machine learning algorithms. For this reason, a lot of effort has been put into distance and similarity learning. However, it is worth noting that this research field lacks theoretical guarantees that can be expected on the generalization capacity of the classifier associated to a learned metric. The theoretical framework of \((\epsilon , \gamma , \tau )\)-good similarity functions [1] provides means to relate the properties of a similarity function and those of a linear classifier making use of it. In this paper, we extend this theory to a method where the metric and the separator are jointly learned in a semi-supervised way, setting that has not been explored before. We furthermore prove the robustness of our algorithm, which allows us to provide a generalization bound for this approach. The behavior of our method is illustrated via some experimental results.

Keywords

Similarity Function Mahalanobis Distance Label Data Unlabeled Data Linear Classifier 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Balcan, M.F., Blum, A., Srebro, N.: Improved guarantees for learning via similarity functions. In: COLT, pp. 287–298. Omnipress (2008)Google Scholar
  2. 2.
    Bao, J.P., Shen, J.Y., Liu, X.D., Liu, H.Y.: Quick asymmetric text similarity measures. ICMLC (2003)Google Scholar
  3. 3.
    Baoli, L., Qin, L., Shiwen, Y.: An adaptive k-nearest neighbor text categorization strategy. ACM TALIP 3, 215–226 (2004)CrossRefGoogle Scholar
  4. 4.
    Bellet, A., Habrard, A., Sebban, M.: Similarity learning for provably accurate sparse linear classification. ICML 2012, 1871–1878 (2012)zbMATHGoogle Scholar
  5. 5.
    Bellet, A., Habrard, A., Sebban, M.: A survey on metric learning for feature vectors and structured data. arXiv preprint arXiv:1306.6709 (2013)
  6. 6.
    Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-theoretic metric learning. In: ICML, pp. 209–216. ACM, New York (2007)Google Scholar
  7. 7.
    Diligenti, M., Maggini, M., Rigutini, L.: Learning similarities for text documents using neural networks. In: ANNPR (2003)Google Scholar
  8. 8.
    Grabowski, M., Szałas, A.: A technique for learning similarities on complex structures with applications to extracting ontologies. In: Szczepaniak, P.S., Kacprzyk, J., Niewiadomski, A. (eds.) AWIC 2005. LNCS (LNAI), vol. 3528, pp. 183–189. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  9. 9.
    Guo, Z.C., Ying, Y.: Guaranteed classification via regularized similarity learning. CoRR abs/1306.3108 (2013)
  10. 10.
    Hoi, S.C.H., Liu, W., Chang, S.F.: Semi-supervised distance metric learning for collaborative image retrieval and clustering. TOMCCAP 6(3), 18 (2010)CrossRefGoogle Scholar
  11. 11.
    Hust, A.: Learning similarities for collaborative information retrieval. In: Machine Learning and Interaction for Text-Based Information Retrieval Workshop, TIR-04, pp. 43–54 (2004)Google Scholar
  12. 12.
    Kolmogorov, A., Tikhomirov, V.: \(\epsilon \)-entropy and \(\epsilon \)-capacity of sets in functional spaces. Am. Math. Soc. Transl. 2(17), 277–364 (1961)Google Scholar
  13. 13.
    Qamar, A.M., Gaussier, É.: Online and batch learning of generalized cosine similarities. In: ICDM, pp. 926–931 (2009)Google Scholar
  14. 14.
    Shalev-Shwartz, S., Singer, Y., Ng, A.Y.: Online and batch learning of pseudo-metrics. In: ICML. ACM, New York (2004)Google Scholar
  15. 15.
    van der Vaart, A., Wellner, J.: Weak Convergence and Empirical Processes. Springer Series in Statistics. Springer, New York (1996) CrossRefzbMATHGoogle Scholar
  16. 16.
    Weinberger, K., Saul, L.: Distance metric learning for large margin nearest neighbor classification. JMLR 10, 207–244 (2009)zbMATHGoogle Scholar
  17. 17.
    Xu, H., Mannor, S.: Robustness and generalization. In: COLT, pp. 503–515 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Maria-Irina Nicolae
    • 1
    • 2
  • Marc Sebban
    • 1
  • Amaury Habrard
    • 1
  • Eric Gaussier
    • 2
  • Massih-Reza Amini
    • 2
  1. 1.Université Jean Monnet, Laboratoire Hubert CurienSaint-ÉtienneFrance
  2. 2.Université Grenoble Alpes, CNRS-LIG/AMASaint-Martin-d’HèresFrance

Personalised recommendations