Skip to main content

On Tree-Based Methods for Similarity Learning

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11943))

  • 1773 Accesses

Abstract

In many situations, the choice of an adequate similarity measure or metric on the feature space dramatically determines the performance of machine learning methods. Building automatically such measures is the specific purpose of metric/similarity learning. In [21], similarity learning is formulated as a pairwise bipartite ranking problem: ideally, the larger the probability that two observations in the feature space belong to the same class (or share the same label), the higher the similarity measure between them. From this perspective, the \(\mathrm{ROC}\) curve is an appropriate performance criterion and it is the goal of this article to extend recursive tree-based \(\mathrm{ROC}\) optimization techniques in order to propose efficient similarity learning algorithms. The validity of such iterative partitioning procedures in the pairwise setting is established by means of results pertaining to the theory of U-processes and from a practical angle, it is discussed at length how to implement them by means of splitting rules specifically tailored to the similarity learning task. Beyond these theoretical/methodological contributions, numerical experiments are displayed and provide strong empirical evidence of the performance of the algorithmic approaches we propose.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    A preorder on a set \(\mathcal {X}\) is any reflexive and transitive binary relationship on \(\mathcal {X}\). A preorder is an order if, in addition, it is antisymmetrical.

References

  1. Bellet, A., Habrard, A.: Robustness and generalization for metric learning. Neurocomputing 151(1), 259–267 (2015)

    Article  Google Scholar 

  2. Bellet, A., Habrard, A., Sebban, M.: Metric Learning. Morgan & Claypool Publishers, San Rafael (2015)

    Book  Google Scholar 

  3. Bousquet, O., Boucheron, S., Lugosi, G.: Introduction to statistical learning theory. In: Advanced Lectures on Machine Learning, pp. 169–207 (2004)

    Google Scholar 

  4. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Wadsworth and Brooks, Monterey (1984)

    MATH  Google Scholar 

  5. Cao, Q., Guo, Z.C., Ying, Y.: Generalization bounds for metric and similarity learning. Mach. Learn. 102(1), 115–132 (2016)

    Article  MathSciNet  Google Scholar 

  6. Clémençon, G., Depecker, M., Vayatis, N.: Ranking forests. J. Mach. Learn. Res. 14, 39–73 (2013)

    MathSciNet  MATH  Google Scholar 

  7. Clémençon, S., Depecker, M., Vayatis, N.: Adaptive partitioning schemes for bipartite ranking. Mach. Learn. 83(1), 31–69 (2011)

    Article  MathSciNet  Google Scholar 

  8. Clémençon, S., Vayatis, N.: Tree-based ranking methods. IEEE Trans. Inf. Theory 55(9), 4316–4336 (2009)

    Article  MathSciNet  Google Scholar 

  9. Clémençon, S., Lugosi, G., Vayatis, N.: Ranking and empirical minimization of U-statistics. Ann. Stat. 36(2), 844–874 (2008)

    Article  MathSciNet  Google Scholar 

  10. Fawcett, T.: An introduction to ROC analysis. Lett. Pattern Recogn. 27(8), 861–874 (2006)

    Article  MathSciNet  Google Scholar 

  11. Huo, J., Gao, Y., Shi, Y., Yin, H.: Cross-modal metric learning for AUC optimization. IEEE Trans. Neural Netw. Learn. Syst. 29(10), 4844–4856 (2018)

    Article  MathSciNet  Google Scholar 

  12. Jain, A., Hong, L., Pankanti, S.: Biometric identification. Commun. ACM 43(2), 90–98 (2000)

    Article  Google Scholar 

  13. Jain, A.K., Ross, A., Prabhakar, S.: An introduction to biometric recognition. IEEE Trans. Circuits Syst. Video Technol. 14(1), 4–20 (2004)

    Article  Google Scholar 

  14. Jain, L., Mason, B., Nowak, R.: Learning low-dimensional metrics. In: NIPS (2017)

    Google Scholar 

  15. Jin, R., Wang, S., Zhou, Y.: Regularized distance metric learning: theory and algorithm. In: NIPS (2009)

    Google Scholar 

  16. Kulis, B.: Metric learning: a survey. Found. Trends Mach. Learn. 5(4), 287–364 (2012)

    Article  Google Scholar 

  17. Lee, A.J.: \({U}\)-statistics: Theory and practice. Marcel Dekker Inc., New York (1990)

    MATH  Google Scholar 

  18. McFee, B., Lanckriet, G.R.G.: Metric learning to rank. In: ICML (2010)

    Google Scholar 

  19. Quinlan, J.: Induction of decision trees. Mach. Learn. 1(1), 1–81 (1986)

    Google Scholar 

  20. Verma, N., Branson, K.: Sample complexity of learning mahalanobis distance metrics. In: NIPS (2015)

    Google Scholar 

  21. Vogel, R., Clémençon, S., Bellet, A.: A probabilistic theory of supervised similarity learning: pairwise bipartite ranking and pointwise ROC curve optimization. In: ICML (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Robin Vogel .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Clémençon, S., Vogel, R. (2019). On Tree-Based Methods for Similarity Learning. In: Nicosia, G., Pardalos, P., Umeton, R., Giuffrida, G., Sciacca, V. (eds) Machine Learning, Optimization, and Data Science. LOD 2019. Lecture Notes in Computer Science(), vol 11943. Springer, Cham. https://doi.org/10.1007/978-3-030-37599-7_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37599-7_56

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37598-0

  • Online ISBN: 978-3-030-37599-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics