Generalization Guarantees for Metric Learning

Abstract

The derivation of guarantees on the generalization performance of the learned model is a wide topic in statistical learning theory [Valiant, 1984, Vapnik and Chervonenkis, 1971]. Assuming that data points are independent and identically distributed (i.i.d.) according to some (unknown but fixed) distribution μ, one essentially aims at bounding the deviation of the true risk of the learned model (its performance on unseen data) from its empirical risk (its performance on the training sample). This deviation is typically a function of the number of training examples, and some notion of complexity of the model such as the VC dimension [Vapnik and Chervonenkis, 1971], the fat-shattering dimension [Alon et al., 1997] or the Rademacher complexity [Bartlett and Mendelson, 2002, Koltchinskii, 2001].