Abstract
We show that for any class of functions H which has a reasonable combinatorial dimension, the vast majority of small subsets of the combinatorial cube can not be represented as a Lipschitz image of a subset of H, unless the Lipschitz constant is very large. We apply this result to the case when H consists of linear functionals of norm at most one on a Hilbert space, and thus show that “most” classification problems can not be represented as a reasonable Lipschitz loss of a kernel class.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barthe, F., Guédon, O., Mendelson, S., Naor, A.: A probabilistic approach to the geometry of the ball, all. Annals of Probability 33(2), 480–513 (2005)
Ben-David, S., Eiron, N., Simon, H.U.: Limitations of learning via embeddings in Euclidean half spaces. Journal of Machine Learning Research 3, 441–461 (2002)
Bollobás, B.: Extremal graph theory. Academic Press, London (1978)
Forster, J., Schmitt, N., Simon, H.U.: Estimating the optimal margins of embeddings in Euclidean halfspaces. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 402–415. Springer, Heidelberg (2001)
Ledoux, M., Talagrand, M.: Probability in Banach spaces. Springer, Heidelberg (1991)
Linial, N., Mendelson, S., Schechtman, G., Shraibman, A.: Complexity measures of sign matrices (preprint)
Mendelson, S., Vershynin, R.: Entropy and the combinatorial dimension. Inventiones Mathmaticae 152(1), 37–55 (2003)
Mendelson, S., Schechtman, G.: The shattering dimension of sets of linear functionals. Annals of Probability 32(3A), 1746–1770 (2004)
Mendelson, S.: Embedding with a Lipschitz function, Random Structures and Algorithms. (available on the journal’s web-page) (to appear)
Milman, V.D., Schechtman, G.: Asymptotic theory of finite dimensional normed spaces. Lecture Notes in Mathematics, vol. 1200. Springer, Heidelberg (1986)
Pajor, A.: Sous espaces _n1 des espaces de Banach. Hermann, Paris (1985)
Pisier, G.: The volume of convex bodies and Banach space geometry. Cambridge University Press, Cambridge (1989)
Van der Vaart, A.W., Wellner, J.A.: Weak convergence and Empirical Processes. Springer, Heidelberg (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mendelson, S. (2005). On the Limitations of Embedding Methods. In: Auer, P., Meir, R. (eds) Learning Theory. COLT 2005. Lecture Notes in Computer Science(), vol 3559. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11503415_24
Download citation
DOI: https://doi.org/10.1007/11503415_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26556-6
Online ISBN: 978-3-540-31892-7
eBook Packages: Computer ScienceComputer Science (R0)