Uniform Convergence of Adaptive Graph-Based Regularization

  • Matthias Hein
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4005)


The regularization functional induced by the graph Laplacian of a random neighborhood graph based on the data is adaptive in two ways. First it adapts to an underlying manifold structure and second to the density of the data-generating probability measure. We identify in this paper the limit of the regularizer and show uniform convergence over the space of Hölder functions. As an intermediate step we derive upper bounds on the covering numbers of Hölder functions on compact Riemannian manifolds, which are of independent interest for the theoretical analysis of manifold-based learning methods.


Riemannian Manifold Uniform Convergence Spectral Cluster Compact Riemannian Manifold Neighborhood Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comp. 15(6), 1373–1396 (2003)CrossRefzbMATHGoogle Scholar
  2. 2.
    Belkin, M., Niyogi, P.: Semi-supervised learning on manifolds. Machine Learning 56, 209–239 (2004)CrossRefzbMATHGoogle Scholar
  3. 3.
    Belkin, M., Niyogi, P.: Towards a theoretical foundation for laplacian-based manifold methods. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 486–500. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Bousquet, O., Chapelle, O., Hein, M.: Measure based regularization. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Adv. in Neur. Inf. Proc. Syst. (NIPS), vol. 16, MIT Press, Cambridge (2004)Google Scholar
  5. 5.
    Canu, S., Elisseeff, A.: Regularization, kernels and sigmoid net (unpublished, 1999)Google Scholar
  6. 6.
    Coifman, S., Lafon, S.: Diffusion maps. Appl. and Comp. Harm. Anal. (January 2005) (preprint) (to appear)Google Scholar
  7. 7.
    Hebey, E.: Nonlinear analysis on manifolds: Sobolev spaces and inequalities. Courant Institute of Mathematical Sciences, New York (1998)Google Scholar
  8. 8.
    Hein, M.: Geometrical aspects of statistical learning theory. PhD thesis, MPI für biologische Kybernetik/Technische Universität Darmstadt (2005),
  9. 9.
    Hein, M., Audibert, J.-Y., von Luxburg, U.: From graphs to manifolds – weak and strong pointwise consistency of graph laplacians. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS, vol. 3559, pp. 470–485. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  10. 10.
    Hendriks, H., Janssen, J.H.M., Ruymgaart, F.H.: Strong uniform convergence of density estimators on compact Euclidean manifolds. Statist. Prob. Lett. 16, 305–311 (1993)CrossRefMathSciNetzbMATHGoogle Scholar
  11. 11.
    Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58, 13–30 (1963)CrossRefMathSciNetzbMATHGoogle Scholar
  12. 12.
    Kolmogorov, A.N., Tihomirov, V.M.: ε-entropy and ε-capacity of sets in functional spaces. Amer. Math. Soc. Transl. 17, 277–364 (1961)MathSciNetGoogle Scholar
  13. 13.
    Schick, T.: Manifolds with boundary of bounded geometry. Math. Nachr. 223, 103–120 (2001)CrossRefMathSciNetzbMATHGoogle Scholar
  14. 14.
    van der Vaart, A.W., Wellner, J.A.: Weak convergence and empirical processes, 2nd edn. Springer, New-York (2001)Google Scholar
  15. 15.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Adv. in Neur. Inf. Proc. Syst. (NIPS), vol. 16. MIT Press, Cambridge (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Matthias Hein
    • 1
  1. 1.Max Planck Institute for Biological CyberneticsTübingenGermany

Personalised recommendations