Abstract
Given a set of n randomly drawn sample points, spectral clustering in its simplest form uses the second eigenvector of the graph Laplacian matrix, constructed on the similarity graph between the sample points, to obtain a partition of the sample. We are interested in the question how spectral clustering behaves for growing sample size n. In case one uses the normalized graph Laplacian, we show that spectral clustering usually converges to an intuitively appealing limit partition of the data space. We argue that in case of the unnormalized graph Laplacian, equally strong convergence results are difficult to obtain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Belkin, M.: Problems of Learning on Manifolds. PhD thesis, University of Chicago (2003)
Belkin, M., Niyogi, P.: Semi-supervised learning on Riemannian manifolds. Machine Learning (to appear), Available at http://people.cs.uchicago.edu/char126/relaxmisha
Bengio, Y. Vincent, P., Paiement, J.-F., Delalleau, O., Ouimet, M., Le Roux, N., Spectral clustering and kernel PCA are learning eigenfunctions. Technical Report TR 1239, University of Montreal (2003)
Bhatia, R.: Matrix Analysis. Springer, New York (1997)
Birman, M., Solomjak, M.: Spectral theory of self-adjoint operators in Hilbert space. Reidel Publishing Company, Dordrecht (1987)
Bousquet, O., Chapelle, O., Hein, M.: Measure based regularization. In: Thrun, S., Saul, L., Scholkopf, B. (eds.) Advances in Neural Information Processing Systems 16, MIT Press, Cambridge (2004)
Chatelin, F.: Spectral Approximation of Linear Operators. Academic Press, New York (1983)
Chung, F.R. K.: Spectral graph theory .In: CBMS Regional Conference Series in Mathematics. Published for the Conference Board of the Mathematical Sciences, Washington, DC, voi .92 (1997)
Donath, W.E., Hoffman, A.J.: Lower bounds for the partitioning of graphs. IBM J. Res. Develop. 17, 420–425 (1973)
Fiedler, M.: Algebraic connectivity of graphs. Czechoslovak Math. J. 23, 298–305 (1973)
Guattery, S., Miller, G.L.: On the quality of spectral separators. SIAM Journal of Matrix Anal. Appl. 19(3) (1998)
Hagen, L., Kahng, A.B.: New spectral methods for ratio cut partitioning and clustering. IEEE Trans. Computer-Aided Design 11(9), 1074–1085 (1992)
Hartigan, J.: Consistency of single linkage for high-density clusters. JASA 76(374), 388–394 (1981)
Hendrickson, B., Leland, R.: An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM J. on Scientific Computing 16, 452–469 (1995)
Kannan, R., Vempala, S., Vetta, A.: On clusterings - good, bad and spectral. Technical report, Computer science Department, Yale University (2000)
Kato, T.: Perturbation theory for linear operators. Springer, Berlin (1966)
Koltchinskii, V.: Asymptotics of spectral projections of some random matrices approximating integral operators. Progress in Probabilty 43 (1998)
Koltchinskii, V., Giné, E.: Random matrix approximation of spectra of integral operators. Bernoulli 6(1), 113–167 (2000)
Meila, M., Shi, J.: A random walks view of spectral segmentation. In: 8th International Workshop on Artificial Intelligence and Statistics (2001)
Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Dietterich, T., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14, MIT Press, Cambridge (2001)
Niyogi, P., Karmarkar, N.K.: An approach to data reduction and clustering with theoretical guarantees. In: Langley, P. (ed.) Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann, San Francisco (2000)
Pollard, D.: Strong consistency of k-means clustering. Annals of Statistics 9(1), 135–140 (1981)
Shawe-Taylor, J., Williams, C., Cristianini, N., Kandola, J.: On the eigenspectrum of the Gram matrix and its relationship to the operator eigenspectrum. In: Cesa- Bianchi, N., Numao, M., Reischuk, R. (eds.) Proceedings of the 13th International Conference on Algorithmic Learning Theory, Springer, Heidelberg (2002)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)
Van Driessche, R., Roose, D.: An improved spectral bisection algorithm and its application to dynamic load balancing. Parallel Comput. 21(1) (1995)
von Luxburg, U., Bousquet, O., Belkin, M.: On the convergence of spectral clustering on random samples: the unnormalized case. Submitted to DAGM (2004), available at http://www.kyb.tuebingen.mpg.de/~ule
Weidmann, J.: Linear Operators in Hilbert spaces. Springer, New York (1980)
Weiss, Y.: Segmentation using eigenvectors: A unifying view. In: Proceedings of the International Conference on Computer Vision, pp. 975–982 (1999)
Williams, C.K.I., Seeger, M.: The effect of the input density distribution on kernel based classifiers. In: Langley, P. (ed.) Proceedings of the 17th International Conference on Machine Learning, pp. 1159–1166. Morgan Kaufmann, San Francisco (2000)
Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using Gaussianfields and harmonic functions. In: Fawcett, T., Mishra, N. (eds.) Proceedings of the 20th International Conference of Machine Learning, AAAI Press, Menlo Park (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
von Luxburg, U., Bousquet, O., Belkin, M. (2004). On the Convergence of Spectral Clustering on Random Samples: The Normalized Case. In: Shawe-Taylor, J., Singer, Y. (eds) Learning Theory. COLT 2004. Lecture Notes in Computer Science(), vol 3120. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27819-1_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-27819-1_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22282-8
Online ISBN: 978-3-540-27819-1
eBook Packages: Springer Book Archive