Advertisement

Generalized Clustering via Kernel Embeddings

  • Stefanie Jegelka
  • Arthur Gretton
  • Bernhard Schölkopf
  • Bharath K. Sriperumbudur
  • Ulrike von Luxburg
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5803)

Abstract

We generalize traditional goals of clustering towards distinguishing components in a non-parametric mixture model. The clusters are not necessarily based on point locations, but on higher order criteria. This framework can be implemented by embedding probability distributions in a Hilbert space. The corresponding clustering objective is very general and relates to a range of common clustering concepts.

Keywords

Generalize Entropy Spectral Cluster Reproduce Kernel Hilbert Space Cluster Assignment Maximum Mean Discrepancy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aghagolzadeh, M., Soltanian-Zadeh, H., Araabi, B., Aghagolzadeh, A.: A hierarchical clustering based on maximizing mutual information. In: ICIP (2007)Google Scholar
  2. Arora, S., Kannan, R.: Learning mixtures of arbitrary Gaussians. In: STOC (2001)Google Scholar
  3. Chaudhuri, K., Rao, S.: Beyond Gaussians: Spectral methods for learning mixtures of heavy-tailed distributions. In: COLT (2008)Google Scholar
  4. Erdogmus, D., Principe, J.C.: From linear adaptive filtering to nonlinear information processing. IEEE Signal Processing Magazine 23(6), 14–33 (2006)CrossRefGoogle Scholar
  5. Fuglede, B., Topsøe, F.: Jensen-Shannon divergence and Hilbert space embedding. In: Proc. of the Int. Symp. on Information Theory, ISIT (2004)Google Scholar
  6. Gokcay, E., Principe, J.C.: Information theoretic clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(2), 158–170 (2002)CrossRefGoogle Scholar
  7. Gretton, A., Bousquet, O., Smola, A., Schölkopf, B.: Measuring statistical dependence with Hilbert-Schmidt norms. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS (LNAI), vol. 3734, pp. 63–77. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. Gretton, A., Borgwardt, K.M., Rasch, M., Schölkopf, B., Smola, A.: A kernel method for the two-sample problem. In: NIPS (2006)Google Scholar
  9. Gretton, A., Borgwardt, K., Rasch, M., Schölkopf, B., Smola, A.: A kernel method for the two-sample problem. Tech. Report 157, MPI for Biol. Cyb. (2008)Google Scholar
  10. Jenssen, R., Erdogmus, D., Principe, J., Eltoft, T.: The laplacian pdf distance: a cost function for clustering in a kernel feature space. In: NIPS (2004)Google Scholar
  11. Lin, J.: Divergence measures based on the Shannon entropy. IEEE Transaction on Information Theory 37(1) (1991)Google Scholar
  12. Newman, M.E.J., Girvan, M.: Finding and evaluating community structure in networks. Physical review E 69 (2004)Google Scholar
  13. Renyi, A.: On measures of entropy and information. In: Proc. Fourth Berkeley Symp. Math., Statistics and Probability, pp. 547–561 (1960)Google Scholar
  14. Rose, K.: A mapping approach to rate-distortion computation and analysis. IEEE Transactions on Information Theory 40(6), 1939–1952 (1994)CrossRefzbMATHGoogle Scholar
  15. Song, L., Smola, A., Gretton, A., Borgwardt, K.: A dependence maximization view of clustering. In: 24th International Conference on Machine Learning, ICML (2007)Google Scholar
  16. Sriperumbudur, B.K., Gretton, A., Fukumizu, K., Lanckriet, G., Schölkopf, B.: Injective Hilbert space embeddings of probability measures. In: COLT (2008)Google Scholar
  17. Wächter, A., Biegler, L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Mathematical Programming 106(1), 25–57 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  18. Xu, L., Neufeld, J., Larson, B., Schuurmans, D.: Maximum margin clustering. In: NIPS (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Stefanie Jegelka
    • 1
  • Arthur Gretton
    • 1
    • 2
  • Bernhard Schölkopf
    • 1
  • Bharath K. Sriperumbudur
    • 3
  • Ulrike von Luxburg
    • 1
  1. 1.Max Planck Institute for Biological CyberneticsTübingenGermany
  2. 2.Carnegie Mellon UniversityPittsburghUSA
  3. 3.Dept. of ECEUC San DiegoLa JollaUSA

Personalised recommendations