MDAI 2007: Modeling Decisions for Artificial Intelligence pp 261-268 | Cite as
c-Means Clustering on the Multinomial Manifold
Abstract
In this paper, we discuss c-means clustering algorithms on the multinomial manifold. Data forms a Riemannian manifold with the Fisher information metric via the probabilistic mapping from datum to a probability distribution. For discrete data, the statistical manifold of the multinomial distribution is appropriate. In general, The euclidean distance is not appropriate on the manifold because the parameter space of the distribution is not flat. We apply the Kullback-Leibler (KL) divergence or the Hellinger distance as approximations of the geodesic distance to hard c-means and fuzzy c-means.
Keywords
Cluster Center Fisher Information Geodesic Distance Multinomial Distribution Lagrange Multiplier MethodPreview
Unable to display preview. Download preview PDF.
References
- 1.Jebara, T., Kondor, R., Howard, A.: Probability product kernels. J. Mach. Learn. Res. 5, 819–844 (2004)MathSciNetGoogle Scholar
- 2.Amari, S., Nagaoka, H. (eds.): Methods of Information Geometry. Translations of Mathematical monographs, vol. 191. Oxford University Press, Oxford (2000)MATHGoogle Scholar
- 3.Lafferty, J., Lebanon, G.: Diffusion kernels on statistical manifolds. J. Mach. Learn. Res. 6, 129–163 (2005)MathSciNetGoogle Scholar
- 4.Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York, NY, USA (1981)MATHGoogle Scholar
- 5.Miyamoto, S., Mukaidono, M.: Fuzzy c-means as a regularization and maximum entropy approach. In: IFSA 1997. Proc.of the 7th International Fuzzy Systems Association World Congres, vol. 2, pp. 86–92 (1997)Google Scholar
- 6.Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. In: KDD 2003. Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 89–98. ACM Press, New York, NY, USA (2003)CrossRefGoogle Scholar
- 7.Pereira, F., Tishby, N., Lee, L.: Distributional clustering of english words. In: Proceedings of the 31st annual meeting on Association for Computational Linguistics, Morristown, NJ, USA, Association for Computational Linguistics, pp. 183–190 (1993)Google Scholar
- 8.Zhang, D., Chen, X., Lee, W.S.: Text classification with kernels on the multinomial manifold. In: SIGIR 2005. Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 266–273. ACM Press, New York, NY, USA (2005)CrossRefGoogle Scholar