Dimension Reduction for Mixtures of Exponential Families
Dimension reduction for a set of distribution parameters has been important in various applications of datamining. The exponential family PCA has been proposed for that purpose, but it cannot be directly applied to mixture models that do not belong to an exponential family. This paper proposes a method to apply the exponential family PCA to mixture models. A key idea is to embed mixtures into a space of an exponential family. The problem is that the embedding is not unique, and the dimensionality of parameter space is not constant when the numbers of mixture components are different. The proposed method finds a sub-optimal solution by linear programming formulation.
KeywordsMixture Model Dimension Reduction Exponential Family Latent Variable Model Linear Programming Formulation
Unable to display preview. Download preview PDF.
- 2.Amari, S.: Information Geometry on Hierarchy of Probability Distributions. IEEE Trans. on Information Theory 41 (2001)Google Scholar
- 3.Collins, M., Dasgupta, S., Schapire, R.: A Generalization of Principal Component Analysis to the Exponential Family. In: Advances in NIPS, vol. 14 (2002)Google Scholar
- 4.Akaho, S.: The e-PCA and m-PCA: dimension reduction by information geometry. In: IJCNN 2004, pp. 129–134 (2004)Google Scholar
- 5.Watanabe, K., Akaho, S., Okada, M.: Clustering on a Subspace of Exponential Family Using Variational Bayes Method. In: Proc. of Worldcomp2008/Information Theory and Statistical Learning (2008)Google Scholar
- 7.Agrawal, R., Srikant, R.: Privacy-Preserving Data Mining. In: Proc. of the ACM SIGMOD, pp. 439–450 (2000)Google Scholar
- 14.Fukumizu, K., Akaho, S., Amari, S.: Critical lines in symmetry of mixture models and its application to component splitting. In: Proc. of NIPS15 (2003)Google Scholar