A Unified Framework for Probabilistic Component Analysis
We present a unifying framework which reduces the construction of probabilistic component analysis techniques to a mere selection of the latent neighbourhood, thus providing an elegant and principled framework for creating novel component analysis models as well as constructing probabilistic equivalents of deterministic component analysis methods. Under our framework, we unify many very popular and well-studied component analysis algorithms, such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Locality Preserving Projections (LPP) and Slow Feature Analysis (SFA), some of which have no probabilistic equivalents in literature thus far. We firstly define the Markov Random Fields (MRFs) which encapsulate the latent connectivity of the aforementioned component analysis techniques; subsequently, we show that the projection directions produced by all PCA, LDA, LPP and SFA are also produced by the Maximum Likelihood (ML) solution of a single joint probability density function, composed by selecting one of the defined MRF priors while utilising a simple observation model. Furthermore, we propose novel Expectation Maximization (EM) algorithms, exploiting the proposed joint PDF, while we generalize the proposed methodologies to arbitrary connectivities via parametrizable MRF products. Theoretical analysis and experiments on both simulated and real world data show the usefulness of the proposed framework, by deriving methods which well outperform state-of-the-art equivalents.
KeywordsUnifying Framework Probabilistic Methods Component Analysis Dimensionality Reduction Random Fields
Unable to display preview. Download preview PDF.
- 1.Akisato, K., Masashi, S., Hitoshi, S., Hirokazu, K.: Designing various multivariate analysis at will via generalized pairwise expression. JIP 6(1), 136–145 (2013)Google Scholar
- 3.Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag New York, Inc., Secaucus (2006)Google Scholar
- 4.Borga, M., Landelius, T., Knutsson, H.: A unified approach to PCA, PLS, MLR and CCA (1997)Google Scholar
- 9.Klampfl, S., Maass, W.: Replacing supervised classification learning by slow feature analysis in spiking neural networks. In: Advances in NIPS, pp. 988–996 (2009)Google Scholar
- 12.Martinez, A.M.: The AR face database. CVC Technical Report 24 (1998)Google Scholar
- 13.Niyogi, X.: Locality preserving projections. In: NIPS 2003, vol. 16, p. 153 (2004)Google Scholar
- 14.Prince, S.J.D., Elder, J.H.: Probabilistic linear discriminant analysis for inferences about identity. In: ICCV (2007)Google Scholar
- 16.Roweis, S.: EM algorithms for PCA and SPCA. In: NIPS 1998, pp. 626–632 (1998)Google Scholar
- 19.Rue, H., Held, L.: Gaussian Markov random fields: Theory and applications. CRC Press (2004)Google Scholar
- 20.Sim, T., Baker, S., Bsat, M.: The CMU Pose, Illumination, and Expression Database. In: Proc. of the IEEE FG 2002 (2002)Google Scholar
- 21.Sun, L., Ji, S., Ye, J.: A least squares formulation for a class of generalized eigenvalue problems in machine learning. In: ICML 2009, pp. 977–984. ACM (2009)Google Scholar