Abstract
We provide a probabilistic and infinitesimal view of how the principal component analysis procedure (PCA) can be generalized to analysis of nonlinear manifold valued data. Starting with the probabilistic PCA interpretation of the Euclidean PCA procedure, we show how PCA can be generalized to manifolds in an intrinsic way that does not resort to linearization of the data space. The underlying probability model is constructed by mapping a Euclidean stochastic process to the manifold using stochastic development of Euclidean semimartingales. The construction uses a connection and bundles of covariant tensors to allow global transport of principal eigenvectors, and the model is thereby an example of how principal fiber bundles can be used to handle the lack of global coordinate system and orientations that characterizes manifold valued statistics. We show how curvature implies non-integrability of the equivalent of Euclidean principal subspaces, and how the stochastic flows provide an alternative to explicit construction of such subspaces. We describe estimation procedures for inference of parameters and prediction of principal components, and we give examples of properties of the model on embedded surfaces.
Similar content being viewed by others
References
Arnaudon, A., Holm, D.D. and Sommer, S. (2018). A Geometric Framework for Stochastic Shape Analysis. accepted for Foundations of Computational Mathematics. arXiv:1703.09971 [cs, math].
Delyon, B. and Hu, Y. (2006). Simulation of conditioned diffusion and application to parameter estimation. Stoch. Process. Appl. 116, 11, 1660–1675. https://doi.org/10.1016/j.spa.2006.04.004.
Eltzner, B., Huckemann, S. and Mardia, K.V. (2015). Torus principal component analysis with an application to RNA structures. arXiv:1511.04993 [q-bio, stat].
Elworthy, D. (1988). Geometric aspects of diffusions on manifolds. Springer, Berlin, Hennequin, P. L. (ed.), p. 277–425. http://link.springer.com/chapter/10.1007/BFb0086183.
Fletcher, P., Lu, C., Pizer, S. and Joshi, S. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/TMI.2004.831793.
Frechet, M. (1948). Les éléments aléatoires de nature quelconque dans un espace distancie. Ann. Inst. H. Poincaré, 10, 215–310.
Hsu, E.P. (2002). Stochastic analysis on manifolds american mathematical soc.
Huckemann, S., Hotz, T. and Munk, A. (2010). Intrinsic shape analysis: Geodesic PCA for Riemannian manifolds modulo isometric Lie group actions. Stat. Sin. 20, 1, 1–100.
Jung, S., Dryden, I.L. and Marron, J.S. (2012). Analysis of principal nested spheres. Biometrika 99, 3, 551–568. https://doi.org/10.1093/biomet/ass022.
Kol, I., Slovk, J. and Michor, P.W. (1993). Natural operations in differential geometry. Springer, Berlin. http://link.springer.com/10.1007/978-3-662-02950-3.
Kühnel, L., Arnaudon, A. and Sommer, S. (2017). Differential geometry and stochastic dynamics with deep learning numerics. arXiv:1712.08364 [cs, stat].
Kühnel, L. and Sommer, S. (2017). Stochastic development regression on non-linear manifolds. Springer, Cham, p. 53–64. https://doi.org/10.1007/978-3-319-59050-9_5.
Marchand, J.L. (2011). Conditioning diffusions with respect to partial observations. arXiv:1105.1608 [math].
Mok, K.P. (1978). On the differential geometry of frame bundles of Riemannian manifolds. Journal Fur Die Reine Und Angewandte Mathematik 1978, 302, 16–31. https://doi.org/10.1515/crll.1978.302.16.
Pennec, X. (2016). Barycentric subspace analysis on manifolds. arXiv:1607.02833 [math, stat].
Roweis, S. (1998). EM algorithms for PCA and SPCA. MIT Press, Cambridge, p. 626–632.
Sommer, S. (2013). Horizontal dimensionality reduction and iterated frame bundle development. Springer, p. 76–83.
Sommer, S. (2014). Diffusion Processes and PCA on Manifolds. Mathematisches Forschungsinstitut Oberwolfach https://www.mfo.de/document/1440a/OWR_2014∖_44.pdf.
Sommer, S. (2015). Anisotropic distributions on manifolds: Template estimation and most probable paths, 9123. Springer, p. 193–204.
Sommer, S. (2015). Evolution equations with anisotropic distributions and diffusion PCA. Springer International Publishing, Nielsen, F. and Barbaresco, F. (eds.), p. 3–11. https://doi.org/10.1007/978-3-319-25040-3_1.
Sommer, S. (2016). Anisotropically weighted and nonholonomically constrained evolutions on manifolds. Entropy 18, 12, 425. https://doi.org/10.3390/e18120425.
Sommer, S. (2018). Diffusion bridge simulation on nonlinear manifolds. In Preparation.
Sommer, S., Arnaudon, A., Kuhnel, L. and Joshi, S. (2017). Bridge simulation and metric estimation on landmark manifolds. Springer, p. 79–91. https://doi.org/10.1007/978-3-319-67675-3_8.
Sommer, S. and Svane, A.M. (2017). Modelling anisotropic covariance using stochastic development and sub-Riemannian frame bundle geometry. Journal of Geometric Mechanics 9, 3, 391–410. https://doi.org/10.3934/jgm.2017015.
Team, T.T.D. (2016). Theano: a python framework for fast computation of mathematical expressions. arXiv:1605.02688 [cs].
Tipping, M.E. and Bishop, C.M. (1999). Probabilistic principal component analysis. Journal of the Royal Statistical Society. Series B 61, 3, 611–622.
Zhang, M. and Fletcher, P. (2013). Probabilistic principal geodesic analysis, p. 1178–1186.
Acknowledgments
The work was supported by the Danish Council for Independent Research, and the CSGB Centre for Stochastic Geometry and Advanced Bioimaging funded by a grant from the Villum foundation. The research was partially performed at the Mathematisches Forschungsinstitut Oberwolfach (MFO), 2014 and 2018.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sommer, S. An Infinitesimal Probabilistic Model for Principal Component Analysis of Manifold Valued Data. Sankhya A 81, 37–62 (2019). https://doi.org/10.1007/s13171-018-0139-5
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13171-018-0139-5
Keywords
- Principal component analysis
- Manifold valued statistics
- Stochastic development
- Probabilistic PCA
- Anisotropic normal distributions
- Frame bundle