Abstract
The Grassmann manifold of linear subspaces is important for the mathematical modelling of a multitude of applications, ranging from problems in machine learning, computer vision and image processing to low-rank matrix optimization problems, dynamic low-rank decompositions and model reduction. With this mostly expository work, we aim to provide a collection of the essential facts and formulae on the geometry of the Grassmann manifold in a fashion that is fit for tackling the aforementioned problems with matrix-based algorithms. Moreover, we expose the Grassmann geometry both from the approach of representing subspaces with orthogonal projectors and when viewed as a quotient space of the orthogonal group, where subspaces are identified as equivalence classes of (orthogonal) bases. This bridges the associated research tracks and allows for an easy transition between these two approaches. Original contributions include a modified algorithm for computing the Riemannian logarithm map on the Grassmannian that is advantageous numerically but also allows for a more elementary, yet more complete description of the cut locus and the conjugate points. We also derive a formula for parallel transport along geodesics in the orthogonal projector perspective, formulae for the derivative of the exponential map, as well as a formula for Jacobi fields vanishing at one point.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Change history
22 March 2024
A Correction to this paper has been published: https://doi.org/10.1007/s10444-024-10115-w
Notes
In these formulae, \([X,Y] = X(Y)-Y(X)\) is the Lie bracket of two vector fields.
References
Absil, P.-A., Edelman, A., Koev, P.: On the largest principal angle between random subspaces. Linear Algebra Appl. 414(1), 288–294 (2006). https://doi.org/10.1016/j.laa.2005.10.004
Absil, P.-A., Mahony, R., Sepulchre, R.: Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Applicandae Mathematica 80(2), 199–220 (2004). https://doi.org/10.1023/B:ACAP.0000013855.14971.91
Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization algorithms on matrix manifolds. Princeton University Press, Princeton, New Jersey (2008). http://press.princeton.edu/titles/8586.html
Afsari, B., Tron, R., Vidal, R.: On the convergence of gradient descent for finding the Riemannian center of mass. SIAM J. Control. Optim. 51(3), 2230–2260 (2013). https://doi.org/10.1137/12086282X
Alexandrino, M.M., Bettiol, R.G.: Lie groups and geometric aspects of isometric actions. Springer International Publishing, Cham (2015). https://doi.org/10.1007/978-3-319-16613-1_2
Alimisis, F., Orvieto, A., Becigneul, G., Lucchi, A.: Momentum improves optimization on Riemannian manifolds. In: Banerjee, A., Fukumizu, K., (eds.) Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, of Proceedings of Machine Learning Research, vol. 130, pp. 1351–1359. PMLR. Accessed 13–15 Apr 2021
Alimisis, F., Vandereycken, B.: Geodesic convexity of the symmetric eigenvalue problem and convergence of Riemannian steepest descent (2023). arXiv:2209.03480
Amsallem, D., Farhat, C.: Interpolation method for adapting reduced-order models and application to aeroelasticity. AIAA J. 46(7), 1803–1813 (2008). https://doi.org/10.2514/1.35374
Balzano, L., Wright, S.J.: Local convergence of an algorithm for subspace identification from partial data. Found. Comput. Math. 15(5), 1279–1314 (2015). https://doi.org/10.1007/s10208-014-9227-7
Batzies, E., Hüper, K., Machado, L., Silva Leite, F.: Geometric mean and geodesic regression on Grassmannians. Linear Algebra Appl. 466, 83–101 (2015). https://doi.org/10.1016/j.laa.2014.10.003
Berceanu, S.: On the geometry of complex Grassmann manifold, its noncompact dual and coherent states. Bull. Belg. Math. Soc. Simon Stevin 4(2), 205–243 (1997). https://doi.org/10.36045/bbms/1105731655
Bergmann, R., Gousenbourger, P.-Y.: A variational model for data fitting on manifolds by minimizing the acceleration of a Bézier curve. Frontiers in Applied Mathematics and Statistics 4, 59 (2018). https://doi.org/10.3389/fams.2018.00059
Bishop, R.L.: Decomposition of cut loci. Proc. Amer. Math. Soc. 65(1), 133–136 (1977). https://doi.org/10.2307/2042008
Borisenko, A.A., Nikolaevskiĭ, Yu.A.: Grassmann manifolds and Grassmann image of submanifolds. Uspekhi Mat. Nauk 46(2(278)), 41–83, 240 (1991). https://doi.org/10.1070/RM1991v046n02ABEH002742
Boumal, N.: Optimization and estimation on manifolds. PhD thesis, Université Catholique de Louvain (2014)
Boumal, N., Absil, P.-A.: Low-rank matrix completion via preconditioned optimization on the Grassmann manifold. Linear Algebra Appl. 475, 200–239 (2015). https://doi.org/10.1016/j.laa.2015.02.027
Chakraborty, R., Vemuri, B.C.: Statistics on the Stiefel manifold: theory and applications. Ann. Stat. 47(1), 415–438 (2019). https://doi.org/10.1214/18-AOS1692
Criscitiello, C., Boumal, N.: Curvature and complexity: better lower bounds for geodesically convex optimization (2023). arXiv:2306.02959
Dieci, L., Eirola, T.: On smooth decompositions of matrices. SIAM J. Matrix Anal. Appl. 20(3), 800–819 (1999). https://doi.org/10.1137/S0895479897330182
do Carmo, M.P.: Riemannian geometry. Mathematics: Theory & Applications. Birkhäuser Boston (1992)
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998). https://doi.org/10.1137/S0895479895290954
Fletcher, P.T.: Geodesic regression and the theory of least squares on Riemannian manifolds. Int. J. Comput. Vision 105(2), 171–185 (2013). https://doi.org/10.1007/s11263-012-0591-y
Gallier, J.H.: Geometric methods and applications: for computer science and engineering. Texts in Applied Mathematics. Springer, New York (2011). https://doi.org/10.1007/978-1-4419-9961-0
Gallivan, K.A., Srivastava, A., Liu, X., Van Dooren, P.: Efficient algorithms for inferences on Grassmann manifolds. In: IEEE workshop on statistical signal processing, pp. 315–318 (2003). https://doi.org/10.1109/SSP.2003.1289408
Gillespie, M.: Variations on a theme of Schubert calculus, pp. 115–158. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-05141-9_4
Godement, R., Ray, U.: Introduction to the theory of lie groups. Universitext. Springer International Publishing (2017). https://doi.org/10.1007/978-3-319-54375-8
Gousenbourger, P.-Y., Massart, E., Absil, P.-A.: Data fitting on manifolds with composite bézier-like curves and blended cubic splines. Journal of Mathematical Imaging and Vision 61(5), 645–671 (2019)
Hairer, E., Lubich, C., Wanner, G.: Geometric numerical integration, of Springer Series in Computational Mathematics, vol. 31, 2nd edn. Springer-Verlag, Berlin (2006). https://doi.org/10.1007/3-540-30666-8
Hall, B.C.: Lie groups, lie algebras, and representations: an elementary introduction. Springer Graduate texts in Mathematics. Springer–Verlag, New York – Berlin – Heidelberg, 2nd edn. (2015). https://doi.org/10.1007/978-3-319-13467-3
Hay, A., Borggaard, J.T., Pelletier, D.: Local improvements to reduced-order models using sensitivity analysis of the proper orthogonal decomposition. J. Fluid Mech. 629, 41–72 (2009). https://doi.org/10.1017/S0022112009006363
Helgason, S.: Differential geometry, lie groups, and symmetric spaces. Crm Proceedings & Lecture Notes. American Mathematical Society (2001)
Helmke, U., Hüper, K., Trumpf, J.: Newton’s method on Graßmann manifolds (2007). arXiv:0709.2205
Helmke, U., Moore, J.B.: Optimization and dynamical systems. Communications & Control Engineering. Springer–Verlag, London (1994). https://doi.org/10.1007/978-1-4471-3467-1
Higham, N.J.: Functions of matrices: theory and computation. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (2008). https://doi.org/10.1137/1.9780898717778
Horn, R.A., Johnson, C.R.: Topics in matrix analysis. Cambridge University Press, Cambridge (1991). https://doi.org/10.1017/CBO9780511840371
Kobayashi, S., Nomizu, K.: Foundations of differential geometry. vol. I & II. Wiley Classics Library. John Wiley & Sons, Inc., New York (1996). Reprint of the 1963 original
Koch, O., Lubich, C.: Dynamical low-rank approximation. SIAM J. Matrix Analysis Applications 29, 434–454 (2007). https://doi.org/10.1137/050639703
Kozlov, E.S.: Geometry of real Grassmann manifolds. Part I - VI. Journal of Mathematical Sciences (2000 – 2001)
Lai, Z., Lim, L.-H., Ye, K.: Simpler Grassmannian optimization (2020). arXiv:2009.13502
Lee, J.M.: Introduction to smooth manifolds. Graduate Texts in Mathematics. Springer New York (2012). https://doi.org/10.1007/978-1-4419-9982-5
Lee, J.M.: Introduction to Riemannian manifolds, of Graduate Texts in Mathematics, vol. 176. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91755-9
Leichtweiss, K.: Zur Riemannschen Geometrie in Grassmannschen Mannigfaltigkeiten. Math. Z. 76, 334–366 (1961). https://doi.org/10.1007/BF01210982
Li, C., Wang, X., Wang, J., Yao, J.-C.: Convergence analysis of gradient algorithms on Riemannian manifolds without curvature constraints and application to Riemannian mass (2019). arXiv:1910.02280
Machado, A., Salavessa, I.: Grassmannian manifolds as subsets of Euclidean spaces. In: Differential geometry (Santiago de Compostela, 1984), of Res. Notes in Math., vol. 131, pp. 85–102. Pitman, Boston, MA (1985)
Man Lui, Y.: Advances in matrix manifolds for computer vision. Image Vis. Comput. 30(6–7), 380–388 (2012). https://doi.org/10.1016/j.imavis.2011.08.002
Minh, H.Q., Murino, V.: Algorithmic advances in Riemannian geometry and applications: for machine learning, computer vision, statistics, and optimization. Advances in Computer Vision and Pattern Recognition. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-45026-1
Nguyen, T.S.: A real time procedure for affinely dependent parametric model order reduction using interpolation on Grassmann manifolds. Int. J. Numer. Meth. Eng. 93(8), 818–833 (2013). https://doi.org/10.1002/nme.4408
Nguyen, T.S., Stykel, T.: Model order reduction of parameterized circuit equations based on interpolation. Adv. Comput. Math. 41, 1321–1342 (2015). https://doi.org/10.1007/s10444-015-9418-z
O’Neill, B.: Semi-Riemannian geometry - with applications to relativity, of Pure and Applied Mathematics, vol. 103. Academic Press, New York (1983)
Qiu, L., Zhang, Y., Li, C.-K.: Unitarily invariant metrics on the Grassmann space. SIAM J. Matrix Anal. Appl. 27(2), 507–25 (2005). https://doi.org/10.1137/040607605
Rahman, I.U., Drori, I., Stodden, V.C., Donoho, D.L., Schröder, P.: Multiscale representations for manifold-valued data. SIAM Journal on Multiscale Modeling and Simulation 4(4), 1201–1232 (2005). https://doi.org/10.1137/050622729
Rentmeesters, Q.: Algorithms for data fitting on some common homogeneous spaces. PhD thesis, Université Catholique de Louvain, Louvain, Belgium (2013). http://hdl.handle.net/2078.1/132587
Saad, Y.: Numerical methods for large eigenvalue problems. Algortihms and Architectures for Advanced Scientific Computing. Manchester University Press, Manchester, UK (1992)
Sakai, T.: On cut loci of compact symmetric spaces. Hokkaido Math. J. 6(1), 136–161 (1977). https://doi.org/10.14492/hokmj/1381758555
Sakai, T.: Riemannian Geometry. Fields Institute Communications. American Mathematical Soc. (1996). https://doi.org/10.1090/mmono/149
Srivastava, A., Liu, X.: Tools for application-driven linear dimension reduction. Neurocomputing 67, 136 – 160 (2005). Geometrical Methods in Neural Networks and Learning. https://doi.org/10.1016/j.neucom.2004.11.036
Srivastava, A., Turaga, P.K.: Riemannian computing in computer vision. Springer International Publishing (2015). https://doi.org/10.1007/978-3-319-22957-7
Usevich, K., Markovsky, I.: Optimization on a Grassmann manifold with application to system identification. Automatica 50(6), 1656–1662 (2014). https://doi.org/10.1016/j.automatica.2014.04.010
Walter, S.F., Lehmann, L., Lamour, R.: On evaluating higher-order derivatives of the QR decomposition of tall matrices with full column rank in forward and reverse mode algorithmic differentiation. Optimization Methods and Software 27(2), 391–403 (2012). https://doi.org/10.1080/10556788.2011.610454
Wong, Y.-C.: Differential geometry of Grassmann manifolds. Proc. Natl. Acad. Sci. U.S.A. 57, 589–594 (1967). https://doi.org/10.1073/pnas.57.3.589
Wong, Y.-C.: Conjugate loci in Grassmann manifolds. Bull. Amer. Math. Soc. 74, 240–245 (1968). https://doi.org/10.1090/S0002-9904-1968-11903-2
Wong, Y.-C.: Sectional curvatures of Grassmann manifolds. Proc. Natl. Acad. Sci. U.S.A. 60(1), 75–79 (1968). https://doi.org/10.1073/pnas.60.1.75
Wu, G.L., Chen, W.H.: A matrix inequality and its geometric applications. Acta Math. Sinica 31(3), 348–355 (1988)
Ye, K., Lim, L.-H.: Schubert varieties and distances between subspaces of different dimensions. SIAM J. Matrix Anal. Appl. 37(3), 1176–1197 (2016). https://doi.org/10.1137/15M1054201
Zhang, D., Balzano, L.: Global convergence of a Grassmannian gradient descent algorithm for subspace estimation. Proceedings of the 19th international conference on artificial intelligence and statistics, AISTATS, pp. 1460–1468. Cadiz, Spain (2016)
Zhang, H., Sra, S.: First-order methods for geodesically convex optimization. In: Feldman, V., Rakhlin, A., Shamir, O. (eds.), 29th Annual Conference on Learning Theory, of Proceedings of Machine Learning Research, vol. 49, pp. 1617–1638, Columbia University, New York, USA. PMLR. Accessed 23–26 Jun 2016 (2016)
Zimmermann, R.: A locally parametrized reduced order model for the linear frequency domain approach to time-accurate computational fluid dynamics. SIAM J. Sci. Comput. 36(3), B508–B537 (2014). https://doi.org/10.1137/130942462
Zimmermann, R., Peherstorfer, B., Willcox, K.: Geometric subspace updates with applications to online adaptive nonlinear model reduction. SIAM Journal on Matrix Analysis and Application 39(1), 234–261 (2018). https://doi.org/10.1137/17M1123286
Zimmermann, R.: Manifold interpolation. In: Benner, P., Grivet-Talocia, S., Quarteroni, A., Rozza, G., Schilders, W., Silveira, L.M. (eds.), vol. 1 system- and data-driven methods and algorithms, pp. 229–274. De Gruyter, Berlin, Boston (2021). https://doi.org/10.1515/9783110498967-007
Zimmermann, R.: Hermite interpolation and data processing errors on Riemannian matrix manifolds. SIAM J. Sci. Comput. 42(5), A2593–A2619 (2020). https://doi.org/10.1137/19M1282878
Acknowledgements
This work was initiated when the first author was at UCLouvain for a research visit, hosted by the third author.
Funding
Open access funding provided by University of Southern Denmark. The third author was supported by the Fonds de la Recherche Scientifique – FNRS and the Fonds Wetenschappelijk Onderzoek – Vlaanderen under EOS Project no 30468160.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by: Felix Krahmer
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised: There is a mixed up in the presentation of the algorithms.
Appendices
Basics from Riemannian geometry
For the reader’s convenience, we recap some fundamentals from Riemannian geometry. Concise introductions can be found in [33, Appendices C.3, C.4, C.5], [23] and [3]. For an in-depth treatment, see for example [20, 36, 41].
An n -dimensional differentiable manifold \(\mathcal {M}\) is a topological space \(\mathcal {M}\) such that for every point \(p\in \mathcal {M}\), there exists a so-called coordinate chart \(x:\mathcal {M}\supset \mathcal {D}_p \rightarrow \mathbb {R}^n\) that bijectively maps an open neighborhood \(\mathcal {D}_p\subset \mathcal {M}\) of a location p to an open neighborhood \(D_{x(p)}\subset \mathbb {R}^n\) around \(x(p)\in \mathbb {R}^n\) with the additional property that the coordinate change
of two such charts \(x,\tilde{x}\) is a diffeomorphism, where their domains of definition overlap, see [23, Fig. 18.2, p. 496]. This enables to transfer the most essential tools from calculus to manifolds. An n-dimensional submanifold of \(\mathbb {R}^{n+d}\) is a subset \(\mathcal {M}\subset \mathbb {R}^{n+d}\) that can be locally smoothly straightened, i.e., satisfies the local n-slice condition [40, Thm. 5.8].
Theorem 5.4
([23, Prop. 18.7, p. 500]) Let \(h: \mathbb {R}^{n+d}\supset \Omega \rightarrow \mathbb {R}^{d}\) be differentiable and \(c_0\in \mathbb {R}^d\) be defined such that the differential \(Dh_p\in \mathbb {R}^{d\times (n+d)}\) has maximum possible rank d at every point \(p\in \Omega \) with \(h(p) = c_0\). Then, the preimage
is an n-dimensional submanifold of \(\mathbb {R}^{n+d}\).
This theorem establishes the Stiefel manifold \(\textrm{St}(n,p) = \left\{ U\in \mathbb {R}^{n\times p}\ \big | \ U^TU=I\right\} \) as an embedded submanifold of \(\mathbb {R}^{n\times p}\), since \(\textrm{St}(n,p) = F^{-1}(I)\) for \(F:U\mapsto U^TU\).
Tangent spaces
The tangent space of a submanifold \(\mathcal {M}\) at a point \(p\in \mathcal {M}\), in symbols \(T_p\mathcal {M}\), is the space of velocity vectors of differentiable curves \(c:t \mapsto c(t)\) passing through p, i.e.,
The tangent space is a vector space of the same dimension n as the manifold \(\mathcal {M}\).
Geodesics and the Riemannian distance function
Riemannian metrics measure the lengths and angles between tangent vectors. Eventually, this allows to measure the lengths of curves on a manifold and the Riemannian distance between two manifold locations.
A Riemannian metric on \(\mathcal {M}\) is a family \((g_p(\cdot ,\cdot ))_{p\in \mathcal {M}}\) of inner products \(g_p(\cdot ,\cdot ): T_p\mathcal {M}\times T_p\mathcal {M}\rightarrow \mathbb {R}\) that is smooth in variations of the base point p, or more precisely, a smooth covariant 2-tensor field, c.f. [41, Chapter 2]. The length of a tangent vector \(v\in T_p\mathcal {M}\) is \(\Vert v\Vert _p := \sqrt{g_p(v,v)}\). The length of a curve \(c:[a,b] \rightarrow \mathcal {M}\) is defined as
A curve is said to be parameterized by the arc length, if \(L(c|_{[a,t]}) = t-a\) for all \(t\in [a,b]\). Obviously, unit-speed curves with \(\Vert \dot{c}(t)\Vert _{c(t)}\equiv 1\) are parameterized by the arc length. Constant-speed curves with \(\Vert \dot{c}(t)\Vert _{c(t)}\equiv \nu _0\) are parameterized proportional to the arc length. The Riemannian distance between two points \(p,q\in \mathcal {M}\) with respect to a given metric is
where, by convention, \(\inf \{\emptyset \} =\infty \). A shortest path between \(p,q\in \mathcal {M}\) is a curve c that connects p and q such that \(L(c) = {{\,\textrm{dist}\,}}_{\mathcal {M}}(p,q)\). Candidates for shortest curves between points are called geodesics and are characterized by a differential equation: A differentiable curve \(c:[a,b]\rightarrow \mathcal {M}\) is a geodesic (w.r.t. to a given Riemannian metric), if the covariant derivative of its velocity vector field vanishes, i.e.,
Intuitively, the covariant derivative can be thought of as the standard derivative (if it exists) followed by a point-wise projection onto the tangent space. In general, a covariant derivative, also known as a linear connection, is a bilinear mapping \((X,Y) \mapsto \nabla _XY\) that maps two vector fields X, Y to a third vector field \(\nabla _XY\) in such a way that it can be interpreted as the directional derivative of Y in the direction of X, [41, §4, §5]. Of importance is the Riemannian connection or Levi-Civita connection that is compatible with a Riemannian metric [3, Thm 5.3.1], [41, Thm 5.10]. It is determined uniquely by the Koszul formula
and is used to define the Riemannian curvature tensorFootnote 1
A Riemannian manifold is flat if and only if it is locally isometric to the Euclidean space, which holds if and only if the Riemannian curvature tensor vanishes identically [41, Thm. 7.10].
Lie groups and orbits
A Lie group is a smooth manifold that is also a group with smooth multiplication and inversion. A matrix Lie group G is a subgroup of the general linear group \(GL(n,\mathbb {C})\) that is closed in \(GL(n,\mathbb {C})\) (but not necessarily in the ambient space \(\mathbb {C}^{n\times n}\)). Basic examples include \(GL(n,\mathbb {R})\) and the orthogonal group \(\textrm{O}(n)\). Any matrix Lie group G is automatically an embedded submanifold of \(\mathbb {C}^{n\times n}\) [29, Corollary 3.45]. The tangent space \(T_IG\) of G at the identity \(I\in G\) has a special role. When endowed with the bracket operator or matrix commutator \([V,W] = VW-WV\) for \(V,W \in T_IG\), the tangent space becomes an algebra, called the Lie algebra associated with the Lie group G, see [29, §3]. As such, it is denoted by \(\mathfrak {g} = T_IG\). For any \(A\in G\), the function “left-multiplication with A” is a diffeomorphism \(L_A:G\rightarrow G,\ L_A(B) = AB\); its differential at a point \(B\in G\) is the isomorphism \(\textrm{d}(L_A)_B:T_BG\rightarrow T_{L_A(B)}G,\ \textrm{d}(L_A)_B(V) = AV\). Using this observation at \(B=I\) shows that the tangent space at an arbitrary location \(A\in G\) is given by the translates (by left-multiplication) of the tangent space at the identity [26, §5.6, p. 160],
A smooth left action of a Lie group G on a manifold M is a smooth map \(\phi :G \times M \rightarrow M\) fulfilling \(\phi (g_1,\phi (g_2,p))=\phi (g_1g_2,p)\) and \(\phi (e,p)=p\) for all \(g_1,g_2 \in G\) and all \(p \in M\), where \(e \in G\) denotes the identity element. One often writes \(\phi (g,p) = g \cdot p\). For each \(p \in M\), the orbit of p is defined as
and the stabilizer of p is defined as
For a detailed introduction see for example [40, Chapters 7 & 21]. We need the following well known result, see for example [33, Section 2.1], where the quotient manifold \(G/G_p\) refers to the set \(\{g G_p \mid g \in G\}\) endowed with the unique manifold structure that turns the quotient map \(g\mapsto g G_p\) into a submersion.
Proposition 2.1
Let G be a compact Lie group acting smoothly on a manifold M. Then for any \(p \in M\), the orbit \(G \cdot p\) is an embedded submanifold of M that is diffeomorphic to the quotient manifold \(G/G_p\).
Proof
The continuous action of a compact Lie group is always proper, [40, Corollary 21.6]. Therefore [5, Proposition 3.41] shows the claim. \(\square \)
B Matrix analysis necessities
Throughout, we consider the matrix space \(\mathbb {R}^{m\times n}\) as a Euclidean vector space with the standard metric
Unless noted otherwise, the singular value decomposition (SVD) of a matrix \(X\in \mathbb {R}^{m\times n}\) is understood to be the compact SVD
The SVD is not unique.
Proposition 3.1
(Ambiguity of the Singular Value Decomposition)[35, Theorem 3.1.1’] Let \(X \in \mathbb {R}^{m \times n}\) have a (full) SVD \(X=U \Sigma V^T\) with singular values in descending order and \({{\,\textrm{rank}\,}}(X)=r\). Let \(\sigma _1>\dots>\sigma _k>0\) be the distinct nonzero singular values with respective multiplicity \(\mu _1,\dots ,\mu _k\). Then \(X=\tilde{U}\Sigma \tilde{V}^T\) is another SVD if and only if \(\tilde{U}=U{{\,\textrm{diag}\,}}(D_1,\dots ,D_k,W_1)\) and \(\tilde{V}=V{{\,\textrm{diag}\,}}(D_1,\dots ,D_k,W_2)\), with \(D_i \in \textrm{O}(\mu _i)\), \(W_1 \in \textrm{O}(m-r)\), and \(W_2 \in \textrm{O}(n-r)\) arbitrary.
Differentiating the singular value decomposition
Let \(p\le n\in \mathbb {N}\) and suppose that \(t \mapsto Y(t)\in \mathbb {R}^{n\times p}\) is a differentiable matrix curve around \(t_0\in \mathbb {R}\). If the singular values of \(Y(t_0)\) are mutually distinct and non-zero, then the singular values and both the left and the right singular vectors depend differentiable on \(t \in [t_0 -\delta t, t_0+\delta t]\) for \(\delta t\) small enough.
Let \( t \mapsto Y(t) = U( t)\Sigma ( t) V( t)^T \in \mathbb {R}^{n\times p}\), where \(U(t)\in \textrm{St}(n,p)\), \(V( t)\in O(p)\) and \(\Sigma ( t)\in \mathbb {R}^{p\times p}\) diagonal and positive definite. Let \(u_j\) and \(v_j\), \(j=1,\ldots ,p\) denote the columns of \(U( t_0)\) and \(V( t_0)\), respectively. For brevity, write \(Y = Y(t_0), \dot{Y} = \frac{\textrm{d}}{\textrm{d}t}\big \vert _{t=t_0}Y(t)\), likewise for the other matrices that feature in the SVD. The derivatives of the matrix factors of the SVD can be calculated with Algorithm 2. A proof can for example be found in [19, 30].
Differentiating the QR-decomposition
Let \(t\mapsto Y(t)\in \mathbb {R}^{n\times r}\) be a differentiable matrix function with Taylor expansion \(Y(t_0+ h) = Y(t_0) + h \dot{Y}(t_0) + \mathcal {O}(h^2)\). Following [59, Proposition 2.2], the QR-decomposition is characterized via the following set of matrix equations.
In the latter, \(P_L = \small \begin{pmatrix} 0 &{} \cdots &{}\cdots &{} 0\\ 1 &{} \ddots &{} &{} \vdots \\ \vdots &{} \ddots &{}\ddots &{}\vdots \\ 1 &{} \cdots &{}1 &{} 0 \end{pmatrix}\) and ‘\(\odot \)’ is the element-wise matrix product so that \(P_L\odot R\) selects the strictly lower triangle of the square matrix R. For brevity, we write \(Y= Y(t_0),\ \dot{Y} = \frac{\textrm{d}}{\textrm{d}t}\big \vert _{t=t_0}Y(t)\), likewise for Q(t), R(t). By the product rule
According to [59, Proposition 2.2], the derivatives \(\dot{Q}, \dot{R}\) can be obtained from Algorithm 3. The trick is to compute \(X = Q^T\dot{Q}\) first and then use this to compute \(\dot{Q} = QQ^T\dot{Q} + (I_n-QQ^T)\dot{Q}\) by exploiting that \(Q^T\dot{Q}\) is skew-symmetric and that \(\dot{R}R^{-1}\) is upper triangular.
Differentiating the QR-decomposition, [59, Proposition 2.2].
Matrix exponential and the principal matrix logarithm
The matrix exponential and the principal matrix logarithm are defined by
The latter is well-defined for matrices that have no eigenvalues on \(\mathbb {R}^-\).
C Computational complexity
For the benefit of the reader, we include Table 1 of the floating point operation (FLOP) counts of some of the most commonly used formulas in this handbook. Note that the FLOP count of the SVD and other operations depends on the specific implementation. Furthermore, we counted \(\sin (\cdot )\), \(\cos (\cdot )\), \(\sqrt{\cdot }\) etc. for scalars as one flop for simplicity.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bendokat, T., Zimmermann, R. & Absil, PA. A Grassmann manifold handbook: basic geometry and computational aspects. Adv Comput Math 50, 6 (2024). https://doi.org/10.1007/s10444-023-10090-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10444-023-10090-8
Keywords
- Grassmann manifold
- Stiefel manifold
- Orthogonal group
- Riemannian exponential
- Geodesic
- Riemannian logarithm
- Cut locus
- Conjugate locus
- Curvature
- Parallel transport
- Quotient manifold
- Horizontal lift
- Subspace
- Singular value decomposition