Abstract
We provide two closed-form geodesic formulas for a family of metrics on Stiefel manifolds recently introduced by Hüper, Markina and Silva Leite, reparameterized by two positive numbers, having both the embedded and canonical metrics as special cases. The closed-form formulas allow us to compute geodesics by matrix exponential in reduced dimension for low-rank Stiefel manifolds. We follow the approach of minimizing the square Frobenius distance between a geodesic ending point to a given point on the manifold to compute the logarithm map and geodesic distance between two endpoints, using Fréchet derivatives to compute the gradient of this objective function. We focus on two optimization methods, gradient descent and L-BFGS. This leads to a new framework to compute the geodesic distance for manifolds with known geodesic formula but no closed-form logarithm map. We show the approach works well for Stiefel as well as flag manifolds. The logarithm map could be used to compute the Riemannian center of mass for these manifolds equipped with the above metrics. The method to translate directional derivatives using Fréchet derivatives to a gradient could potentially be applied to other matrix equations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Al-Mohy, A.H., Higham, N.J.: Computing the Fréchet derivative of the matrix exponential, with an application to condition number estimation. SIAM J. Matrix Anal. Appl. 30(4), 1639–1657 (2009). https://doi.org/10.1137/080716426
Bryner, D.: Endpoint geodesics on the Stiefel manifold embedded in Euclidean space. SIAM J. Matrix Anal. Appl. 38(4), 1139–1159 (2017). https://doi.org/10.1137/16M1103099
Chakraborty, R., Vemuri, B.C.: Statistics on the Stiefel manifold: theory and applications. Ann. Stat. 47(1), 415–438 (2019). https://doi.org/10.1214/18-AOS1692
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1999). https://doi.org/10.1137/S0895479895290954
Gallier, J., Quaintance, J.: Differential Geometry and Lie Groups, a Computational Perspective. Springer, Berlin (2020)
Helmke, U., Moore, J.: Optimization and Dynamical Systems. Springer, London (1994)
Hüper, K., Markina, I., Silva Leite, F.: A Lagrangian approach to extremal curves on Stiefel manifolds. J. Geom. Mech. (2020). https://doi.org/10.3934/jgm.2020031
Karcher, H.: Riemannian center of mass and mollifier smoothing. Commun. Pure Appl. Math. (1977). https://doi.org/10.1002/cpa.3160300502
Ma, X., Kirby, M., Peterson, C.: Self-organizing mappings on the flag manifold with applications to hyper-spectral image data analysis. Neural Comput. Appl. (2021). https://doi.org/10.1007/s00521-020-05579-y
Mathias, R.: A chain rule for matrix functions and applications. SIAM J. Matrix Anal. Appl. 17, 610–620 (1996). https://doi.org/10.1137/S0895479895283409
Miolane, N., Brigant, A.L., Mathe, J., Hou, B., Guigui, N., Thanwerdas, Y., Heyder, S., Peltre, O., Koep, N., Zaatiti, H., Hajri, H., Cabanes, Y., Gerald, T., Chauchat, P., Shewmake, C., Kainz, B., Donnat, C., Holmes, S., Pennec, X.: Geomstats: a python package for riemannian geometry in machine learning. arxiv:2004.04667 (2020)
Najfeld, I., Havel, T.F.: Derivatives of the matrix exponential and their computation. Adv. Appl. Math. 16, 321–375 (1995). https://doi.org/10.1006/aama.1995.1017
Nguyen, D.: Operator-valued formulas for Riemannian gradient and Hessian and families of tractable metrics in optimization and machine learning. arXiv:2009.10159 (2020)
Nguyen, D.: Project StiefelLog. https://github.com/dnguyend/StiefelLog (2021)
Nishimori, Y., Akaho, S.: Learning algorithms utilizing quasi-geodesics flows on Stiefel manifold. Neurocomputing 67, 106–135 (2005). https://doi.org/10.1016/j.neucom.2004.11.035
Nishimori, Y., Akaho, S., Plumbley, M.D.: Riemannian optimization method on the flag manifold for independent subspace analysis. In: Rosca, J., Erdogmus, D., Príncipe, J.C., Haykin, S. (eds.) Independent Component Analysis and Blind Signal Separation, pp. 295–302. Springer, Berlin (2006). https://doi.org/10.1007/11679363_37
Nocedal, J., Wright, S.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering, Springer, New York (2000)
Pennec, X.: Statistical computing on manifolds: from Riemannian geometry to computational anatomy. In: LNCS, vol. 5416, pp. 347–386. Springer, Berlin (2008)
Pennec, X.: Barycentric subspace analysis on manifolds. Ann. Stat. 46(6A), 2711–2746 (2018). https://doi.org/10.1214/17-AOS1636
Rentmeesters, Q.: Algorithms for data fitting on some common homogeneous spaces. Ph.D. thesis, Université Catholique de Louvain, Louvain, Belgium (2013). http://hdl.handle.net/2078.1/132587
Schmidt, M.: minFunc: Unconstrained differentiable multivariate optimization in MATLAB. http://www.cs.ubc.ca/~schmidtm/Software/minFunc.html (2005)
Srivastava, A., Klassen, E.: Functional and Shape Data Analysis. Springer, Berlin (2016)
Sterck, H., Howse, A.: Nonlinearly preconditioned L-BFGS as an acceleration mechanism for alternating least squares, with application to tensor decomposition. Numer. Linear Algebra Appl. 25 (2018). https://doi.org/10.1002/nla.2202
Sundaramoorthi, G., Mennucci, A., Soatto, S., Yezzi, A.: A new geometric metric in the space of curves, and applications to tracking deforming objects by prediction and filtering. SIAM J. Imaging Sci. 4, 109–145 (2011). https://doi.org/10.1137/090781139
Turaga, P., Veeraraghavan, A., Chellappa, R.: Statistical analysis on Stiefel and Grassmann manifolds with applications in computer vision. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1–8 (2008). https://doi.org/10.1109/CVPR.2008.4587733
Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M., Wilson, J., Millman, K.J., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R., Larson, E., Carey, C.J., Polat, İ., Feng, Y., Moore, E.W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E.A., Harris, C.R., Archibald, A.M., Ribeiro, A.H., Pedregosa, F., van Mulbregt, P., SciPy 1.0 Contributors: Scipy 1.0: Fundamental algorithms for scientific computing in Python (2020)
Ye, K., Wong, K., Lim, L.: Optimization on flag manifolds. arXiv:1907.00949 (2019)
Zimmermann, R.: A matrix-algebraic algorithm for the Riemannian logarithm on the Stiefel manifold under the canonical metric. SIAM J. Matrix Anal. Appl. 38(2), 322–342 (2017). https://doi.org/10.1137/16M1074485
Acknowledgements
We thank the editors and reviewers for helpful comments, inspiring the gradient-descent algorithm and improving the article significantly. We would like to thank Professor Overton for his interests and very kind advice, Professor Higham for answering our question on Lemma 4.1, our friend John Tillinghast and our family for their support in this project.
Funding
No funds, grants or other support were received.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author declares that he has no conflict of interest.
Additional information
Communicated by Alexandru Kristály.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Nguyen, D. Closed-form Geodesics and Optimization for Riemannian Logarithms of Stiefel and Flag Manifolds. J Optim Theory Appl 194, 142–166 (2022). https://doi.org/10.1007/s10957-022-02012-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-022-02012-3
Keywords
- Stiefel manifold
- Geodesic
- Computer vision
- Flag manifold
- Logarithm map
- Riemannian center of mass
- Fréchet derivative