Skip to main content
Log in

Intrinsic representation of tangent vectors and vector transports on matrix manifolds

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Abstract

The quasi-Newton methods on Riemannian manifolds proposed thus far do not appear to lend themselves to satisfactory convergence analyses unless they resort to an isometric vector transport. This prompts us to propose a computationally tractable isometric vector transport on the Stiefel manifold of orthonormal p-frames in \(\mathbb {R}^n\). Specifically, it requires \(O(np^2)\) flops, which is considerably less expensive than existing alternatives in the frequently encountered case where \(n\gg p\). We then build on this result to also propose computationally tractable isometric vector transports on other manifolds, namely the Grassmann manifold, the fixed-rank manifold, and the positive-semidefinite fixed-rank manifold. In the process, we also propose a convenient way to represent tangent vectors to these manifolds as elements of \(\mathbb {R}^d\), where d is the dimension of the manifold. We call this an “intrinsic” representation, as opposed to “extrinsic” representations as elements of \(\mathbb {R}^w\), where w is the dimension of the embedding space. Finally, we demonstrate the performance of the proposed isometric vector transport in the context of a Riemannian quasi-Newton method applied to minimizing the Brockett cost function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Throughout this paper, the computational complexity is measured by flop counts. A flop is a floating point operation [12, Section 1.2.4].

  2. It should be noted that for those Z such that \(\phi (Z)\) is nonsmooth, there always exists a permutation matrix P, which is usually obtained by row pivoting QR decomposition, such that \(\phi (PZ)\) is smooth. Therefore, one can define \(\tilde{\phi }(Z) = P^T \phi (PZ)\), which is also smooth and satisfies \(\tilde{\phi }(Z) = \begin{bmatrix} Z&Z_\perp \end{bmatrix}\).

  3. For gradient based methods, \(\mathrm {E2D}\) is always applied before \(\mathrm {D2E}\) since the gradient at a new iterate is computed almost immediately after the new iterate is obtained, and \(\mathrm {E2D}\) need be invoked to obtain the intrinsic representation of the gradient.

  4. Note that \(Alg1(X, Z) = Alg1(X, P_X (Z))\), where \(X \in {{\mathrm{\mathrm {St}}}}(p, n)\) and \(Alg1: ({{\mathrm{\mathrm {St}}}}(p, n), \mathbb {R}^{n \times p}) \rightarrow \mathbb {R}^{np - p(p+1)/2}\) is the function defined by Algorithm 1.

References

  1. Anderson, E., Bai, Z., Bischof, C., Blackford, L., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide. Society for Industrial and Applied Mathematics, 3rd edn (1999)

  2. Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  3. Adler, R.L., Dedieu, J.-P., Margulies, J.Y.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  4. Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization algorithms on matrix manifolds. Princeton University Press, Princeton, NJ (2008)

    Book  MATH  Google Scholar 

  5. Absil, P.-A., Oseledets, I.V.: Low-rank retractions: a survey and new results. Comput. Optim. Appl. 62(1), 5–29 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Afsari, B., Tron, R., Vidal, R.: On the convergence of gradient descent for finding the Riemannian center of mass. SIAM J. Control Optim. 51(3), 2230–2260 (2013). arXiv:1201.0925v1

  7. Boumal, N., Absil, P.A.: RTRMC: A Riemannian trust-region method for low-rank matrix completion. Adv. Neural Inf. Process. Syst. 24(NIPS), 406–414 (2011)

  8. Bini, D.A., Iannazzo, B.: Computing the Karcher mean of symmetric positive definite matrices. Linear Algebra Appl. 438(4), 1700–1710 (2013). doi:10.1016/j.laa.2011.08.052

    Article  MathSciNet  MATH  Google Scholar 

  9. Boothby, W.M.: An introduction to differentiable manifolds and Riemannian geometry, 2nd edn. Academic Press, USA (1986)

  10. Dai, W., Kerman, E., Milenkovic, O.: A geometric approach to low-rank matrix completion. IEEE Trans. Inf. Theory 58(1), 237–247 (2012). arXiv:1006.2086v1

  11. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998). doi:10.1137/S0895479895290954

    Article  MathSciNet  MATH  Google Scholar 

  12. Golub, G.H., Van Loan, C.F.: Matrix computations. Johns Hopkins Studies in the Mathematical Sciences, 3rd edn. Johns Hopkins University Press, USA (1996)

  13. Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM 42(6), 1115–1145 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  14. Huang, W., Absil, P.-A., Gallivan, K.A.: A Riemannian symmetric rank-one trust-region method. Math. Program. 150(2), 179–216 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  15. Huang, W., Absil, P.A., Gallivan, K.A.: A Riemannian BFGS method for nonconvex optimization problems. Lect. Notes Comput. Sci. Eng. (2016) (to appear)

  16. Huang, W., Gallivan, K.A., Absil, P.A.: A Broyden Class of Quasi-Newton Methods for Riemannian Optimization. SIAM J. Optim. 25(3), 1660–1685 (2015)

  17. Huang, W., Gallivan, K.A., Srivastava, A., Absil, P.A.: Riemannian optimization for registration of curves in elastic shape analysis. J. Math. Imaging Vision 54(3), 320–343 (2015). doi:10.1007/s10851-015-0606-8

  18. Huang, W., Gallivan, K.A., Zhang, X.: Solving phaselift by low rank Riemannian optimization methods. In: Proceedings of the International Conference on Computational Science (ICCS2016) (2016) (accepted)

  19. Kleinsteuber, M., Shen, H.: Blind source separation with compressively sensed linear mixtures. IEEE Signal Process. Lett. 19(2), 107–110 (2012). arXiv:1110.2593v1

  20. Lee, J.M.: Introduction to smooth manifolds, vol. 36 of Graduate Texts in Mathematics. Springer, New York (2011). doi:10.1016/B978-0-12-387667-6.00013-0

  21. Mishra, B., Meyer, G., Sepulchre, R.: Low-rank optimization for distance matrix completion. In: Proceeding of 50th IEEE Conference on Decision and Control and European Control Conference, pp. 4455–4460 (2011). doi:10.1109/CDC.2011.6160810

  22. Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd ed. Springer, New York (2006)

  23. Qi, C., Gallivan, K.A., Absil, P.-A.: Riemannian BFGS algorithm with applications. Recent Advances in Optimization and its Applications in Engineering, pp. 183–192 (2010)

  24. Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012). doi:10.1137/11082885X

    Article  MathSciNet  MATH  Google Scholar 

  25. Selvan, S.E., Amato, U., Gallivan, K.A., Qi, C.: Descent algorithms on oblique manifold for source-adaptive ICA contrast. IEEE Trans. Neural Netw. Learn. Syst. 23(12), 1930–1947 (2012)

    Article  Google Scholar 

  26. Sander, O.: Geodesic finite elements for Cosserat rods. Int. J. Numer. Meth. Eng. 82(13), 1645–1670 (2010). doi:10.1002/nme.2814

    MathSciNet  MATH  Google Scholar 

  27. Sato, H.: A Dai-Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions. Comput. Optim. Appl. (2015) (to appear)

  28. Sato, H., Iwai, T.: A new, globally convergent Riemannian conjugate gradient method. Optimization 64(4), 1011–1031 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Turaga, P., Veeraraghavan, A., Srivastava, A., Chellappa, R.: Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition. IEEE Trans. Pattern Anal. Mach. Intell. 33(11), 2273–2286 (2011). doi:10.1109/TPAMI.2011.52

  30. Vandereycken, B.: Low-rank matrix completion by Riemannian optimization–extended version. SIAM J. Optim. 23(2), 1214–1236 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. Vandereycken, B., Absil, P.A., Vandewalle, S.: Embedded geometry of the set of symmetric positive semidefinite matrices of fixed rank. In: Proceedings of the IEEE 15th Workshop on Statistical Signal Processing, pp. 389–392 (2009)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wen Huang.

Additional information

This paper presents research results of the Belgian Network DYSCO (Dynamical Systems, Control, and Optimization), funded by the Interuniversity Attraction Poles Programme initiated by the Belgian Science Policy Office. This work was supported by Grant FNRS PDR T.0173.13.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, W., Absil, PA. & Gallivan, K.A. Intrinsic representation of tangent vectors and vector transports on matrix manifolds. Numer. Math. 136, 523–543 (2017). https://doi.org/10.1007/s00211-016-0848-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00211-016-0848-4

Keywords

Navigation