Skip to main content
SpringerLink
Log in

Low-rank retractions: a survey and new results

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Retractions are a prevalent tool in Riemannian optimization that provides a way to smoothly select a curve on a manifold with given initial position and velocity. We review and propose several retractions on the manifold \({\mathcal {M}}_r\) of rank-\(r\) \(m\times n\) matrices. With the exception of the exponential retraction (for the embedded geometry), which is clearly the least efficient choice, the retractions considered do not differ much in terms of run time and flop count. However, considerable differences are observed according to properties such as domain of definition, boundedness, first/second-order property, and symmetry.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The Matlab code that we used to generate the tables of this paper is available from http://sites.uclouvain.be/absil/2013.04.

References

  1. Absil, P.-A., Amodei, L., Meyer, G.: Two Newton methods on the manifold of fixed-rank matrices endowed with riemannian quotient geometries. Comput. Stat. 29(3–4), 569–590 (2014). doi:10.1007/s00180-013-0441-6

    Article  MathSciNet  Google Scholar 

  2. Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007). doi:10.1007/s10208-005-0179-9

    Article  MathSciNet  MATH  Google Scholar 

  3. Adler, R.L., Dedieu, J.-P., Margulies, J.Y., Martens, M., Shub, Mike: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002). doi:10.1093/imanum/22.3.359

    Article  MathSciNet  MATH  Google Scholar 

  4. Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012). doi:10.1137/100802529

    Article  MathSciNet  MATH  Google Scholar 

  5. Absil, P.-A., Mahony, R., Sepulchre. R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ (2008). http://press.princeton.edu/titles/8586.html

  6. Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: Manopt, a matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014). http://jmlr.org/papers/v15/boumal14a.html

  7. Boothby, W.M.: An Introduction to Differentiable Manifolds and Riemannian Geometry, Revised Second Edition. Academic Press, London (2003)

    Google Scholar 

  8. Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, 3rd edn. Johns Hopkins University Press, Baltimore, MD (1996)

    Google Scholar 

  9. Koch, O., Lubich, C.: Dynamical low-rank approximation. SIAM J. Matrix Anal. Appl. 29(2), 434–454 (2007). doi:10.1137/050639703

    Article  MathSciNet  Google Scholar 

  10. Kressner, Daniel, Steinlechner, Michael, Vandereycken, Bart: Low-rank tensor completion by Riemannian optimization. BIT Numer. Math. 54(2), 447–468 (2014). doi:10.1007/s10543-013-0455-z

    Article  MathSciNet  MATH  Google Scholar 

  11. Lee, J.M.: Introduction to Smooth Manifolds, Volume 218 of Graduate Texts in Mathematics. Springer, New York (2003)

    Book  Google Scholar 

  12. Lubich, C., Oseledets, I.V.: A projector-splitting integrator for dynamical low-rank approximation (2013) arXiv:1301.1058v2

  13. Luenberger, D.G.: The gradient projection method along geodesics. Manag. Sci. 18, 620–631 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  14. Mishra, B., Meyer, G., Bach, F., Sepulchre, R.: Low-rank optimization with trace norm penalty. SIAM J. Optim. 23(4), 2124–2149 (2013). doi:10.1137/110859646

    Article  MathSciNet  MATH  Google Scholar 

  15. Mishra, B., Meyer, G., Bonnabel, S., Sepulchre, R.: Fixed-rank matrix factorizations and Riemannian low-rank optimization (2013). arXiv:1209.0430v2

  16. Mishra, B., Sepulchre, R.: R3MC: a Riemannian three-factor algorithm for low-rank matrix completion, 2014. Accepted for publication in the proceedings of the 53rd IEEE Conference on Decision and Control (2014). arXiv:1306.2672v2

  17. Rosen, J.B.: The gradient projection method for nonlinear programming. II. Nonlinear constraints. J. Soc. Ind. Appl. Math. 9, 514–532 (1961)

    Article  MATH  Google Scholar 

  18. Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012). doi:10.1137/11082885X

    Article  MathSciNet  MATH  Google Scholar 

  19. Shub, M.: Some remarks on dynamical systems and numerical analysis. In L. Lara-Carrero and J. Lewowicz, (eds), Proceedings of the VII ELAM. Equinoccio, U. Simón Bolívar, Caracas, pp. 69–92 (1986)

  20. Shalit, U., Weinshall, D., Chechik, G.: Online learning in the embedded manifold of low-rank matrices. J. Mach. Learn. Res. 13, 429–458 (2013)

    MathSciNet  Google Scholar 

  21. Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23(2), 1214–1236 (2013). doi:10.1137/110845768

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

We are grateful to the anonymous referees and to Bart Vandereycken for several useful comments on the first version of this paper. This work was financially supported by the Belgian FRFC (Fonds de la Recherche Fondamentale Collective). The work of I.O. was supported by Russian Science Foundation Grant 14-11-00659

Author information

Authors and Affiliations

  1. Department of Mathematical Engineering, ICTEAM Institute, Université catholique de Louvain, 1348, Louvain-la-Neuve, Belgium

    P.-A. Absil

  2. Skolkovo Institute of Science and Technology, Novaya St. 100, Skolkovo, 143025, Moscow, Russia

    I. V. Oseledets

Authors
  1. P.-A. Absil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. V. Oseledets
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to P.-A. Absil.

Additional information

This paper presents research results of the Belgian Network DYSCO (Dynamical Systems, Control, and Optimization), funded by the Interuniversity Attraction Poles Programme initiated by the Belgian Science Policy Office.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Absil, PA., Oseledets, I.V. Low-rank retractions: a survey and new results. Comput Optim Appl 62, 5–29 (2015). https://doi.org/10.1007/s10589-014-9714-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-014-9714-4

Keywords

Search

Navigation