Computational Optimization and Applications

, Volume 62, Issue 1, pp 5–29 | Cite as

Low-rank retractions: a survey and new results

  • P.-A. AbsilEmail author
  • I. V. Oseledets


Retractions are a prevalent tool in Riemannian optimization that provides a way to smoothly select a curve on a manifold with given initial position and velocity. We review and propose several retractions on the manifold \({\mathcal {M}}_r\) of rank-\(r\) \(m\times n\) matrices. With the exception of the exponential retraction (for the embedded geometry), which is clearly the least efficient choice, the retractions considered do not differ much in terms of run time and flop count. However, considerable differences are observed according to properties such as domain of definition, boundedness, first/second-order property, and symmetry.


Low-rank manifold Fixed-rank manifold Low-rank optimization Retraction Geodesic Quasi-geodesic Projective retraction Orthographic retraction Lie–Trotter splitting 



We are grateful to the anonymous referees and to Bart Vandereycken for several useful comments on the first version of this paper. This work was financially supported by the Belgian FRFC (Fonds de la Recherche Fondamentale Collective). The work of I.O. was supported by Russian Science Foundation Grant 14-11-00659


  1. 1.
    Absil, P.-A., Amodei, L., Meyer, G.: Two Newton methods on the manifold of fixed-rank matrices endowed with riemannian quotient geometries. Comput. Stat. 29(3–4), 569–590 (2014). doi: 10.1007/s00180-013-0441-6 MathSciNetCrossRefGoogle Scholar
  2. 2.
    Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007). doi: 10.1007/s10208-005-0179-9 MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Adler, R.L., Dedieu, J.-P., Margulies, J.Y., Martens, M., Shub, Mike: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002). doi: 10.1093/imanum/22.3.359 MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012). doi: 10.1137/100802529 MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Absil, P.-A., Mahony, R., Sepulchre. R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ (2008).
  6. 6.
    Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: Manopt, a matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014).
  7. 7.
    Boothby, W.M.: An Introduction to Differentiable Manifolds and Riemannian Geometry, Revised Second Edition. Academic Press, London (2003)Google Scholar
  8. 8.
    Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, 3rd edn. Johns Hopkins University Press, Baltimore, MD (1996)Google Scholar
  9. 9.
    Koch, O., Lubich, C.: Dynamical low-rank approximation. SIAM J. Matrix Anal. Appl. 29(2), 434–454 (2007). doi: 10.1137/050639703 MathSciNetCrossRefGoogle Scholar
  10. 10.
    Kressner, Daniel, Steinlechner, Michael, Vandereycken, Bart: Low-rank tensor completion by Riemannian optimization. BIT Numer. Math. 54(2), 447–468 (2014). doi: 10.1007/s10543-013-0455-z MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Lee, J.M.: Introduction to Smooth Manifolds, Volume 218 of Graduate Texts in Mathematics. Springer, New York (2003)CrossRefGoogle Scholar
  12. 12.
    Lubich, C., Oseledets, I.V.: A projector-splitting integrator for dynamical low-rank approximation (2013) arXiv:1301.1058v2
  13. 13.
    Luenberger, D.G.: The gradient projection method along geodesics. Manag. Sci. 18, 620–631 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Mishra, B., Meyer, G., Bach, F., Sepulchre, R.: Low-rank optimization with trace norm penalty. SIAM J. Optim. 23(4), 2124–2149 (2013). doi: 10.1137/110859646 MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Mishra, B., Meyer, G., Bonnabel, S., Sepulchre, R.: Fixed-rank matrix factorizations and Riemannian low-rank optimization (2013). arXiv:1209.0430v2
  16. 16.
    Mishra, B., Sepulchre, R.: R3MC: a Riemannian three-factor algorithm for low-rank matrix completion, 2014. Accepted for publication in the proceedings of the 53rd IEEE Conference on Decision and Control (2014). arXiv:1306.2672v2
  17. 17.
    Rosen, J.B.: The gradient projection method for nonlinear programming. II. Nonlinear constraints. J. Soc. Ind. Appl. Math. 9, 514–532 (1961)CrossRefzbMATHGoogle Scholar
  18. 18.
    Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012). doi: 10.1137/11082885X MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Shub, M.: Some remarks on dynamical systems and numerical analysis. In L. Lara-Carrero and J. Lewowicz, (eds), Proceedings of the VII ELAM. Equinoccio, U. Simón Bolívar, Caracas, pp. 69–92 (1986)Google Scholar
  20. 20.
    Shalit, U., Weinshall, D., Chechik, G.: Online learning in the embedded manifold of low-rank matrices. J. Mach. Learn. Res. 13, 429–458 (2013)MathSciNetGoogle Scholar
  21. 21.
    Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23(2), 1214–1236 (2013). doi: 10.1137/110845768 MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Mathematical Engineering, ICTEAM InstituteUniversité catholique de LouvainLouvain-la-NeuveBelgium
  2. 2.Skolkovo Institute of Science and TechnologyMoscowRussia

Personalised recommendations