Computational Optimization and Applications

, Volume 62, Issue 1, pp 5–29 | Cite as

Low-rank retractions: a survey and new results

Article

Abstract

Retractions are a prevalent tool in Riemannian optimization that provides a way to smoothly select a curve on a manifold with given initial position and velocity. We review and propose several retractions on the manifold \({\mathcal {M}}_r\) of rank-\(r\)\(m\times n\) matrices. With the exception of the exponential retraction (for the embedded geometry), which is clearly the least efficient choice, the retractions considered do not differ much in terms of run time and flop count. However, considerable differences are observed according to properties such as domain of definition, boundedness, first/second-order property, and symmetry.

Keywords

Low-rank manifold Fixed-rank manifold Low-rank optimization Retraction Geodesic Quasi-geodesic Projective retraction Orthographic retraction Lie–Trotter splitting 

Notes

Acknowledgments

We are grateful to the anonymous referees and to Bart Vandereycken for several useful comments on the first version of this paper. This work was financially supported by the Belgian FRFC (Fonds de la Recherche Fondamentale Collective). The work of I.O. was supported by Russian Science Foundation Grant 14-11-00659

References

  1. 1.
    Absil, P.-A., Amodei, L., Meyer, G.: Two Newton methods on the manifold of fixed-rank matrices endowed with riemannian quotient geometries. Comput. Stat. 29(3–4), 569–590 (2014). doi:10.1007/s00180-013-0441-6 MathSciNetCrossRefGoogle Scholar
  2. 2.
    Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007). doi:10.1007/s10208-005-0179-9 MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Adler, R.L., Dedieu, J.-P., Margulies, J.Y., Martens, M., Shub, Mike: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002). doi:10.1093/imanum/22.3.359 MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012). doi:10.1137/100802529 MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Absil, P.-A., Mahony, R., Sepulchre. R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ (2008). http://press.princeton.edu/titles/8586.html
  6. 6.
    Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: Manopt, a matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014). http://jmlr.org/papers/v15/boumal14a.html
  7. 7.
    Boothby, W.M.: An Introduction to Differentiable Manifolds and Riemannian Geometry, Revised Second Edition. Academic Press, London (2003)Google Scholar
  8. 8.
    Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, 3rd edn. Johns Hopkins University Press, Baltimore, MD (1996)Google Scholar
  9. 9.
    Koch, O., Lubich, C.: Dynamical low-rank approximation. SIAM J. Matrix Anal. Appl. 29(2), 434–454 (2007). doi:10.1137/050639703 MathSciNetCrossRefGoogle Scholar
  10. 10.
    Kressner, Daniel, Steinlechner, Michael, Vandereycken, Bart: Low-rank tensor completion by Riemannian optimization. BIT Numer. Math. 54(2), 447–468 (2014). doi:10.1007/s10543-013-0455-z MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Lee, J.M.: Introduction to Smooth Manifolds, Volume 218 of Graduate Texts in Mathematics. Springer, New York (2003)CrossRefGoogle Scholar
  12. 12.
    Lubich, C., Oseledets, I.V.: A projector-splitting integrator for dynamical low-rank approximation (2013) arXiv:1301.1058v2
  13. 13.
    Luenberger, D.G.: The gradient projection method along geodesics. Manag. Sci. 18, 620–631 (1972)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Mishra, B., Meyer, G., Bach, F., Sepulchre, R.: Low-rank optimization with trace norm penalty. SIAM J. Optim. 23(4), 2124–2149 (2013). doi:10.1137/110859646 MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Mishra, B., Meyer, G., Bonnabel, S., Sepulchre, R.: Fixed-rank matrix factorizations and Riemannian low-rank optimization (2013). arXiv:1209.0430v2
  16. 16.
    Mishra, B., Sepulchre, R.: R3MC: a Riemannian three-factor algorithm for low-rank matrix completion, 2014. Accepted for publication in the proceedings of the 53rd IEEE Conference on Decision and Control (2014). arXiv:1306.2672v2
  17. 17.
    Rosen, J.B.: The gradient projection method for nonlinear programming. II. Nonlinear constraints. J. Soc. Ind. Appl. Math. 9, 514–532 (1961)CrossRefMATHGoogle Scholar
  18. 18.
    Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012). doi:10.1137/11082885X MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Shub, M.: Some remarks on dynamical systems and numerical analysis. In L. Lara-Carrero and J. Lewowicz, (eds), Proceedings of the VII ELAM. Equinoccio, U. Simón Bolívar, Caracas, pp. 69–92 (1986)Google Scholar
  20. 20.
    Shalit, U., Weinshall, D., Chechik, G.: Online learning in the embedded manifold of low-rank matrices. J. Mach. Learn. Res. 13, 429–458 (2013)MathSciNetGoogle Scholar
  21. 21.
    Vandereycken, B.: Low-rank matrix completion by Riemannian optimization. SIAM J. Optim. 23(2), 1214–1236 (2013). doi:10.1137/110845768 MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Mathematical Engineering, ICTEAM InstituteUniversité catholique de LouvainLouvain-la-NeuveBelgium
  2. 2.Skolkovo Institute of Science and TechnologyMoscowRussia

Personalised recommendations