Advertisement

Computational Optimization and Applications

, Volume 72, Issue 2, pp 293–308 | Cite as

Cholesky QR-based retraction on the generalized Stiefel manifold

  • Hiroyuki SatoEmail author
  • Kensuke Aihara
Article

Abstract

When optimizing on a Riemannian manifold, it is important to use an efficient retraction, which maps a point on a tangent space to a point on the manifold. In this paper, we prove a map based on the QR factorization to be a retraction on the generalized Stiefel manifold. In addition, we propose an efficient implementation of the retraction based on the Cholesky QR factorization. Numerical experiments show that the proposed retraction is more efficient than the existing one based on the polar factorization.

Keywords

Riemannian optimization Generalized Stiefel manifold Retraction Cholesky QR factorization 

Mathematics Subject Classification

90C30 65K05 65F30 

Notes

Acknowledgements

The authors would like to thank the editor and the reviewer for their careful reading and constructive comments, especially on Theorem 3.2. The authors would also like to thank Dr. Akira Imakura (University of Tsukuba) and Dr. Yusaku Yamamoto (The University of Electro-Communications) for their helpful advice. This study was supported in part by Grant Numbers JP16K17647 and JP18K18064 from the Grants-in-Aid for Scientific Research Program (KAKENHI) of the Japan Society for the Promotion of Science (JSPS).

References

  1. 1.
    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)zbMATHCrossRefGoogle Scholar
  2. 2.
    Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Adler, R.L., Dedieu, J.P., Margulies, J.Y., Martens, M., Shub, M.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Aihara, K., Sato, H.: A matrix-free implementation of Riemannian Newton’s method on the Stiefel manifold. Optim. Lett. 11(8), 1729–1741 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Bendory, T., Eldar, Y.C., Boumal, N.: Non-convex phase retrieval from STFT measurements. IEEE Trans. Inf. Theory 64(1), 467–484 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Boumal, N., Absil, P.A., Cartis, C.: Global rates of convergence for nonconvex optimization on manifolds. IMA J. Numer. Anal. https://doi.org/10.1093/imanum/drx080 (2018)
  7. 7.
    Boumal, N., Mishra, B., Absil, P.A., Sepulchre, R.: Manopt, a Matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014)zbMATHGoogle Scholar
  8. 8.
    Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Imakura, A., Yamamoto, Y.: Efficient implementations of the modified Gram–Schmidt orthogonalization with a non-standard inner product. arXiv preprint arXiv:1703.10440 (2017)
  10. 10.
    Kasai, H., Sato, H., Mishra, B.: Riemannian stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis. In: International Conference on Machine Learning, pp. 2521–2529 (2018)Google Scholar
  11. 11.
    Lowery, B.R., Langou, J.: Stability analysis of QR factorization in an oblique inner product. arXiv preprint arXiv:1401.5171 (2014)
  12. 12.
    Manton, J.H.: Optimization algorithms exploiting unitary constraints. IEEE Trans. Signal Process. 50(3), 635–650 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Mishra, B., Sepulchre, R.: Riemannian preconditioning. SIAM J. Optim. 26(1), 635–660 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Rozložník, M., Tůma, M., Smoktunowicz, A., Kopal, J.: Numerical stability of orthogonalization methods with a non-standard inner product. BIT Numer. Math. 52(4), 1035–1058 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Sato, H.: A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions. Comput. Optim. Appl. 64(1), 101–118 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Sato, H., Iwai, T.: A Riemannian optimization approach to the matrix singular value decomposition. SIAM J. Optim. 23(1), 188–212 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Sato, H., Sato, K.: Riemannian optimal system identification algorithm for linear MIMO systems. IEEE Control Syst. Lett. 1(2), 376–381 (2017)CrossRefGoogle Scholar
  19. 19.
    Yger, F., Berar, M., Gasso, G., Rakotomamonjy, A.: Adaptive canonical correlation analysis based on matrix manifolds. In: Proceedings of the 29th International Conference on Machine Learning, pp. 299–306 (2012)Google Scholar
  20. 20.
    Yger, F., Berar, M., Gasso, G., Rakotomamonjy, A.: Oblique principal subspace tracking on manifold. In: Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2429–2432. IEEE (2012)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.The Hakubi Center for Advanced ResearchKyoto UniversityKyotoJapan
  2. 2.Department of Applied Mathematics and PhysicsKyoto UniversityKyotoJapan
  3. 3.Department of Computer ScienceTokyo City UniversityTokyoJapan

Personalised recommendations