Cholesky QR-based retraction on the generalized Stiefel manifold

Abstract

When optimizing on a Riemannian manifold, it is important to use an efficient retraction, which maps a point on a tangent space to a point on the manifold. In this paper, we prove a map based on the QR factorization to be a retraction on the generalized Stiefel manifold. In addition, we propose an efficient implementation of the retraction based on the Cholesky QR factorization. Numerical experiments show that the proposed retraction is more efficient than the existing one based on the polar factorization.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)

    Google Scholar 

  2. 2.

    Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012)

    MathSciNet  MATH  Article  Google Scholar 

  3. 3.

    Adler, R.L., Dedieu, J.P., Margulies, J.Y., Martens, M., Shub, M.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002)

    MathSciNet  MATH  Article  Google Scholar 

  4. 4.

    Aihara, K., Sato, H.: A matrix-free implementation of Riemannian Newton’s method on the Stiefel manifold. Optim. Lett. 11(8), 1729–1741 (2017)

    MathSciNet  MATH  Article  Google Scholar 

  5. 5.

    Bendory, T., Eldar, Y.C., Boumal, N.: Non-convex phase retrieval from STFT measurements. IEEE Trans. Inf. Theory 64(1), 467–484 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  6. 6.

    Boumal, N., Absil, P.A., Cartis, C.: Global rates of convergence for nonconvex optimization on manifolds. IMA J. Numer. Anal. https://doi.org/10.1093/imanum/drx080 (2018)

  7. 7.

    Boumal, N., Mishra, B., Absil, P.A., Sepulchre, R.: Manopt, a Matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014)

    MATH  Google Scholar 

  8. 8.

    Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)

    MathSciNet  MATH  Article  Google Scholar 

  9. 9.

    Imakura, A., Yamamoto, Y.: Efficient implementations of the modified Gram–Schmidt orthogonalization with a non-standard inner product. arXiv preprint arXiv:1703.10440 (2017)

  10. 10.

    Kasai, H., Sato, H., Mishra, B.: Riemannian stochastic recursive gradient algorithm with retraction and vector transport and its convergence analysis. In: International Conference on Machine Learning, pp. 2521–2529 (2018)

  11. 11.

    Lowery, B.R., Langou, J.: Stability analysis of QR factorization in an oblique inner product. arXiv preprint arXiv:1401.5171 (2014)

  12. 12.

    Manton, J.H.: Optimization algorithms exploiting unitary constraints. IEEE Trans. Signal Process. 50(3), 635–650 (2002)

    MathSciNet  MATH  Article  Google Scholar 

  13. 13.

    Mishra, B., Sepulchre, R.: Riemannian preconditioning. SIAM J. Optim. 26(1), 635–660 (2016)

    MathSciNet  MATH  Article  Google Scholar 

  14. 14.

    Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22(2), 596–627 (2012)

    MathSciNet  MATH  Article  Google Scholar 

  15. 15.

    Rozložník, M., Tůma, M., Smoktunowicz, A., Kopal, J.: Numerical stability of orthogonalization methods with a non-standard inner product. BIT Numer. Math. 52(4), 1035–1058 (2012)

    MathSciNet  MATH  Article  Google Scholar 

  16. 16.

    Sato, H.: A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions. Comput. Optim. Appl. 64(1), 101–118 (2016)

    MathSciNet  MATH  Article  Google Scholar 

  17. 17.

    Sato, H., Iwai, T.: A Riemannian optimization approach to the matrix singular value decomposition. SIAM J. Optim. 23(1), 188–212 (2013)

    MathSciNet  MATH  Article  Google Scholar 

  18. 18.

    Sato, H., Sato, K.: Riemannian optimal system identification algorithm for linear MIMO systems. IEEE Control Syst. Lett. 1(2), 376–381 (2017)

    Article  Google Scholar 

  19. 19.

    Yger, F., Berar, M., Gasso, G., Rakotomamonjy, A.: Adaptive canonical correlation analysis based on matrix manifolds. In: Proceedings of the 29th International Conference on Machine Learning, pp. 299–306 (2012)

  20. 20.

    Yger, F., Berar, M., Gasso, G., Rakotomamonjy, A.: Oblique principal subspace tracking on manifold. In: Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2429–2432. IEEE (2012)

Download references

Acknowledgements

The authors would like to thank the editor and the reviewer for their careful reading and constructive comments, especially on Theorem 3.2. The authors would also like to thank Dr. Akira Imakura (University of Tsukuba) and Dr. Yusaku Yamamoto (The University of Electro-Communications) for their helpful advice. This study was supported in part by Grant Numbers JP16K17647 and JP18K18064 from the Grants-in-Aid for Scientific Research Program (KAKENHI) of the Japan Society for the Promotion of Science (JSPS).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hiroyuki Sato.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sato, H., Aihara, K. Cholesky QR-based retraction on the generalized Stiefel manifold. Comput Optim Appl 72, 293–308 (2019). https://doi.org/10.1007/s10589-018-0046-7

Download citation

Keywords

  • Riemannian optimization
  • Generalized Stiefel manifold
  • Retraction
  • Cholesky QR factorization

Mathematics Subject Classification

  • 90C30
  • 65K05
  • 65F30