Advertisement

Riemannian Trust Regions with Finite-Difference Hessian Approximations are Globally Convergent

  • Nicolas Boumal
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9389)

Abstract

The Riemannian trust-region algorithm (RTR) is designed to optimize differentiable cost functions on Riemannian manifolds. It proceeds by iteratively optimizing local models of the cost function. When these models are exact up to second order, RTR boasts a quadratic convergence rate to critical points. In practice, building such models requires computing the Riemannian Hessian, which may be challenging. A simple idea to alleviate this difficulty is to approximate the Hessian using finite differences of the gradient. Unfortunately, this is a nonlinear approximation, which breaks the known convergence results for RTR.

We propose RTR-FD: a modification of RTR which retains global convergence when the Hessian is approximated using finite differences. Importantly, RTR-FD reduces gracefully to RTR if a linear approximation is used. This algorithm is available in the Manopt toolbox.

Keywords

RTR-FD Optimization on manifolds Convergence Manopt 

Notes

Acknowledgment

The author thanks P.-A. Absil for numerous helpful discussions.

References

  1. 1.
    Absil, P.A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)CrossRefzbMATHGoogle Scholar
  3. 3.
    Boumal, N.: Interpolation and regression of rotation matrices. In: Nielsen, F., Barbaresco, F. (eds.) GSI 2013. LNCS, vol. 8085, pp. 345–352. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  4. 4.
    Boumal, N., Mishra, B., Absil, P.A., Sepulchre, R.: Manopt, a Matlab toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014). http://www.manopt.org zbMATHGoogle Scholar
  5. 5.
    Conn, A., Gould, N., Toint, P.: Trust-region methods. MPS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Philadelphia (2000) CrossRefzbMATHGoogle Scholar
  6. 6.
    Huang, W., Absil, P.A., Gallivan, K.: A Riemannian symmetric rank-one trust-region method. Math. Program. 150(2), 179–216 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Huang, W., Gallivan, K., Absil, P.A.: A Broyden class of quasi-Newton methods for Riemannian optimization. Technical report UCL-INMA-2014.01, Université catholique de Louvain (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Inria & D.I., UMR 8548Ecole Normale SupérieureParisFrance

Personalised recommendations