Advertisement

Numerische Mathematik

, Volume 129, Issue 1, pp 91–125 | Cite as

A framework for generalising the Newton method and other iterative methods from Euclidean space to manifolds

  • Jonathan H. MantonEmail author
Article

Abstract

The Newton iteration is a popular method for minimising a cost function on Euclidean space. Various generalisations to cost functions defined on manifolds appear in the literature. In each case, the convergence rate of the generalised Newton iteration needed establishing from first principles. The present paper presents a framework for generalising iterative methods from Euclidean space to manifolds that ensures local convergence rates are preserved. It applies to any (memoryless) iterative method computing a coordinate independent property of a function (such as a zero or a local minimum). All possible Newton methods on manifolds are believed to come under this framework. Changes of coordinates, and not any Riemannian structure, are shown to play a natural role in lifting the Newton method to a manifold. The framework also gives new insight into the design of Newton methods in general.

Mathematics Subject Classification

49M15 

Notes

Acknowledgments

This work was funded in part by the Australian Research Council. Special thanks to Dr Jochen Trumpf for insightful and thought-provoking discussions during the preliminary stages of this paper, and to the two anonymous reviewers for excellent guidance on improving the presentation.

References

  1. 1.
    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)CrossRefzbMATHGoogle Scholar
  2. 2.
    Absil, P.A., Mahony, R., Sepulchre, R., Van Dooren, P.: A Grassmann-Rayleigh quotient iteration for computing invariant subspaces. SIAM Rev. Publ. Soc. Indus Appl. Math. 44(1), 57–73 (2002)zbMATHGoogle Scholar
  3. 3.
    Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. Siam J. Optim. 22(1), 135–158 (2012)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Absil, P.A., Sepulchre, R., Van Dooren, P., Mahony, R.: Cubically convergent iterations for invariant subspace computation. SIAM J. Matrix Anal. Appl. 26(1), 70–96 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Adler, R.L., Dedieu, J.-P., Margulies, J.Y., Martens, M., Shub, M.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22(3), 359–390 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  6. 6.
    Alvarez, F., Bolte, J., Munier, J.: A unifying local convergence result for Newton’s method in Riemannian manifolds. Found. Comput. Math. 8(2), 197–226 (2008)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Argyros, I.K.: An improved unifying convergence analysis of Newton’s method in Riemannian manifolds. J. Appl. Math. Comput. 25(1–2), 345–351 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Deuflhard, P.: Newton methods for nonlinear problems: affine invariance and adaptive algorithms. In: Springer Series in Computational Mathematics. Springer, Berlin (2004)Google Scholar
  9. 9.
    Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J Matrix Anal. Appl. 20(2), 303–353 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Ferreira, O., Svaiter, B.: Kantorovich’s theorem on Newton’s method in Riemannian manifolds. J. Complex. 18(1), 304–329 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Gabay, D.: Minimizing a differentiable function over a differentiable manifold. J. Optim. Theory Appl. 37(2), 177–219 (1982)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Helmke, U., Moore, J.B.: Optimization and dynamical systems. In: Communications and Control Engineering Series. Springer-Verlag London Ltd., London (1994)Google Scholar
  13. 13.
    Hirsch, M.W., Smale, S.: Differential Equations, Dynamical Systems, and Linear Algebra. Academic Press, New York (1974)zbMATHGoogle Scholar
  14. 14.
    Kantorovich, L., Akhilov, G.: Functional Analysis in Normed Spaces. Fizmatgiz, Moscow (1959)zbMATHGoogle Scholar
  15. 15.
    Manton, J.H.: Optimisation algorithms exploiting unitary constraints. IEEE Trans. Signal Process. 50(3), 635–650 (2002)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Manton, J.H.: Optimisation geometry. In: Hüper, K., Trumpf, J. (eds.) Mathematical System Theory-Festschrift in Honor of Uwe Helmke on the Occasion of his Sixtieth Birthday, pp. 261–274. Create Space (2013)Google Scholar
  17. 17.
    Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970)zbMATHGoogle Scholar
  18. 18.
    Polak, E.: Optimization: Algorithms and Consistent Approximations. Springer, New York (1997)CrossRefzbMATHGoogle Scholar
  19. 19.
    Shub, M.: Some remarks on dynamical systems and numerical analysis. In: Dynamical Systems and Partial Differential Equations (Caracas, 1984), pp. 69–91. Universidad Simon Bolivar, Caracas (1986)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of Electrical and Electronic EngineeringThe University of MelbourneVictoriaAustralia

Personalised recommendations