# A framework for generalising the Newton method and other iterative methods from Euclidean space to manifolds

- 386 Downloads
- 8 Citations

## Abstract

The Newton iteration is a popular method for minimising a cost function on Euclidean space. Various generalisations to cost functions defined on manifolds appear in the literature. In each case, the convergence rate of the generalised Newton iteration needed establishing from first principles. The present paper presents a framework for generalising iterative methods from Euclidean space to manifolds that ensures local convergence rates are preserved. It applies to any (memoryless) iterative method computing a coordinate independent property of a function (such as a zero or a local minimum). All possible Newton methods on manifolds are believed to come under this framework. Changes of coordinates, and not any Riemannian structure, are shown to play a natural role in lifting the Newton method to a manifold. The framework also gives new insight into the design of Newton methods in general.

## Mathematics Subject Classification

49M15## Notes

### Acknowledgments

This work was funded in part by the Australian Research Council. Special thanks to Dr Jochen Trumpf for insightful and thought-provoking discussions during the preliminary stages of this paper, and to the two anonymous reviewers for excellent guidance on improving the presentation.

## References

- 1.Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)CrossRefzbMATHGoogle Scholar
- 2.Absil, P.A., Mahony, R., Sepulchre, R., Van Dooren, P.: A Grassmann-Rayleigh quotient iteration for computing invariant subspaces. SIAM Rev. Publ. Soc. Indus Appl. Math.
**44**(1), 57–73 (2002)zbMATHGoogle Scholar - 3.Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. Siam J. Optim.
**22**(1), 135–158 (2012)CrossRefzbMATHMathSciNetGoogle Scholar - 4.Absil, P.A., Sepulchre, R., Van Dooren, P., Mahony, R.: Cubically convergent iterations for invariant subspace computation. SIAM J. Matrix Anal. Appl.
**26**(1), 70–96 (2004)CrossRefzbMATHMathSciNetGoogle Scholar - 5.Adler, R.L., Dedieu, J.-P., Margulies, J.Y., Martens, M., Shub, M.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal.
**22**(3), 359–390 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 6.Alvarez, F., Bolte, J., Munier, J.: A unifying local convergence result for Newton’s method in Riemannian manifolds. Found. Comput. Math.
**8**(2), 197–226 (2008)CrossRefzbMATHMathSciNetGoogle Scholar - 7.Argyros, I.K.: An improved unifying convergence analysis of Newton’s method in Riemannian manifolds. J. Appl. Math. Comput.
**25**(1–2), 345–351 (2007)CrossRefzbMATHMathSciNetGoogle Scholar - 8.Deuflhard, P.: Newton methods for nonlinear problems: affine invariance and adaptive algorithms. In: Springer Series in Computational Mathematics. Springer, Berlin (2004)Google Scholar
- 9.Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J Matrix Anal. Appl.
**20**(2), 303–353 (1998)CrossRefzbMATHMathSciNetGoogle Scholar - 10.Ferreira, O., Svaiter, B.: Kantorovich’s theorem on Newton’s method in Riemannian manifolds. J. Complex.
**18**(1), 304–329 (2002)CrossRefzbMATHMathSciNetGoogle Scholar - 11.Gabay, D.: Minimizing a differentiable function over a differentiable manifold. J. Optim. Theory Appl.
**37**(2), 177–219 (1982)CrossRefzbMATHMathSciNetGoogle Scholar - 12.Helmke, U., Moore, J.B.: Optimization and dynamical systems. In: Communications and Control Engineering Series. Springer-Verlag London Ltd., London (1994)Google Scholar
- 13.Hirsch, M.W., Smale, S.: Differential Equations, Dynamical Systems, and Linear Algebra. Academic Press, New York (1974)zbMATHGoogle Scholar
- 14.Kantorovich, L., Akhilov, G.: Functional Analysis in Normed Spaces. Fizmatgiz, Moscow (1959)zbMATHGoogle Scholar
- 15.Manton, J.H.: Optimisation algorithms exploiting unitary constraints. IEEE Trans. Signal Process.
**50**(3), 635–650 (2002)CrossRefMathSciNetGoogle Scholar - 16.Manton, J.H.: Optimisation geometry. In: Hüper, K., Trumpf, J. (eds.) Mathematical System Theory-Festschrift in Honor of Uwe Helmke on the Occasion of his Sixtieth Birthday, pp. 261–274. Create Space (2013)Google Scholar
- 17.Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Academic Press, New York (1970)zbMATHGoogle Scholar
- 18.Polak, E.: Optimization: Algorithms and Consistent Approximations. Springer, New York (1997)CrossRefzbMATHGoogle Scholar
- 19.Shub, M.: Some remarks on dynamical systems and numerical analysis. In: Dynamical Systems and Partial Differential Equations (Caracas, 1984), pp. 69–91. Universidad Simon Bolivar, Caracas (1986)Google Scholar