On Convergence of Minimization Methods: Attraction, Repulsion, and Selection

Abstract

In this paper, we revisit the convergence properties of the iterationprocess x^k+1=x^k−α(x^k)B(x^k)⁻¹∇f(x^k)for minimizing a function f(x). After reviewing some classic results andintroducing the notion of strong attraction, we give necessary andsufficient conditions for a stationary point of f(x) to be a point of strongattraction for the iteration process. Not only this result gives a newalgorithmic interpretation to the classic Ostrowski theorem, but alsoprovides insight into the interesting phenomenon called selectiveminimization. We present also illustrative numerical examples for nonlinearleast squares problems.