Skip to main content
SpringerLink
Log in

On the convergence of Newton iterations to non-stationary points

  • Published:
Mathematical Programming Submit manuscript

Abstract.

We study conditions under which line search Newton methods for nonlinear systems of equations and optimization fail due to the presence of singular non-stationary points. These points are not solutions of the problem and are characterized by the fact that Jacobian or Hessian matrices are singular. It is shown that, for systems of nonlinear equations, the interaction between the Newton direction and the merit function can prevent the iterates from escaping such non-stationary points. The unconstrained minimization problem is also studied, and conditions under which false convergence cannot occur are presented. Several examples illustrating failure of Newton iterations for constrained optimization are also presented. The paper also shows that a class of line search feasible interior methods cannot exhibit convergence to non-stationary points.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Armand, P., Gilbert, J.-Ch., Jan-Jégou, S.: A feasible BFGS interior point algorithm for solving strongly convex minimization problems. SIAM J. Optim. 11, 199–222 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  2. Balay, S., Gropp, W.D., McInnes, L.C., Smith, B.F.: PETSc users manual. Technical Report Report ANL-95/11, Revision 2.1.1, Argonne National Laboratory, Argonne, Illinois, USA, 2001

  3. Byrd, R.H., Marazzi, M., Nocedal, J.: Proofs for ‘‘On the convergence of Newton iterations to non-stationary points’’. Technical Report OTC 2001/7, Optimization Technology Center, Northwestern University, Evanston, Illinois, USA, March 2001

  4. Conn, A.R., Gould, N.I.M., Toint, Ph.: Trust-region methods. MPS-SIAM Series on Optimization. SIAM publications, Philadelphia, Pennsylvania, USA, 2000

  5. Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs, New Jersey, USA, 1983. Reprinted as Classics in Applied Mathematics 16, SIAM, Philadelphia, Pennsylvania, USA, 1996

  6. El-Bakry, A.S., Tapia, R.A., Tsuchiya, T., Zhang, Y.: On the formulation and theory of the Newton interior-point method for nonlinear programming. J. Optim. Theory Appl. 89(3), 507–541 (1996)

    MATH  Google Scholar 

  7. Fletcher, R.: Practical Methods of Optimization. J. Wiley and Sons, Chichester, England, second edition, 1987

  8. Forsgren, A., Gill, P.E.: Primal-dual interior methods for nonconvex nonlinear programming. SIAM J. Optim. 8(4), 1132–1152 (1998)

    Article  MATH  Google Scholar 

  9. Gay, D.M., Overton, M.L., Wright, M.H.: A primal-dual interior method for nonconvex nonlinear programming. In: Y. Yuan, editor, Advances in Nonlinear Programming, pages 31–56, Dordrecht, The Netherlands, 1998. Kluwer Academic Publishers.

  10. Marazzi, M., Nocedal, J.: Feasibility control in nonlinear optimization. In: A. DeVore, A. Iserles, E. Suli, editors, Foundations of Computational Mathematics, London Mathematical Society Lecture Note Series, volume 284, pages 125–154, Cambridge, UK, 2001. Cambridge University Press.

  11. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: User guide for MINPACK-1. Technical Report 80–74, Argonne National Laboratory, Argonne, Illinois, USA, 1980

  12. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operations Research. Springer, 1999

  13. Powell, M.J.D.: A hybrid method for nonlinear equations. In: P. Rabinowitz, editor, Numerical Methods for Nonlinear Algebraic Equations, pages 87–114, London, 1970. Gordon and Breach

  14. Vanderbei, R.J., Shanno, D.F.: An interior point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13, 231–252 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  15. Wächter, A., Biegler, L.T.: Failure of global convergence for a class of interior point methods for nonlinear programming. Math. Programming 88(3), 565–574 (2000)

    Google Scholar 

  16. Wächter, A., Biegler, L.T.: Global and local convergence of line search filter methods for nonlinear programming. Technical Report CAPD B-01-09, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, August 2000

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Colorado, Boulder, CO 80309

    Richard H. Byrd

  2. The MathWorks, 3 Apple Hill Drive, Natick, Massachusetts, 01760-2098, USA

    Marcelo Marazzi

  3. Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, 60208-3118

    Jorge Nocedal

Authors
  1. Richard H. Byrd
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcelo Marazzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jorge Nocedal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jorge Nocedal.

Additional information

This author was supported by Air Force Office of Scientific Research grant F49620-00-1-0162, Army Research Office Grant DAAG55-98-1-0176, and National Science Foundation grant INT-9726199.

This author was supported by Department of Energy grant DE-FG02-87ER25047-A004.

This author was supported by National Science Foundation grant CCR-9987818 and Department of Energy grant DE-FG02-87ER25047-A004.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Byrd, R., Marazzi, M. & Nocedal, J. On the convergence of Newton iterations to non-stationary points. Math. Program., Ser. A 99, 127–148 (2004). https://doi.org/10.1007/s10107-003-0376-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10107-003-0376-8

Keywords

Search

Navigation