Skip to main content
SpringerLink
Log in

Continuation Newton methods with the residual trust-region time-stepping scheme for nonlinear equations

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

For nonlinear equations, the homotopy methods (continuation methods) are popular in engineering fields since their convergence regions are large and they are quite reliable to find a solution. The disadvantage of the classical homotopy methods is that their computational time is heavy since they need to solve many auxiliary nonlinear systems during the intermediate continuation processes. In order to overcome this shortcoming, we consider the special explicit continuation Newton method with the residual trust-region time-stepping scheme for this problem. According to our numerical experiments, the new method is more robust and faster than the traditional optimization method (the built-in subroutine fsolve.m of the MATLAB environment) and the homotopy continuation methods (HOMPACK90 and NAClab). Furthermore, we analyze the global convergence and the local superlinear convergence of the new method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Allgower, E.L., Georg, K.: Introduction to numerical continuation methods. SIAM, Philadelphia (2003)

    MATH  Google Scholar 

  2. Ascher, U.M., Petzold, L.R.: Computer methods for ordinary differential equations and differential-algebraic equations. SIAM, Philadelphia (1998)

    MATH  Google Scholar 

  3. Axelsson, O., Sysala, S.: Continuation Newton methods. Comput. Math. Appl. 70, 2621–2637 (2015)

    MathSciNet  MATH  Google Scholar 

  4. Branin, F.H.: Widely convergent method for finding multiple solutions of simultaneous nonlinear equations. IBM J. Res. Dev. 16, 504–521 (1972)

    MathSciNet  MATH  Google Scholar 

  5. Brenan, K.E., Campbell, S.L., Petzold, L.R.: Numerical solution of initial-value problems in differential-algebraic equations. SIAM, Philadelphia (1996)

    MATH  Google Scholar 

  6. Conn, A.R., Gould, N., Toint, P.L.: Trust-region methods. SIAM, Philadelphia (2000)

    MATH  Google Scholar 

  7. Davidenko, D.F.: On a new method of numerical solution of systems of nonlinear equations (in Russian). Dokl. Akad. Nauk SSSR 88, 601–602 (1953)

    Google Scholar 

  8. Deuflhard, P.: Newton methods for nonlinear problems: affine invariance and adaptive algorithms. Springer, Berlin (2004)

    MATH  Google Scholar 

  9. Deuflhard, P., Pesch, H.J., Rentrop, P.: A modified continuation method for the numerical solution of nonlinear two-point boundary value problems by shooting techniques. Numer. Math. 26, 327–343 (1975)

    MathSciNet  MATH  Google Scholar 

  10. Dennis, J.E., Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations. SIAM, Philadelphia (1996)

    MATH  Google Scholar 

  11. Dennis, J.E., Moré, J.J.: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28, 549–560 (1974)

    MathSciNet  MATH  Google Scholar 

  12. Doedel, E.J.: Lecture notes in numerical analysis of nonlinear equations. In: Krauskopf, B., Osinga, H.M., Galán-Vioque, J (eds.) Numerical Continuation Methods for Dynamical Systems, pp 1–50. Springer, Berlin (2007)

  13. Golub, G.H., Van Loan, C.F.: Matrix Computation, 4th edn. The John Hopkins University Press, Baltimore (2013)

    MATH  Google Scholar 

  14. Griewank, A.: On solving nonlinear equations with simple singularities or nearly singular solutions. SIAM Rev. 27, 537–563 (1985)

    MathSciNet  MATH  Google Scholar 

  15. Hairer, E., Nørsett, S.P., Wanner, G.: Solving ordinary differential equations I, Nonstiff Problems, 2nd edn. Springer, Berlin (1993)

    MATH  Google Scholar 

  16. Hairer, E., Wanner, G.: Solving ordinary differential equations II, stiff and differential-algebraic problems, 2nd edn. Springer, Berlin (1996)

    MATH  Google Scholar 

  17. Hansen, P.C.: Regularization Tools: A MATLAB package for analysis and solution of discrete ill-posed problems. Numer. Algorithms 6, 1–35 (1994)

    MathSciNet  MATH  Google Scholar 

  18. Hiebert, K.L.: An evaluation of mathematical software that solves systems of nonlinear equations. ACM Trans. Math. Softw. 8, 5–20 (1982)

    MATH  Google Scholar 

  19. Higham, D.J.: Trust region algorithms and timestep selection. SIAM J. Numer. Anal. 37, 194–210 (1999)

    MathSciNet  MATH  Google Scholar 

  20. Kalaba, R.F., Zagustin, E., Holbrow, W., Huss, R.: A modification of Davidenko’s method for nonlinear systems. Comput. Math. Appl. 3, 315–319 (1977)

    MathSciNet  MATH  Google Scholar 

  21. Kelley, C.T., Keyes, D.E.: Convergence analysis of pseudo-transient continuation. SIAM J. Numer. Anal. 35, 508–523 (1998)

    MathSciNet  MATH  Google Scholar 

  22. Kelley, C.T.: Solving nonlinear equations with Newton’s method. SIAM, Philadelphia (2003)

    MATH  Google Scholar 

  23. Kelley, C.T.: Numerical methods for nonlinear equations. Acta Numer. 27, 207–287 (2018)

    MathSciNet  MATH  Google Scholar 

  24. Luo, X.-L.: Singly diagonally implicit Runge-Kutta methods combining line search techniques for unconstrained optimization. J. Comput. Math. 23, 153–164 (2005)

    MathSciNet  MATH  Google Scholar 

  25. Lee, T.L., Li, T.Y., Tsai, C.H.: HOM4PS-2.0: a software package for solving polynomial systems by the polyhedral homotopy continuation method. Computing 83, 109–133 (2008)

    MathSciNet  MATH  Google Scholar 

  26. Levenberg, K.: A method for the solution of certain problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    MathSciNet  MATH  Google Scholar 

  27. Liao, S.J.: Homotopy analysis method in nonlinear differential equations. Springer, Berlin (2012)

    MATH  Google Scholar 

  28. Logan, S.R.: Fundamentals of chemical kinetics. Longman Group Limited, London (1996)

    Google Scholar 

  29. Lukšan, L.: Inexact trust region method for large sparse systems of nonlinear equations. J. Optim. Theory Appl. 81, 569–590 (1994)

    MathSciNet  MATH  Google Scholar 

  30. Luo, X.-L., Liao, L.-Z., Tam, H.-W.: Convergence analysis of the Levenberg-Marquardt method. Optim. Methods Softw. 22, 659–678 (2007)

    MathSciNet  MATH  Google Scholar 

  31. Luo, X.-L.: A trajectory-following method for solving the steady state of chemical reaction rate equations. J. Theor. Comput. Chem. 8, 1025–1044 (2009)

    Google Scholar 

  32. Luo, X.-L.: A second-order pseudo-transient method for steady-state problems. Appl. Math. Comput. 216, 1752–1762 (2010)

    MathSciNet  MATH  Google Scholar 

  33. Luo, X.-L., Yao, Y. -Y.: Primal-dual path-following methods and the trust-region updating strategy for linear programming with noisy data. J. Comput. Math., published online at https://doi.org/10.4208/jcm.2101-m2020-0173 or available at arXiv:2006.07568 (2021)

  34. Luo, X.-L., Lv, J.-H., Sun, G.: Continuation methods with the trusty time-stepping scheme for linearly constrained optimization with noisy data, Optim. Eng. published online at https://doi.org/10.1007/s11081-020-09590-z (2021)

  35. Luo, X.-L., Xiao, H., Lv, J.H., Zhang, S.: Explicit pseudo-transient continuation and the trust-region updating strategy for unconstrained optimization. Appl. Numer. Math. 165, 290–302 (2021). https://doi.org/10.1016/j.apnum.2021.02.019

    Article  MathSciNet  MATH  Google Scholar 

  36. Luo, X.-L., Xiao, H.: Generalized continuation Newton methods and the trust-region updating strategy for the underdetermined system, arXiv preprint available at https://arxiv.org/abs/2103.05829, submitted (2021)

  37. MATLAB 9.6.0 (R2019a): The MathWorks Inc..http://www.mathworks.com (2019)

  38. Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math. 11(1963), 431–441 (1963)

    MathSciNet  MATH  Google Scholar 

  39. Meintjes, K., Morgan, A.P.: Chemical equilibrium systems as numerical test problems. ACM Trans. Math. Softw. 16, 143–151 (1990)

    MATH  Google Scholar 

  40. Moré, J.J.: The Levenberg-Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis, Lecture Notes in Mathematics, vol. 630, pp 105–116. Springer, Berlin (1978)

  41. Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981)

    MathSciNet  MATH  Google Scholar 

  42. Nocedal, J., Wright, S.J.: Numerical optimization. Springer, Berlin (1999)

    MATH  Google Scholar 

  43. Ortega, J.M., Rheinboldt, W.C.: Iteration solution of nonlinear equations in several variables. SIAM, Philadelphia (2000)

    MATH  Google Scholar 

  44. Powell, M.J.D.: Convergence properties of a class of minimization algorithms. In: Mangasarian, O.L., Meyer, R.R., Robinson, S.M. (eds.) Nonlinear Programming 2, pp 1–27. Academic Press, New York (1975)

  45. Robertson, H.H.: The solution of a set of reaction rate equations. In: Walsh, J (ed.) Numerical Analysis, an Introduction, pp 178–182. Academic Press, New York (1966)

  46. Shampine, L.F.: Linear conservation laws for ODEs. Comput. Math. Appl. 35, 45–53 (1998)

    MathSciNet  MATH  Google Scholar 

  47. Shampine, L.F.: Conservation laws and the numerical solution of ODEs II. Comput. Math. Appl. 38, 61–72 (1999)

    MathSciNet  MATH  Google Scholar 

  48. Shampine, L.F., Gladwell, I., Thompson, S.: Solving ODEs with MATLAB. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  49. Shampine, L.F., Thompson, S., Kierzenka, J.A., Byrne, G.D.: Non-negative solutions of ODEs. Appl. Math. Comput. 170, 556–569 (2005)

    MathSciNet  MATH  Google Scholar 

  50. Tanabe, K.: Continuous Newton-Raphson method for solving an underdetermined system of nonlinear equations. Nonlinear Anal. 3, 495–503 (1979)

    MathSciNet  MATH  Google Scholar 

  51. Verwer, J.G., Van Loon, M.: An evaluation of explicit pseudo-steady state approximation schemes for stiff ODE systems from chemical kinetics. J. Comput. Phys. 113, 347–352 (1994)

    MathSciNet  MATH  Google Scholar 

  52. Watson, L.T., Sosonkina, M., Melville, R.C., Morgan, A.P., Walker, H.F.: HOMPACK90: A suite of fortran 90 codes for globally convergent homotopy algorithms. ACM Trans. Math. Softw. 23, 514–549 (1997)

    MathSciNet  MATH  Google Scholar 

  53. Yuan, Y.X.: Trust region algorithms for nonlinear equations. Information 1, 7–20 (1998)

    MathSciNet  MATH  Google Scholar 

  54. Yuan, Y.X.: Recent advances in trust region algorithms. Math. Program. 151, 249–281 (2015)

    MathSciNet  MATH  Google Scholar 

  55. Zeng, Z.G., Li, T.Y.: NAClab: A Matlab toolbox for numerical algebraic computation, vol. 47. http://homepages.neiu.edu/zzeng/naclab.html (2013)

  56. Zeng, Z.G.: A Newton’s iteraton convergence quadratically to nonisolated solutions too. the preprint of Department of Mathematics, Northeastern Illionis University, Chicago, http://homepages.neiu.edu/zzeng (2019)

Download references

Acknowledgements

The authors are grateful to two anonymous referees for their helpful comments and suggestions.

Funding

This work was supported in part by Grant 61876199 from National Natural Science Foundation of China, and Grant YJCB2011003HI from the Innovation Research Program of Huawei Technologies Co., Ltd.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Beijing University of Posts and Telecommunications, P. O. Box 101, Xitucheng Road No. 10, Haidian District, 100876, Beijing, China

    Xin-long Luo, Hang Xiao & Jia-hui Lv

Authors
  1. Xin-long Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hang Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia-hui Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin-long Luo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, Xl., Xiao, H. & Lv, Jh. Continuation Newton methods with the residual trust-region time-stepping scheme for nonlinear equations. Numer Algor 89, 223–247 (2022). https://doi.org/10.1007/s11075-021-01112-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-021-01112-x

Keywords

Mathematics Subject Classification (2010)

Search

Navigation