Skip to main content
SpringerLink
Log in

A Numerical Study of Basins of Attraction of Zero-Finding Neural Nets Designed Using Control Theory

  • Original Research
  • Published:
Differential Equations and Dynamical Systems Aims and scope Submit manuscript

Abstract

This article carries out a comparative study of zero-finding neural networks for nonlinear functions. By taking the view that artificial recurrent neural nets are dynamical systems described by ordinary differential equations (ODEs), new neural nets were recently derived in the literature using a unified control Liapunov function (CLF) approach, after interpreting the zero finding problem as a regulation problem for a closed-loop continuous-time dynamical system. The resulting neural net or continuous-time ODE is discretized by Euler’s method and the discretization step size interpreted as a control which is chosen so as to optimize the decrement in the chosen CLF, along system trajectories. Given the viewpoint adopted in this article, the words dynamical system, ODE and neural net are used interchangeably. For standard test functions of two variables, the basins of attraction are found by numerical simulation, starting from a uniformly distributed grid of initial points. For the chosen test functions, analysis of the basins shows a correlation between regularity of the basin boundaries and the predictability of convergence to a zero. In addition, this analysis suggests how to construct a team algorithm with favorable convergence properties.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Attouch H., Cominetti R.: A dynamical approach to convex minimization coupling approximation with the steepest descent method. J. Differ. Equ. 128, 519–540 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  2. Attouch H., Teboulle M.: Regularized Lotka–Volterra dynamical system as continuous proximal-like method in optimization. J. Optim. Theory Appl. 121(3), 541–570 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  3. Baran B., Kaszkurewicz E., Bhaya A.: Parallel asynchronous team algorithms: convergence and performance analysis. IEEE Trans. Parallel Distrib. Syst. 7(7), 677–688 (1996)

    Article  Google Scholar 

  4. Bhaya A., Kaszkurewicz E.: Iterative methods as dynamical systems with feedback control. In: Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii, pp. 2374–2380 (2003)

  5. Bhaya A., Kaszkurewicz E.: Control Perspectives on Numerical Algorithms and Matrix Problems, Advances in Control. SIAM, Philadelphia (2006)

    Book  Google Scholar 

  6. Bhaya A., Kaszkurewicz E.: A control-theoretic approach to the design of zero finding numerical methods. IEEE Trans. Autom. Control 52(6), 1014–1026 (2007)

    Article  MathSciNet  Google Scholar 

  7. Branin F.H.: Widely convergent method to finding multiple solutions of simultaneous nonlinear equations. IBM J. Res. Develop. 16(5), 504–522 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chao K.S., de Figueiredo R.J.P.: Optimally controlled iterative schemes for obtaining the solution of a non-linear equation. Int. J. Control 18(2), 377–384 (1973)

    Article  MATH  Google Scholar 

  9. Dixon L.C.W., Gomulka J., Szegö G.P.: Towards a global optimization technique. In: Dixon, L.C.W., Szegö, G.P. (eds) Towards a Global Optimization, pp. 29–54. North Holland, Amsterdam (1975)

    Google Scholar 

  10. Goh B.S.: Algorithms for unconstrained optimization via control theory. J. Optim. Theory Appl. 92(3), 581–604 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  11. Gomulka J.: Remarks on Branin’s method for solving nonlinear equations. In: Dixon, L.C.W., Szegö, G.P. (eds) Towards a Global Optimization, pp. 96–106. North Holland, Amsterdam (1975)

    Google Scholar 

  12. Hardy J.W.: An implemented extension of Branin’s method. In: Dixon, L.C.W., Szegö, G.P. (eds) Towards a Global Optimization, pp. 117–142. North Holland, Amsterdam (2002)

    Google Scholar 

  13. Hauser R., Nedić J.: The continuous Newton–Raphson method can look ahead. SIAM J. Optim. 15(3), 915–925 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hedge S., Kacera S.: Safeguarded Zero-Finding Methods. Computer Science Department Research Report. Courant Institute of Mathematical Sciences, New York University, New York (2006)

    Google Scholar 

  15. Laskari E.C., Parsopoulos K.E., Vrahatis M.N.: Evolutionary operators in global optimization with dynamic search trajectories. Numer. Algorithms 34, 393–403 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  16. Leake D., Mukai H.: Acceleration of one-step stationary root-finding algorithms. J. Numer. Anal. 9, 81–93 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  17. Riaza R., Zufiria P.J.: Weak singularities and the continuous Newton method. J. Math. Anal. Appl. 236(2–3), 438–462 (1999)

    Article  MathSciNet  Google Scholar 

  18. Treccani G.: On the convergence of Branin’s method: a counter-example. In: Dixon, L.C.W., Szegö, G.P. (eds) Towards a Global Optimization, pp. 107–116. North Holland, Amsterdam (1975)

    Google Scholar 

  19. Zufiria P.J., Guttalu R.S.: On the role of singularities in the Branin’s method from dynamic and continuation perspectives. Appl. Math. Comput. 130, 593–618 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for High Performance Computing (NACAD), NACAD/COPPE/UFRJ, PO Box 68504, Rio de Janeiro, RJ, 21945-970, Brazil

    Fernando A. Pazos

  2. Department of Electrical Engineering and Center for High Performance Computing (NACAD), PEE/COPPE/UFRJ, PO Box 68504, Rio de Janeiro, RJ, 21945-970, Brazil

    Amit Bhaya

Authors
  1. Fernando A. Pazos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amit Bhaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amit Bhaya.

Additional information

This research was partially supported by grants from FAPERJ (CNE), CNPq (BPP, PNPD, Universal), CAPES (ProEx).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pazos, F.A., Bhaya, A. A Numerical Study of Basins of Attraction of Zero-Finding Neural Nets Designed Using Control Theory. Differ Equ Dyn Syst 19, 63–86 (2011). https://doi.org/10.1007/s12591-010-0068-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12591-010-0068-9

Keywords

Search

Navigation