Neural Processing Letters

, Volume 26, Issue 3, pp 145–158

Comparison of Stochastic Global Optimization Methods to Estimate Neural Network Weights


DOI: 10.1007/s11063-007-9048-7

Cite this article as:
Hamm, L., Brorsen, B.W. & Hagan, M.T. Neural Process Lett (2007) 26: 145. doi:10.1007/s11063-007-9048-7


Training a neural network is a difficult optimization problem because of numerous local minima. Many global search algorithms have been used to train neural networks. However, local search algorithms are more efficient with computational resources, and therefore numerous random restarts with a local algorithm may be more effective than a global algorithm. This study uses Monte-Carlo simulations to determine the efficiency of a local search algorithm relative to nine stochastic global algorithms when using a neural network on function approximation problems. The computational requirements of the global algorithms are several times higher than the local algorithm and there is little gain in using the global algorithms to train neural networks. Since the global algorithms only marginally outperform the local algorithm in obtaining a lower local minimum and they require more computational resources, the results in this study indicate that with respect to the specific algorithms and function approximation problems studied, there is little evidence to show that a global algorithm should be used over a more traditional local optimization routine for training neural networks. Further, neural networks should not be estimated from a single set of starting values whether a global or local optimization method is used.


Evolutionary algorithms Function approximation Neural networks Simulated annealing Stochastic global optimization 

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Lonnie Hamm
    • 1
  • B. Wade Brorsen
    • 2
  • Martin T. Hagan
    • 3
  1. 1.Straumur-Burdaras Investment BankLondonUK
  2. 2.Department of Agricultural EconomicsOklahoma State UniversityStillwaterUSA
  3. 3.School of Electrical and Computer EngineeringOklahoma State UniversityStillwaterUSA

Personalised recommendations