ODE-LM: A Hybrid Training Algorithm for Feedforward Neural Networks

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 215)

Abstract

A hybrid training algorithm named ODE-LM, in which the orthogonal differential evolution (ODE) algorithm is combined with the Levenberg-Marquardt (LM) method, is proposed to optimize feedforward neural network weights and biases. The ODE is first applied to globally optimize the network weights in a large space to some extent (the ODE will stop after a certain generation), and then LM is used to further learn until the maximum number of iterations is reached. The performance of ODE-LM has been evaluated on several benchmarks. The results demonstrate that ODE-LM is capable to overcome the slow training of traditional evolutionary neural network with lower learning error.

Keywords

Feedforward neural network Differential evolution  Orthogonal crossover Levenberg-Marquardt method 

Notes

Acknowledgments

This work was supported by the Fundamental Research Funds for the Central Universities (K50511700004).

References

  1. 1.
    Ilonen J, Kamarainen JK, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17:93–105Google Scholar
  2. 2.
    Yu J, Xi L, Wang S (2007) An improved particle swarm optimization for evolving feedforward artificial neural networks. Neural Process Lett 26:217–231CrossRefGoogle Scholar
  3. 3.
    Su CL, Yang SM, Huang WL (2011) A two-stage algorithm integrating genetic algorithm and modified newton method for neural network training in engineering systems. Expert Syst Appl 38:12189–12194CrossRefGoogle Scholar
  4. 4.
    Ding S, Su C, Yu J (2011) An optimizing BP neural network algorithm based on genetic algorithm. Artif Intell Rev 36:153–162CrossRefGoogle Scholar
  5. 5.
    Jing X (2012) Robust adaptive learning of feedforward neural networks via LMI optimizations. Neural Netw 31:33–45CrossRefMATHGoogle Scholar
  6. 6.
    Szczuka M, Slezak D (2011) Feedforward neural networks for compound signals. Theoret Comput Sci 412:5960–5973CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Nasr MB, Chtourou M (2006) A hybrid training algorithm for feedforward neural networks. Neural Process Lett 24:107–117CrossRefGoogle Scholar
  8. 8.
    Ahmed SU, Shahjahan M, Murase K (2011) Injecting chaos in feedforward neural networks. Neural Process Lett 34:87–100CrossRefGoogle Scholar
  9. 9.
    Mirjalili S, Mohd Hashim SZ, Moradian Sardroudi H (2012) Training feedforword neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput 218:11125–11137Google Scholar
  10. 10.
    Ng SC, Cheung CC, Leung SH (2004) Magnified gradient function with deterministic weight modification in adaptive learning. IEEE Trans Neural Netw 15:1411–1423CrossRefGoogle Scholar
  11. 11.
    Chandra R, Frean M, Zhang M (2012) Crossover-based local search in cooperative co-evolutionary feedforward neural networks. Appl Soft Comput 12:2924–2932CrossRefGoogle Scholar
  12. 12.
    Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8:694–713CrossRefGoogle Scholar
  13. 13.
    Yu J, Wang S, Xi L (2008) Evolving artificial neural networks using an improved PSO and DPSO. Neurocomputing 71:1054–1060CrossRefGoogle Scholar
  14. 14.
    Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Global Optimiz 11:341–359CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Tsai JT, Liu TK, Chou JH (2004) Hybrid Taguchi-genetic algorithm for global numerical optimization. IEEE Trans Evol Comput 8:365–377CrossRefGoogle Scholar
  16. 16.
    Ho SY, Shu LS, Chen JH (2004) Intelligent evolutionary algorithms for large parameter optimization problems. IEEE Trans Evol Comput 8:522–540CrossRefGoogle Scholar
  17. 17.
    Asuncion A, Newman DJ (2007) UCI machine learning repository. School of information and Computer Science, University of California, Irvine. http://www.ics.uci.edu/

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of MathematicsXidian UniversityXi’anChina
  2. 2.National Lab of Radar Signal ProcessingXidian UniversityXi’anChina

Personalised recommendations