Advertisement

Artificial Bee Colony Training of Neural Networks

  • John A. BullinariaEmail author
  • Khulood AlYahya
Chapter
Part of the Studies in Computational Intelligence book series (SCI, volume 512)

Abstract

The Artificial Bee Colony (ABC) is a recently introduced swarm intelligence algorithm for optimization, that has previously been applied successfully to the training of neural networks. This paper explores more carefully the performance of the ABC algorithm for optimizing the connection weights of feed-forward neural networks for classification tasks, and presents a more rigorous comparison with the traditional Back-Propagation (BP) training algorithm. The empirical results show that using the standard “stopping early” approach with optimized learning parameters leads to improved BP performance over the previous comparative study, and that a simple variation of the ABC approach provides improved ABC performance too. With both improvements applied, we conclude that the ABC approach does perform very well on small problems, but the generalization performances achieved are only significantly better than standard BP on one out of six datasets, and the training times increase rapidly as the size of the problem grows.

Keywords

Hide Unit Generalization Performance Training Neural Network Swarm Intelligence Algorithm Particle Swarm Optimiza 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)Google Scholar
  2. 2.
    Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases. University of California (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
  3. 3.
    Bullinaria, J.A.: Using evolution to improve neural network learning: Pitfalls and solutions. Neural Computing and Applications 16, 209–226 (2007)CrossRefGoogle Scholar
  4. 4.
    Cantu-Paz, E., Kamath, C.: An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics 35, 915–927 (2005)CrossRefGoogle Scholar
  5. 5.
    Duch, W., Maszczyk, T., Jankowski, N.: Make it cheap: learning with O(nd) complexity. In: Proceedings of the World Congress on Computational Intelligence, pp. 132–135 (2012)Google Scholar
  6. 6.
    Engelbrecht, A.P.: Computational Intelligence: An Introduction. Wiley, Sussex (2007)Google Scholar
  7. 7.
    Karaboga, D.: An idea based on honey bee swarm for numerical optimization. Technical Report TR06, Computer Engineering Department, Erciyes University, Turkey (2005)Google Scholar
  8. 8.
    Karaboga, D., Akay, B.: A comparative study of Artificial Bee Colony algorithm. Applied Mathematics and Computation 214, 108–132 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Karaboga, D., Akay, B., Ozturk, C.: Artificial Bee Colony (ABC) optimization algorithm for training feed-forward neural networks. In: Torra, V., Narukawa, Y., Yoshida, Y. (eds.) MDAI 2007. LNCS (LNAI), vol. 4617, pp. 318–329. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  10. 10.
    Karaboga, D., Basturk, B.: On the performance of Artificial Bee Colony (ABC) algorithm. Applied Soft Computing 8, 687–697 (2008)CrossRefGoogle Scholar
  11. 11.
    Karaboga, D., Ozturk, C.: Neural networks training by Artificial Bee Colony algorithm on pattern classification. Neural Network World 19, 279–292 (2009)Google Scholar
  12. 12.
    Ozturk, C., Karaboga, D.: Hybrid Artificial Bee Colony algorithm for neural network training. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 84–88 (2011)Google Scholar
  13. 13.
    Prechelt, L.: PROBEN1 – A set of benchmarks and benchmarking rules for neural network training algorithms. Technical Report 21/94, Universitat Karlsruhe, Fakult at fur Informatik, Germany (1994)Google Scholar
  14. 14.
    Qiongshuai, L., Shiqing, W.: A hybrid model of neural network and classification in wine. In: Proceedings of the 3rd International Conference on Computer Research and Development, pp. 58–61 (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.School of Computer ScienceUniversity of BirminghamBirminghamUK

Personalised recommendations