Advertisement

Cognitive Computation

, Volume 10, Issue 6, pp 1096–1134 | Cite as

A Cognitively Inspired Hybridization of Artificial Bee Colony and Dragonfly Algorithms for Training Multi-layer Perceptrons

  • Waheed A. H. M. GhanemEmail author
  • Aman Jantan
Article

Abstract

The objective of this article is twofold. On the one hand, we introduce a cognitively inspired hybridization metaheuristic that combines the strengths of two existing metaheuristics: the artificial bee colony (ABC) algorithm and the dragonfly algorithm (DA). The aim of this hybridization is to reduce the problems of slow convergence and trapping into local optima, by striking a good balance between global and local search components of the constituent algorithms. On the other hand, we use the proposed metaheuristic to train a multi-layer perceptron (MLP) as an alternative to existing traditional- and metaheuristic-based learning algorithms; this is for the purpose of improving overall accuracy by optimizing the set of MLP weights and biases. The proposed hybrid ABC/DA (HAD) algorithm comprises three main components: the static and dynamic swarming behavior phase in DA and two global search phases in ABC. The first one performs global search (DA phase), the second one performs local search (onlooker phase), and the third component implements global search (modified scout bee phase). The resultant metaheuristic optimizer is employed to train an MLP to reach a set of weights and biases that can yield high performance compared to traditional learning algorithms or even other metaheuristic optimizers. The proposed algorithm was first evaluated using 33 benchmark functions to test its performance in numerical optimization problems. Later, using HAD for training MLPs was evaluated against six standard classification datasets. In both cases, the performance of HAD was compared with the performance of several new and old metaheuristic methods from swarm intelligence and evolutionary computing. Experimental results show that HAD algorithm is clearly superior to the standard ABC and DA algorithms, as well as to other well-known algorithms, in terms of achieving the best optimal value, convergence speed, avoiding local minima and accuracy of trained MLPs. The proposed algorithm is a promising metaheuristic technique for general numerical optimization and for training MLPs. Specific applications and use cases are yet to be explored fully but they are supported by the encouraging results in this study.

Keywords

Artificial intelligence (AI) Swarm intelligence (SI) Computation intelligence (CI) Global optimization problem Artificial bee colony algorithm (ABC) Dragonfly algorithm (DA) Multi-layer perceptron (MLP) 

Notes

Acknowledgements

The authors sincerely thanks the editors and anonymous reviewers for their helpful suggestions on how to improve the presentation of the article.

Funding

This study was funded by Universiti Sains Malaysia Fellowship (grant number [APEX (308/AIPS/415401)]) and the RUI grant, Account No. [1001/PKOMP/8014017] also under the Universiti Sains Malaysia.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

References

  1. 1.
    Kim S-S, McLoone S, Byeon J-H, Lee S, Liu H. Cognitively inspired artificial bee colony clustering for cognitive wireless sensor networks. Cogn Comput. 2017;9(2):207–24.CrossRefGoogle Scholar
  2. 2.
    Fernández-Caballero A, González P, Navarro E. Cognitively-inspired computing for gerontechnology. Cogn Comput. 2016;8(2):297–8.CrossRefGoogle Scholar
  3. 3.
    Bonabeau E, Dorigo M, Theraulaz G. Swarm intelligence: from natural to artificial systems. Press: Oxford Univ; 1999.Google Scholar
  4. 4.
    Mavrovouniotis M, Li C, Yang S. A survey of swarm intelligence for dynamic optimization: algorithms and applications. Swarm Evol Comput. 2017;33:1–17.CrossRefGoogle Scholar
  5. 5.
    Chen J, Zeng Z, Jiang P, Tang H. Deformation prediction of landslide based on functional network. Neurocomputing. 2015;149:151–7.CrossRefGoogle Scholar
  6. 6.
    Ghanem WAHM, Jantan A. Using hybrid artificial bee colony algorithm and particle swarm optimization for training feed-forward neural networks. J Theor Appl Inf Technol. 2014;67(3)Google Scholar
  7. 7.
    Mirjalili SA, Hashim SZM, Sardroudi HM. Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput. 2012;218(22):11125–37.Google Scholar
  8. 8.
    Ghanem WAHM, Jantan A. Novel multi-objective artificial bee Colony optimization for wrapper based feature selection in intrusion detection. Int J Adv Soft Comput Appl. 2016;8(1)Google Scholar
  9. 9.
    Bandaru S, Ng AHC, Deb K. Data mining methods for knowledge discovery in multi-objective optimization: Part A-Survey. Expert Syst Appl. 2017;70:139–59.CrossRefGoogle Scholar
  10. 10.
    Ali MH, Al Mohammed BAD, Ismail A, Zolkipli MF. A new intrusion detection system based on Fast Learning Network and Particle swarm optimization. IEEE Access. 2018;6:20255–61.CrossRefGoogle Scholar
  11. 11.
    Ghanem WAHM, Jantan A. New approach to improve anomaly detection using a neural network optimized by hybrid abc and pso algorithms. Pak J Stat. 2018;34(1)Google Scholar
  12. 12.
    Han X, Chang X, Quan L, Xiong X, Li J, Zhang Z, et al. Feature subset selection by gravitational search algorithm optimization. Inf Sci. 2014;281:128–46.CrossRefGoogle Scholar
  13. 13.
    Aljarah I, Al-Zoubi A’M, Faris H, Hassonah MA, Mirjalili S, Saadeh H. Simultaneous feature selection and support vector machine optimization using the grasshopper optimization algorithm. Cogn Comput. 2018:1–18.Google Scholar
  14. 14.
    Yang, Xin-She. A new metaheuristic bat-inspired algorithm. In Nature inspired cooperative strategies for optimization (NICSO 2010), pp. 65–74. Springer, Berlin, Heidelberg, 2010.Google Scholar
  15. 15.
    Yang, Xin-She, and Suash Deb. Cuckoo search via Lévy flights. In Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on, pp. 210–214. IEEE, 2009.Google Scholar
  16. 16.
    An J, Kang Q, Wang L, Qidi W. Mussels wandering optimization: an ecologically inspired algorithm for global optimization. Cogn Comput. 2013;5(2):188–99.CrossRefGoogle Scholar
  17. 17.
    Eberhart, Russell, and James Kennedy. A new optimizer using particle swarm theory. In Micro Machine and Human Science, 1995. MHS'95., Proceedings of the Sixth International Symposium on, pp. 39–43. IEEE, 1995.Google Scholar
  18. 18.
    Wang, Gai-Ge, Suash Deb, and Leandro dos S. Coelho. Elephant herding optimization. In Computational and Business Intelligence (ISCBI), 2015 3rd International Symposium on, pp. 1–5. IEEE, 2015.Google Scholar
  19. 19.
    Mirjalili S. Moth-flame optimization algorithm: a novel nature-inspired heuristic paradigm. Knowl-Based Syst. 2015;89:228–49.CrossRefGoogle Scholar
  20. 20.
    Mirjalili S, Lewis A. The whale optimization algorithm. Adv Eng Softw. 2016;95:51–67.CrossRefGoogle Scholar
  21. 21.
    Wang G-G, Deb S, Cui Z. Monarch butterfly optimization. Neural Comput Appl. 2015:1–20.Google Scholar
  22. 22.
    Gandomi, Amir Hossein, and Amir Hossein Alavi. Krill herd: a new bio-inspired optimization algorithm. Communications in Nonlinear Science and Numerical Simulation 17, no. 12 (2012): 4831–4845.Google Scholar
  23. 23.
    Wang G-G. Moth search algorithm: a bio-inspired metaheuristic algorithm for global optimization problems. Memetic Computing. 2016:1–14.Google Scholar
  24. 24.
    Mirjalili S. The ant lion optimizer. Adv Eng Softw. 2015;83:80–98.CrossRefGoogle Scholar
  25. 25.
    Storn R, Price K. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim. 1997;11(4):341–59.CrossRefGoogle Scholar
  26. 26.
    Simon D. Biogeography-based optimization. IEEE Trans Evol Comput. 2008;12(6):702–13.CrossRefGoogle Scholar
  27. 27.
    Beyer H-G, Schwefel H-P. Evolution strategies–a comprehensive introduction. Nat Comput. 2002;1(1):3–52.CrossRefGoogle Scholar
  28. 28.
    Goldberg DE, Holland JH. Genetic algorithms and machine learning. Mach Learn. 1988;3(2):95–9.CrossRefGoogle Scholar
  29. 29.
    Rashedi E, Nezamabadi-Pour H, Saryazdi S. GSA: a gravitational search algorithm. Inf Sci. 2009;179(13):2232–48.CrossRefGoogle Scholar
  30. 30.
    Geem ZW, Kim JH, Loganathan GV. A new heuristic optimization algorithm: harmony search. Simulation. 2001;76(2):60–8.CrossRefGoogle Scholar
  31. 31.
    Mirjalili S. SCA: a sine cosine algorithm for solving optimization problems. Knowl-Based Syst. 2016;96:120–33.CrossRefGoogle Scholar
  32. 32.
    Ghanem, Waheed Ali HM, and Aman Jantan. An enhanced Bat algorithm with mutation operator for numerical optimization problems" Neural Comput & Applic (2017): 1–35.Google Scholar
  33. 33.
    Ghanem WAHM, Jantan A. A novel hybrid artificial bee colony with monarch butterfly optimization for global optimization problems. In: Modeling, Simulation, and Optimization. Cham: Springer; 2018. p. 27–38.CrossRefGoogle Scholar
  34. 34.
    Ghanem WAHM, Jantan A. Hybridizing Bat algorithm with modified pitch adjustment operator for numerical optimization problems. In: Modeling, Simulation, and Optimization. Cham: Springer; 2018. p. 57–69.CrossRefGoogle Scholar
  35. 35.
    Ghanem WAHM, Jantan A. Hybridizing artificial bee colony with monarch butterfly optimization for numerical optimization problems. Neural Comput & Applic. 2018;30(1):163–81.CrossRefGoogle Scholar
  36. 36.
    Wang G-G, Gandomi AH, Alavi AH, Hao G-S. Hybrid krill herd algorithm with differential evolution for global numerical optimization. Neural Comput & Applic. 2014;25(2):297–308.CrossRefGoogle Scholar
  37. 37.
    Mirjalili, Seyedali, and Siti Zaiton Mohd Hashim. A new hybrid PSOGSA algorithm for function optimization. In Computer and information application (ICCIA), 2010 international conference on, pp. 374–377. IEEE, 2010.Google Scholar
  38. 38.
    Siddique N, Adeli H. Nature-inspired chemical reaction optimisation algorithms. Cogn Comput. 2017;9(4):411–22.CrossRefGoogle Scholar
  39. 39.
    Wu T, Yao M, Yang J. Dolphin swarm extreme learning machine. Cogn Comput. 2017;9(2):275–84.CrossRefGoogle Scholar
  40. 40.
    Karaboga, Dervis. An idea based on honey bee swarm for numerical optimization. Vol. 200. Technical report-tr06, Erciyes university, engineering faculty, computer engineering department, 2005.Google Scholar
  41. 41.
    Karaboga D, Basturk B. On the performance of artificial bee colony (ABC) algorithm. Appl Soft Comput. 2008;8(1):687–97.CrossRefGoogle Scholar
  42. 42.
    Mirjalili S. Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems. Neural Comput & Applic. 2016;27(4):1053–73.CrossRefGoogle Scholar
  43. 43.
    Floudas, Christodoulos A. Deterministic global optimization: theory, methods and applications. Vol. 37. Springer Science & Business Media, 2013.Google Scholar
  44. 44.
    Horst, Reiner, and Hoang Tuy. Global optimization: deterministic approaches. Springer Science & Business Media, 2013.Google Scholar
  45. 45.
    Ojha VK, Abraham A, Snášel V. Metaheuristic design of feedforward neural networks: a review of two decades of research. Eng Appl Artif Intell. 2017;60:97–116.CrossRefGoogle Scholar
  46. 46.
    Zhang N, Ding S, Shi Z. Denoising Laplacian multi-layer extreme learning machine. Neurocomputing. 2016;171:1066–74.CrossRefGoogle Scholar
  47. 47.
    Meng L, Ding S, Yu X. Research on denoising sparse autoencoder. Int J Mach Learn Cybern. 2017;8(5):1719–29.CrossRefGoogle Scholar
  48. 48.
    Malakooti B, Zhou Y. Approximating polynomial functions by feedforward artificial neural networks: capacity analysis and design. Appl Math Comput. 1998;90(1):27–51.Google Scholar
  49. 49.
    Isa NAM, Mamat WMFW. Clustered-hybrid multilayer perceptron network for pattern recognition application. Appl Soft Comput. 2011;11(1):1457–66.CrossRefGoogle Scholar
  50. 50.
    Melin P, Sánchez D, Castillo O. Genetic optimization of modular neural networks with fuzzy response integration for human recognition. Inf Sci. 2012;197:1–19.CrossRefGoogle Scholar
  51. 51.
    Guo ZX, Wong WK, Li M. Sparsely connected neural network-based time series forecasting. Inf Sci. 2012;193:54–71.CrossRefGoogle Scholar
  52. 52.
    Suganuma, Masanori, Mete Ozay, and Takayuki Okatani. Exploiting the potential of standard convolutional autoencoders for image restoration by evolutionary search. arXiv preprint arXiv:1803.003 70 (2018).Google Scholar
  53. 53.
    Wang, Yunhe, Chang Xu, Jiayan Qiu, Chao Xu, and Dacheng Tao.Towards evolutional compression arXiv preprint arXiv:1707.08005 (2017).Google Scholar
  54. 54.
    Real, Esteban, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc Le, and Alex Kurakin. Large-scale evolution of image classifiers. arXiv preprint arXiv:1703.01041 (2017).Google Scholar
  55. 55.
    Zhang J-R, Zhang J, Lok T-M, Lyu MR. A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Appl Math Comput. 2007;185(2):1026–37.Google Scholar
  56. 56.
    Mirjalili S, Mirjalili SM, Lewis A. Let a biogeography-based optimizer train your multi-layer perceptron. Inf Sci. 2014;269:188–209.CrossRefGoogle Scholar
  57. 57.
    Ampazis N, Perantonis SJ, Drivaliaris D. Improved Jacobian Eigen-analysis scheme for accelerating learning in feedforward neural networks. Cognitive Computation. 2015;7(1):86–102.CrossRefGoogle Scholar
  58. 58.
    Mirjalili S. How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl Intell. 2015;43(1):150–61.CrossRefGoogle Scholar
  59. 59.
    Zhu G, Kwong S. Gbest-guided artificial bee colony algorithm for numerical function optimization. Appl Math Comput. 2010;217(7):3166–73.Google Scholar
  60. 60.
    Li Z, Wang W, Yan Y, Zheng L. PS–ABC: a hybrid algorithm based on particle swarm and artificial bee colony for high-dimensional optimization problems. Expert Syst Appl. 2015;42(22):8881–95.CrossRefGoogle Scholar
  61. 61.
    Yılmaz S, Küçüksille EU. A new modification approach on bat algorithm for solving optimization problems. Appl Soft Comput. 2015;28:259–75.CrossRefGoogle Scholar
  62. 62.
    Wang G-G, Guo L, Gandomi AH, Hao G-S, Wang H. Chaotic krill herd algorithm. Inf Sci. 2014;274:17–34.CrossRefGoogle Scholar
  63. 63.
    Dheeru D and Karra Taniskidou E., {UCI} Machine Learning Repository. 2017.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computer ScienceUniversiti Sains MalaysiaPenangMalaysia
  2. 2.Faculty of EngineeringUniversity of AdenAdenYemen
  3. 3.Faculty of Education-SaberUniversity of AdenAdenYemen

Personalised recommendations