Advertisement

Selfish herds optimization algorithm with orthogonal design and information update for training multi-layer perceptron neural network

  • Ruxin ZhaoEmail author
  • Yongli WangEmail author
  • Peng Hu
  • Hamed Jelodar
  • Chi Yuan
  • YanChao Li
  • Isma Masood
  • Mahdi Rabbani
Article
First Online:
  • 30 Downloads

Abstract

Selfish herd optimization algorithm is a novel meta-heuristic optimization algorithm, which simulates the group behavior of herds when attacked by predators in nature. With the further research of algorithm, it is found that the algorithm cannot get a better global optimal solution in solving some problems. In order to improve the optimization ability of the algorithm, we propose a selfish herd optimization algorithm with orthogonal design and information update (OISHO) in this paper. Through using orthogonal design method, a more competitive candidate solution can be generated. If the candidate solution is better than the global optimal solution, it will replace the global optimal solution. At the same time, at the end of each iteration, we update the population information of the algorithm. The purpose is to increase the diversity of the population, so that the algorithm expands its search space to find better solutions. In order to verify the effectiveness of the proposed algorithm, it is used to train multi-layer perceptron (MLP) neural network. For training multi-layer perceptron neural network, this is a challenging task to present a satisfactory and effective training algorithm. We chose twenty different datasets from UCI machine learning repository as training dataset, and the experimental results are compared with SSA, GG-GSA, GSO, GOA, WOA and SOS, respectively. Experimental results show that the proposed algorithm has better optimization accuracy, convergence speed and stability compared with other algorithms for training multi-layer perceptron neural network.

Keywords

Selfish herd optimization algorithm Orthogonal design Multi-layer perceptron (MLP) neural network Information update Meta-heuristic optimization algorithm 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions. This article has been awarded by the National Natural Science Foundation of China (61170035, 61272420, 81674099), the Fundamental Research Fund for the Central Universities (30916011328, 30918015103), and Nanjing Science and Technology Development Plan Project (201805036).

References

  1. 1.
    McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133MathSciNetzbMATHGoogle Scholar
  2. 2.
    Verpoort PC, MacDonald P, Conduit GJ (2018) Materials data validation and imputation with an artificial neural network. Comput Mater Sci 147:176–185Google Scholar
  3. 3.
    Manngard M, Kronqvist J, Jari M (2018) Böling. Structural learning in artificial neural networks using sparse optimization. Neurocomputing 272(10):660–667Google Scholar
  4. 4.
    Tavana M, Abtahi A-R, Di Caprio D, Poortarigh M (2018) An Artificial Neural Network and Bayesian Network model for liquidity risk assessment in banking. Neurocomputing 275(31):2525–2554Google Scholar
  5. 5.
    Leśniak A, Juszczyk M (2018) Prediction of site overhead costs with the use of artificial neural network based model. Arch Civil Mech Eng 18(3):973–982Google Scholar
  6. 6.
    Mulero Á, Pierantozzi M, Cachadiña I, Di Nicola G (2017) An Artificial Neural Network for the surface tension of alcohols. Fluid Phase Equilibria 449(15):28–40Google Scholar
  7. 7.
    Kim K-KK, Patrón ER, Braatz RD (2018) Standard representation and unified stability analysis for dynamic artificial neural network models. Neural Netw 98:251–262Google Scholar
  8. 8.
    Valero D, Bung DB (2018) Artificial Neural Networks and pattern recognition for air-water flow velocity estimation using a single-tip optical fibre probe. J Hydro-Environ Res 19:150–159Google Scholar
  9. 9.
    Lai CC (2018) Kuo L.Su. Development of an intelligent mobile robot localization system using Kinect RGB-D mapping and neural network. Comput Electr Eng 67:620–628Google Scholar
  10. 10.
    Reale C, Gavin K, Librić L, Jurić-Kaćunić D (2018) Automatic classification of fine-grained soils using CPT measurements and Artificial Neural Networks. Adv Eng Inform 36:207–215Google Scholar
  11. 11.
    Kaymak S, Helwan A, Uzun D (2017) Breast cancer image classification using artificial neural networks. Proc Comput Sci 120:126–131Google Scholar
  12. 12.
    Sitton JD, Zeinali Y, Brett A (2017) Story. Rapid soil classification using artificial neural networks for use in constructing compressed earth blocks. Construct Build Mater 138(1):214–221Google Scholar
  13. 13.
    Krishnan R, Dharani A (2016) Classification Analysis of Topographical Features Using Artificial Neural Network. Proc Technol 25:399–404Google Scholar
  14. 14.
    Hiew BY, Tan SC, Lim WS (2016) Intra-specific competitive co-evolutionary artificial neural network for data classification. Neurocomputing 185(12):220–230Google Scholar
  15. 15.
    Bardou D, Zhang K, Ahmad SM (2018) Lung sounds classification using convolutional neural networks. Artif Intell Med 88:58–69Google Scholar
  16. 16.
    Erkaymaz O, Ozer M, Perc M (2017) Performance of small-world feedforward neural networks for the diagnosis of diabetes. Appl Math Comput 311(15):22–28MathSciNetGoogle Scholar
  17. 17.
    Yang F, Yan L, Ling L (2018) Doubly stochastic radial basis function methods. J Comput Phys 363(15):87–97MathSciNetzbMATHGoogle Scholar
  18. 18.
    Kulkarni SR, Rajendran B (2018) Spiking neural networks for handwritten digit recognition—Supervised learning and network optimization. Neural Netw 103:118–127Google Scholar
  19. 19.
    Cheng S, Chi-Man P (2018) Multi-scale hierarchical recurrent neural networks for hyperspectral image classification. Neurocomputing 294(14):82–93Google Scholar
  20. 20.
    Nan W, Wenxiao S, Shaoshuai F, Shuxiang L (2011) PSO-FNN-based vertical handoff decision algorithm in heterogeneous wireless networks. Proc Environ Sci 11:55–62Google Scholar
  21. 21.
    Hameed AA, Karlik B, Salman MS (2016) Back-propagation algorithm with variable adaptive momentum. Knowledge-Based Syst 114(15):79–87Google Scholar
  22. 22.
    Montana DJ, Davis L (1989) Training feedforward neural networks using genetic algorithms. Proc Int Joint Conf Artif Intell (IJCAI '89), Detroit, Mich USA 89:762–767zbMATHGoogle Scholar
  23. 23.
    Li W (2018) Improving particle swarm optimization based on neighborhood and historical memory for training multi-layer perceptron. Information 20.  https://doi.org/10.3390/info9010016.
  24. 24.
    Gambhir S, Malik SK, Kumar Y (2017) PSO-ANN based diagnostic model for the early detection of dengue disease. New Horizons Translat Med 4(4):1–8Google Scholar
  25. 25.
    Wu H, Zhou Y, Luo Q, Basset MA (2016) Training feedforward neural networks using symbiotic organisms search algorithm. Comput Intell Neurosci Article ID 9063065, 14 pagesGoogle Scholar
  26. 26.
    Aljarah I, Faris H, Mirjalili S (2016) Optimizing connection weights in neural networks using the whale optimization algorithm. Soft Comput 22(1):1–15Google Scholar
  27. 27.
    Bohat VK, Arya KV (2018) An effective gbest-guide d gravitational search algorithm for real-parameter optimization and its application in training of feedforward neural networks. Knowledge-Based Syst 143:192–207Google Scholar
  28. 28.
    Alboaneen DA, Tianfield H, Zhang Y (2017) Glowworm Swarm Optimization for Training Multi-Layer Perceptrons. BDCAT’17, Session: Deep Learning, Austin, Texas, USA: 131–138Google Scholar
  29. 29.
    Heidari AA, Faris H, Aljarah I, Mirjalili S (2018) An efficient hybrid multilayer perceptron neural network with grasshopper optimization. Soft Computing.: 1–18Google Scholar
  30. 30.
    Abusnaina AA, Ahmad S, Jarrar R, Mafarja M (2018) Training neural networks using salp swarm algorithm for pattern classification. ICFNDS’18, Amman, JordanGoogle Scholar
  31. 31.
    Valian E, Mohanna S, Tavakoli S (2011) Improved cuckoo search algorithm for feedforward neural network training. Int J Artif Intell Appl 2(3):36–43Google Scholar
  32. 32.
    Mirjalili S (2015) How effective is the Grey Wolf optimizer in training multi-layer perceptions. Appl Intell 43(1):150–161Google Scholar
  33. 33.
    Moallem P, Razmjooy N (2012) A multi-layer perceptron neural network trained by invasive weed optimization for potato color image segmentation. Trends Appl Sci Res 7(6):445–455Google Scholar
  34. 34.
    Karaboga D, Akay B, Ozturk C (2007) Artificial Bee Colony (ABC) optimization algorithm for training feed-forward neural networks. Proc Int Conf Model Decisions Artif Intell(MDAI ’07), Springer, Kitakyushu, Japan: 318–329Google Scholar
  35. 35.
    Ozturk C, Karaboga D (2011) Hybrid artificial bee colony algorithm for neural network training. Proc IEEE Congress Evol Comput (CEC ’11), IEEE, New Orleans, LA, USA: 84–88Google Scholar
  36. 36.
    Griffiths EJ, Orponen P (2005) Optimization, block designs and No Free Lunch theorems. Inform Process Lett 94(2):55–61MathSciNetzbMATHGoogle Scholar
  37. 37.
    Fausto F, Cuevas E, Valdivia A, González A (2017) A global optimization algorithm inspired in the behavior of selfish herds. Biosystems 160:39–55Google Scholar
  38. 38.
    Araújo R d A, Oliveira ALI, Meira S (2017) A morphological neural network for binary classification problems. Eng Appl Artif Intell 65:12–28Google Scholar
  39. 39.
    Hamilton WD (1971) Geometry to the selfish herd. J Theory Biol 31(2):295–311Google Scholar
  40. 40.
    Feng Z k, Niu W j, Cheng C t, Liao S l (2017) Hydropower system operation optimization by discrete differential dynamic programming based on orthogonal experiment design. Energy 126(1):720–732Google Scholar
  41. 41.
    Deng L, Feng B, Zhang Y (2018) An optimization method for multi-objective and multi-factor designing of ceramic slurry: Combining orthogonal experimental design with artificial neural networks. Ceramics Int.  https://doi.org/10.1016/j.ceramint.2018.06.010
  42. 42.
    Ghaderpour E (2018) Constructions for orthogonal designs using signed group orthogonal designs. Discrete Math 341(1):277–285MathSciNetzbMATHGoogle Scholar
  43. 43.
    Tawfak L, Al-Bahrani, Jagdish C (2018) Patra. A novel orthogonal PSO algorithm based on orthogonal diagonalization. Swarm Evolution Comput 40:1–23Google Scholar
  44. 44.
    Xu K, Zhou J, Zhang Y, Gu R (2012) Differential evolution based on ε-domination and orthogonal design method for power environmentally-friendly dispatch. Expert Syst Applic 39(4):3956–3963Google Scholar
  45. 45.
    Juan D, Yang, Man-Ni, Yang, Shi-Fang (2016) Correlations and optimization of a heat exchanger with offset fins by genetic algorithm combining orthogonal design. Appl Thermal Eng 107(25):1091–1103Google Scholar
  46. 46.
    Villanueva J (2008) Kolmogorov theorem revisited. J Differ Equations 244(9):2251–2276MathSciNetzbMATHGoogle Scholar
  47. 47.
    Samet H, Hashemi F, Ghanbari T (2015) Minimum non detection zone for islanding detection using an optimal Artificial Neural Network algorithm based on PSO. Renew Sustain Energy Rev 52:1–18Google Scholar
  48. 48.
    Yeh I-C, Yang K-J, Ting T-M (2009) Knowledge discovery on RFM model using Bernoulli sequence. Expert Syst Appl 36(3):5866–5871Google Scholar
  49. 49.
    Siegler RS (1976) Three aspects of cognitive development. Cognit Psychol 8(4):481–520Google Scholar
  50. 50.
    Blake CL, Merz CJ (1998) UCI repository of machine learning databases. http://archive.ics.uci.edu/ml/datasets.html. Accessed 20 May 2018
  51. 51.
    Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugenics 7(Part 2):179–188Google Scholar
  52. 52.
    Niknam T, Olamaie J, Amiri B (2008) A hybrid evolutionary algorithm based on ACO and SA for cluster analysis. J Appl Sci 8(15):2695–2702Google Scholar
  53. 53.
    Derrac J, Gracie S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1:3–18Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringNanjing University of Science and TechnologyNanjingChina

Personalised recommendations

Cite article

Buy options