Skip to main content
SpringerLink
Log in

Advances of metaheuristic algorithms in training neural networks for industrial applications

  • Application of soft computing
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

In recent decades, researches on optimizing the parameter of the artificial neural network (ANN) model has attracted significant attention from researchers. Hybridization of superior algorithms helps improving optimization performance and capable of solving complex applications. As a traditional gradient-based learning algorithm, ANN suffers from a slow learning rate and is easily trapped in local minima when training techniques such as gradient descent (GD) and back-propagation (BP) algorithm are used. The characteristics of randomization and selection of the best or near-optimal solution of metaheuristic algorithm provide an effective and robust solution; therefore, it has always been used in training of ANN to improve and overcome the above problems. New metaheuristic algorithms are proposed every year. Therefore, the review of its latest developments is essential. This article attempts to summarize the metaheuristic algorithms which have been proposed from the year 1975 to 2020 from various journals, conferences, technical papers, and books. The comparison of the popularity of the metaheuristic algorithm is presented in two time frames, such as algorithms proposed in the recent 20 years and those proposed earlier. Then, some of the popular metaheuristic algorithms and their working principle are reviewed. This article further categorizes the latest metaheuristic search algorithm in the literature to indicate their efficiency in training ANN for various industry applications. More and more researchers tend to develop new hybrid optimization tools by combining two or more metaheuristic algorithms to optimize the training parameters of ANN. Generally, the algorithm’s optimal performance must be able to achieve a fine balance of their exploration and exploitation characteristics. Hence, this article tries to compare and summarize the properties of various metaheuristic algorithms in terms of their convergence rate and the ability to avoid the local minima. This information is useful for researchers working on algorithm hybridization by providing a good understanding of the convergence rate and the ability to find a global optimum.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Availability of data and material

NA.

Code availability

NA.

References

  • Abdual-Salam ME, Abdul-Kader HM, Abdel-Wahed WF (2010) Comparative study between differential evolution and particle swarm optimization algorithms in training of feed-forward neural network for stock price prediction. In: 2010 The 7th International Conference on Informatics and Systems (INFOS). IEEE

  • Abedinia O, Amjady N, Ghasemi A (2016) A new metaheuristic algorithm based on shark smell optimization. Complexity 21(5):97–116

    MathSciNet  Google Scholar 

  • Abraham A, Grosan C, Pedrycz W (2008) Engineering evolutionary intelligent systems. Springer, Berlin Heidelberg

    MATH  Google Scholar 

  • Afrakhteh S et al (2020) Accurate classification of EEG signals using neural networks trained by hybrid population-physic-based algorithm. Int J Autom Comput 17(1):108–122

    Google Scholar 

  • Aguilar-Rivera R, Valenzuela-Rendón M, Rodríguez-Ortiz JJ (2015) Genetic algorithms and Darwinian approaches in financial applications: a survey. Expert Syst Appl 42(21):7684–7697

    Google Scholar 

  • Ahmed MH, Hasan S, Ali A (2015) Learning enhancement of radial basis function neural network with harmony search algorithm. Int J Adv Soft Comput Appl. 7(1):98

    Google Scholar 

  • Akinosho TD et al (2020) Deep learning in the construction industry: a review of present status and future innovations. J Build Eng 32:101827

    Google Scholar 

  • Al-Betar MA et al (2016) Tournament-based harmony search algorithm for non-convex economic load dispatch problem. Appl Soft Comput 47:449–459

    Google Scholar 

  • Al-Betar MA et al (2018) Economic load dispatch problems with valve-point loading using natural updated harmony search. Neural Comput Appl 29(10):767–781

    Google Scholar 

  • Aladag CH (2011) A new architecture selection method based on tabu search for artificial neural networks. Expert Syst Appl 38(4):3287–3293

    Google Scholar 

  • Alia OM, Mandava R, Aziz ME (2010) A hybrid Harmony Search algorithm to MRI brain segmentation. In: 2010 9th IEEE International Conference on Cognitive Informatics (ICCI)

  • Alpaydin E (2004) Introduction to machine learning. MIT Press, Cambridge

    MATH  Google Scholar 

  • Alweshah M (2014) Firefly algorithm with artificial neural network for time series problems. Res J Appl Sci Eng Technol 7(19):3978–3982

    Google Scholar 

  • Amali DGB, Dinakaran M (2016) A review of heuristic global optimization based artificial neural network training approaches. Int J Pharm Technol 8(4):21670–21679

    Google Scholar 

  • Ansari A et al (2020) A hybrid metaheuristic method in training artificial neural network for bankruptcy prediction. IEEE Access 8:176640–176650

    Google Scholar 

  • Apostolopoulos T, Vlachos A (2010) Application of the firefly algorithm for solving the economic emissions load dispatch problem. Int J Combin. https://doi.org/10.1155/2011/523806

    Article  MATH  Google Scholar 

  • Arora S, Singh S (2019) Butterfly optimization algorithm: a novel approach for global optimization. Soft Comput 23(3):715–734

    Google Scholar 

  • Askari Q, Younas I, Saeed M (2020) Political optimizer: a novel socio-inspired meta-heuristic for global optimization. Knowl-Based Syst 195:105709

    Google Scholar 

  • Askarzadeh A (2016) A novel metaheuristic method for solving constrained engineering optimization problems: crow search algorithm. Comput Struct 169:1–12

    Google Scholar 

  • Askarzadeh A, Rezazadeh A (2013) Artificial neural network training using a new efficient optimization algorithm. Appl Soft Comput 13(2):1206–1213

    Google Scholar 

  • Atashpaz-Gargari E, Lucas C (2007) Imperialist competitive algorithm: An algorithm for optimization inspired by imperialistic competition. In: 2007 IEEE Congress on Evolutionary Computation

  • Awadallah MA et al (2017) Hybridization of harmony search with hill climbing for highly constrained nurse rostering problem. Neural Comput Appl 28(3):463–482

    Google Scholar 

  • Ayvaz MT (2009) Application of harmony search algorithm to the solution of groundwater management models. Adv Water Resour 32(6):916–924

    Google Scholar 

  • Bahrami S, Doulati Ardejani F, Baafi E (2016) Application of artificial neural network coupled with genetic algorithm and simulated annealing to solve groundwater inflow problem to an advancing open pit mine. Journal of Hydrology 536:471–484

    Google Scholar 

  • Bai Q (2010) Analysis of particle swarm optimization algorithm. Comput Inf Sci. https://doi.org/10.5539/cis.v3n1p180

    Article  Google Scholar 

  • Bashir ZA, El-Hawary ME (2009) Applying wavelets to short-term load forecasting using PSO-based neural networks. IEEE Trans Power Syst 24(1):20–27

    Google Scholar 

  • Basu M (2016) Quasi-oppositional differential evolution for optimal reactive power dispatch. Int J Electr Power Energy Syst 78:29–40

    Google Scholar 

  • Battiti R, Tecchiolli G (1994) The reactive tabu search. ORSA J Comput 6(2):126–140

    MATH  Google Scholar 

  • Beheshti Z, Shamsuddin SMH (2013) A review of population-based meta-heuristic algorithms. Int J Adv Soft Comput Appl 5(1):1–35

    Google Scholar 

  • Bensingh RJ et al (2019) Injection molding process optimization of a bi-aspheric lens using hybrid artificial neural networks (ANNs) and particle swarm optimization (PSO). Measurement 134:359–374

    Google Scholar 

  • Benyelloul K, Aourag H (2013) Bulk modulus prediction of austenitic stainless steel using a hybrid GA–ANN as a data mining tools. Comput Mater Sci 77:330–334

    Google Scholar 

  • Bhargava V, Fateen S-EK, Bonilla-Petriciolet A (2013) Cuckoo search: a new nature-inspired optimization method for phase equilibrium calculations. Fluid Phase Equilib 337:191–200

    Google Scholar 

  • Bhesdadiya R et al (2018) Training multilayer perceptrons in neural network using interior search algorithm. Advances in Computer and Computational Sciences. Springer, pp 69–77

    Google Scholar 

  • Bhoskar MT et al (2015) Genetic algorithm and its applications to mechanical engineering: a review. Mater Today Proc 2(4–5):2624–2630

    Google Scholar 

  • Biglari M et al (2013) Solving blasius differential equation by using hybrid neural network and gravitational search algorithm (HNNGSA). Global J Sci Eng Technol 11:29–36

    Google Scholar 

  • Bolaji AL et al (2014) University course timetabling using hybridized artificial bee colony with hill climbing optimizer. J Comput Sci 5(5):809–818

    Google Scholar 

  • Bolaji AL, Ahmad AA, Shola PB (2018) Training of neural network for pattern classification using fireworks algorithm. Int J Syst Assur Eng Manag 9(1):208–215

    Google Scholar 

  • Bousmaha R, Hamou RM, Amine A (2019) Training feedforward neural networks using hybrid particle swarm optimization, multi-verse optimization. In: CITSC

  • Brajevic I, Tuba M (2013) Training feed-forward neural networks using firefly algorithm. In: Proceedings of the 12th International Conference on Artificial Intelligence, Knowledge Engineering and Data Bases (AIKED’13)

  • Brammya G et al (2019) Deer hunting optimization algorithm: a new nature-inspired meta-heuristic paradigm. Comput J. https://doi.org/10.1093/comjnl/bxy133

    Article  Google Scholar 

  • Busetti F (2003) Simulated annealing overview. JP Morgan, Italy

    Google Scholar 

  • Buyukozkan K et al (2016) Lexicographic bottleneck mixed-model assembly line balancing problem: artificial bee colony and tabu search approaches with optimised parameters. Expert Syst Appl 50:151–166

    Google Scholar 

  • Cantu-Paz E, Kamath C (2005) An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Trans Syst Man Cybern Part b Cybern 35(5):915–927

    Google Scholar 

  • Cao C et al (2018) Deep learning and its applications in biomedicine. Genom Proteom Bioinf 16(1):17–32

    Google Scholar 

  • Carvalho M, Ludermir TB (2007) Particle swarm optimization of neural network architectures andweights. In: 7th international conference on hybrid intelligent systems, 2007. HIS 2007

  • Castellani M, Rowlands H (2009) Evolutionary Artificial Neural Network Design and Training for wood veneer classification. Eng Appl Artif Intell 22(4–5):732–741

    Google Scholar 

  • Catalbas MC, Gulten A (2018) Circular structures of puffer fish: a new metaheuristic optimization algorithm. In: 2018 Third international conference on electrical and biomedical engineering, clean energy and green computing (EBECEGC)

  • Ceylan H et al (2008) Transport energy modeling with meta-heuristic harmony search algorithm, an application to Turkey. Energy Policy 36(7):2527–2535

    Google Scholar 

  • Chaki S, Ghosal S (2011) Application of an optimized SA-ANN hybrid model for parametric modeling and optimization of LASOX cutting of mild steel. Prod Eng Res Devel 5(3):251–262

    Google Scholar 

  • Chakraborty S, Mitra A (2018) Parametric optimization of abrasive water-jet machining processes using grey wolf optimizer. Mater Manuf Processes 33(13):1471–1482

    Google Scholar 

  • Chandrasekar K, Ramana N (2012) Performance comparison of GA, DE, PSO and SA approaches in enhancement of total transfer capability using facts devices. J Electr Eng Technol 7(4):493–500

    Google Scholar 

  • Chatterjee A, Mahanti GK, Chatterjee A (2012) Design of a fully digital controlled reconfigurable switched beam concentric ring array antenna using firefly and particle swarm optimization algorithm. Prog Electromagn Res B 36:113–131

    Google Scholar 

  • Chen R et al (2018) Intelligent fault diagnosis of gearbox based on improved fireworks algorithm and probabilistic neural network. Trans Chin Soc Agric Eng 34(17):192–198

    Google Scholar 

  • Chen J, Cai H, Wang W (2018) A new metaheuristic algorithm: car tracking optimization algorithm. Soft Comput 22(12):3857–3878

    Google Scholar 

  • Chen QH Do, Hsieh HN (2015) Training artificial neural networks by a hybrid PSO-CS algorithm. Algorithms 8(2):292–308

    MathSciNet  MATH  Google Scholar 

  • Chen et al (2008) A novel hybrid Evolutionary Algorithm based on PSO and AFSA for feedforward neural network training. In: 4th International Conference on Wireless Communications, Networking and Mobile Computing, 2008. WiCOM'08. IEEE

  • Cheng M-Y, Prayogo DJC (2014) Structures, symbiotic organisms search: a new metaheuristic optimization algorithm. Comput Struct 139:98–112

    Google Scholar 

  • Civicioglu P (2012) Transforming geocentric cartesian coordinates to geodetic coordinates by using differential search algorithm. Comput Geosci 46:229–247

    Google Scholar 

  • Civicioglu P (2013) Artificial cooperative search algorithm for numerical optimization problems. Inf Sci 229:58–76

    MATH  Google Scholar 

  • Cobo P, Moraes E, Simón F (2015) Inverse estimation of the non-acoustical parameters of loose granular absorbers by simulated annealing. Build Environ 94:859–866

    Google Scholar 

  • Codreanu I (2005) A parallel between differential evolution and genetic algorithms with exemplification in a microfluidics optimization problem. In: 2005 International Semiconductor Conference, 2005. CAS 2005 Proceedings. IEEE

  • Cogill R, Hindi H (2007) Optimal routing and scheduling in flexible manufacturing systems using integer programming. In: 2007 46th IEEE Conference on Decision and Control

  • Dai L et al (2017) Deep learning for speech recognition: review of state-of-the-arts technologies and prospects. J Data Acquisit Process 2(2):1004–9037

    Google Scholar 

  • Das S et al (2009) Differential evolution using a neighborhood-based mutation operator. IEEE Trans Evol Comput 13(3):526–553

    Google Scholar 

  • Das R et al (2016) Application of artificial bee colony algorithm for maximizing heat transfer in a perforated fin. Proc Inst Mech Eng Part E: J Process Mech Eng. 232(1):38–48

    Google Scholar 

  • Das S, Mullick SS, Suganthan P (2016) Recent advances in differential evolution—an updated survey. Swarm Evol Comput 27:1–30

    Google Scholar 

  • Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–31

    Google Scholar 

  • Dastanpour A et al (2014) Using gravitational search algorithm to support artificial neural network in intrusion detection system. Smart Comput Rev 4(6):426–434

    Google Scholar 

  • de Lima AMM et al (2008) A nuclear reactor core fuel reload optimization using artificial ant colony connective networks. Ann Nucl Energy 35(9):1606–1612

    Google Scholar 

  • Deng Y, Wu J, Tan Y-J (2016) Optimal attack strategy of complex networks based on tabu search. Phys A 442:74–81

    MathSciNet  MATH  Google Scholar 

  • Devendiran S et al (2015) Bearing fault diagnosis using CWT, BGA and Artificial Bee Colony Algorithm. Int J Mech Mechatron Eng. 15(3)

  • Dhiman G, Kumar V (2017) Spotted hyena optimizer: a novel bio-inspired based metaheuristic technique for engineering applications. Adv Eng Softw 114:48–70

    Google Scholar 

  • Diop L et al (2020) Annual rainfall forecasting using hybrid artificial intelligence model: integration of multilayer perceptron with whale optimization algorithm. Water Resour Manage 34(2):733–746

    Google Scholar 

  • Do Q (2015) A hybrid gravitational search algorithm and back-propagation for training feedforward neural networks. In: Nguyen V-H, Le A-C, Huynh V-N (eds) Knowledge and Systems Engineering. Springer International Publishing, Berlin, pp 381–392

    Google Scholar 

  • Dorigo M (1992) Optimization, learning and natural algorithms. Ph. D. Thesis, Politecnico di Milano, Italy

  • Doğan B, Ölmez T (2015) A new metaheuristic for numerical function optimization: vortex search algorithm. Comput Oper Res 293:125–145

    Google Scholar 

  • Duman E, Uysal M, Alkaya AF (2012) Migrating birds Optimization: a new metaheuristic approach and its performance on quadratic assignment problem. Inf Sci 217:65–77

    MathSciNet  Google Scholar 

  • Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 1995. MHS '95

  • Ebrahimi A, Khamehchi E (2016) Sperm whale algorithm: an effective metaheuristic algorithm for production optimization problems. J Nat Gas Sci Eng 29:211–222

    Google Scholar 

  • Eker E et al (2020) Training multi-layer perceptron using harris hawks optimization. In: 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA)

  • Esen H et al (2008) Artificial neural networks and adaptive neuro-fuzzy assessments for ground-coupled heat pump system. Energy Buildings 40(6):1074–1083

    Google Scholar 

  • Fan Q, Zhang Y (2016) Self-adaptive differential evolution algorithm with crossover strategies adaptation and its application in parameter estimation. Chemom Intell Lab Syst 151:164–171

    Google Scholar 

  • Fan C, Zhou Y, Tang Z (2020) Neighborhood centroid opposite-based learning Harris Hawks optimization for training neural networks. Evol Intell 2020:1–21

    Google Scholar 

  • Fang S, Zhang X (2016) A Hybrid Algorithm of Particle swarm optimization and tabu search for distribution network reconfiguration. Math Problems Eng 2016:1–7

    Google Scholar 

  • Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell 45(2):322–332

    Google Scholar 

  • Faris H et al (2018) Grey wolf optimizer: a review of recent variants and applications. Neural Comput Appl 30(2):413–435

    Google Scholar 

  • Farmer JD, Packard NH, Perelson AS (1986) The immune system, adaptation, and machine learning. Phys D 22(1):187–204

    MathSciNet  Google Scholar 

  • Farshidpour S, Keynia F (2012) Using artificial bee colony algorithm for MLP training on software defect prediction. Oriental J Comput Sci Technol 5(2):231–239

    Google Scholar 

  • Fathollahi-Fard AM, Hajiaghaei-Keshteli M, Tavakkoli-Moghaddam R (2020) Red deer algorithm (RDA): a new nature-inspired meta-heuristic. Soft Comput 24(19):14637–14665

    Google Scholar 

  • Fesanghary M, Damangir E, Soleimani I (2009) Design optimization of shell and tube heat exchangers using global sensitivity analysis and harmony search algorithm. Appl Therm Eng 29(5):1026–1031

    Google Scholar 

  • Gambhir S, Malik SK, Kumar Y (2017) PSO-ANN based diagnostic model for the early detection of dengue disease. New Horizons Trans Med 4(1):1–8

    Google Scholar 

  • Gandomi AH (2014) Interior search algorithm (ISA): a novel approach for global optimization. ISA Trans 53(4):1168–1183

    Google Scholar 

  • Gandomi AH, Alavi AH (2012) Krill herd: a new bio-inspired optimization algorithm. Commun Nonlinear Sci Numer Simul 17(12):4831–4845

    MathSciNet  MATH  Google Scholar 

  • Gandomi AH, Yang X-S, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29(1):17–35

    Google Scholar 

  • Gandomi et al (2015) Optimization of retaining wall design using recent swarm intelligence techniques. Eng Struct 103:72–84

    Google Scholar 

  • Gao H et al (2014) A hybrid particle-swarm tabu search algorithm for solving job shop scheduling problems. IEEE Trans Industr Inf 10(4):2044–2054

    Google Scholar 

  • Geem ZW (2009a) Multiobjective optimization of time-cost trade-off using harmony search. J Construct Eng Manag 136(6):711–716

    Google Scholar 

  • Geem ZW (2009b) Harmony search algorithms for structural design optimization. Springer, Berlin Heidelberg

    Google Scholar 

  • Geem ZW, Kim JH, Loganathan G (2001) A new heuristic optimization algorithm: harmony search. Simulation 76(2):60–68

    Google Scholar 

  • Gen M, Cheng R (1997) Genetic algorithms and engineering design. Wiley, Newyork

    Google Scholar 

  • Ghalambaz M et al (2011) A hybrid neural network and gravitational search algorithm (HNNGSA) method to solve well known Wessinger’s equation. World Acad Sci Eng Technol 73:803–807

    Google Scholar 

  • Ghanem WAH, Jantan A (2014) Using hybrid artificial bee colony algorithm and particle swarm optimization for training feed-forward neural networks. J Theor Appl Inf Technol 67(3)

  • Ghanem WAHM et al (2020) An efficient intrusion detection model based on hybridization of artificial bee colony and dragonfly algorithms for training multilayer perceptrons. IEEE Access 8:130452–130475

    Google Scholar 

  • Gholizadeh S, Barati H (2012) A comparative study of three metaheuristics for optimum design of trusses. Int J Optim Civil Eng 3(3):423–441

    Google Scholar 

  • Glover FJD (1977) Heuristics for integer programming using surrogate constraints. Decis Sci 8(1):156–166

    Google Scholar 

  • Glover F (1986) Future paths for integer programming and links to artificial intelligence. Comput Oper Res 13(5):533–549

    MathSciNet  MATH  Google Scholar 

  • Glover F (1989) Tabu search—part I. ORSA J Comput 1(3):190–206

    MATH  Google Scholar 

  • Glover F (1990) Tabu search—part II. ORSA J Comput 2(1):4–32

    MATH  Google Scholar 

  • Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley Publishing Company, Boston

    MATH  Google Scholar 

  • González B et al (2015) Fuzzy logic in the gravitational search algorithm for the optimization of modular neural networks in pattern recognition. Expert Syst Appl 42(14):5839–5847

    Google Scholar 

  • Grzechca D (2011) Simulated annealing with artificial neural network fitness function for ECG amplifier testing. In: 2011 20th European Conference on Circuit Theory and Design (ECCTD)

  • Gudise VG, Venayagamoorthy GK (2003) Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks. In: Swarm Intelligence Symposium, 2003. SIS '03. Proceedings of the 2003 IEEE

  • Haddad OB, Afshar A, Mariño MA (2006) Honey-bees mating optimization (HBMO) algorithm: a new heuristic approach for water resources optimization. Water Resour Manage 20(5):661–680

    Google Scholar 

  • Haghnegahdar L, Wang Y (2020) A whale optimization algorithm-trained artificial neural network for smart grid cyber intrusion detection. Neural Comput Appl 32(13):9427–9441

    Google Scholar 

  • Hai-Jew S (2014) Enhancing qualitative and mixed methods research with technology. IGI Global, Pennsylvania

    Google Scholar 

  • Halliday D, Resnick R, Walker J (1994) Fundamentals of physics. Wiley, New York

    MATH  Google Scholar 

  • Hamdan S et al (2017) On the performance of artificial neural network with sine-cosine algorithm in forecasting electricity load demand. In: 2017 International Conference on Electrical and Computing Technologies and Applications (ICECTA)

  • Hanseth O, Aanestad M (2001) Bootstrapping networks, communities and infrastructures. On the evolution of ICT solutions in health care. In: Proceedings of the 1st International Conference on Information Technology in Health Care (ITHC’01)

  • Harifi S et al (2019) Emperor Penguins colony: a new metaheuristic algorithm for optimization. Evol Intel 12(2):211–226

    Google Scholar 

  • Hassanin MF, Shoeb AM, Hassanien AE (2017) Designing multilayer feedforward neural networks using multi-verse optimizer. Handbook of Research on Machine Learning Innovations and Trends. IGI Global, Pennyslyvia, pp 1076–1093

    Google Scholar 

  • Hassanzadeh T, Vojodi H, Moghadam AME (2011) An image segmentation approach based on maximum variance intra-cluster method and firefly algorithm. In: 2011 Seventh International Conference on Natural Computation (ICNC). IEEE

  • Hatamlou A, Abdullah S, Nezamabadi-pour H (2011) Application of gravitational search algorithm on data clustering. In: Yao J et al (eds) Rough sets and knowledge technology. Springer, Berlin Heidelberg, pp 337–346

    Google Scholar 

  • Haykin S (1998) Neural networks: a comprehensive foundation. Hoboken, Prentice Hall PTR, p 842

    Google Scholar 

  • He Y et al (2005) Optimizing weights of neural network using an adaptive tabu search approach. In: Wang J, Liao X, Yi Z (Eds) Advances in neural networks—ISNN 2005: Second International Symposium on Neural Networks, Chongqing, China, May 30–June 1, 2005, Proceedings, Part I. Springer, Berlin, Heidelberg, pp 672–676

  • He S, Wu Q, Saunders J (2006) A novel group search optimizer inspired by animal behavioural ecology. In: 2006 IEEE international conference on evolutionary computation. IEEE

  • Heidari AA et al (2019) Harris hawks optimization: algorithm and applications. Futur Gener Comput Syst 97:849–872

    Google Scholar 

  • Hinton GE, Osindero S, Teh Y-WJN (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    MathSciNet  MATH  Google Scholar 

  • Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press, Ann Arbor

    MATH  Google Scholar 

  • Horng M-H et al (2012) Firefly meta-heuristic algorithm for training the radial basis function network for data classification and disease diagnosis. INTECH Open Access Publisher, London

    Google Scholar 

  • Horng M-H (2012) Vector quantization using the firefly algorithm for image compression. Expert Syst Appl 39(1):1078–1091

    Google Scholar 

  • Hosseini HS (2007) Problem solving by intelligent water drops. In: 2007 IEEE congress on evolutionary computation. IEEE

  • Hussain K et al (2018) Metaheuristic research: a comprehensive survey. Artif Intell Rev 52(4):2191–2233

    Google Scholar 

  • Husseinzadeh Kashan A (2015) A new metaheuristic for optimization: optics inspired optimization (OIO). Comput Oper Res 55:99–125

    MathSciNet  MATH  Google Scholar 

  • Ingber L (1993) Simulated annealing: practice versus theory. Math Comput Model 18(11):29–57

    MathSciNet  MATH  Google Scholar 

  • Irani R, Nasimi R (2011) Application of artificial bee colony-based neural network in bottom hole pressure prediction in underbalanced drilling. J Petrol Sci Eng 78(1):6–12

    Google Scholar 

  • Jadon SS et al (2017) Hybrid artificial bee colony algorithm with differential evolution. Appl Soft Comput 58:11–24

    Google Scholar 

  • Jain M, Singh V, Rani A (2019) A novel nature-inspired algorithm for optimization: squirrel search algorithm. Swarm Evol Comput 44:148–175

    Google Scholar 

  • Janakiraman S (2018) A hybrid ant colony and artificial bee colony optimization algorithm-based cluster head selection for IoT. Proc Comput Sci 143:360–366

    Google Scholar 

  • Jarraya B, Bouri A (2012) Metaheuristic optimization backgrounds: a literature review. Int J Contemp Bus Stud 3(12):31-44

    Google Scholar 

  • Javidy B, Hatamlou A, Mirjalili S (2015) Ions motion algorithm for solving optimization problems. Appl Soft Comput 32:72–79

    Google Scholar 

  • Jayaswal S, Agarwal P (2014) Balancing U-shaped assembly lines with resource dependent task times: a simulated annealing approach. J Manuf Syst 33(4):522–534

    Google Scholar 

  • Kalogirou SA (2000) Applications of artificial neural-networks for energy systems. Appl Energy 67(1–2):17–35

    Google Scholar 

  • Kankal M, Uzlu E (2017) Neural network approach with teaching–learning-based optimization for modeling and forecasting long-term electric energy demand in Turkey. Neural Comput Appl 28(1):737–747

    Google Scholar 

  • Karaboga D, Basturk B (2008) On the performance of artificial bee colony (ABC) algorithm. Appl Soft Comput 8(1):687–697

    Google Scholar 

  • Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. Technical Report-tr06, Erciyes University, Engineering Faculty: Kayseri, Turkey

  • Kashan AH (2009) League championship algorithm: a new algorithm for numerical function optimization. In: International Conference of Soft Computing and Pattern Recognition, 2009. SOCPAR'09. IEEE

  • Kassim N et al (2014) Harmony search-based optimization of artificial neural network for predicting AC power from a photovoltaic system. In: 2014 IEEE 8th International Power Engineering and Optimization Conference (PEOCO). IEEE

  • Kattan A, Abdullah R (2013) Training feed-forward artificial neural networks for pattern-classification using the harmony search algorithm. In: The Second International Conference on Digital Enterprise and Information Systems (DEIS2013). 2013. The Society of Digital Information and Wireless Communication

  • Kaveh A, Bakhshpoori T (2013) Optimum design of steel frames using Cuckoo Search algorithm with Lévy flights. Struct Design Tall Spec Build 22(13):1023–1036

    Google Scholar 

  • Kaveh A, Bakhshpoori T (2016) Water Evaporation Optimization: a novel physically inspired optimization algorithm. Comput Struct 167:69–85

    Google Scholar 

  • Kaveh A, Dadras A (2017) A novel meta-heuristic optimization algorithm: thermal exchange optimization. Adv Eng Softw 110:69–84

    Google Scholar 

  • Kaveh M, Khishe M, Mosavi MR (2019) Design and implementation of a neighborhood search biogeography-based optimization trainer for classifying sonar dataset using multi-layer perceptron neural network. Analog Integr Circuits Signal Process 100(2):405–428

    Google Scholar 

  • Kaveh A, Mahdavi VRJC (2014) Colliding bodies optimization: a novel meta-heuristic method. Comput Struct 139:18–27

    Google Scholar 

  • Kayarvizhy N, Kanmani S, Uthariaraj RV (2014) ANN models optimized using swarm intelligence algorithms. WSEAS Trans Comput 13:501–519

    Google Scholar 

  • Kennedy J, Eberhart R (1995) Particle swarm optimization. In IEEE International Conference on Neural Networks, 1995. Proceedings

  • Khajeh M, Golzary AR (2014) Synthesis of zinc oxide nanoparticles–chitosan for extraction of methyl orange from water samples: cuckoo optimization algorithm–artificial neural network. Spectrochim Acta Part A Mol Biomol Spectrosc 131:189–194

    Google Scholar 

  • Khajeh M, Hezaryan S (2013) Combination of ACO-artificial neural network method for modeling of manganese and cobalt extraction onto nanometer SiO2 from water samples. J Ind Eng Chem 19(6):2100–2107

    Google Scholar 

  • Khajeh M, Jahanbin E (2014) Application of cuckoo optimization algorithm–artificial neural network method of zinc oxide nanoparticles–chitosan for extraction of uranium from water samples. Chemom Intell Lab Syst 135:70–75

    Google Scholar 

  • Khajehzadeh M et al (2011) A survey on meta-heuristic global optimization algorithms. Res J Appl Sci Eng Technol 3(6):569–578

    Google Scholar 

  • Khalilpourazari S, Khalilpourazary S (2018) Optimization of production time in the multi-pass milling process via a Robust Grey Wolf Optimizer. Neural Comput Appl 29(12):1321–1336

    Google Scholar 

  • Khishe M, Mosavi M, Kaveh M (2017) Improved migration models of biogeography-based optimization for sonar dataset classification by using neural network. Appl Acoust 118:15–29

    Google Scholar 

  • Khishe M, Mosavi M, Moridi A (2018) Chaotic fractal walk trainer for sonar data set classification using multi-layer perceptron neural network and its hardware implementation. Appl Acoust 137:121–139

    Google Scholar 

  • Kiranyaz S et al (2009) Evolutionary artificial neural networks by multi-dimensional particle swarm optimization. Neural Netw 22(10):1448–1462

    Google Scholar 

  • Kirkpatrick S, Gelatt CD, Vecchi MPJ (1983) Optimization by simulated annealing. Science 220(4598):671–680

    MathSciNet  MATH  Google Scholar 

  • Kirkpatrick S, Vecchi MP (1983) Optimization by simmulated annealing. Science 220(4598):671–680

    MathSciNet  MATH  Google Scholar 

  • Kowalski PA, Łukasik S (2016) Training neural networks with krill herd algorithm. Neural Process Lett 44(1):5–17

    Google Scholar 

  • Koziel S, Yang XS (2011) Computational optimization, methods and algorithms. Springer, Berlin Heidelberg

    MATH  Google Scholar 

  • Krishnanand KN, Ghose D (2005) Detection of multiple source locations using a glowworm metaphor with applications to collective robotics. In: Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005. 2005. Pasadena, CA, USA: IEEE

  • Kulkarni O, Kulkarni S (2018) Process parameter optimization in WEDM by grey wolf optimizer. Mater Today Proc 5(2 part 1):4402–4412

    Google Scholar 

  • Kulluk S, Ozbakir L, Baykasoglu A (2012) Training neural networks with harmony search algorithms for classification problems. Eng Appl Artif Intell 25(1):11–19

    Google Scholar 

  • Kumar M, Kulkarni AJ, Satapathy SC (2018) Socio evolution and learning optimization algorithm: a socio-inspired optimization methodology. Futur Gener Comput Syst 81:252–272

    Google Scholar 

  • Kumar A, Pant S, Ram M (2019) Gray wolf optimizer approach to the reliability-cost optimization of residual heat removal system of a nuclear power plant safety system. Quality Reliab Eng Int 35(7):2228–2239

    Google Scholar 

  • Kumar K, Thakur GSM (2012) Advanced applications of neural networks and artificial intelligence: a review. Int J Inform Technol Comput Sci (IJITCS) 4(6):57–68

    Google Scholar 

  • Kumar A, Chakarverty S (2011) Design optimization for reliable embedded system using Cuckoo search. In: 2011 3rd International Conference on Electronics Computer Technology (ICECT). IEEE

  • Kumar et al (2010) Decision level biometric fusion using Ant Colony Optimization. In: 2010 17th IEEE international conference on image processing (ICIP)

  • Lahiri A, Chakravorti S (2005) A novel approach based on simulated annealing coupled to artificial neural network for 3-D electric-field optimization. IEEE Trans Power Delivery 20(3):2144–2152

    Google Scholar 

  • Lai DSW, Caliskan Demirag O, Leung JMY (2016) A tabu search heuristic for the heterogeneous vehicle routing problem on a multigraph. Transp Res Part E: Logis Transp Rev 86:32–52

    Google Scholar 

  • Lalithamma GA, Puttaswamy PS (2013) Literature review of applications of neural network in control systems. Int J Sci Res Publ 3(9):1–6

    Google Scholar 

  • LeCun Y, Bengio Y, Hinton GJ (2015) Deep learning. Nature 521(7553):436–444

    Google Scholar 

  • Lee KS, Geem ZW (2004) A new structural optimization method based on the harmony search algorithm. Comput Struct 82(9):781–798

    Google Scholar 

  • Lenin K, Ravindhranath Reddy B, Suryakalavathi M (2016) Hybrid Tabu search-simulated annealing method to solve optimal reactive power problem. Int J Electr Power Energy Syst 2016(82):87–91

    Google Scholar 

  • Li S et al (2007) A GA-based NN approach for makespan estimation. Appl Math Comput 185(2):1003–1014

    MATH  Google Scholar 

  • Li Z et al (2015) PS–ABC: a hybrid algorithm based on particle swarm and artificial bee colony for high-dimensional optimization problems. Exp Syst Appl 42(22):8881–8895

    Google Scholar 

  • Li XL, Shao ZJ, Qian JX (2002) An optimizing method based on autonomous animats: fish-swarm algorithm. Syst Eng Theory Pract 22(11):32–38

    Google Scholar 

  • Li et al (2013) Evaluation of an environment-aware sequence-based localization algorithm for building fire emergency scenarios. In: Proc of the 30th International Conference on Application of IT in the AEC Industry (CIB W78 2013)

  • Lin S-W, Vincent FY (2015) A simulated annealing heuristic for the multiconstraint team orienteering problem with multiple time windows. Appl Soft Comput 37:632–642

    Google Scholar 

  • Liptak BG (2005) Instrument engineers’ handbook, fourth edition, volume two: process control and optimization. CRC Press, Boca Raton

    Google Scholar 

  • Louis YHT et al (2019) Development of whale optimization neural network for daily water level forecasting. Int J Adv Trends Comput Sci Eng 8(3):354–362

    Google Scholar 

  • Lu C et al (2016) An effective multi-objective discrete grey wolf optimizer for a real-world scheduling problem in welding production. Adv Eng Softw 99:161–176

    Google Scholar 

  • Luo Q et al (2021) Using spotted hyena optimizer for training feedforward neural networks. Cogn Syst Res 65:1–16

    Google Scholar 

  • Lutfy OF (2020) A wavelet functional link neural network controller trained by a modified sine cosine algorithm using the feedback error learning strategy. J Eng Sci Technol 15(1):709–727

    MathSciNet  Google Scholar 

  • Lv L et al (2018) Solving vehicle routing problem through a tabu bee colony-based genetic algorithm. In: International Conference on Swarm Intelligence. Springer

  • Madić M, Marković D, Radovanović M (2013) Comparison of meta-heuristic algorithms for solving machining optimization problems. Facta Univ Ser Mech Eng 11(1):29–44

    Google Scholar 

  • Mahmood M, Al-Khateeb B (2019) The blue monkey: a new nature inspired metaheuristic optimization algorithm. Period Eng Nat Sci (PEN) 7(3):1054–1066

    Google Scholar 

  • Malinak P, Jaksa R (2007) Simultaneous gradient and evolutionary neural network weights adaptation methods. In: IEEE Congress on Evolutionary Computation, 2007. CEC 2007

  • Mandloi M, Bhatia V (2016) A low-complexity hybrid algorithm based on particle swarm and ant colony optimization for large-MIMO detection. Expert Syst Appl 50:66–74

    Google Scholar 

  • Manoochehri M, Kolahan F (2014) Integration of artificial neural network and simulated annealing algorithm to optimize deep drawing process. Int J Adv Manuf Technol 73(1–4):241–249

    Google Scholar 

  • McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133

    MathSciNet  MATH  Google Scholar 

  • Meetei KT (2014) A survey: swarm intelligence vs genetic algorithm. Int J Sci Res (IJSR) 3(5):231–5

    Google Scholar 

  • Meng A-B et al (2014) Crisscross optimization algorithm and its application. Knowl-Based Syst 67:218–229

    Google Scholar 

  • Minsky ML, Papert S (1988) Perceptrons: an introduction to computational geometry. MIT Press, Cambridge

    MATH  Google Scholar 

  • Mirjalili S (2015) How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl Intell 43(1):150–161

    Google Scholar 

  • Mirjalili S (2016a) Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems. Neural Comput Appl 27(4):1053–1073

    Google Scholar 

  • Mirjalili S (2016b) SCA: a sine cosine algorithm for solving optimization problems. Knowl-Based Syst 96:120–133

    Google Scholar 

  • Mirjalili S et al (2016) Multi-verse optimizer: a nature-inspired algorithm for global optimization. Neural Comput Appl 27(2):495–513

    Google Scholar 

  • Mirjalili S et al (2017) Salp Swarm algorithm: a bio-inspired optimizer for engineering design problems. Adv Eng Softw 114:163–191

    Google Scholar 

  • Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67

    Google Scholar 

  • Mirjalili S, Mirjalili SM, Lewis A (2014a) Let a biogeography-based optimizer train your multi-layer perceptron. Inf Sci 269:188–209

    MathSciNet  Google Scholar 

  • Mirjalili S, Mirjalili SM, Lewis A (2014b) Grey wolf optimizer. Adv Eng Softw 69:46–61

    Google Scholar 

  • Mirjalili SA, Sardroudi HM, Hashim SZM (2012) Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput 218(22):11125–11137

    MathSciNet  MATH  Google Scholar 

  • Moghdani R, Salimifard K (2018) Volleyball premier league algorithm. Appl Soft Comput 64:161–185

    Google Scholar 

  • Mohammadhassani J et al (2015) Prediction and reduction of diesel engine emissions using a combined ANN–ACO method. Appl Soft Comput 34:139–150

    Google Scholar 

  • Moharam A, El-Hosseini MA, Ali HA (2016) Design of optimal PID controller using hybrid differential evolution and particle swarm optimization with an aging leader and challengers. Appl Soft Comput 38:727–737

    Google Scholar 

  • Moravej Z, Akhlaghi A (2013) A novel approach based on cuckoo search for DG allocation in distribution network. Int J Electr Power Energy Syst 44(1):672–679

    Google Scholar 

  • Mosavi M et al (2016) Classification of sonar target using hybrid particle swarm and gravitational search. Mar Technol 3(1):1–13

    Google Scholar 

  • Mosavi M et al (2017) Training radial basis function neural network using stochastic fractal search algorithm to classify sonar dataset. Iran J Electr Electron Eng 13(1):100–111

    Google Scholar 

  • Mosavi MR, Khishe M, Akbarisani M (2017) Neural network trained by biogeography-based optimizer with chaos for sonar data set classification. Wireless Pers Commun 95(4):4623–4642

    Google Scholar 

  • Mosavi M, Khishe M, Ghamgosar A (2016) Classification of sonar data set using neural network trained by Gray Wolf Optimization. Neural Network World 26(4):393

    Google Scholar 

  • Mosbah H, El-Hawary M (2017) Optimization of neural network parameters by stochastic fractal search for dynamic state estimation under communication failure. Electr Power Syst Res 147:288–301

    Google Scholar 

  • Moscato PJC (1989) On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithms. C3P Report 826:1989

    Google Scholar 

  • Mostafaeipour A, Goli A, Qolipour M (2018) Prediction of air travel demand using a hybrid artificial neural network (ANN) with Bat and Firefly algorithms: a case study. J Supercomput 74(10):5461–5484

    Google Scholar 

  • Mousavirad SJ, Ebrahimpour-Komleh H (2017) Human mental search: a new population-based metaheuristic optimization algorithm. Appl Intell 47(3):850–887

    Google Scholar 

  • Muangkote N, Sunat K, Chiewchanwattana S (2014) An improved grey wolf optimizer for training q-Gaussian Radial Basis Functional-link nets. In: 2014 international computer science and engineering conference (ICSEC). IEEE

  • Mucherino A, Seref O (2007) Monkey search: a novel metaheuristic search for global optimization. In: AIP conference proceedings, American Institute of Physics

  • Murat YS, Ceylan H (2006) Use of artificial neural networks for transport energy demand modeling. Energy Policy 34(17):3165–3172

    Google Scholar 

  • Muro C et al (2011) Wolf-pack (Canis lupus) hunting strategies emerge from simple rules in computational simulations. Behav Proc 88(3):192–197

    Google Scholar 

  • Murugan R et al (2018) Hybridizing bat algorithm with artificial bee colony for combined heat and power economic dispatch. Appl Soft Comput 72:189–217

    Google Scholar 

  • Nakrani S, Tovey C (2004) On honey bees and dynamic server allocation in internet hosting centers. Adapt Behav 12(3–4):223–240

    Google Scholar 

  • Nassif AB et al (2019) Speech recognition using deep neural networks: A systematic review. IEEE Access 7:19143–19165

    Google Scholar 

  • Nawi NM, Khan A, Rehman M (2013) A new Levenberg Marquardt based back propagation algorithm trained with cuckoo search. Proc Technol 11:18–23

    Google Scholar 

  • Nawi NM, Rehman M (2014) CSBPRNN: a new hybridization technique using cuckoo search to train back propagation recurrent neural network. In: Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng-2013). Springer

  • Nur AS, Radzi NHM, Ibrahim AO (2014) Artificial neural network weight optimization: a review. TELKOMNIKA Indonesian J Electr Eng 12(9):6897–6902

    Google Scholar 

  • Nur AS, Radzi NHM, Shamsuddin SM (2015) Near optimal convergence of back-propagation method using harmony search algorithm. TELKOMNIKA Indonesian J Electr Eng 14(1):163–172

    Google Scholar 

  • Ojha VK, Abraham A, Snášel V (2017) Metaheuristic design of feedforward neural networks: a review of two decades of research. Eng Appl Artif Intell 60:97–116

    Google Scholar 

  • Ojugo A et al (2013) A hybrid artificial neural network gravitational search algorithm for rainfall runoffs modeling and simulation in hydrology. Progress Intell Comput Appl 2:22–33

    Google Scholar 

  • Osman IH, Laporte G (1996) Metaheuristics: A bibliography. Ann Oper Res 63(5):511–623

    MATH  Google Scholar 

  • Ouyang H-B et al (2016) Hybrid harmony search particle swarm optimization with global dimension selection. Inf Sci 346–347:318–337

    Google Scholar 

  • Özkaraca O (2018) A comparative evaluation of Gravitational Search Algorithm (GSA) against Artificial Bee Colony (ABC) for thermodynamic performance of a geothermal power plant. Energy 165:1061–1077

    Google Scholar 

  • Panchal A (2009) Harmony search in therapeutic medical physics. Music-inspired Harmony search algorithm. Springer, pp 189–203

    Google Scholar 

  • Panda M, Priyadarshini R, Pradhan S (2016) Autonomous mobile robot path planning using hybridization of particle swarm optimization and Tabu search. In: 2016 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC). IEEE

  • Parsopoulos KE (2010) Particle swarm optimization and intelligence: advances and applications. IGI Global, Pennsylvania

    Google Scholar 

  • Passino KM (2002) Biomimicry of bacterial foraging for distributed optimization and control. IEEE Control Syst Mag 22(3):52–67

    Google Scholar 

  • Pawar PJ, Vidhate US, Khalkar MY (2018) Improving the quality characteristics of abrasive water jet machining of marble material using multi-objective artificial bee colony algorithm. J Comput Design Eng 5(3):319–328

    Google Scholar 

  • Payne RB, Sorenson MD, Klitz K (2005) The Cuckoos. Oxford University Press, UK

    Google Scholar 

  • Pisinger D, Ropke S (2007) A general heuristic for vehicle routing problems. Comput Oper Res 34(8):2403–2435

    MathSciNet  MATH  Google Scholar 

  • Poli R, Kennedy J, Blackwell T (2007) Particle swarm optimization. Swarm Intell 1(1):33–57

    Google Scholar 

  • Punnathanam V, Kotecha P (2016) Yin-Yang-pair optimization: a novel lightweight optimization algorithm. Eng Appl Artif Intell 54:62–79

    Google Scholar 

  • Rabanal P, Rodríguez I, Rubio F (2007) Using river formation dynamics to design heuristic algorithms. In: International conference on unconventional computation. Springer, Berlin

  • Rao RV, Savsani VJ, Vakharia DP (2011) Teaching–learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43(3):303–315

    Google Scholar 

  • Rashedi E, Nezamabadi-pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179(13):2232–2248

    MATH  Google Scholar 

  • Ravakhah S et al (2017) Sonar false alarm rate suppression using classification methods based on interior search algorithm. Int J Comput Sci Netw Secur 17(7):58–65

    Google Scholar 

  • Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29(9):2352–2449

    MathSciNet  MATH  Google Scholar 

  • Razmjooy N, Ramezani M, Namadchian A (2016) A New LQR optimal control for a single-link flexible joint robot manipulator based on grey wolf optimizer. Majlesi J Electr Eng 10(3):53

    Google Scholar 

  • Rere LR, Fanany MI, Arymurthy AM (2015) Simulated annealing algorithm for deep learning. Proc Comput Sci 72:137–144

    Google Scholar 

  • Rere LMR, Fanany MI, Arymurthy AM (2016) Metaheuristic algorithms for convolution neural network. Comput Intell Neurosci 2016:1537325

    Google Scholar 

  • Reynolds CW (1987) Flocks, herds and schools: a distributed behavioral model. SIGGRAPH Comput Graph 21(4):25–34

    Google Scholar 

  • Rodan A, Faris H (2016) Optimizing feedforward neural networks using biogeography based optimization for e-mail spam identification. Int J Commun Netw Syst Sci 9(1):19–28

    Google Scholar 

  • Rubinstein RY (1997) Optimization of computer simulation models with rare events. Eur J Oper Res 99(1):89–112

    MATH  Google Scholar 

  • Saba S, Ahsan F, Mohsin S (2017) BAT-ANN based earthquake prediction for Pakistan region. Soft Comput 21(19):5805–5813

    Google Scholar 

  • Sadati N, Amraee T, Ranjbar AM (2009) A global particle swarm-based-simulated annealing optimization technique for under-voltage load shedding problem. Appl Soft Comput 9(2):652–657

    Google Scholar 

  • Said GA, Mahmoud AM, El-Horbaty ES (2014) A comparative study of meta-heuristic algorithms for solving quadratic assignment problem. arXiv preprint arXiv:1407.4863

  • Sait SM, Oughali FC, Al-Asli M (2016) Design partitioning and layer assignment for 3D integrated circuits using tabu search and simulated annealing. J Appl Res Technol 14(1):67–76

    Google Scholar 

  • Salimi H (2015) Stochastic fractal search: a powerful metaheuristic algorithm. Knowl-Based Syst 75:1–18

    Google Scholar 

  • Salmani MH, Eshghi K (2017) A smart structural algorithm SSA based on infeasible region to solve mixed integer problems. Int J Appl Metaheuristic Comput (IJAMC) 8(1):24–44

    Google Scholar 

  • Sangwan KS, Saxena S, Kant G (2015) Optimization of machining parameters to minimize surface roughness using integrated ANN-GA approach. Procedia CIRP 29:305–310

    Google Scholar 

  • Santhosh M, Venkaiah C, Kumar DV (2020) A hybrid forecasting model based on artificial neural network and teaching learning based optimization algorithm for day-ahead wind speed prediction. Intelligent Computing Techniques for Smart Energy Systems. Springer, pp 455–463

    Google Scholar 

  • Saremi S, Mirjalili S, Lewis A (2017) Grasshopper optimisation algorithm: theory and application. Adv Eng Softw 105:30–47

    Google Scholar 

  • Sattar D, Salim R (2020) A smart metaheuristic algorithm for solving engineering problems. Eng Comput. https://doi.org/10.1007/s00366-020-00951-x

    Article  Google Scholar 

  • Sayadi M, Ramezanian R, Ghaffari-Nasab N (2010) A discrete firefly meta-heuristic with local search for makespan minimization in permutation flow shop scheduling problems. Int J Ind Eng Comput 1(1):1–10

    Google Scholar 

  • Schutz B (2003) Gravity from the ground up: an introductory guide to gravity and general relativity. Cambridge University Press, Cambridge

    Google Scholar 

  • Sentinella MR (2007) Comparison and integrated use of differential evolution and genetic algorithms for space trajectory optimisation. In: IEEE Congress on Evolutionary Computation, 2007. CEC 2007. IEEE

  • Serani A, Diez M (2017) Dolphin pod optimization. In: International workshop on machine learning, optimization, and big data. Springer, Cham

  • Sexton RS et al (1998) Global optimization for artificial neural networks: a tabu search application. Eur J Oper Res 106(2–3):570–584

    MATH  Google Scholar 

  • Shadravan S, Naji HR, Bardsiri VK (2019) The sailfish optimizer: a novel nature-inspired metaheuristic algorithm for solving constrained engineering optimization problems. Eng Appl Artif Intell 80:20–34

    Google Scholar 

  • Shah-Hosseini H (2011) Principal components analysis by the galaxy-based search algorithm: a novel metaheuristic for continuous optimisation. Int J Comput Sci Eng 6(1/2):132–140

    Google Scholar 

  • Sharma N, Sharma H, Sharma A (2018) Beer froth artificial bee colony algorithm for job-shop scheduling problem. Appl Soft Comput 68:507–524

    Google Scholar 

  • Shen Q, Shi W-M, Kong W (2008) Hybrid particle swarm optimization and tabu search approach for selecting genes for tumor classification using gene expression data. Comput Biol Chem 32(1):53–60

    MATH  Google Scholar 

  • Simon D (2008) Biogeography-based optimization. IEEE Trans Evol Comput 12(6):702–713

    Google Scholar 

  • Simon D (2013) Evolutionary optimization algorithms. Wiley, New York

    Google Scholar 

  • Smith SF (1980) A learning system based on genetic adaptive algorithms. University of Pittsburgh, Pittsburgh

    Google Scholar 

  • Socha K, Blum C (2007) An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Comput Appl 16(3):235–247

    Google Scholar 

  • Soleimani H, Kannan G (2015) A hybrid particle swarm optimization and genetic algorithm for closed-loop supply chain network design in large-scale networks. Appl Math Model 39(14):3990–4012

    MathSciNet  MATH  Google Scholar 

  • Soltani HMAZ, Haghighat AT, Chegini T (2011) A couple of algorithms for k-coverage problem in visual sensor networks. In: International Conference on Communication Engineering and Networks

  • Spears WM (2000) Evolutionary algorithms: the role of mutation and recombination. Springer, Berlin Heidelberg

    MATH  Google Scholar 

  • Sreeraj P, Kannan T, Maji S (2013) Simulated annealing algorithm for optimization of welding variables for percentage of dilution and application of ANN for prediction of weld bead geometry in GMAW process. Int J Eng Res Appl (IJERA). 3(1):1360–1373

    Google Scholar 

  • Storn R, Price K (1997) Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359

    MathSciNet  MATH  Google Scholar 

  • Sudha L et al (2016) Optimization of process parameters in feed manufacturing using artificial neural network. Comput Electron Agric 120:1–6

    Google Scholar 

  • Sulaiman SI et al (2014) Cuckoo search for determining Artificial Neural Network training parameters in modeling operating photovoltaic module temperature. In: Proceedings of 2014 International Conference on Modelling, Identification & Control

  • Sulaiman SI et al (2015) Optimization of an Artificial Neural Network using Firefly Algorithm for modeling AC power from a photovoltaic system. In: SAI Intelligent Systems Conference (IntelliSys), 2015. IEEE

  • Taillard ÉD, Voss S (2002) POPMUSIC—Partial optimization metaheuristic under special intensification conditions. Essays and surveys in metaheuristics. Springer, pp 613–629

    Google Scholar 

  • Talbi E-G (2009) Metaheuristics: from design to implementation, vol 74. Wiley, NewYork

    MATH  Google Scholar 

  • Tamura K, Yasuda K (2011) Primary study of spiral dynamics inspired optimization. IEEJ Trans Electr Electron Eng 6(S1):S98–S100

    Google Scholar 

  • Tan Y, Zhu Y (2010) Fireworks algorithm for optimization. In: International conference in swarm intelligence. Springer

  • Tayarani-N MH, Akbarzadeh-T MR (2008) Magnetic optimization algorithms a new synthesis. In: 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence). IEEE

  • Tilahun SJP (2013) Prey predator algorithm: a new metaheuristic optimization approach. Universiti Sains Malaysia, Penang, Malaysia

    Google Scholar 

  • Ülker ED, Haydar A (2012) Comparison of the performances of differential evolution, particle swarm optimization and harmony search algorithms on benchmark functions. Acad Res Int 3(2):85–92

    Google Scholar 

  • Utamima A et al (2015) Distribution route optimization of gallon water using genetic algorithm and tabu search. Proc Comput Sci 72:503–510

    Google Scholar 

  • Uzlu E et al (2014) Estimates of hydroelectric generation using neural networks with the artificial bee colony algorithm for Turkey. Energy 69:638–647

    Google Scholar 

  • Verma SK, Yadav S, Nagar SK (2017) Optimization of fractional order PID controller using grey wolf optimizer. J Control Autom Electr Syst 28(3):314–322

    Google Scholar 

  • Vincent FY, Lin S-Y (2015) A simulated annealing heuristic for the open location-routing problem. Comput Oper Res 62:184–196

    MathSciNet  MATH  Google Scholar 

  • Voulodimos A et al (2018) Recent developments in deep learning for engineering applications. Comput Intell Neurosci 2018:8141259

    Google Scholar 

  • Walton S et al (2013) A review of the development and applications of the Cuckoo search algorithm. Swarm intelligence and bio-inspired computation theory and applications. Elsevier, New York, pp 257–271

    Google Scholar 

  • Wang JC, Chen TY (2015) A simulated annealing-based permutation method and experimental analysis for multiple criteria decision analysis with interval type-2 fuzzy sets. Appl Soft Comput 36:57–69

    Google Scholar 

  • Wang J-S, Li S-X (2019) An improved grey wolf optimizer based on differential evolution and elimination mechanism. Sci Rep 9(1):7181

    Google Scholar 

  • Wang X, Tang L (2016) An adaptive multi-population differential evolution algorithm for continuous multi-objective optimization. Inf Sci 348:124–141

    Google Scholar 

  • Wang G, Deb S, Coelho LDS (2015) Elephant herding optimization. In: 2015 3rd International Symposium on Computational and Business Intelligence (ISCBI)

  • Wasukar AR (2014) Artificial neural network—an important asset for future computing. Int J Res Emerg Sci Technol 1(1):28–34

    Google Scholar 

  • Whitley D (2001) An overview of evolutionary algorithms: practical issues and common pitfalls. Inf Softw Technol 43(14):817–831

    Google Scholar 

  • Wilbert S, Philip P (2012) Artificial neural networks—a review of applications of neural networks in the modeling of HIV epidemic. Int J Comput Appl 44(16):1–19

    Google Scholar 

  • Wu H et al (2016) Training feedforward neural networks using symbiotic organisms search algorithm. Comput Intell Neurosci 2016:9063065

    Google Scholar 

  • Wu J, Wei C (2015) Training artificial neural network using hybrid optimization algorithm for rainfall-runoff forecasting. In: Huang D-S, Bevilacqua V, Premaratne P (eds) Intelligent computing theories and methodologies: 11th international conference, ICIC 2015, Fuzhou, China, August 20–23, 2015, Proceedings, Part I. Springer International Publishing, Cham, pp 576–586

    Google Scholar 

  • Xhafa F et al (2015) Solving mesh router nodes placement problem in wireless mesh networks by tabu search algorithm. J Comput Syst Sci 81(8):1417–1428

    MathSciNet  MATH  Google Scholar 

  • Xie J (1997) A brief review on evolutionary computation. Control Decis 1:000

    Google Scholar 

  • Xin B et al (2010) An adaptive hybrid optimizer based on particle swarm and differential evolution for global optimization. Science China Inf Sci 53(5):980–989

    MathSciNet  MATH  Google Scholar 

  • Xu C et al (2019) Improving performance of retaining walls under dynamic conditions developing an optimized ann based on ant colony optimization technique. IEEE Access 7:94692–94700

    Google Scholar 

  • Xu BC, Zhang Y-Y (2014) An improved gravitational search algorithm for dynamic neural network identification. Int J Autom Comput 11(4):434–440

    Google Scholar 

  • Xu X, Li Y (2007) Comparison between particle swarm optimization, differential evolution and multi-parents crossover. In: 2007 International Conference on Computational Intelligence and Security. IEEE

  • Yaghini M, Khoshraftar MM, Fallahi M (2013) A hybrid algorithm for artificial neural network training. Eng Appl Artif Intell 26(1):293–301

    Google Scholar 

  • Yang XS (2008) Nature-inspired metaheuristic algorithms. Luniver Press, Beckington, p 128

    Google Scholar 

  • Yang XS (2010) Firefly algorithm, stochastic test functions and design optimisation. Int J Bio-Inspired Comput 2(2):78–84

    Google Scholar 

  • Yang XS, Deb S (2010) Engineering optimisation by cuckoo search. Int J Math Modell Numer Optim 1(4):330–343

    MATH  Google Scholar 

  • Yang X-S (2010) A new metaheuristic bat-inspired algorithm. In: González JR et al (eds) Nature inspired cooperative strategies for optimization (NICSO 2010). Springer, Berlin, pp 65–74

    Google Scholar 

  • Yang XS, Deb S (2009) Cuckoo search via Lévy flights. In: World Congress on nature and biologically inspired computing. NaBIC 2009. IEEE

  • Yang X-S (2012) Flower pollination algorithm for global optimization. In: International conference on unconventional computing and natural computation. Springer

  • Yao X (1993) A review of evolutionary artificial neural networks. Int J Intell Syst 8(4):539–567

    Google Scholar 

  • Yi W et al (2016) An improved adaptive differential evolution algorithm for continuous optimization. Expert Syst Appl 44:1–12

    Google Scholar 

  • Young T et al (2018) Recent trends in deep learning based natural language processing. IEEE Comput Intell Mag 13(3):55–75

    Google Scholar 

  • Yu JJQ, Li VOK (2015) A social spider algorithm for global optimization. Appl Soft Comput 30:614–627

    Google Scholar 

  • Yuce B et al (2013) Honey bees inspired optimization method: the bees algorithm. Insects 4(4):646–662

    Google Scholar 

  • Yuce B, Rezgui Y, Mourshed M (2016) ANN–GA smart appliance scheduling for optimised energy management in the domestic sector. Energy Build 111:311–325

    Google Scholar 

  • Yue Y et al (2016) Optimization-based artificial bee colony algorithm for data collection in large-scale mobile wireless sensor networks. J Sensors. https://doi.org/10.1155/2016/7057490

    Article  Google Scholar 

  • Zaman MA, Matin A (2012) Nonuniformly spaced linear antenna array design using firefly algorithm. Int J Microw Sci Technol. https://doi.org/10.1155/2012/256759

    Article  Google Scholar 

  • Zeidi JR et al (2013) A hybrid multi-objective approach based on the genetic algorithm and neural network to design an incremental cellular manufacturing system. Comput Ind Eng 66(4):1004–1014

    Google Scholar 

  • Zhang J et al (2018) Queuing search algorithm: a novel metaheuristic algorithm for solving engineering optimization problems. Appl Math Model 63:464–490

    MathSciNet  MATH  Google Scholar 

  • Zhang W et al (2020) An inspired machine-learning algorithm with a hybrid whale optimization for power transformer PHM. Energies 13(12):3143

    Google Scholar 

  • Zhang Y, Jin Z (2020) Group teaching optimization algorithm: a novel metaheuristic method for solving global optimization problems. Expert Syst Appl 148:113246

    Google Scholar 

  • Zheng Y-J (2015) Water wave optimization: a new nature-inspired metaheuristic. Comput Oper Res 55:1–11

    MathSciNet  MATH  Google Scholar 

  • Zhou Z-H (2004) Rule extraction: using neural networks or for neural networks? J Comput Sci Technol 19(2):249–253

    Google Scholar 

  • Zhou Y et al (2020) Teaching learning-based whale optimization algorithm for multi-layer perceptron neural network training. Math Biosci Eng 17(5):5987–6025

    MATH  Google Scholar 

  • Zorarpacı E, Özel SA (2016) A hybrid approach of differential evolution and artificial bee colony for feature selection. Expert Syst Appl 62:91–103

    Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge University of Malaya’s financial support of this project under Impact-Oriented Interdisciplinary Research Grant Programme (Grant No: IIRG001A-19IISS) and Ministry of Higher Education Malaysia (program myBrain15).

Funding

This research was funded by the University of Malaya Impact-Oriented Interdisciplinary Research Grant Programme (Grant No: IIRG001A-19IISS) and Ministry of Higher Education Malaysia (program myBrain15).

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Faculty of Engineering, University Malaya, 50603, Kuala Lumpur, Malaysia

    Hue Yee Chong & Hwa Jen Yap

  2. Faculty of Information Science and Technology, Multimedia University, 75450, Melaka, Malaysia

    Shing Chiang Tan

  3. Department of Electronics and Communication Engineering, College of Engineering, University Tenaga Nasional, 43000, Selangor, Malaysia

    Keem Siah Yap

  4. Department of Electrical and Electronics Engineering, Xiamen University Malaysia, 43900, Sepang, Malaysia

    Shen Yuong Wong

Authors
  1. Hue Yee Chong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hwa Jen Yap
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shing Chiang Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Keem Siah Yap
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shen Yuong Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

None.

Corresponding author

Correspondence to Hwa Jen Yap.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chong, H.Y., Yap, H.J., Tan, S.C. et al. Advances of metaheuristic algorithms in training neural networks for industrial applications. Soft Comput 25, 11209–11233 (2021). https://doi.org/10.1007/s00500-021-05886-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-05886-z

Keywords

Search

Navigation