Skip to main content

A modified whale optimization algorithm to overcome delayed convergence in artificial neural networks

Abstract

Artificial neural network (ANN) is modeled to predict and classify problems. However, in the training phase of ANNs discovering faultless values of the weights of a network is extremely troublesome. Traditional weight updating methods often get stuck into local optima and converge to optimal solutions very slowly. Therefore, to overcome these drawbacks a modified version of a nature-based algorithm which merges meta-heuristics with weight-updating technique of ANN has been used in this paper. Whale optimization algorithm (WOA) is a well-established, efficient and competitive algorithm inspired by the hunting mechanism of the whales including their behavior in finding and attacking their prey with their bubble-net feeding technique. In WOA, the next location of the search individuals or whales is modified depending on some probability. Due to the high exploration rate of WOA, there is a disproportion between exploration and exploitation in the WOA and it also converges to the solution slowly. Thus, to establish an equilibrium between exploration and exploitation a new variant of WOA called modified whale optimization algorithm (MWOA) is proposed to overcome the problem of delayed convergence. In MWOA, roulette wheel selection is combined with WOA to enhance the convergence speed of WOA. MWOA is tested on 11 benchmark functions, and the outcomes are compared with WOA. The results prove that MWOA has gained success in overcoming the problem of the slow convergence of WOA. Also, the results show that the proposed MWOA technique, when applied to ANN, can overcome the problems of traditional techniques and has improved the results.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  • Abedinpourshotorban H, Shamsuddin SM, Beheshti Z, Jawawi DN (2016) Electromagnetic field optimization: a physics-inspired metaheuristic optimization algorithm. Swarm Evol Comput 26:8–22

    Google Scholar 

  • Abualigah L (2020) Group search optimizer: a nature-inspired meta-heuristic optimization algorithm with its results, variants, and applications. Neural Comput Appl 33:1–24

  • Aljarah I, Faris H, Mirjalili S (2018) Optimizing connection weights in neural networks using the whale optimization algorithm. Soft Comput 22(1):1–15

    Google Scholar 

  • Baluja S (1994) Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning. School of Computer Science, Carnegie Mellon University, 1994

  • Basheer IA, Hajmeer M (2000) Artificial neural networks: fundamentals, computing, design, and application. J Microbiol Methods 43(1):3–31

    Google Scholar 

  • Birjandi AK, Akhyani F, Sheikh R, Sana SS (2019) Evaluation and selecting the contractor in bidding with incomplete information using MCGDM method. Soft Comput 23(20):10569–10585

    Google Scholar 

  • Braik M, Sheta A, Arieqat, A (2008). A comparison between GAs and PSO in training ANN to model the TE chemical process reactor. In Proceedings of the AISB symposium on swarm intelligence algorithms and applications (pp. 24–30)

  • Chatterjee S, Sarkar S, Hore S, Dey N, Ashour AS, Balas VE (2017) Particle swarm optimization trained neural network for structural failure prediction of multistoried RC buildings. Neural Comput Appl 28(8):2005–2016

    Google Scholar 

  • Chou JS, Pham AD (2017) Nature-inspired metaheuristic optimization in least squares support vector regression for obtaining bridge scour information. Inf Sci 399:64–80

    Google Scholar 

  • Crepinsek M, Liu SH, Mernik M (2013) Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput Surv (CSUR) 45(3):1–33

    MATH  Google Scholar 

  • Dhiman G (2019) ESA: a hybrid bio-inspired metaheuristic optimization approach for engineering problems. Eng Comput 37:1–31

  • Dorigo M, Stutzle T (2019) Ant colony optimization: overview and recent advances. Handbook of metaheuristics, pp. 311–351

  • Erol OK, Eksin I (2006) A new optimization method: big bang-big crunch. Adv Eng Softw 37(2):106–111

    Google Scholar 

  • Faramarzi A, Heidarinejad M, Mirjalili S, Gandomi AH (2020) Marine Predators Algorithm: a nature-inspired metaheuristic. Expert Syst Appl 152:113377

    Google Scholar 

  • Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell 45(2):322–332

    Google Scholar 

  • Formato RA (2007) Central force optimization: a new metaheuristic with applications in applied electromagnetics. Prog Electromagn Res 77:425–491

    Google Scholar 

  • Ghasemiyeh R, Moghdani R, Sana SS (2017) A hybrid artificial neural network with metaheuristic algorithms for predicting stock price. Cybern Syst 48(4):365–392

    Google Scholar 

  • Haseli G, Sheikh R, Sana SS (2020) Base-criterion on multi-criteria decision-making method and its applications. Int J Manag Sci Eng Manag 15(2):79–88

    Google Scholar 

  • Jamali G, Sana SS, Moghdani R (2018) Hybrid improved cuckoo search algorithm and genetic algorithm for solving Markov-modulated demand. RAIRO-Operations Res 52(2):473–497

    MathSciNet  Google Scholar 

  • Kaur G, Arora S (2018) Chaotic whale optimization algorithm. J Comput Design Eng 5(3):275–284

    Google Scholar 

  • Kaveh A, Talatahari S (2010) A novel heuristic optimization method: charged system search. Acta Mechanica 213(3):267–289

    MATH  Google Scholar 

  • Kennedy J (2011) Encyclopedia of machine learning. Particle Swarm Optimization (pp. 760–766)

  • Kennedy J (2006) Swarm intelligence. Handbook of nature-inspired and innovative computing. Springer, Berlin, pp 187–219

    Google Scholar 

  • Kim JS, Jung S (2015) Implementation of the RBF neural chip with the back-propagation algorithm for on-line learning. Appl Soft Comput 29:233–244

    Google Scholar 

  • Kushwah R, Tapaswi S, Kumar A (2019) A detailed study on Internet connectivity schemes for mobile ad hoc network. Wireless Personal Commun 104(4):1433–1471

    Google Scholar 

  • Lee KC, Lu PT (2020) Application of Whale Optimization Algorithm to Inverse Scattering of an Imperfect Conductor with Corners. Int J Antennas Propagation 2020:1–9

  • Lee JG, Senel G, Lim PK, Kim J, Hur K (2020) Octahedron sets. Ann Fuzzy Math Inform 19(3):211–238

    MathSciNet  MATH  Google Scholar 

  • Li S, Gong W, Yan X, Hu C, Bai D, Wang L, Gao L (2019) Parameter extraction of photovoltaic models using an improved teaching-learning-based optimization. Energy Convers Manag 186:293–305

    Google Scholar 

  • Li H, Huang Z, Liu X, Zeng C, Zou P (2020) Multi-fidelity meta-optimization for nature inspired optimization algorithms. Appl Soft Comput 96:106619

    Google Scholar 

  • Ling Y, Zhou Y, Luo Q (2017) Levy flight trajectory-based whale optimization algorithm for global optimization. IEEE Access 5:6168–6186

    Google Scholar 

  • Lipowski A, Lipowska D (2012) Roulette-wheel selection via stochastic acceptance. Phys A Stat Mech Appl 391(6):2193–2196

    Google Scholar 

  • Mafarja M, Aljarah I, Heidari AA, Faris H, Fournier-Viger P, Li X, Mirjalili S (2018) Binary dragonfly optimization for feature selection using time-varying transfer functions. Knowledge-Based Syst 161:185–204

    Google Scholar 

  • Mareli M, Twala B (2018) An adaptive Cuckoo search algorithm for optimisation. Appl Comput Inf 14(2):107–115

    Google Scholar 

  • Maulik U, Bandyopadhyay S (2000) Genetic algorithm-based clustering technique. Pattern Recognit 33(9):1455–1465

    Google Scholar 

  • Mirjalili S (2019) Ant Colony Optimisation. In: Evolutionary Algorithms and Neural Networks, vol 780. pp 33–42. https://doi.org/10.1007/978-3-319-93025-1_3

  • Mirjalili S (2015) How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl Intell 43(1):150–161

    Google Scholar 

  • Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Software 95:51–67

    Google Scholar 

  • Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Advances in engineering software 69:46–61

    Google Scholar 

  • Mohammadi-Balani A, Nayeri MD, Azar A, Taghizadeh-Yazdi M (2021) Golden eagle optimizer: a nature-inspired metaheuristic algorithm. Comput Ind Eng 152:107050

    Google Scholar 

  • Ospina-Mateus H, Jimenez LAQ, Lopez-Valdes FJ, Garcia SB, Barrero LH, Sana SS (2021) Extraction of decision rules using genetic algorithms and simulated annealing for prediction of severity of traffic accidents by motorcyclists. J Ambient Intell Humanized comput 1–22. https://doi.org/10.1007/s12652-020-02759-5

  • Pandey AC, Rajpoot DS, Saraswat M (2017). Hybrid step size based cuckoo search. In 2017 Tenth International Conference on Contemporary Computing (IC3) (pp. 1–6). IEEE

  • Pandey AC, Rajpoot DS (2019) Spam review detection using spiral cuckoo search clustering method. Evol Intell 12(2):147–164

    Google Scholar 

  • Pencheva T, Atanassov K, Shannon A (2009) Modelling of a roulette wheel selection operator in genetic algorithms using generalized nets. Int J Bioautomation 13(4):257–264

    Google Scholar 

  • Rakitianskaia AS, Engelbrecht AP (2012) Training feedforward neural networks with dynamic particle swarm optimisation. Swarm Intell 6(3):233–270

    Google Scholar 

  • Rashedi E, Nezamabadi-Pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179(13):2232–2248

    MATH  Google Scholar 

  • Sana SS, Ospina-Mateus H, Arrieta FG, Chedid JA (2019) Application of genetic algorithm to job scheduling under ergonomic constraints in manufacturing industry. J Ambient Intell Humanized Comput 10(5):2063–2090

  • Sanel G, Lee JG, Hur K (2020) Distance and similarity measures for octahedron sets and their application to MCGDM problems. Mathematics 8(10):1690

    Google Scholar 

  • Sarkar BK, Sana SS, Chaudhuri K (2012) A genetic algorithm-based rule extraction system. Appl Soft Comput 12(1):238–254

    Google Scholar 

  • Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Networks 61:85–117

    Google Scholar 

  • Senel G (2016) A new approach to Hausdorff space theory via the soft sets. Math Problems Eng 2016:1–6. https://doi.org/10.1155/2016/2196743

  • Senel G (2018) The relation between soft topological space and soft ditopological space. Commun Faculty Sci Univ Ankara-series A1 Math Stat 67(2):209–219

    MathSciNet  MATH  Google Scholar 

  • Sharif M, Amin J, Raza M, Yasmin M, Satapathy SC (2020) An integrated design of particle swarm optimization (PSO) with fusion of features for detection of brain tumor. Pattern Recogn Letts 129:150–157

    Google Scholar 

  • Shukla A, Pandey H M, Mehrotra D (2015) Comparative review of selection techniques in genetic algorithm. In 2015 International Conference on Futuristic Trends on Computational Analysis and Knowledge Management (ABLAZE) (pp. 515–519). IEEE

  • Simon D (2008) Biogeography-based optimization. IEEE Trans Evol Comput 12(6):702–713

    Google Scholar 

  • Storn R, Price K (1997) Differential evolution a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359

    MathSciNet  MATH  Google Scholar 

  • Takami MA, Sheikh R, Sana SS (2016) Product portfolio optimisation using teaching-learning-based optimisation algorithm: a new approach in supply chain management. Int J Syst Sci Oper Logistics 3(4):236–246

    Google Scholar 

  • Tinkle DW, Wilbur HM, Tilley SG (1970) Evolutionary strategies in lizard reproduction. Evolution 24(1):55–74

    Google Scholar 

  • Weimer W, Nguyen T, Le Goues C, Forrest S (2009). Automatically finding patches using genetic programming. In 31st International Conference on Software Engineering (pp. 364–374) IEEE

  • Wu G, Shen X, Li H, Chen H, Lin A, Suganthan PN (2018) Ensemble of differential evolution variants. Inf Sci 423:172–186

    MathSciNet  Google Scholar 

  • Xue Y, Jiang J, Zhao B, Ma T (2018) A self-adaptive artificial bee colony algorithm based on global best for global optimization. Soft Comput 22(9):2935–2952

    Google Scholar 

  • Yan Z, Zhang J, Zeng J, Tang J (2021) Nature-inspired approach: an enhanced whale optimization algorithm for global optimization. Math Comput Simul 185:17–46

    MathSciNet  MATH  Google Scholar 

  • Yang XS (2020) Nature-inspired optimization algorithms: challenges and open problems. J Comput Sci 46:10114

    MathSciNet  Google Scholar 

  • Yang XS (2020) Nature-inspired optimization algorithms. Academic Press, Cambridge

    MATH  Google Scholar 

  • Yazdani M, Jolai F (2016) Lion optimization algorithm (LOA): a nature-inspired metaheuristic algorithm. J Comput Design Eng 3(1):24–36

    Google Scholar 

Download references

Conflict of interest

The authors declare that they have no conflict of interest. This article does not contain any studies with human participants or animals performed by any of the authors.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rashmi Kushwah.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest regarding the publication of this paper.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kushwah, R., Kaushik, M. & Chugh, K. A modified whale optimization algorithm to overcome delayed convergence in artificial neural networks . Soft Comput 25, 10275–10286 (2021). https://doi.org/10.1007/s00500-021-05983-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-05983-z

Keywords

  • Artificial neural network
  • Whale optimization algorithm
  • Meta-heuristic
  • Roulette wheel selection