Skip to main content
SpringerLink
Log in

A modified whale optimization algorithm to overcome delayed convergence in artificial neural networks

  • Optimization
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Artificial neural network (ANN) is modeled to predict and classify problems. However, in the training phase of ANNs discovering faultless values of the weights of a network is extremely troublesome. Traditional weight updating methods often get stuck into local optima and converge to optimal solutions very slowly. Therefore, to overcome these drawbacks a modified version of a nature-based algorithm which merges meta-heuristics with weight-updating technique of ANN has been used in this paper. Whale optimization algorithm (WOA) is a well-established, efficient and competitive algorithm inspired by the hunting mechanism of the whales including their behavior in finding and attacking their prey with their bubble-net feeding technique. In WOA, the next location of the search individuals or whales is modified depending on some probability. Due to the high exploration rate of WOA, there is a disproportion between exploration and exploitation in the WOA and it also converges to the solution slowly. Thus, to establish an equilibrium between exploration and exploitation a new variant of WOA called modified whale optimization algorithm (MWOA) is proposed to overcome the problem of delayed convergence. In MWOA, roulette wheel selection is combined with WOA to enhance the convergence speed of WOA. MWOA is tested on 11 benchmark functions, and the outcomes are compared with WOA. The results prove that MWOA has gained success in overcoming the problem of the slow convergence of WOA. Also, the results show that the proposed MWOA technique, when applied to ANN, can overcome the problems of traditional techniques and has improved the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Abedinpourshotorban H, Shamsuddin SM, Beheshti Z, Jawawi DN (2016) Electromagnetic field optimization: a physics-inspired metaheuristic optimization algorithm. Swarm Evol Comput 26:8–22

    Article  Google Scholar 

  • Abualigah L (2020) Group search optimizer: a nature-inspired meta-heuristic optimization algorithm with its results, variants, and applications. Neural Comput Appl 33:1–24

  • Aljarah I, Faris H, Mirjalili S (2018) Optimizing connection weights in neural networks using the whale optimization algorithm. Soft Comput 22(1):1–15

    Article  Google Scholar 

  • Baluja S (1994) Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning. School of Computer Science, Carnegie Mellon University, 1994

  • Basheer IA, Hajmeer M (2000) Artificial neural networks: fundamentals, computing, design, and application. J Microbiol Methods 43(1):3–31

    Article  Google Scholar 

  • Birjandi AK, Akhyani F, Sheikh R, Sana SS (2019) Evaluation and selecting the contractor in bidding with incomplete information using MCGDM method. Soft Comput 23(20):10569–10585

    Article  Google Scholar 

  • Braik M, Sheta A, Arieqat, A (2008). A comparison between GAs and PSO in training ANN to model the TE chemical process reactor. In Proceedings of the AISB symposium on swarm intelligence algorithms and applications (pp. 24–30)

  • Chatterjee S, Sarkar S, Hore S, Dey N, Ashour AS, Balas VE (2017) Particle swarm optimization trained neural network for structural failure prediction of multistoried RC buildings. Neural Comput Appl 28(8):2005–2016

    Article  Google Scholar 

  • Chou JS, Pham AD (2017) Nature-inspired metaheuristic optimization in least squares support vector regression for obtaining bridge scour information. Inf Sci 399:64–80

    Article  Google Scholar 

  • Crepinsek M, Liu SH, Mernik M (2013) Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput Surv (CSUR) 45(3):1–33

    Article  MATH  Google Scholar 

  • Dhiman G (2019) ESA: a hybrid bio-inspired metaheuristic optimization approach for engineering problems. Eng Comput 37:1–31

  • Dorigo M, Stutzle T (2019) Ant colony optimization: overview and recent advances. Handbook of metaheuristics, pp. 311–351

  • Erol OK, Eksin I (2006) A new optimization method: big bang-big crunch. Adv Eng Softw 37(2):106–111

    Article  Google Scholar 

  • Faramarzi A, Heidarinejad M, Mirjalili S, Gandomi AH (2020) Marine Predators Algorithm: a nature-inspired metaheuristic. Expert Syst Appl 152:113377

    Article  Google Scholar 

  • Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell 45(2):322–332

    Article  Google Scholar 

  • Formato RA (2007) Central force optimization: a new metaheuristic with applications in applied electromagnetics. Prog Electromagn Res 77:425–491

    Article  Google Scholar 

  • Ghasemiyeh R, Moghdani R, Sana SS (2017) A hybrid artificial neural network with metaheuristic algorithms for predicting stock price. Cybern Syst 48(4):365–392

    Article  Google Scholar 

  • Haseli G, Sheikh R, Sana SS (2020) Base-criterion on multi-criteria decision-making method and its applications. Int J Manag Sci Eng Manag 15(2):79–88

    Google Scholar 

  • Jamali G, Sana SS, Moghdani R (2018) Hybrid improved cuckoo search algorithm and genetic algorithm for solving Markov-modulated demand. RAIRO-Operations Res 52(2):473–497

    Article  MathSciNet  Google Scholar 

  • Kaur G, Arora S (2018) Chaotic whale optimization algorithm. J Comput Design Eng 5(3):275–284

    Article  Google Scholar 

  • Kaveh A, Talatahari S (2010) A novel heuristic optimization method: charged system search. Acta Mechanica 213(3):267–289

    Article  MATH  Google Scholar 

  • Kennedy J (2011) Encyclopedia of machine learning. Particle Swarm Optimization (pp. 760–766)

  • Kennedy J (2006) Swarm intelligence. Handbook of nature-inspired and innovative computing. Springer, Berlin, pp 187–219

    Book  Google Scholar 

  • Kim JS, Jung S (2015) Implementation of the RBF neural chip with the back-propagation algorithm for on-line learning. Appl Soft Comput 29:233–244

    Article  Google Scholar 

  • Kushwah R, Tapaswi S, Kumar A (2019) A detailed study on Internet connectivity schemes for mobile ad hoc network. Wireless Personal Commun 104(4):1433–1471

    Article  Google Scholar 

  • Lee KC, Lu PT (2020) Application of Whale Optimization Algorithm to Inverse Scattering of an Imperfect Conductor with Corners. Int J Antennas Propagation 2020:1–9

  • Lee JG, Senel G, Lim PK, Kim J, Hur K (2020) Octahedron sets. Ann Fuzzy Math Inform 19(3):211–238

    MathSciNet  MATH  Google Scholar 

  • Li S, Gong W, Yan X, Hu C, Bai D, Wang L, Gao L (2019) Parameter extraction of photovoltaic models using an improved teaching-learning-based optimization. Energy Convers Manag 186:293–305

    Article  Google Scholar 

  • Li H, Huang Z, Liu X, Zeng C, Zou P (2020) Multi-fidelity meta-optimization for nature inspired optimization algorithms. Appl Soft Comput 96:106619

    Article  Google Scholar 

  • Ling Y, Zhou Y, Luo Q (2017) Levy flight trajectory-based whale optimization algorithm for global optimization. IEEE Access 5:6168–6186

    Article  Google Scholar 

  • Lipowski A, Lipowska D (2012) Roulette-wheel selection via stochastic acceptance. Phys A Stat Mech Appl 391(6):2193–2196

    Article  Google Scholar 

  • Mafarja M, Aljarah I, Heidari AA, Faris H, Fournier-Viger P, Li X, Mirjalili S (2018) Binary dragonfly optimization for feature selection using time-varying transfer functions. Knowledge-Based Syst 161:185–204

    Article  Google Scholar 

  • Mareli M, Twala B (2018) An adaptive Cuckoo search algorithm for optimisation. Appl Comput Inf 14(2):107–115

    Google Scholar 

  • Maulik U, Bandyopadhyay S (2000) Genetic algorithm-based clustering technique. Pattern Recognit 33(9):1455–1465

    Article  Google Scholar 

  • Mirjalili S (2019) Ant Colony Optimisation. In: Evolutionary Algorithms and Neural Networks, vol 780. pp 33–42. https://doi.org/10.1007/978-3-319-93025-1_3

  • Mirjalili S (2015) How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Appl Intell 43(1):150–161

    Article  Google Scholar 

  • Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Software 95:51–67

    Article  Google Scholar 

  • Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Advances in engineering software 69:46–61

    Article  Google Scholar 

  • Mohammadi-Balani A, Nayeri MD, Azar A, Taghizadeh-Yazdi M (2021) Golden eagle optimizer: a nature-inspired metaheuristic algorithm. Comput Ind Eng 152:107050

    Article  Google Scholar 

  • Ospina-Mateus H, Jimenez LAQ, Lopez-Valdes FJ, Garcia SB, Barrero LH, Sana SS (2021) Extraction of decision rules using genetic algorithms and simulated annealing for prediction of severity of traffic accidents by motorcyclists. J Ambient Intell Humanized comput 1–22. https://doi.org/10.1007/s12652-020-02759-5

  • Pandey AC, Rajpoot DS, Saraswat M (2017). Hybrid step size based cuckoo search. In 2017 Tenth International Conference on Contemporary Computing (IC3) (pp. 1–6). IEEE

  • Pandey AC, Rajpoot DS (2019) Spam review detection using spiral cuckoo search clustering method. Evol Intell 12(2):147–164

    Article  Google Scholar 

  • Pencheva T, Atanassov K, Shannon A (2009) Modelling of a roulette wheel selection operator in genetic algorithms using generalized nets. Int J Bioautomation 13(4):257–264

    Google Scholar 

  • Rakitianskaia AS, Engelbrecht AP (2012) Training feedforward neural networks with dynamic particle swarm optimisation. Swarm Intell 6(3):233–270

    Article  Google Scholar 

  • Rashedi E, Nezamabadi-Pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179(13):2232–2248

    Article  MATH  Google Scholar 

  • Sana SS, Ospina-Mateus H, Arrieta FG, Chedid JA (2019) Application of genetic algorithm to job scheduling under ergonomic constraints in manufacturing industry. J Ambient Intell Humanized Comput 10(5):2063–2090

  • Sanel G, Lee JG, Hur K (2020) Distance and similarity measures for octahedron sets and their application to MCGDM problems. Mathematics 8(10):1690

    Article  Google Scholar 

  • Sarkar BK, Sana SS, Chaudhuri K (2012) A genetic algorithm-based rule extraction system. Appl Soft Comput 12(1):238–254

    Article  Google Scholar 

  • Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Networks 61:85–117

    Article  Google Scholar 

  • Senel G (2016) A new approach to Hausdorff space theory via the soft sets. Math Problems Eng 2016:1–6. https://doi.org/10.1155/2016/2196743

  • Senel G (2018) The relation between soft topological space and soft ditopological space. Commun Faculty Sci Univ Ankara-series A1 Math Stat 67(2):209–219

    Article  MathSciNet  MATH  Google Scholar 

  • Sharif M, Amin J, Raza M, Yasmin M, Satapathy SC (2020) An integrated design of particle swarm optimization (PSO) with fusion of features for detection of brain tumor. Pattern Recogn Letts 129:150–157

    Article  Google Scholar 

  • Shukla A, Pandey H M, Mehrotra D (2015) Comparative review of selection techniques in genetic algorithm. In 2015 International Conference on Futuristic Trends on Computational Analysis and Knowledge Management (ABLAZE) (pp. 515–519). IEEE

  • Simon D (2008) Biogeography-based optimization. IEEE Trans Evol Comput 12(6):702–713

    Article  Google Scholar 

  • Storn R, Price K (1997) Differential evolution a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359

    Article  MathSciNet  MATH  Google Scholar 

  • Takami MA, Sheikh R, Sana SS (2016) Product portfolio optimisation using teaching-learning-based optimisation algorithm: a new approach in supply chain management. Int J Syst Sci Oper Logistics 3(4):236–246

    Google Scholar 

  • Tinkle DW, Wilbur HM, Tilley SG (1970) Evolutionary strategies in lizard reproduction. Evolution 24(1):55–74

    Article  Google Scholar 

  • Weimer W, Nguyen T, Le Goues C, Forrest S (2009). Automatically finding patches using genetic programming. In 31st International Conference on Software Engineering (pp. 364–374) IEEE

  • Wu G, Shen X, Li H, Chen H, Lin A, Suganthan PN (2018) Ensemble of differential evolution variants. Inf Sci 423:172–186

    Article  MathSciNet  Google Scholar 

  • Xue Y, Jiang J, Zhao B, Ma T (2018) A self-adaptive artificial bee colony algorithm based on global best for global optimization. Soft Comput 22(9):2935–2952

    Article  Google Scholar 

  • Yan Z, Zhang J, Zeng J, Tang J (2021) Nature-inspired approach: an enhanced whale optimization algorithm for global optimization. Math Comput Simul 185:17–46

    Article  MathSciNet  MATH  Google Scholar 

  • Yang XS (2020) Nature-inspired optimization algorithms: challenges and open problems. J Comput Sci 46:10114

    Article  MathSciNet  Google Scholar 

  • Yang XS (2020) Nature-inspired optimization algorithms. Academic Press, Cambridge

    MATH  Google Scholar 

  • Yazdani M, Jolai F (2016) Lion optimization algorithm (LOA): a nature-inspired metaheuristic algorithm. J Comput Design Eng 3(1):24–36

    Article  Google Scholar 

Download references

Conflict of interest

The authors declare that they have no conflict of interest. This article does not contain any studies with human participants or animals performed by any of the authors.

Author information

Authors and Affiliations

  1. Jaypee Institute of Information Technology, Noida, India

    Rashmi Kushwah, Manika Kaushik & Kashish Chugh

Authors
  1. Rashmi Kushwah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Manika Kaushik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kashish Chugh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rashmi Kushwah.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest regarding the publication of this paper.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kushwah, R., Kaushik, M. & Chugh, K. A modified whale optimization algorithm to overcome delayed convergence in artificial neural networks . Soft Comput 25, 10275–10286 (2021). https://doi.org/10.1007/s00500-021-05983-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-05983-z

Keywords

Search

Navigation