Skip to main content
SpringerLink
Log in

An Improved Cuckoo Search Algorithm for Optimization of Artificial Neural Network Training

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Artificial neural networks are widely used for solving engineering design problems of various disciplines due to its simplicity, efficiency, and adaptability. It predicts promising and accurate results. Artificial neural network solves these problems with weights and biases obtained in the training process. In training, the weights and biases have to be updated such that the difference between predicted and actual values has to be minimized. The artificial neural network uses stochastic gradient steepest descent methods to update the weights and biases for optimizing it. These methods are good at finding the optimum solution. However, they suffer from the drawbacks of vanishing gradient at local minima and critical points and are sensitive to initial weights and biases. As a result, it falls into local minima, the training time becomes high, and accuracy becomes low. One of the best solutions to overcome these problems is to use metaheuristics algorithms instead of stochastic gradient descent methods. Among metaheuristics, the cuckoo search algorithm is widely used in many applications due to its simplicity and efficiency. In this work, we proposed an improved Cuckoo search algorithm by incorporating Voronoi diagram with Cuckoo search to strengthen the weak areas of Cuckoo search and to overcome the addressed problems of the artificial neural network. The proposed Cuckoo search algorithm performance is tested on higher dimensional benchmark functions and on benchmark data sets. Moreover, its performance is compared with variants of Cuckoo search and other metaheuristic algorithms. The proposed algorithm has shown better results in terms of the number of generations, accuracy, cross-entropy, and root mean square error (RMSE).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Algorithm 2
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability and access

The datasets generated during and/or analysed during the current study are available in the UCI machine learning repository, https://archive.ics.uci.edu/ml/index.php.

References

  1. Yu X, Efe MO, Kaynak O (2002) A general backpropagation algorithm for feedforward neural networks learning. IEEE Trans Neural Netw 13(1):251–254

    Article  Google Scholar 

  2. Trujillo-Romero F (2013) Generation of neural networks using a genetic algorithm approach. Int J Bio-Inspired Comput 5(5):289–302. https://doi.org/10.1504/IJBIC.2013.057183

    Article  Google Scholar 

  3. Zhou G, Zhou Y, Huang H, Tang Z (2019) Functional networks and applications: a survey. Neurocomputing 335:384–399

    Article  Google Scholar 

  4. Zhang J-R, Zhang J, Lok T-M, Lyu MR (2007) A hybrid particle swarm optimization–back-propagation algorithm for feedforward neural network training. Appl Math Comput 185(2):1026–1037. https://doi.org/10.1016/j.amc.2006.07.025

    Article  Google Scholar 

  5. Liu C, Ding W, Li Z, Yang C (2017) Prediction of high-speed grinding temperature of titanium matrix composites using BP neural network based on PSO algorithm. Int J Adv Manuf Technol 89(5):2277–2285. https://doi.org/10.1007/s00170-016-9267-z

    Article  Google Scholar 

  6. Si T, Hazra S, Jana N (2012) Artificial neural network training using differential evolutionary algorithm for classification. In: Proceedings of the International Conference on Information Systems Design and Intelligent Applications 2012 (INDIA 2012) Held in Visakhapatnam, India, January 2012, pp. 769–778. Springer

  7. Mirjalili S, Mohd Hashim SZ, Moradian Sardroudi H (2012) Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl Math Comput 218(22):11125–11137. https://doi.org/10.1016/j.amc.2012.04.069

    Article  MathSciNet  Google Scholar 

  8. Agrawal U, Arora J, Singh R, Gupta D, Khanna A, Khamparia A (2020) Hybrid Wolf-Bat algorithm for optimization of connection weights in multi-layer perceptron. ACM . Multimedia Comput Commun Appl 16(1s):37–13720. https://doi.org/10.1145/3350532

    Article  Google Scholar 

  9. Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell 45(2):322–332. https://doi.org/10.1007/s10489-016-0767-1

    Article  Google Scholar 

  10. Luo Q, Li J, Zhou Y, Liao L (2021) Using spotted hyena optimizer for training feedforward neural networks. Cognitive Syst Res 65:1–16

    Article  Google Scholar 

  11. Fan C, Zhou Y, Tang Z (2021) Neighborhood centroid opposite-based learning Harris Hawks optimization for training neural networks. Evol Intel 14(4):1847–1867. https://doi.org/10.1007/s12065-020-00465-x

    Article  Google Scholar 

  12. Zhou G, Miao F, Tang Z, Zhou Y, Luo Q (2023) Kohonen neural network and symbiotic-organism search algorithm for intrusion detection of network viruses. Front Comput Neurosci 17

  13. Liu Q, Li N, Jia H, Qi Q, Abualigah L (2023) A chimp-inspired remora optimization algorithm for multilevel thresholding image segmentation using cross entropy. Artificial Intell Rev. https://doi.org/10.1007/s10462-023-10498-0

    Article  Google Scholar 

  14. Liu Q, Li N, Jia H, Qi Q, Abualigah L, Liu Y (2022) A hybrid arithmetic optimization and golden sine algorithm for solving industrial engineering design problems. Mathematics 10(9):1567. https://doi.org/10.3390/math10091567

    Article  Google Scholar 

  15. Chen Y-t (2014) Novel back propagation optimization by cuckoo search algorithm. Sci World J. https://doi.org/10.1155/2014/878262

    Article  Google Scholar 

  16. Wang H, Zeng Q, Zhang Z, Wang H (2022) Research on temperature compensation of multi-channel pressure scanner based on an improved cuckoo search optimizing a BP neural network. Micromachines 13(8):1351. https://doi.org/10.3390/mi13081351

    Article  Google Scholar 

  17. Zhu X, Wang N (2017) Cuckoo search algorithm with membrane communication mechanism for modeling overhead crane systems using RBF neural networks. Appl Soft Comput 56:458–471. https://doi.org/10.1016/j.asoc.2017.03.019

    Article  Google Scholar 

  18. Cheng J, Wang L, Xiong Y (2018) An improved cuckoo search algorithm and its application in vibration fault diagnosis for a hydroelectric generating unit. Eng Optim 50(9):1593–1608. https://doi.org/10.1080/0305215X.2017.1401067

    Article  Google Scholar 

  19. Zhang W, Han G, Wang J, Liu Y (2019) A BP neural network prediction model based on dynamic cuckoo search optimization algorithm for industrial equipment fault prediction. IEEE Access 7:11736–11746. https://doi.org/10.1109/ACCESS.2019.2892729

    Article  Google Scholar 

  20. Cheng J, Wang L, Xiong Y (2019) Cuckoo search algorithm with memory and the vibrant fault diagnosis for hydroelectric generating unit. Eng Comput 35(2):687–702. https://doi.org/10.1007/s00366-018-0627-1

    Article  Google Scholar 

  21. Bala A, Ismail I, Ibrahim R, Sait SM, Salami HO (2019) Prediction using cuckoo search optimized echo state network. Arab J Sci Eng 44(11):9769–9778. https://doi.org/10.1007/s13369-019-04008-0

    Article  Google Scholar 

  22. Chiroma H, Abdul-kareem S, Khan A, Nawi NM, Gital AY, Shuib L, Abubakar AI, Rahman MZ, Herawan T (2015) Global warming: predicting opec carbon dioxide emissions from petroleum consumption using neural network and hybrid cuckoo search algorithm. PLoS ONE 10(8):0136140. https://doi.org/10.1371/journal.pone.0136140

    Article  Google Scholar 

  23. ENIREDDY VAMSIDHAR, KUMAR RK, (2015) Improved cuckoo search with particle swarm optimization for classification of compressed images. Sadhana 40(8):2271–2285. https://doi.org/10.1007/s12046-015-0440-0

  24. Syed Mustafa A, Kumaraswamy YS (2016) Hybrid cuckoo optimized multi layer perceptron for hospital management information system web-services quality classification. J Med Imag Health Inform 6(7):1641–1645. https://doi.org/10.1166/jmihi.2016.1864

    Article  Google Scholar 

  25. Khan A, Shah R, Imran M, Khan A, Bangash JI, Shah K (2019) An alternative approach to neural network training based on hybrid bio meta-heuristic algorithm. J Ambient Intell Human Comput 10(10):3821–3830. https://doi.org/10.1007/s12652-019-01373-4

    Article  Google Scholar 

  26. Cristin R, Kumar BS, Priya C, Karthick K (2020) Deep neural network based Rider-Cuckoo Search Algorithm for plant disease detection. Artif Intell Rev 53(7):4993–5018. https://doi.org/10.1007/s10462-020-09813-w

    Article  Google Scholar 

  27. Abbas S, Khan M, Alhaisoni M, Tariq U, Armghan A, Alenezi F, Majumdar A, Thinnukool O (2022) Crops Leaf Diseases Recognition: A Framework of Optimum Deep Learning Features. cmc 74(1), 1139–1159 https://doi.org/10.32604/cmc.2023.028824

  28. Aziz RM, Desai NP, Baluch MF (2022) Computer vision model with novel cuckoo search based deep learning approach for classification of fish image. Multimed Tools Appl. https://doi.org/10.1007/s11042-022-13437-3

    Article  Google Scholar 

  29. Chen X, Jin S, Qin S, Li L (2015) Short-term wind speed forecasting study and its application using a hybrid model optimized by cuckoo search. Math Problem Eng. https://doi.org/10.1155/2015/608597

    Article  Google Scholar 

  30. Heng J, Wang C, Zhao X, Wang J (2016) A hybrid forecasting model based on empirical mode decomposition and the cuckoo search algorithm: a case study for power load. Math Problem Eng. https://doi.org/10.1155/2016/3205396

    Article  Google Scholar 

  31. Tran-Ngoc H, Khatir S, De Roeck G, Bui-Tien T, Abdel Wahab M (2019) An efficient artificial neural network for damage detection in bridges and beam-like structures by improving training parameters using cuckoo search algorithm. Eng Struct 199:109637. https://doi.org/10.1016/j.engstruct.2019.109637

    Article  Google Scholar 

  32. Alqahtani F, Al-Makhadmeh Z, Tolba A, Said W (2020) Internet of things-based urban waste management system for smart cities using a Cuckoo Search Algorithm. Cluster Comput 23(3):1769–1780. https://doi.org/10.1007/s10586-020-03126-x

    Article  Google Scholar 

  33. Feng C, Xu L, Zhao L, Han Y, Su M, Peng C (2022) Prediction of welded joint fatigue properties based on a novel hybrid SPDTRS-CS-ANN method. Eng Fracture Mech 275:108824. https://doi.org/10.1016/j.engfracmech.2022.108824

    Article  Google Scholar 

  34. Shatnawi M, Nasrudin MF (2011) Starting configuration of Cuckoo Search algorithm using Centroidal Voronoi Tessellations. In: 2011 11th International Conference on Hybrid Intelligent Systems (HIS), pp. 40–45. https://doi.org/10.1109/HIS.2011.6122077

  35. O’Rourke J et al. (1998) Computational Geometry in C. Cambridge University Press

  36. Sarhani M, Voß S, Jovanovic R Initialization of metaheuristics: Comprehensive review, critical analysis, and research directions. International Transactions in Operational Research (n/a) https://doi.org/10.1111/itor.13237

  37. Yang X-S, Deb S (2009) Cuckoo Search via Lévy flights. In: 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC), pp. 210–214. https://doi.org/10.1109/NABIC.2009.5393690

  38. Aurenhammer F (1991) Voronoi diagrams– a survey of a fundamental geometric data structure. ACM Comput Surv 23(3):345–405. https://doi.org/10.1145/116873.116880

    Article  Google Scholar 

  39. Fortune S (1987) A sweepline algorithm for Voronoi diagrams. Algorithmica 2(1):153–174. https://doi.org/10.1007/BF01840357

    Article  MathSciNet  Google Scholar 

  40. Dua D, Graff C (2017) UCI Machine learning repository. http://archive.ics.uci.edu/ml

  41. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95 - International Conference on Neural Networks 4, 1942–19484. https://doi.org/10.1109/ICNN.1995.488968

  42. Storn R, Price K (1997) Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359. https://doi.org/10.1023/A:1008202821328. (Accessed 2022-06-21)

    Article  MathSciNet  Google Scholar 

  43. Cheng J, Xiong Y (2022) Parameter control based cuckoo search algorithm for numerical optimization. Neural Process Lett 54(4):3173–3200. https://doi.org/10.1007/s11063-022-10758-0

    Article  Google Scholar 

  44. Surjanovic S, Bingham D Virtual library of simulation experiments: test functions and datasets. Retrieved from http://www.sfu.ca/~ssurjano

  45. Arya S, Malamatos T, Mount DM (2002) Space-efficient approximate Voronoi diagrams. In: Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing. STOC ’02, pp. 721–730. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/509907.510011

  46. Arya S, Malamatos T (2002) Linear-size approximate voronoi diagrams. In: Proceedings of the Thirteenth Annual ACM-SIAM Symposium on Discrete Algorithms. SODA ’02, pp. 147–155. Society for Industrial and Applied Mathematics, USA

Download references

Funding

Funding No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Calicut, Calicut, Kerala, 673601, India

    Pedda Nagyalla Maddaiah & Pournami Pulinthanathu Narayanan

Authors
  1. Pedda Nagyalla Maddaiah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pournami Pulinthanathu Narayanan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors confirm their contribution to the paper: study conception and design, data collection, analysis and interpretation of results, and manuscript preparation.

Corresponding author

Correspondence to Pedda Nagyalla Maddaiah.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical and informed consent for data used

The authors declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maddaiah, P.N., Narayanan, P.P. An Improved Cuckoo Search Algorithm for Optimization of Artificial Neural Network Training. Neural Process Lett 55, 12093–12120 (2023). https://doi.org/10.1007/s11063-023-11411-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11411-0

Keywords

Search

Navigation