Skip to main content
SpringerLink
Log in

An adaptive hybrid differential evolution algorithm for continuous optimization and classification problems

  • S. I : Hybridization of Neural Computing with Nature Inspired Algorithms
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Differential evolution (DE) is recognized as a simplistic yet robust evolutionary algorithm: it has been utilized to tackle different challenging optimization problems in various science and engineering disciplines. DE has some disadvantages, such as premature convergence and slow convergence rate, leading to the worst DE execution arrangement. Two DE variations named adaptive parameter selection-based DE (APSDE) and chaotic map hybridization based on DE and PSO (CMHDE-PSO) have been proposed to tackle the issues mentioned above. The proposed variants contain three unique advantages for APSDE: (1) new population initialization scheme to keep up the decent variety of the population diversity; (2) controlled mutation factor technique followed by adaptive decreasing parameter selection procedure; (3) novel mutation strategy with a specific weighted pattern to determine the mutant vector for the mutation operation. Similarly, for CMHDE-PSO (1) novel distribution called Torus for the selection of initial population located in the search space; (2) new parameter adoption technique based on chaotic circle maps defined by chaos theory; (3) average pattern means of two different mutation strategies; (4) and lastly the hybridization of proposed improved DE with PSO to supports DE escaping local minima. Both APSDE and CMHDE-PSO are compared with several standard non-DE old-fashioned optimization algorithms and various advanced DE variants. We accomplish definite experiments behind the powerful searching technique by applying the APSDE and CMHDE-PSO-based mutation and parameter selection strategy for the function optimization and weight optimization of feed-forward neural networks (FFNN) on real-world data classification problems. For data classification performance evaluation, 10 data sets are utilized from the repository of UCI machine learning. Experimental results showed that APSDE and CMHDE-PSO extensively beat different EAs in all test functions and obtained higher accuracy with the recent state-of-the-art algorithms for weight optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Ali AF, Tawhid MA (2016) A hybrid pso and de algorithm for solving engineering optimization problems. Appl Math Inf Sci 10(2):431–449

    Article  Google Scholar 

  2. Asuncion A, Newman D (2007) Uci machine learning repository

  3. Awad NH, Ali MZ, Mallipeddi R, Suganthan PN (2018) An improved differential evolution algorithm using efficient adapted surrogate model for numerical optimization. Inform Sci 451:326–347

    Article  MathSciNet  Google Scholar 

  4. Brest J, Greiner S, Boskovic B, Mernik M, Zumer V (2006) Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657

    Article  Google Scholar 

  5. Chakri A, Khelif R, Benouaret M, Yang XS (2017) New directional bat algorithm for continuous optimization problems. Expert Syst Appl 69:159–175

    Article  Google Scholar 

  6. Cui L, Li G, Lin Q, Chen J, Lu N (2016) Adaptive differential evolution algorithm with novel mutation strategies in multiple sub-populations. Comput Operat 67:155–173

    Article  MathSciNet  Google Scholar 

  7. Das S, Suganthan PN (2010) Differential evolution: A survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–31

    Article  Google Scholar 

  8. Davis L (1991) Handbook of genetic algorithms. CUMINCAD 1

  9. Drugan MM (2019) Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms. Swarm Evol Comput 44:228–246

    Article  Google Scholar 

  10. Fan Q, Yan X (2015) Self-adaptive differential evolution algorithm with discrete mutation control parameters. Expert Syst Appl 42(3):1551–1572

    Article  Google Scholar 

  11. García-Martínez C, Lozano M, Herrera F, Molina D, Sánchez AM (2008) Global and local real-coded genetic algorithms based on parent-centric crossover operators. Eur J Oper Res 185(3):1088–1113

    Article  Google Scholar 

  12. Guo Z, Liu G, Li D, Wang S (2017) Self-adaptive differential evolution with global neighborhood search. Soft Comput 21(13):3759–3768

    Article  Google Scholar 

  13. Islam SM, Das S, Ghosh S, Roy S, Suganthan PN (2011) An adaptive differential evolution algorithm with novel mutation and crossover strategies for global numerical optimization. IEEE Trans Syst Man Cybern Part B 42(2):482–500

    Article  Google Scholar 

  14. Jadon SS, Tiwari R, Sharma H, Bansal JC (2017) Hybrid artificial bee colony algorithm with differential evolution. Appl Soft Comput 58:11–24

    Article  Google Scholar 

  15. Jørgensen MS, Larsen UF, Jacobsen KW, Hammer B (2018) Exploration versus exploitation in global atomistic structure optimization. J Phys Chem A 122(5):1504–1509

    Article  Google Scholar 

  16. Łapa K, Cpałka K, Przybył A, Grzanek K (2018) Negative space-based population initialization algorithm (nspia). In: International Conference on Artificial Intelligence and Soft Computing, Springer, pp 449–461

  17. Laskar NM, Guha K, Chatterjee I, Chanda S, Baishnab KL, Paul PK (2019) Hwpso: A new hybrid whale-particle swarm optimization algorithm and its application in electronic design optimization problems. Appl Intell 49(1):265–291

    Article  Google Scholar 

  18. Lee CY, Yao X (2004) Evolutionary programming using mutations based on the lévy probability distribution. IEEE Trans Evol Comput 8(1):1–13

    Article  Google Scholar 

  19. Liang JJ, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295

    Article  Google Scholar 

  20. Lin GH, Zhang J, Liu ZH (2018) Hybrid particle swarm optimization with differential evolution for numerical and engineering optimization. Int J Autom Comput 15(1):103–114

    Article  Google Scholar 

  21. Lu H, Wang X, Fei Z (2014) Qiu M (2014) The effects of using chaotic map on improving the performance of multiobjective evolutionary algorithms. Math Prob Eng

  22. Lynn N, Ali MZ, Suganthan PN (2018) Population topologies for particle swarm optimization and differential evolution. Swarm Evol Comput 39:24–35

    Article  Google Scholar 

  23. Mallipeddi R, Suganthan PN, Pan QK, Tasgetiren MF (2011) Differential evolution algorithm with ensemble of parameters and mutation strategies. Appl Soft Comput 11(2):1679–1696

    Article  Google Scholar 

  24. Mandal JK, Dutta P, Mukhopadhyay S (2019) Advances in Intelligent Computing. Springer, Berlin

    Book  Google Scholar 

  25. Meng Z, Pan JS, Tseng KK (2019) Pade: An enhanced differential evolution algorithm with novel control parameter adaptation schemes for numerical optimization. Knowl Based Syst 168:80–99

    Article  Google Scholar 

  26. Mezura-Montes E, Velázquez-Reyes J, Coello CC (2006) Modified differential evolution for constrained optimization. In: 2006 IEEE International Conference on Evolutionary Computation, IEEE, pp 25–32

  27. Mohamed AK, Mohamed AW, Elfeky EZ, Saleh M (2019a) Solving constrained non-linear integer and mixed-integer global optimization problems using enhanced directed differential evolution algorithm. In: Machine Learning Paradigms: Theory and Application, Springer, Berlin, pp 327–349

  28. Mohamed AW (2018) A novel differential evolution algorithm for solving constrained engineering optimization problems. J Intell Manuf 29(3):659–692

    Article  Google Scholar 

  29. Mohamed AW, Mohamed AK, Elfeky EZ, Saleh M (2019b) Enhanced directed differential evolution algorithm for solving constrained engineering optimization problems. Int J Appl Metaheuristic Comput (IJAMC) 10(1):1–28

    Article  Google Scholar 

  30. Nakib A, Daachi B, Siarry P (2012) Hybrid differential evolution using low-discrepancy sequences for image segmentation. In: 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum, IEEE, pp 634–640

  31. Nenavath H, Jatoth RK (2018) Hybridizing sine cosine algorithm with differential evolution for global optimization and object tracking. Appl Soft Comput 62:1019–1043

    Article  Google Scholar 

  32. Noman N, Bollegala D, Iba H (2011) An adaptive differential evolution algorithm. In: 2011 IEEE Congress of Evolutionary Computation (CEC), IEEE, pp 2229–2236

  33. Pahner U, Hameyer K (2000) Adaptive coupling of differential evolution and multiquadrics approximation for the tuning of the optimization process. IEEE Trans Magn 36(4):1047–1051

    Article  Google Scholar 

  34. Parouha RP, Das KN (2016) A novel hybrid optimizer for solving economic load dispatch problem. Int J Elect Power Energy Syst 78:108–126

    Article  Google Scholar 

  35. Qin AK, Huang VL, Suganthan PN (2008) Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans Evol Comput 13(2):398–417

    Article  Google Scholar 

  36. Qiu X, Xu JX, Xu Y, Tan KC (2017) A new differential evolution algorithm for minimax optimization in robust design. IEEE Trans Cybern 48(5):1355–1368

    Article  Google Scholar 

  37. Sacco WF, Rios-Coelho AC (2019) On initial populations of differential evolution for practical optimization problems. Computer Intelligence. Optimization and Inverse Problems with Applications in Engineering. Springer, Berlin, pp 53–62

    Chapter  Google Scholar 

  38. Shi Y, Eberhart RC (1998) Parameter selection in particle swarm optimization. In: International conference on evolutionary programming, Springer, pp 591–600

  39. Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359

    Article  MathSciNet  Google Scholar 

  40. Tam JH, Ong ZC, Ismail Z, Ang BC, Khoo SY (2019) A new hybrid ga- aco- pso algorithm for solving various engineering design problems. Int J Comput Math 96(5):883–919

    Article  MathSciNet  Google Scholar 

  41. Tawhid MA, Ali AF (2017) Multi-directional bat algorithm for solving unconstrained optimization problems. Opsearch 54(4):684–705

    Article  MathSciNet  Google Scholar 

  42. Wang H, Rahnamayan S, Sun H, Omran MG (2013) Gaussian bare-bones differential evolution. IEEE Trans Cybern 43(2):634–647

    Article  Google Scholar 

  43. Wang Y, Cai Z, Zhang Q (2011) Differential evolution with composite trial vector generation strategies and control parameters. IEEE Trans Evol Comput 15(1):55–66

    Article  Google Scholar 

  44. Wang Y, Liu ZZ, Li J, Li HX, Wang J (2018) On the selection of solutions for mutation in differential evolution. Front Comput Sci 12(2):297–315

    Article  Google Scholar 

  45. Wu G, Mallipeddi R, Suganthan PN, Wang R, Chen H (2016) Differential evolution with multi-population based ensemble of mutation strategies. Inf Sci 329:329–345

    Article  Google Scholar 

  46. Wu G, Shen X, Li H, Chen H, Lin A, Suganthan PN (2018) Ensemble of differential evolution variants. Inf Sci 423:172–186

    Article  MathSciNet  Google Scholar 

  47. Wu L, Wang Y, Yuan X (2011) Design of 2-d recursive filters using self-adaptive mutation differential evolution algorithm. International Journal of Computational Intelligence Systems 4(4):644–654. https://doi.org/10.1080/18756891.2011.9727819

    Article  Google Scholar 

  48. Yılmaz S, Kücüksille EU (2015) A new modification approach on bat algorithm for solving optimization problems. Appl Soft Comput 28:259–275

    Article  Google Scholar 

  49. Zhang J, Sanderson AC (2009) Jade: adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958

    Article  Google Scholar 

  50. Zou F, Wang L, Hei X, Chen D, Yang D (2014) Teaching-learning-based optimization with dynamic group strategy for global optimization. Inf Sci 273:112–131

    Article  Google Scholar 

Download references

Acknowledgements

We would like to express our gratitude to the anonymous reviewer for helping out to furnish this research article more precisely.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Gujrat, Gujrat, Pakistan

    Hafiz Tayyab Rauf & Waqas Haider Khan Bangyal

  2. Department of Information Sciences, University of Education, Lahore, Pakistan

    M. Ikramullah Lali

Authors
  1. Hafiz Tayyab Rauf
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Waqas Haider Khan Bangyal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Ikramullah Lali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Ikramullah Lali.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rauf, H.T., Bangyal, W.H.K. & Lali, M.I. An adaptive hybrid differential evolution algorithm for continuous optimization and classification problems. Neural Comput & Applic 33, 10841–10867 (2021). https://doi.org/10.1007/s00521-021-06216-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06216-y

Keywords

Search

Navigation