Skip to main content
SpringerLink
Log in

Incorporating Q-learning and gradient search scheme into JAYA algorithm for global optimization

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Many swarm intelligence techniques are facing rigorous challenges since they cannot exploit useful information well during the evolutionary procedure. To remedy this issue, this paper raises a reinforced JAYA algorithm (QLJAYA) that employs the Q-learning and gradient search scheme. In QLJAYA, to balance convergence and diversity, a modified search formula and gradient search scheme are adaptively selected to generate solutions under the control of Q-learning. In addition, to strengthen the rotational invariance of JAYA, the covariance matrix learning strategy is adopted to construct an eigen coordinate system for each solution. Experimental simulations on CEC2017 and CEC2019 test suites and application to identify parameters of photovoltaic systems suggest that QLJAYA can exhibit a better or at least competitive overall performance compared to several typical JAYA variants and other well-known metaheuristics. The source code of  QLJAYA is publicly available at https://github.com/denglingyun123/QLJAYA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Alexandru ZI, Radu-Emil P, Raul-Cristian R, Petriu Emil M (2022) Reinforcement learning-based control using q-learning and gravitational search algorithm with experimental validation on a nonlinear servo system. Inf Sci 583:99–120

    Google Scholar 

  • Ali EE, El-Hameed MA, El-Fergany AA, El-Arini MM (2016) Parameter extraction of photovoltaic generating units using multi-verse optimizer. Sustain Energy Technol Assess 17:68–76

    Google Scholar 

  • Chen X, Tianfield H, Wenli D, Liu G (2016) Biogeography-based optimization with covariance matrix based migration. Appl Soft Comput 45:71–85

    Google Scholar 

  • Cheng S, Qin Q, Chen J, Shi Y (2016) Brain storm optimization algorithm: a review. Artif Intell Rev 46:12

    Google Scholar 

  • Dan S (2009) Biogeography-based optimization. IEEE Trans Evol Comput 12(6):702–713

    Google Scholar 

  • Deng L, Liu S (2023) An enhanced slime mould algorithm based on adaptive grouping technique for global optimization. Expert Syst Appl 222:119877

    Google Scholar 

  • Deng L, Liu S (2023) A multi-strategy improved slime mould algorithm for global optimization and engineering design problems. Comput Methods Appl Mech Eng 404:115764

    MathSciNet  MATH  Google Scholar 

  • Dhruv K, Pallavi S, Ashish B, Rana KPS, Kumar V (2017) Pv cell and module efficient parameters estimation using evaporation rate based water cycle algorithm. Swarm Evol Comput 35:93–110

    Google Scholar 

  • Diego O, El AMA, Ella HA (2017) Parameter estimation of photovoltaic cells using an improved chaotic whale optimization algorithm. Appl Energy 200:141–154

    Google Scholar 

  • Easwarakhanthan T, Bottin J, Bouhouch I, Boutrit C (1986) Nonlinear minimization algorithm for determining the solar cell parameters with microcomputers. Int J Solar Energy 4(1):1–12

    Google Scholar 

  • Gan C, Cao W, Min W, Chen X (2018) A new bat algorithm based on iterative local search and stochastic inertia weight. Expert Syst Appl 104:202–212

    Google Scholar 

  • Gao W, Dang Q, Gong M (2022) An adaptive framework to select the coordinate systems for evolutionary algorithms. Appl Soft Comput 129:109585

    Google Scholar 

  • García-Martínez C, Gutiérrez PD, Molina D, Lozano M, Herrera F (2017) Since 2005 competition on real-parameter optimisation: a decade of research, progress and comparative analysis’s weakness. Soft Comput 21:5573–5583

    Google Scholar 

  • Gupta S, Deep K, Mirjalili S (2020) An efficient equilibrium optimizer with mutation strategy for numerical optimization. Appl Soft Comput 96:106542

    Google Scholar 

  • Hao C, Asghar HA, Xuehua Z, Lejun Z, Huiling C (2020) Advanced orthogonal learning-driven multi-swarm sine cosine optimization: framework and case studies. Expert Syst Appl 144:113113

    Google Scholar 

  • Jafar G, Reza KM, Somayeh M, Elahe H, Somayeh A (2022) Powerful enhanced jaya algorithm for efficiently optimizing numerical and engineering problems. Soft Comput 26(11):5315–5333

    Google Scholar 

  • Joaquín D, Salvador G, Daniel M, Francisco H (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1:3–18

    Google Scholar 

  • Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (abc) algorithm. J Glob Optim 39:459–471

    MathSciNet  MATH  Google Scholar 

  • Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95—International Conference on Neural Networks, Vol 4, pp 1942–1948

  • Kisan IK, Kumar JR (2020) An efficient jaya algorithm with lévy flight for non-linear channel equalization. Expert Syst Appl 145:2970

    Google Scholar 

  • Kunjie Yu, Liang JJ, Qu BY, Chen X, Heshan W (2017) Parameters identification of photovoltaic models using an improved jaya optimization algorithm. Energy Convers Manag 150:742–753

    Google Scholar 

  • Kutlu OF, Aydemr SB (2022) Chaotic hunger games search optimization algorithm for global optimization and engineering problems. Math Comput Simul 192:514–536

    MathSciNet  MATH  Google Scholar 

  • Laith A, Ali D (2022) Improved multi-core arithmetic optimization algorithm-based ensemble mutation for multidisciplinary applications. J Intell Manuf 34:1833–1874

    Google Scholar 

  • Li S, Gong W, Qiong G (2021) A comprehensive survey on meta-heuristic algorithms for parameter extraction of photovoltaic models. Renew Sustain Energy Rev 141:110828

    Google Scholar 

  • Liang J, Qiao K, Yuan M, Kunjie Yu, Boyang Q, Ge S, Li Y, Chen G (2020) Evolutionary multi-task optimization for parameters extraction of photovoltaic models. Energy Convers Manag 207:112509

    Google Scholar 

  • Long W, Cai S, Jiao J, Ming X, Tiebin W (2020) A new hybrid algorithm based on grey wolf optimizer and cuckoo search for parameter extraction of solar photovoltaic models. Energy Convers Manag 203:112243

    Google Scholar 

  • Long W, Jiao J, Liang X, Ming X, Tang M, Cai S (2022) Parameters estimation of photovoltaic models using a novel hybrid seagull optimization algorithm. Energy 249:123760

    Google Scholar 

  • Long W, Tiebin W, Ming X, Tang M, Cai S (2021) Parameters identification of photovoltaic models by using an enhanced adaptive butterfly optimization algorithm. Energy 229:120750

    Google Scholar 

  • Ma L, Cheng S, Shi Y (2020) Enhancing learning efficiency of brain storm optimization via orthogonal learning design. IEEE Trans Syst Man Cybernetics Syst 51(11):6723–6742

    Google Scholar 

  • Mahdi A, Arash MGS, Goli ER, Siamak T (2020) Optimum design of fuzzy controller using hybrid ant lion optimizer and jaya algorithm. Artif Intell Rev 53:1553–1584

    Google Scholar 

  • Meng Z, Zhong Y, Mao G, Liang Y (2022) Pso-sono: a novel pso variant for single-objective numerical optimization. Inf Sci 586:176–191

    Google Scholar 

  • Noel Mathew M (2012) A new gradient based particle swarm optimization algorithm for accurate computation of global minimum. Appl Soft Comput 12(1):353–359

    Google Scholar 

  • Patel SJ, Panchal AK, Kheraj V (2014) Extraction of solar cell parameters from a single current-voltage characteristic using teaching learning based optimization algorithm. Appl Energy 119:384–393

    Google Scholar 

  • Premkumar M, Pradeep J, Sowmya R, Madurai ER, Santhosh KB (2021) Enhanced chaotic jaya algorithm for parameter estimation of photovoltaic cell/modules. ISA Trans 116:139–166

    Google Scholar 

  • Price KV, Awad NH, Ali MZ, Suganthan PN (2018) Problem definitions and evaluation criteria for the 100-digit challenge special session and competition on single objective numerical optimization. In: Technical report. Nanyang Technological University Singapore

  • Qaraad M, Amjad S, Hussein NK, Badawy M, Mirjalili S, Elhosseini MA (2023) Photovoltaic parameter estimation using improved moth flame algorithms with local escape operators. Comput Electr Eng 106:8603

    Google Scholar 

  • Qin AK, Huang VL, Suganthan PN (2009) Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans Evol Comput 13(2):398–417

    Google Scholar 

  • Rao RV, Savsani VJ, Vakharia DP (2012) Teaching-learning-based optimization: an optimization method for continuous non-linear large scale problems. Inf Sci 183(1):1–15

    MathSciNet  Google Scholar 

  • Rao Ravipudi Venkata (2016) Jaya: a simple and new optimization algorithm for solving constrained and unconstrained optimization problems. Int J Ind Eng Comput 7:19–34

    Google Scholar 

  • Reginaldo S, Gilvan B, Adam S, Moisés S, Claudomiro S, Costa João CWA (2018) A semi-autonomous particle swarm optimizer based on gradient information and diversity control for global optimization. Appl Soft Comput 69:330–343

    Google Scholar 

  • Salomon R (1998) Evolutionary algorithms and gradient search: similarities and differences. IEEE Trans Evol Comput 2(2):45–55

    Google Scholar 

  • Shi S, Xiong H (2021) A hybrid immune genetic algorithm with tabu search for minimizing the tool switch times in cnc milling batch-processing. Appl Intell 52(7):7793–7807

    MathSciNet  Google Scholar 

  • Sutton Richard S, Doina P, Satinder S (1999) Between mdps and semi-mdps: a framework for temporal abstraction in reinforcement learning. Artif Intell 112(1–2):181–211

    MathSciNet  MATH  Google Scholar 

  • Tanabe R, Fukunaga A (2013) Success-history based parameter adaptation for differential evolution. In: 2013 IEEE Congress on Evolutionary Computation, pp 71–78

  • Venkata RR, Ankit S (2017) A self-adaptive multi-population based jaya algorithm for engineering optimization. Swarm Evol Comput 37:1–26

    Google Scholar 

  • Wang B-C, Li H-X, Feng Y (2018) An improved teaching-learning-based optimization for constrained evolutionary optimization. Inf Sci 456:131–144

    MathSciNet  Google Scholar 

  • Wang J, Yang B, Li D, Zeng C, Chen Y, Guo Z, Zhang X, Tan T, Shu H, Tao Yu (2021) Photovoltaic cell parameter estimation based on improved equilibrium optimizer algorithm. Energy Convers Manag 236:114051

    Google Scholar 

  • Xu P, Wenjian L, Xin L, Shi C, Yuhui S (2021) Bso20: efficient brain storm optimization for real-parameter numerical optimization. Complex Intell Syst 7(5):2415–2436

    Google Scholar 

  • Yang X, Gong W, Wang L (2019) Comparative study on parameter extraction of photovoltaic models via differential evolution. Energy Convers Manag 201:112113

    Google Scholar 

  • Ye T, Haowen C, Haiping M, Xingyi Z, Chen TK, Yaochu J (2022) Integrating conjugate gradients into evolutionary algorithms for largE−scale continuous multi-objective optimization. IEEE/CAA J Autom Sin 9(10):1801–1817

    Google Scholar 

  • Yiying Z, Zhigang J (2022) Comprehensive learning jaya algorithm for engineering design optimization problems. J Intell Manuf 33:1229–1253

    Google Scholar 

  • Yiying Z (2021) Neural network algorithm with reinforcement learning for parameters extraction of photovoltaic models. In: IEEE Transactions on Neural Networks and Learning Systems

  • Yu K, Qu B, Yue C, Ge S, Chen X, Liang J (2019) A performancE−guided jaya algorithm for parameters identification of photovoltaic cell and module. Appl Energy 237:241–257

    Google Scholar 

  • Zhang Y, Chi A, Mirjalili S (2021) Enhanced jaya algorithm: a simple but efficient optimization method for constrained engineering design problems. Knowl-Based Syst 233:107555

    Google Scholar 

  • Zhang J, Sanderson AC (2009) Jade: Adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958

    Google Scholar 

  • Zhao S, Yulu W, Tan S, Jinran W, Cui Z, Wang Y-G (2023) Qqlmpa: a quasi-opposition learning and q-learning based marine predators algorithm. Expert Syst Appl 213:119246

    Google Scholar 

  • Zhao F, Zhang H, Ling Wang R, Ma TX, Zhu N et al (2022) A surrogatE−assisted jaya algorithm based on optimal directional guidance and historical learning mechanism. Eng Appl Artif Intell 111:104775

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 12271419).

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Xidian University, Xi’an, 710126, China

    Lingyun Deng & Sanyang Liu

Authors
  1. Lingyun Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LD Conceptualization, Software, Writing—original draft, Visualization. SL: Funding acquisition, Conceptualization, Validation, Writing—review & editing. LD Conceptualization, Software, Writing—original draft, Visualization. SL: Funding acquisition, Conceptualization, Validation, Writing—Review and editing.

Corresponding author

Correspondence to Lingyun Deng.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, L., Liu, S. Incorporating Q-learning and gradient search scheme into JAYA algorithm for global optimization. Artif Intell Rev 56 (Suppl 3), 3705–3748 (2023). https://doi.org/10.1007/s10462-023-10613-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-023-10613-1

Keywords

Search

Navigation