Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement learning marine predators algorithm for global optimization

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Given the weak convergence, limited balance capacity, and optimization limitations observed in the Marine Predators Algorithm (MPA), which draws inspiration from the predatory behavior of marine organisms during evolutionary processes, this study introduces a Reinforcement Learning Marine Predators Algorithm (RLMPA). Firstly, based on the predatory characteristics at different stages, we have designed three location update strategies for search agents aimed at creating high-quality candidate solutions from three perspectives. In particular, ranking paired mutually beneficial learning is specifically designed to expand the scope of exploration to generate as many high-quality candidate solutions as possible for future generations. The Gaussian random walk learning is specifically designed to achieve better optimization in the transitional phase by adjusting the step-size control parameters, successfully completing the transition from exploration to local exploitation phase. Additionally, modified somersault foraging strategy is introduced to accelerate local convergence and perform more extensive local exploitation. Secondly, we integrate reinforcement learning into MPA and use Q-learning mechanism to adaptively select location update strategies. Agents fully utilize the collected information to evaluate the next action of the agents, coordinate the exploration phase and exploitation phase, and enhance the global optimization ability. Finally, compared with 10 competitive algorithms, RLMPA achieves better comprehensive performance in global optimization ability, search efficiency and convergence speed on 41 test functions and 5 practical engineering problems. In the Friedman rank sum tests, RLMPA achieves a preferable overall ranking, and has certain ascendant preponderances in solving practical problems with stability, effectiveness and robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

All data for this study are available from the corresponding author.

References

  1. Zhou, Y., He, X., Chen, Z., Jiang, S.: A neighborhood regression optimization algorithm for computationally expensive optimization problems. IEEE Trans. Cybern. 52(5), 3018–3031 (2022)

    Article  Google Scholar 

  2. Zhu, D., et al.: Human memory optimization algorithm: a memory-inspired optimizer for global optimization problems. Expert Syst. App. 237, 121597 (2023)

    Article  Google Scholar 

  3. Sun, G., Han, R., Deng, L., Li, C., Yang, G.: Hierarchical structure-based joint operations algorithm for global optimization. Swarm Evol. Comput. 79, 101311 (2023)

    Article  Google Scholar 

  4. Li, C., Sun, G., Deng, L., Qiao, L., Yang, G.: A population state evaluation-based improvement framework for differential evolution. Inf. Sci. 629, 15–38 (2023)

    Article  Google Scholar 

  5. Zhu, D., et al.: A multi-strategy particle swarm algorithm with exponential noise and fitness-distance balance method for low-altitude penetration in secure space. J. Comput. Sci. 74, 102149 (2023)

    Article  Google Scholar 

  6. Zhu, D., Wang, S., Zhou, C., et al.: Manta ray foraging optimization based on mechanics game and progressive learning for multiple optimization problems. Appl. Soft Comput. (2023). https://doi.org/10.1016/j.asoc.2023.110561

    Article  Google Scholar 

  7. Shami, T.M., Mirjalili, S., Al-Eryani, Y., Daoudi, K., Izadi, S., Abualigah, L.: Velocity pausing particle swarm optimization: a novel variant for global optimization. Neural Comput App 35(12), 1–31 (2023)

    Google Scholar 

  8. Ma, C., Huang, H., Fan, Q., Wei, J., Du, Y., Gao, W.: Grey wolf optimizer based on Aquila exploration method. Expert Syst. App. 205, 117629 (2022)

    Article  Google Scholar 

  9. Houssein, E.H., Oliva, D., Çelik, E., Emam, M.M., Ghoniem, R.M.: Boosted sooty tern optimization algorithm for global optimization and feature selection. Expert Syst App 213, 119015 (2023)

    Article  Google Scholar 

  10. Too, J., Mafarja, M., Mirjalili, S.: Spatial bound whale optimization algorithm: an efficient high-dimensional feature selection approach. Neural Comput App 33, 16229–16250 (2021)

    Article  Google Scholar 

  11. Wang, J., Bei, J., Song, H., Zhang, H., Zhang, P.: A whale optimization algorithm with combined mutation and removing similarity for global optimization and multilevel thresholding image segmentation. Appl. Soft Comput. 137, 110130 (2023)

    Article  Google Scholar 

  12. Elgamal, Z., Md Sabri, A.Q., Tubishat, M., Tbaishat, D., Makhadmeh, S.N., Alomari, O.A.: Improved reptile search optimization algorithm using chaotic map and simulated annealing for feature selection in medical field. IEEE Access 10, 51428–51446 (2022)

    Article  Google Scholar 

  13. Nematollahi, A.F., Rahiminejad, A., Vahidi, B.: A novel meta-heuristic optimization method based on golden ratio in nature. Soft. Comput. 24(2), 1117–1151 (2020)

    Article  Google Scholar 

  14. Li Y, Zhao L, Zhou S. Review of genetic algorithm. Mater. Sci Eng, PTS1–22011, 365–367

  15. Storn, R., Price, K.: Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997). https://doi.org/10.1023/A:1008202821328

    Article  MathSciNet  Google Scholar 

  16. Hwang, C.-R.: Simulated annealing: theory and applications. Acta Appl. Math. 12(1), 108–111 (1988). https://doi.org/10.1007/BF00047572

    Article  MathSciNet  Google Scholar 

  17. Saryazdi, S.: GSA: a gravitational search algorithm. Inf. Sci. 179(13), 2232–2248 (2009)

    Article  Google Scholar 

  18. Rao, R.V., Savsani, V.J., Vakharia, D.P.: Teaching-learning-based optimization: an optimization method for continuous non-linear large scale problems (article). Inf. Sci. 183(1), 1–15 (2012)

    Article  Google Scholar 

  19. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: IEEE International Conference on Neural Networks, pp. 1942–1948. IEEE (1995)

    Google Scholar 

  20. Mirjalili, S., Lewis, A.: Grey wolf optimizer. Adv. Eng. Softw. 69(3), 46–61 (2014)

    Article  Google Scholar 

  21. Mirjalili, S., Lewis, A.: The whale optimization algorithm. Adv. Eng. Softw. 95(5), 51–67 (2016)

    Article  Google Scholar 

  22. Gandomi, A.H., Yang, X.-S., Alavi, A.H.: Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng. Comput. 29(1), 17–35 (2013)

    Article  Google Scholar 

  23. Karaboga, D., Akay, B.: A comparative study of artificial bee colony algorithm. Appl. Math. Comput. 214(1), 108–132 (2009)

    MathSciNet  Google Scholar 

  24. Heidari, A.A., Mirjalili, S., Faris, H., Aljarah, I., Mafarja, M., Chen, H.: Harris hawks optimization: algorithm and applications. Fut Gen Comput Syst 97, 849–872 (2019)

    Article  Google Scholar 

  25. Zhao, W., Zhang, Z., Wang, L.: Manta ray foraging optimization: an effective bio-inspired optimizer for engineering applications. Eng App Artif Intell 87, 103300 (2020)

    Article  Google Scholar 

  26. Yang, X.S.: Firefly algorithm, stochastic test functions and design optimization. Int J Bio-Inspired Comput 2(2), 78–84 (2010)

    Article  Google Scholar 

  27. Mirjalili, S., Gandomi, A.H., Mirjalili, S.Z., Saremi, S.: Salp swarm algorithm: a bio-inspired optimizer for engineering design problems. Adv. Eng. Softw. 114, 163–191 (2017)

    Article  Google Scholar 

  28. Faramarzi, A., Heidarinejad, M., Mirjalili, S., Gandomi, A.H.: Marine predators algorithm: a nature-inspired metaheuristic. Expert Syst. Appl. 152, 113377 (2020)

    Article  Google Scholar 

  29. Yousri, D., AbdElaziz, M., Oliva, D., Abraham, A., Alotaibi, M.A., Hossain, M.A.: Fractional-order comprehensive learning marine predators algorithm for global optimization and feature selection. Knowl Based Syst 235, 107603 (2022)

    Article  Google Scholar 

  30. Abd Elaziz, M., Mohammadi, D., Oliva, D., Salimifard, K.: Quantum marine predators algorithm for addressing multilevel image segmentation. Appl. Soft Comput. 110, 107598 (2021)

    Article  Google Scholar 

  31. ZakiDiab, A.A., Tolba, M.A., El-Magd, A.G.A., Zaky, M.M., El-Rifaie, A.M.: Fuel cell parameters estimation via marine predators and political optimizers. IEEE Access 8, 166998–167018 (2020)

    Article  Google Scholar 

  32. Fan, Q., Huang, H., Chen, Q., Yao, L., Yang, K., Huang, D.: A modified self-adaptive marine predators algorithm: framework and engineering applications. Eng. Comput. 38(4), 3269–3294 (2022)

    Article  Google Scholar 

  33. Shaheen, A.M., Elsayed, A.M., Ginidi, A.R., El-Sehiemy, R.A., Alharthi, M.M., Ghoneim, S.S.M.: A novel improved marine predators algorithm for combined heat and power economic dispatch problem. Alexandria Eng J 61(3), 1834–1851 (2022)

    Article  Google Scholar 

  34. Qin, C., Han, B.: A novel hybrid quantum particle swarm optimization with marine predators for engineering design problems. IEEE Access 10, 1 (2022)

    Article  MathSciNet  Google Scholar 

  35. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Article  Google Scholar 

  36. Shakya, A.K., Pillai, G., Chakrabarty, S.: Reinforcement learning algorithms: a brief survey. Expert Syst App 231, 120495 (2023)

    Article  Google Scholar 

  37. Song, Y., Wei, L., Yang, Q., Wu, J., Xing, L., Chen, Y.: RL-GA: a reinforcement learning-based genetic algorithm for electromagnetic detection satellite scheduling problem. Swarm Evol. Comput. 77, 101236 (2023)

    Article  Google Scholar 

  38. Zamfirache, I.A., Precup, R.-E., Roman, R.-C., Petriu, E.M.: Reinforcement Learning-based control using Q-learning and gravitational search algorithm with experimental validation on a nonlinear servo system. Inf. Sci. 583, 99–120 (2022)

    Article  Google Scholar 

  39. Wang, B., Feng, K., Wang, X.: Bi-objective scenario-guided swarm intelligent algorithms based on reinforcement learning for robust unrelated parallel machines scheduling with setup times. Swarm Evol. Comput. 80, 101321 (2023)

    Article  Google Scholar 

  40. Li, W., Liang, P., Sun, B., Sun, Y., Huang, Y.: Reinforcement learning-based particle swarm optimization with neighborhood differential mutation strategy. Swarm Evol. Comput. 78, 101274 (2023)

    Article  Google Scholar 

  41. Kumar, S., Yildiz, B.S., Mehta, P., Panagant, N., Sait, S.M., Mirjalili, S., Yildiz, A.R.: Chaotic marine predators algorithm for global optimization of real-world engineering problems. Knowl Based Syst 261, 110192 (2023)

    Article  Google Scholar 

  42. Zhao, S., Wu, Y., Tan, S., Wu, J., Cui, Z., Wang, Y.-G.: QQLMPA: a quasi-opposition learning and Q-learning based marine predators algorithm. Expert Syst App 213, 119246 (2022)

    Article  Google Scholar 

  43. Yousri, D.A., Fathy, A.A., Rezk, H.: A new comprehensive learning marine predator algorithm for extracting the optimal parameters of supercapacitor model. J Energy Storage 42, 103035 (2021)

    Article  Google Scholar 

  44. Shen, B., Khishe, M., Mirjalili, S.: Evolving marine predators algorithm by dynamic foraging strategy for real-world engineering optimization problems. Eng App Artif Intell 123, 106207 (2023)

    Article  Google Scholar 

  45. Han, M., Du, Z., Zhu, H., Li, Y., Yuan, Q., Zhu, H.: Golden-sine dynamic marine predator algorithm for addressing engineering design optimization. Expert Syst App 210, 118460 (2022)

    Article  Google Scholar 

  46. AS Sadiq; AA Dehkordi; S Mirjalili; Q-V Pham. Nonlinear marine predator algorithm: a cost-effective optimizer for fair power allocation in NOMA-VLC-B5G networks. Expert Syst App, 2022, 203.

  47. Oszust, M.: Enhanced marine predators algorithm with local escaping operator for global optimization. Knowl Based Syst 232, 107467 (2021)

    Article  Google Scholar 

  48. Hassan, M.H., Yousri, D., Kamel, S., Rahmann, C.: A modified Marine predators algorithm for solving single- and multi-objective combined economic emission dispatch problems. Comput. Ind. Eng. 164, 107906 (2022)

    Article  Google Scholar 

  49. Wang, M., Li, X., Chen, L., Chen, H.: Medical machine learning based on multiobjective evolutionary algorithm using learning decomposition. Expert Syst App 216, 119450 (2023)

    Article  Google Scholar 

  50. Gibson, S., Issac, B., Zhang, L., Jacob, S.M.: Detecting spam email with machine learning optimized with bio-inspired metaheuristic algorithms. IEEE Access 8, 187914–187932 (2020)

    Article  Google Scholar 

  51. Ma, J., Xia, D., Wang, Y., Niu, X., Jiang, S., Liu, Z., Guo, H.: A comprehensive comparison among metaheuristics (MHs) for geohazard modeling using machine learning: Insights from a case study of landslide displacement prediction. Eng App Artif Intell 114, 105150 (2022)

    Article  Google Scholar 

  52. Esmaeili, H., Bidgoli, B.M., Hakami, V.: CMML: combined metaheuristic-machine learning for adaptable routing in clustered wireless sensor networks. Appl. Soft Comput. 118, 108477 (2022)

    Article  Google Scholar 

  53. Deng, H., Peng, L., Zhang, H., Yang, B., Chen, Z.: Ranking-based biased learning swarm optimizer for large-scale optimization. Inf Sci: Int J 493, 120–137 (2019)

    Article  MathSciNet  Google Scholar 

  54. Onay, F.K.: A novel improved chef-based optimization algorithm with Gaussian random walk-based diffusion process for global optimization and engineering problems. Math Comput Simulation 212, 195–223 (2023)

    Article  MathSciNet  Google Scholar 

  55. Peng, H., Zeng, Z., Deng, C., Wu, Z.: Multi-strategy serial cuckoo search algorithm for global optimization. Knowl Based Syst 214, 106729 (2021)

    Article  Google Scholar 

  56. Duan, Y., Liu, C., Li, S., Guo, X., Yang, C.: Manta ray foraging and Gaussian mutation-based elephant herding optimization for global optimization. Eng. Comput. 39(2), 1085–1125 (2023)

    Article  Google Scholar 

  57. Liu, J., Wu, Y.: An improved lion swarm optimization algorithm with chaotic mutation strategy and boundary mutation strategy for global optimization. IEEE Access 10, 1 (2022)

    Article  Google Scholar 

  58. Feng, Z.-k, Duan, J.-f, Niu, W.-j, Jiang, Z.-q, Liu, Y.: Enhanced sine cosine algorithm using opposition learning, adaptive evolution and neighborhood search strategies for multivariable parameter optimization problems. Appl. Soft Comput. 119, 108562 (2022)

    Article  Google Scholar 

  59. Ai, C., He, S., Fan, X.: Parameter estimation of fractional-order chaotic power system based on lens imaging learning strategy state transition algorithm. IEEE Access 11, 13724–13737 (2023)

    Article  Google Scholar 

  60. Jiao, K., Chen, J., Xin, B., Li, L.: A reference vector based multiobjective evolutionary algorithm with Q-learning for operator adaptation. Swarm Evol. Comput. 76, 101225 (2023)

    Article  Google Scholar 

  61. Zhang, H., Sun, J., Bäck, T., Zhang, Q., Xu, Z.: Controlling sequential hybrid evolutionary algorithm by Q-learning. IEEE Comput Intell Magaz 18(1), 84–103 (2023)

    Article  Google Scholar 

  62. Hamad, Q.S., Samma, H., Suandi, S.A., Mohamad-Saleh, J.: Q-learning embedded sine cosine algorithm (QLESCA). Expert Syst App 193, 116417 (2022)

    Article  Google Scholar 

  63. Baykasoglu, A., Ozsoydan, F.B.: Adaptive firefly algorithm with chaos for mechanical design optimization problems. Appl. Soft Comput. 36(1), 152–164 (2015)

    Article  Google Scholar 

  64. Kalyanmoy, D.: An efficient constraint handling method for genetic algorithms. Comput. Methods Appl. Mech. Eng. 186, 311–338 (2000)

    Article  Google Scholar 

Download references

Funding

This research was funded by the Natural Science Foundation of China (Grant Nos. 62062037, 61562037, 72261018); the Natural Science Foundation of Jiangxi Province (Grant Nos. 20212BAB202014, 20171BAB202026).

Author information

Authors and Affiliations

  1. School of Information Engineering, Jaingxi University of Science and Technology, Ganzhou, 341000, China

    Jianlan Wang, Zhendong Wang, Shuxin Yang, Junling Wang & Dahai Li

  2. College of Mathematics and Computer Science, Zhejiang Normal University, Jinhua, 321004, China

    Donglin Zhu

Authors
  1. Jianlan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhendong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Donglin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuxin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junling Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dahai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JW: Conceptualization, Methodology, Software, Data curation, Writing—Original draft preparation. ZW: Conceptualization, Supervision, Funding acquisition. DZ: Visualization, Investigation. SY: Conceptualization, Methodology. JW: Methodology, supervision. DL: Methodology, supervision.

Corresponding author

Correspondence to Zhendong Wang.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Figs.

Fig. 14
figure 14figure 14

The box plots of 11 algorithms based on CEC 2022 (20-dimenison)

Full size image

14,

Fig. 15
figure 15figure 15figure 15

The box plots of 11 algorithms based on CEC 2017 (30-dimenison)

Full size image

15,

Fig. 16
figure 16figure 16figure 16

The box plots of 11 algorithms based on CEC 2017 (50-dimenison)

Full size image

16,

Fig. 17
figure 17figure 17

The convergence curves of 11 algorithms based on CEC 2022 (20-dimension)

Full size image

17,

Fig. 18
figure 18figure 18figure 18figure 18figure 18

The convergence curves of 11 algorithms based on CEC 2017 (30-dimension)

Full size image

18, and

Fig. 19
figure 19figure 19figure 19figure 19figure 19

The convergence curves of 11 algorithms based on CEC 2017 (50-dimension)

Full size image

19; Table 

Table 15 The results of 11 algorithms on CEC 2022 (20-dimension)
Full size table

15 and

Table 16 The results of 11 algorithms on CEC 2017 (50-dimension)
Full size table

16.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Wang, Z., Zhu, D. et al. Reinforcement learning marine predators algorithm for global optimization. Cluster Comput (2024). https://doi.org/10.1007/s10586-024-04381-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10586-024-04381-y

Keywords

Search

Navigation