Skip to main content
SpringerLink
Log in

NPROS: A Not So Pure Random Orthogonal search algorithm—A suite of random optimization algorithms driven by reinforcement learning

  • Original Paper
  • Published:
Optimization Letters Aims and scope Submit manuscript

Abstract

We live in a world where waves of novel nature-inspired metaheuristic algorithms keep hitting the shore repeatedly. This never-ending surge of new metaheuristic algorithms is overwhelming to the extent that their novelty is being criticized. In this paper, instead of focusing on metaheuristics, we focus on pure random search algorithms for global optimization. Pure Random Orthogonal Search (PROS) is a recently published random optimization algorithm, which is strikingly simple, involves no parameter tuning, but is very effective in solving global optimization problems. In this paper, we propose a modified version of the PROS algorithm, which injects a flavor of exploitation into the otherwise purely explorative PROS algorithm. Further, the concepts of reinforcement learning are utilized to provide the proposed algorithm the ability to ‘learn’ to take the optimal actions, to find the global optima. The source code of NPROS is publicly available at: https://github.com/Shahul-Rahman/NPROS

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Sörensen, Kenneth: Metaheuristics-the metaphor exposed. Int. Trans. Oper. Res. 22(1), 3–18 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  2. Stork, J., Eiben, A.E., Bartz-Beielstein, T.: A new taxonomy of global optimization algorithms. Nat. Comput. 21, 219–242 (2020)

    Article  MathSciNet  Google Scholar 

  3. Locatelli, M., Schoen, F.: Global optimization: theory, algorithms, and applications. Society for Industrial and Applied Mathematics, (2013)

  4. Rastrigin, L.A.: The convergence of the random search method in the extremal control of a many parameter system. Automat. Remote Control 24, 1337–1342 (1963)

    Google Scholar 

  5. Matyas, J.: Random optimization. Automat. Remote control 26(2), 246–253 (1965)

    MathSciNet  MATH  Google Scholar 

  6. White, R.C., Jr.: A survey of random methods for parameter optimization. Simulation 17(5), 197–205 (1971)

    Article  MathSciNet  Google Scholar 

  7. Schumer, M. A., Steiglitz, Kenneth: ”Adaptive step size random search.” IEEE Transactions on Automatic Control 13.3: 270-276 (1968)

  8. Weise, T.: ”Global optimization algorithms-theory and application.” Self-Published Thomas Weise 361 (2009)

  9. Locatelli, Marco, Schoen, Fabio: (Global) Optimization: historical notes and recent developments. EURO J. Comput. Optim. 9, 100012 (2021)

    Article  MathSciNet  Google Scholar 

  10. Plevris, Vagelis, Bakas, Nikolaos P., Solorzano, German: Pure random orthogonal search (PROS): a plain and elegant parameterless algorithm for global optimization. Appl. Sci. 11(11), 5053 (2021)

    Article  Google Scholar 

  11. Hameed, A.S.S.S., Rajagopalan, N.: SPGD: search party gradient descent algorithm, a simple gradient-based parallel algorithm for bound-constrained optimization. Mathematics 10(5), 800 (2022)

    Article  Google Scholar 

  12. Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT press, Cambridge (2018)

    MATH  Google Scholar 

  13. Wikipedia contributors. ”Normal distribution.” Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 23 Jun. 2022. Web. (11 Jul. 2022)

  14. Casella, G., Berger, R.L.: Statistical Inference, 2nd edn. Duxbury, London (2001). (ISBN 978-0-534-24312-8)

    MATH  Google Scholar 

  15. Maroti, A.: ”Rbed: Reward based epsilon decay.” arXiv preprint arXiv:1910.13701 (2019)

  16. Jamil, M., Yang, X.-S.: A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4, 150–194 (2013)

    MATH  Google Scholar 

  17. Surjanovic, S., Bingham, D.: Virtual Library of Simulation Experiments: Test Functions and Datasets. 2013. Available online: http://www.sfu.ca/ ssurjano .Web. 11 Jul. (2022)

  18. Nathanrooy. Landscapes/Single_Objective.Py at Master Nathanrooy/Landscapes. GitHub. Available online: https://github.com/nathanrooy/landscapes/blob/master/landscapes/single_objective.py .Web. 11 Jul. (2022)

  19. Mahapatra, G. S., Maneckshaw, Balakrishnan, Barker, Kash: ”Multi-objective reliability redundancy allocation using MOPSO under hesitant fuzziness.” Expert Systems with Applications 198: 116696 (2022)

  20. Mohamed, A.W., Hadi, A.A., Mohamed, A.K.: Gaining-sharing knowledge based algorithm for solving optimization problems: a novel nature-inspired algorithm. Int. J. Mach. Learn. Cybernet. 11(7), 1501–1529 (2020)

    Article  Google Scholar 

  21. Gupta, R., Mahendran, V., Badarla, V.: Optimal Searching of Prefetched DASH Segments in Fog Nodes: A Multi-Armed Bandit Approach. In: Proceedings of the 17th ACM Symposium on QoS and Security for Wireless and Mobile Networks. (2021)

  22. Glowacka, D.: Bandit algorithms in interactive information retrieval. In: Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval. (2017)

  23. Vanaret, C. et al.: Certified global minima for a benchmark of difficult optimization problems. arXiv preprint arXiv:2003.09867 (2020)

  24. Thieu, N.V., Mirjalili, S.: MEALPY: a Framework of The State-of-The- Art Meta-Heuristic Algorithms in Python. (2022) URL https://doi.org/10.5281/zenodo.5665747

  25. Grosso, A., Locatelli, M., Schoen, F.: A population-based approach for hard global optimization problems based on dissimilarity measures. Math. Progr. 110, 373–404 (2007)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering, National Institute of Technology Puducherry, Thiruvettakudy, Karaikal, Puducherry, 609609, India

    A. S. Syed Shahul Hameed & Narendran Rajagopalan

Authors
  1. A. S. Syed Shahul Hameed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Narendran Rajagopalan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. S. Syed Shahul Hameed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hameed, A.S.S.S., Rajagopalan, N. NPROS: A Not So Pure Random Orthogonal search algorithm—A suite of random optimization algorithms driven by reinforcement learning. Optim Lett (2023). https://doi.org/10.1007/s11590-023-02038-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11590-023-02038-0

Keywords

Search

Navigation