Skip to main content
SpringerLink
Log in

Lipschitz-inspired HALRECT algorithm for derivative-free global optimization

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

This article considers a box-constrained global optimization problem for Lipschitz-continuous functions with an unknown Lipschitz constant. Motivated by the famous DIRECT (DIviding RECTangles), a new HALRECT (HALving RECTangles) algorithm is introduced. A new deterministic approach combines halving (bisection) with a new multi-point sampling scheme in contrast to trisection and midpoint sampling used in most existing DIRECT-type algorithms. A new partitioning and sampling scheme uses more comprehensive information on the objective function. Four different strategies for selecting potentially optimal hyper-rectangles are introduced to exploit the objective function’s information effectively. The original algorithm HALRECT and other introduced HALRECT variations (twelve in total) are tested and compared with the other twelve recently introduced DIRECT-type algorithms on 96 box-constrained benchmark functions from DIRECTGOLib v1.1, and 96 perturbed their versions. Extensive experimental results are advantageous compared to state-of-the-art DIRECT-type global optimization. New HALRECT approaches offer high robustness across problems of different degrees of complexity, varying from simple—uni-modal and low dimensional to complex—multi-modal and higher dimensionality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availibility

DIRECTGOLib—DIRECT Global Optimization test problems Library is designed as a continuously-growing open-source GitHub repository to which anyone can easily contribute. The exact data underlying this article from DIRECTGOLib v1.1 can be accessed either on GitHub or at Zenodo: -GitHub: (https://github.com/blockchain-group/DIRECTGOLib/tree/v1.1), -Zenodo: (https://doi.org/10.5281/zenodo.6491951), and used under the MIT license. We welcome contributions and corrections to this work.

References

  1. Baker, C.A., Watson, L.T., Grossman, B., Mason, W.H., Haftka, R.T.: Parallel Global Aircraft Configuration Design Space Exploration, pp. 79–96. Nova Science Publishers, Inc. (2001)

    Google Scholar 

  2. Bishop, C.M., Nasrabadi, N.M.: Pattern Recognition and Machine Learning, vol. 4. Springer, New York (2006)

    Google Scholar 

  3. Booker, A.J., Dennis, J., Frank, P.D., Serafini, D.B., Torczon, V.: Optimization using surrogate objectives on a helicopter test example. In: Computational Methods for Optimal Design and Control, pp. 49–58. Springer, New York (1998). https://doi.org/10.1007/978-1-4612-1780-0_3

    Chapter  Google Scholar 

  4. Costa, M.F.P., Rocha, A.M.A.C., Fernandes, E.M.G.P.: Filter-based direct method for constrained global optimization. J. Glob. Optim. 71(3), 517–536 (2018). https://doi.org/10.1007/s10898-017-0596-8

    Article  MathSciNet  Google Scholar 

  5. Finkel, D.E.: MATLAB source code for DIRECT. http://www4.ncsu.edu/~ctk/Finkel_Direct/ (2004). Accessed 22 Mar 2017

  6. Finkel, D.E., Kelley, C.T.: Additive scaling and the DIRECT algorithm. J. Glob. Optim. 36(4), 597–608 (2006). https://doi.org/10.1007/s10898-006-9029-9

    Article  MathSciNet  Google Scholar 

  7. Floudas, C.A.: Deterministic global optimization: theory, methods and applications. In: Nonconvex Optimization and its Applications, vol. 37. Springer, Boston, MA (1999). https://doi.org/10.1007/978-1-4757-4949-6

  8. Gablonsky, J.M.: Modifications of the DIRECT algorithm. Ph.D. thesis, North Carolina State University (2001)

  9. Gablonsky, J.M., Kelley, C.T.: A locally-biased form of the DIRECT algorithm. J. Glob. Optim. 21(1), 27–37 (2001). https://doi.org/10.1023/A:1017930332101

    Article  MathSciNet  Google Scholar 

  10. Gaviano, M., Kvasov, D.E., Lera, D., Sergeyev, Y.D.: Algorithm 829: software for generation of classes of test functions with known local and global minima for global optimization. ACM Trans. Math. Softw. (TOMS) 29(4), 469–480 (2003). https://doi.org/10.1145/962437.962444

    Article  MathSciNet  Google Scholar 

  11. Grishagin, V.A.: Operating characteristics of some global search algorithms. In: Problems of Stochastic Search, vol. 7, pp. 198–206. Zinatne, Riga (1978). (In Russian)

  12. He, J., Verstak, A., Watson, L.T., Sosonkina, M.: Design and implementation of a massively parallel version of direct. Comput. Optim. Appl. (2008). https://doi.org/10.1007/s10589-007-9092-2

    Article  MathSciNet  Google Scholar 

  13. Holmstrom, K., Goran, A.O., Edvall, M.M.: User’s guide for tomlab 7 (2010). https://tomopt.com/

  14. Horst, R., Pardalos, P.M., Thoai, N.V.: Introduction to global optimization. In: Nonconvex Optimization and its Application. Kluwer Academic Publishers, Berlin (1995)

  15. Jones, D.R.: The direct global optimization algorithm. In: Floudas, C.A., Pardalos, P.M. (eds.) The Encyclopedia of Optimization, pp. 431–440. Kluwer Academic Publishers, Dordrect (2001)

    Chapter  Google Scholar 

  16. Jones, D.R., Martins, J.R.R.A.: The DIRECT algorithm: 25 years later. J. Glob. Optim. 79, 521–566 (2021). https://doi.org/10.1007/s10898-020-00952-6

    Article  MathSciNet  Google Scholar 

  17. Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993). https://doi.org/10.1007/BF00941892

    Article  MathSciNet  Google Scholar 

  18. Lera, D., Sergeyev, Y.D.: Lipschitz and Hölder global optimization using space-filling curves. Appl. Numer. Math. 60(1–2), 115–129 (2010). https://doi.org/10.1016/j.apnum.2009.10.004

    Article  MathSciNet  Google Scholar 

  19. Lera, D., Sergeyev, Y.D.: Acceleration of univariate global optimization algorithms working with Lipschitz functions and Lipschitz first derivatives. SIAM J. Optim. 23(1), 508–529 (2013). https://doi.org/10.1137/110859129

    Article  MathSciNet  Google Scholar 

  20. Lera, D., Sergeyev, Y.D.: Deterministic global optimization using space-filling curves and multiple estimates of Lipschitz and Hölder constants. Commun. Nonlinear Sci. Numer. Simul. 23(1), 328–342 (2015). https://doi.org/10.1016/j.cnsns.2014.11.015

    Article  MathSciNet  Google Scholar 

  21. Lera, D., Sergeyev, Y.D.: Gosh: derivative-free global optimization using multi-dimensional space-filling curves. J. Glob. Optim. 71, 193–211 (2018). https://doi.org/10.1007/s10898-017-0589-7

    Article  MathSciNet  Google Scholar 

  22. Liberti, L., Kucherenko, S.: Comparison of deterministic and stochastic approaches to global optimization. Int. Trans. Oper. Res. 12(3), 263–285 (2005). https://doi.org/10.1111/j.1475-3995.2005.00503.x

    Article  MathSciNet  Google Scholar 

  23. Liu, H., Xu, S., Chen, X., Wang, X., Ma, Q.: Constrained global optimization via a direct-type constraint-handling technique and an adaptive metamodeling strategy. Struct. Multidiscip. Optim. 55(1), 155–177 (2017). https://doi.org/10.1007/s00158-016-1482-6

    Article  MathSciNet  Google Scholar 

  24. Liuzzi, G., Lucidi, S., Piccialli, V.: A direct-based approach exploiting local minimizations for the solution for large-scale global optimization problems. Comput. Optim. Appl. 45(2), 353–375 (2010). https://doi.org/10.1007/s10589-008-9217-2

    Article  MathSciNet  Google Scholar 

  25. Liuzzi, G., Lucidi, S., Piccialli, V.: Exploiting derivative-free local searches in direct-type algorithms for global optimization. Comput. Optim. Appl. 65, 449–475 (2016). https://doi.org/10.1007/s10589-015-9741-9

    Article  MathSciNet  Google Scholar 

  26. Na, J., Lim, Y., Han, C.: A modified direct algorithm for hidden constraints in an LNG process optimization. Energy 126, 488–500 (2017). https://doi.org/10.1016/j.energy.2017.03.047

    Article  Google Scholar 

  27. Paulavičius, R., Chiter, L., Žilinskas, J.: Global optimization based on bisection of rectangles, function values at diagonals, and a set of Lipschitz constants. J. Glob. Optim. 71(1), 5–20 (2018). https://doi.org/10.1007/s10898-016-0485-6

    Article  MathSciNet  Google Scholar 

  28. Paulavičius, R., Sergeyev, Y.D., Kvasov, D.E., Žilinskas, J.: Globally-biased DISIMPL algorithm for expensive global optimization. J. Glob. Optim. 59(2–3), 545–567 (2014). https://doi.org/10.1007/s10898-014-0180-4

    Article  MathSciNet  Google Scholar 

  29. Paulavičius, R., Sergeyev, Y.D., Kvasov, D.E., Žilinskas, J.: Globally-biased BIRECT algorithm with local accelerators for expensive global optimization. Expert Syst. Appl. 144, 11305 (2020). https://doi.org/10.1016/j.eswa.2019.113052

    Article  Google Scholar 

  30. Paulavičius, R., Žilinskas, J.: Analysis of different norms and corresponding Lipschitz constants for global optimization. Technol. Econ. Dev. Econ. 36(4), 383–387 (2006). https://doi.org/10.1080/13928619.2006.9637758

    Article  Google Scholar 

  31. Paulavičius, R., Žilinskas, J.: Analysis of different norms and corresponding Lipschitz constants for global optimization in multidimensional case. Inf. Technol. Control 36(4), 383–387 (2007)

    Google Scholar 

  32. Paulavičius, R., Žilinskas, J.: Simplicial Lipschitz optimization without the Lipschitz constant. J. Glob. Optim. 59(1), 23–40 (2014). https://doi.org/10.1007/s10898-013-0089-3

    Article  MathSciNet  Google Scholar 

  33. Paulavičius, R., Žilinskas, J.: Advantages of simplicial partitioning for Lipschitz optimization problems with linear constraints. Optim. Lett. 10(2), 237–246 (2016). https://doi.org/10.1007/s11590-014-0772-4

    Article  MathSciNet  Google Scholar 

  34. Pillo, G.D., Liuzzi, G., Lucidi, S., Piccialli, V., Rinaldi, F.: A DIRECT-type approach for derivative-free constrained global optimization. Comput. Optim. Appl. 65(2), 361–397 (2016). https://doi.org/10.1007/s10589-016-9876-3

    Article  MathSciNet  Google Scholar 

  35. Pillo, G.D., Lucidi, S., Rinaldi, F.: An approach to constrained global optimization based on exact penalty functions. J. Optim. Theory Appl. 54(2), 251–260 (2010). https://doi.org/10.1007/s10898-010-9582-0

    Article  MathSciNet  Google Scholar 

  36. Pintér, J.D.: Global optimization in action: continuous and Lipschitz optimization: algorithms, implementations and applications. In: Nonconvex Optimization and its Applications, vol. 6. Springer, Berlin (1996). https://doi.org/10.1007/978-1-4757-2502-5

  37. Piyavskii, S.A.: An algorithm for finding the absolute minimum of a function. Theory Optim. Solut. 2, 13–24 (1967). https://doi.org/10.1016/0041-5553(72)90115-2. (in Russian)

    Article  MathSciNet  Google Scholar 

  38. Posypkin, M.A., Sergeyev, Y.D.: Efficient smooth minorants for global optimization of univariate functions with the first derivative satisfying the interval lipschitz condition. J. Glob. Optim. (2022). https://doi.org/10.1007/s10898-022-01251-y

    Article  Google Scholar 

  39. Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Glob. Optim. 56(3), 1247–1293 (2013). https://doi.org/10.1007/s10898-012-9951-y

    Article  MathSciNet  Google Scholar 

  40. Sergeyev, Y.D., Kvasov, D., Mukhametzhanov, M.: On the efficiency of nature-inspired metaheuristics in expensive global optimization with limited budget. Sci. Rep. 8(1), 1–9 (2018). https://doi.org/10.1038/s41598-017-18940-4

    Article  Google Scholar 

  41. Sergeyev, Y.D., Kvasov, D.E.: Global search based on diagonal partitions and a set of Lipschitz constants. SIAM J. Optim. 16(3), 910–937 (2006). https://doi.org/10.1137/040621132

    Article  MathSciNet  Google Scholar 

  42. Sergeyev, Y.D., Kvasov, D.E.: Diagonal Global Optimization Methods. FizMatLit, Moscow (2008). (In Russian)

    Google Scholar 

  43. Sergeyev, Y.D., Kvasov, D.E.: Lipschitz global optimization. In: Cochran, J.J., Cox, L.A., Keskinocak, P., Kharoufeh, J.P., Smith, J.C. (eds.) Wiley Encyclopedia of Operations Research and Management Science (in 8 volumes), vol. 4, pp. 2812–2828. Wiley, New York, NY (2011)

    Google Scholar 

  44. Sergeyev, Y.D., Kvasov, D.E.: Deterministic global optimization: an introduction to the diagonal approach. In: Springer Briefs in Optimization. Springer, Berlin (2017). https://doi.org/10.1007/978-1-4939-7199-2

  45. Sergeyev, Y.D., Nasso, M.C., Lera, D.: Numerical methods using two different approximations of space-filling curves for black-box global optimization. J. Glob. Optim. (2022). https://doi.org/10.1007/s10898-022-01216-1

    Article  Google Scholar 

  46. Sergeyev, Y.D., Nasso, M.C., Mukhametzhanov, M.S., Kvasov, D.E.: Novel local tuning techniques for speeding up one-dimensional algorithms in expensive global optimization using lipschitz derivatives. J. Comput. Appl. Math. 383, 113134 (2021). https://doi.org/10.1016/j.cam.2020.113134

    Article  MathSciNet  Google Scholar 

  47. Sergeyev, Y.D., Strongin, R.G., Lera, D.: Introduction to global optimization exploiting space-filling curves. In: Springer Briefs in Optimization. Springer, New York, NY (2013). https://doi.org/10.1007/978-1-4614-8042-6

  48. Shubert, B.O.: A sequential method seeking the global maximum of a function. SIAM J. Numer. Anal. 9, 379–388 (1972). https://doi.org/10.1137/0709036

    Article  MathSciNet  Google Scholar 

  49. Stripinis, L., Paulavičius, R.: A new DIRECT-GLh algorithm for global optimization with hidden constraints. Optim. Lett. 15(6), 1865–1884 (2021). https://doi.org/10.1007/s11590-021-01726-z

    Article  MathSciNet  Google Scholar 

  50. Stripinis, L., Paulavičius, R.: An empirical study of various candidate selection and partitioning techniques in the DIRECT framework. J. Glob. Optim. (2022). https://doi.org/10.1007/s10898-022-01185-5

    Article  Google Scholar 

  51. Stripinis, L., Paulavičius, R.: Directgo: a new direct-type matlab toolbox for derivative-free global optimization, version v1.1.0, GitHub. (2022) https://github.com/blockchain-group/DIRECTGO/releases/tag/v1.1.0

  52. Stripinis, L., Paulavičius, R.: DIRECTGOLib - DIRECT Global Optimization test problems Library, Version v1.1, GitHub. (2022) https://github.com/blockchain-group/DIRECTGOLib/tree/v1.1

  53. Stripinis, L., Paulavičius, R., Žilinskas, J.: Improved scheme for selection of potentially optimal hyper-rectangles in DIRECT. Optim. Lett. 12(7), 1699–1712 (2018). https://doi.org/10.1007/s11590-017-1228-4

    Article  MathSciNet  Google Scholar 

  54. Stripinis, L., Paulavičius, R., Žilinskas, J.: Penalty functions and two-step selection procedure based DIRECT-type algorithm for constrained global optimization. Struct. Multidiscip. Optim. 59(6), 2155–2175 (2019). https://doi.org/10.1007/s00158-018-2181-2

    Article  MathSciNet  Google Scholar 

  55. Stripinis, L., Paulavičius, R.: Directgo: a new direct-type matlab toolbox for derivative-free global optimization. ACM Trans. Math. Softw. 48(4), 1–46 (2022). https://doi.org/10.1145/3559755

    Article  MathSciNet  Google Scholar 

  56. Stripinis, L., Paulavičius, R.: DIRECTGOLib - DIRECT Global Optimization test problems Library, Version v1.1, Zenodo (2022). https://doi.org/10.5281/zenodo.6491951

  57. Stripinis, L., Paulavičius, R.: Experimental study of excessive local refinement reduction techniques for global optimization DIRECT-type algorithms. Mathematics (2022). https://doi.org/10.3390/math10203760

    Article  Google Scholar 

  58. Stripinis, L., Žilinskas, J., Casado, L.G., Paulavičius, R.: On matlab experience in accelerating DIRECT-GLce algorithm for constrained global optimization through dynamic data structures and parallelization. Appl. Math. Comput. 390, 125596 (2021). https://doi.org/10.1016/j.amc.2020.125596

    Article  MathSciNet  Google Scholar 

  59. Strongin, R.G., Sergeyev, Y.D.: Global Optimization with Non-convex Constraints: Sequential and Parallel Algorithms. Kluwer Academic Publishers, Dordrecht (2000)

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Data Science and Digital Technologies, Vilnius University, Akademijos 4, 08663, Vilnius, Lithuania

    Linas Stripinis & Remigijus Paulavičius

Authors
  1. Linas Stripinis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Remigijus Paulavičius
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Remigijus Paulavičius.

Ethics declarations

Code availability

All implemented versions of the HALRECT algorithm are available at the GitHub repository: (https://github.com/blockchain-group/DIRECTGO) and can be used under the MIT license. We welcome contributions and corrections to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stripinis, L., Paulavičius, R. Lipschitz-inspired HALRECT algorithm for derivative-free global optimization. J Glob Optim 88, 139–169 (2024). https://doi.org/10.1007/s10898-023-01296-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-023-01296-7

Keywords

Mathematics Subject Classification

Search

Navigation