Skip to main content
SpringerLink
Log in

GOSH: derivative-free global optimization using multi-dimensional space-filling curves

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

Global optimization is a field of mathematical programming dealing with finding global (absolute) minima of multi-dimensional multiextremal functions. Problems of this kind where the objective function is non-differentiable, satisfies the Lipschitz condition with an unknown Lipschitz constant, and is given as a “black-box” are very often encountered in engineering optimization applications. Due to the presence of multiple local minima and the absence of differentiability, traditional optimization techniques using gradients and working with problems having only one minimum cannot be applied in this case. These real-life applied problems are attacked here by employing one of the mostly abstract mathematical objects—space-filling curves. A practical derivative-free deterministic method reducing the dimensionality of the problem by using space-filling curves and working simultaneously with all possible estimates of Lipschitz and Hölder constants is proposed. A smart adaptive balancing of local and global information collected during the search is performed at each iteration. Conditions ensuring convergence of the new method to the global minima are established. Results of numerical experiments on 1000 randomly generated test functions show a clear superiority of the new method w.r.t. the popular method DIRECT and other competitors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Barkalov, K.A., Gergel, V.P.: Parallel global optimization on GPU. J. Glob. Optim. 66(1), 3–20 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  2. Butz, A.R.: Space filling curves and mathematical programming. Inf. Control 12(4), 313–330 (1968)

    Article  MathSciNet  MATH  Google Scholar 

  3. Calvin, J.M., Žilinskas, A.: One-dimensional p-algorithm with convergence rate \(o(n^{- 3+\delta })\) for smooth functions. J. Optim. Theory Appl. 106(2), 297–307 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  4. Evtushenko, Y.G., Posypkin, M.: A deterministic approach to global box-constrained optimization. Optim. Lett. 7(4), 819–829 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  5. Famularo, D., Pugliese, P., Sergeyev, Ya D.: A global optimization technique for checking parametric robustness. Automatica 35, 1605–1611 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  6. Finkel, D.E., Kelley, C.T.: Additive scaling and the DIRECT algorithm. J. Glob. Optim. 36(4), 597–608 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Gablonsky, M.J.: DIRECT v2.04 FORTRAN code with documentation. Technical report (2001). http://www4.ncsu.edu/ctk/SOFTWARE/DIRECTv204.tar.gz

  8. Gablonsky, M. J.: Modifications of the DIRECT algorithm. Technical report, Ph.D thesis, North Carolina State University, Raleigh, NC (2001)

  9. Gablonsky, M.J., Kelley, C.T.: A locally-biased form of the DIRECT algorithm. J. Glob. Optim. 21, 27–37 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. Gaviano, M., Kvasov, D.E., Lera, D., Sergeyev, Ya D.: Algorithm 829: Software for generation of classes of test functions with known local and global minima for global optimization. ACM Trans. Math. Softw. 29(4), 469–480 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  11. Gergel, V.P., Gergel, V.A.A.V.: Adaptive nested optimization scheme for multidimensional global search. J. Glob. Optim. 66(1), 35–51 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Gergel, V.P., Grishagin, V.A., Israfilov, R.A.: Local tuning in nested scheme of global optimization. Proced. Comput. Sci. 51, 865–874 (2015). (International Conference on Computational Science ICCS 2015—Computational Science at the Gates of Nature)

    Article  Google Scholar 

  13. Gillard, J.W., Kvasov, D.E.: Lipschitz optimization methods for fitting a sum of damped sinusoids to a series of observations. Stat. Interface 10(1), 59–70 (2016)

    Article  MathSciNet  Google Scholar 

  14. Gourdin, E., Jaumard, B., Ellaia, R.: Global optimization of Hölder functions. J. Glob. Optim. 8, 323–348 (1996)

    Article  MATH  Google Scholar 

  15. Grishagin, V.A., Israfilov, R.A.: Global search acceleration in the nested optimization scheme. AIP Conf. Proc. 1738, 400010 (2016)

    Article  Google Scholar 

  16. Grishagin, V.A., Israfilov, R.A., Sergeyev, Ya D.: Convergence conditions and numerical comparison of global optimization methods based on dimensionality reduction schemes. Appl. Math. Comput. 318, 270–280 (2018)

    MathSciNet  Google Scholar 

  17. Horst, R., Pardalos, P.M. (eds.): Handbook of Global Optimization, vol. 1. Kluwer Academic Publishers, Dordrecht (1995)

    MATH  Google Scholar 

  18. Horst, R., Tuy, H.: Global Optimization—Deterministic Approaches. Springer-Verlag, Berlin (1996)

    MATH  Google Scholar 

  19. Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79, 157–181 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  20. Kvasov, D.E., Pizzuti, C., Sergeyev, Ya D.: Local tuning and partition strategies for diagonal GO methods. Numer. Math. 94(1), 93–106 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Kvasov, D.E., Sergeyev, Ya D.: Lipschitz global optimization methods in control problems. Autom. Remote Control 74(9), 1435–1448 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kvasov, D.E., Sergeyev, Ya D.: Deterministic approaches for solving practical black-box global optimization problems. Adv. Eng. Softw. 80, 58–66 (2015)

    Article  Google Scholar 

  23. Kvasov, D.E., Sergeyev, Ya D.: A univariate global search working with a set of Lipschitz constants for the first derivative. Optim. Lett. 3(2), 303–318 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  24. Lera, D., Sergeyev, Ya D.: Global minimization algorithms for Hölder functions. BIT 42(1), 119–133 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  25. Lera, D., Sergeyev, Ya D.: Acceleration of univariate global optimization algorithms working with Lipschitz functions and Lipschitz first derivatives. SIAM J. Optim. 23(1), 508–529 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lera, D., Sergeyev, Ya D.: Deterministic global optimization using space-filling curves and multiple estimates of Lipschitz and Hölder constants. Commun. Nonlinear Sci. Numer. Simul. 23, 328–342 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lera, D., Sergeyev, Ya D.: An information global minimization algorithm using the local improvement technique. J. Glob. Optim. 48(1), 99–112 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  28. Lera, D., Sergeyev, Ya D.: Lipschitz and Hölder global optimization using space-filling curves. Appl. Numer. Maths. 60, 115–129 (2010)

    Article  MATH  Google Scholar 

  29. Liuzzi, G., Lucidi, S., Piccialli, V.: A direct-based approach exploiting local minimizations for the solution for large-scale global optimization problem. Comput. Optim. Appl. 45(2), 353–375 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  30. Liuzzi, G., Lucidi, S., Piccialli, V.: A partition-based global optimization algorithm. J. Glob. Optim. 48(1), 113–128 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  31. Paulavičius, R., Chiter, L., Žilinskas, J.: Global optimization based on bisection of rectangles, function values at diagonals, and a set of Lipschitz constants. J. Glob. Optim. (2017). https://doi.org/10.1007/s10898-016-0485-6

  32. Paulavičius, R., Sergeyev, Ya D., Kvasov, D.E., Žilinskas, J.: Globally-biased DISIMPL algorithm for expensive global optimization. J. Glob. Optim. 59(2–3), 545–567 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  33. Paulavičius, R., Žilinskas, J.: Simplicial Global Optimization. SpringerBriefs in Optimization. Springer, New York (2014)

    Book  MATH  Google Scholar 

  34. Pintér, J.D.: Global Optimization in Action (Continuous and Lipschitz Optimization: Algorithms, Implementations and Applications). Kluwer Academic Publishers, Dordrecht (1996)

    Book  MATH  Google Scholar 

  35. Pintér, J.D.: Global optimization: software, test problems, and applications. In: Pardalos, P.M., Romeijn, H.E. (eds.) Handbook of Global Optimization, vol. 2, pp. 515–569. Kluwer Academic Publishers, Dordrecht (2002)

    Chapter  Google Scholar 

  36. Piyavskij, S.A.: An algorithm for finding the absolute extremum of a function. USSR Comput. Math. Math. Phys. 12(4), 57–67 (1972). (in Russian: Zh. Vychisl. Mat. Mat. Fiz., 12(4) (1972), pp. 888–896)

    Article  Google Scholar 

  37. Preparata, F.P., Shamos, M.I.: Computational Geometry: An Introduction. Springer-Verlag, New York (1993)

    MATH  Google Scholar 

  38. Sagan, H.: Space-Filling Curves. Springer, New York (1994)

    Book  MATH  Google Scholar 

  39. Sergeyev, Ya D.: An information global optimization algorithm with local tuning. SIAM J. Optim. 5(4), 858–870 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  40. Sergeyev, Ya D.: A one-dimensional deterministic global minimization algorithm. Comput. Math. Math. Phys. 35(5), 705–717 (1995)

    MathSciNet  Google Scholar 

  41. Sergeyev, Ya D., Daponte, P., Grimaldi, D., Molinaro, A.: Two methods for solving optimization problems arising in electronic measurements and electrical engineering. SIAM J. Optim. 10(1), 1–21 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  42. Sergeyev, Ya D., Grishagin, V.A.: Sequential and parallel algorithms for global optimization. Optim. Methods Softw. 3, 111–124 (1994)

    Article  MATH  Google Scholar 

  43. Sergeyev, Ya D., Kvasov, D.E.: Deterministic Global Optimization: An Introduction to the Diagonal Approach. Springer, New York (2017)

    Book  MATH  Google Scholar 

  44. Sergeyev, Ya D., Strongin, R.G., Lera, D.: Introduction to Global Optimization Exploiting Space-Filling Curves. SpringerBriefs in Optimization. Springer, New York (2013)

    Book  MATH  Google Scholar 

  45. Strongin, R.G.: Numerical Methods in Multiextremal Problems: Information-Statistical Algorithms. Nauka, Moscow (1978). (In Russian)

    MATH  Google Scholar 

  46. Strongin, R.G., Sergeyev, Ya D.: Global optimization: fractal approach and non-redundant parallelism. J. Glob. Optim. 27, 25–50 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  47. Strongin, R .G., Sergeyev, Ya D.: Global Optimization with Non-Convex Constraints: Sequential and Parallel Algorithms. Kluwer Academic Publishers, Dordrecht (2000). (2nd ed., 2012; 3rd ed., 2014, Springer, New York)

    Book  MATH  Google Scholar 

  48. Žilinskas, A.: On similarities between two models of global optimization: statistical models and radial basis functions. J. Glob. Optim. 48(1), 173–182 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  49. Žilinskas, A., Žilinskas, J.: Parallel hybrid algorithm for global optimization of problems occurring in MDS-based visualization. Comput. Math. Appl. 52(1–2), 211–224 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  50. Žilinskas, A., Žilinskas, J.: A hybrid global optimization algorithm for non-linear least squares regression. J. Glob. Optim. 56(2), 265–277 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  51. Zhigljavsky, A.A.: Theory of Global Random Search. Kluwer Academic Publishers, Dordrecht (1991)

    Book  Google Scholar 

  52. Zhigljavsky, A.A., Žilinskas, A.: Stochastic Global Optimization. Springer, New York (2008)

    MATH  Google Scholar 

Download references

Acknowledgements

The authors thank the unknown reviewers for their very useful comments that have allowed the authors to improve the manuscript. The research of Ya. D. Sergeyev was supported by the Russian Science Foundation, project No 15-11-30022 “Global optimization, supercomputing computations, and applications”.

Author information

Authors and Affiliations

  1. Dipartimento di Matematica e Informatica, Università di Cagliari, Cagliari, Italy

    Daniela Lera

  2. Dipartimento di Ingegneria Informatica, Modellistica, Elettronica e Sistemistica, Università della Calabria and the Institute of High Performance Computing and Networking of the National Research Council of Italy, Via Pietro Bucci 42C, 87036, Rende, CS, Italy

    Yaroslav D. Sergeyev

  3. Department of Software and Supercomputing, Lobachevskiy University of Nizhni Novgorod, Gagarin Av. 23, Nizhni Novgorod, Russia

    Yaroslav D. Sergeyev

Authors
  1. Daniela Lera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaroslav D. Sergeyev
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yaroslav D. Sergeyev.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lera, D., Sergeyev, Y.D. GOSH: derivative-free global optimization using multi-dimensional space-filling curves. J Glob Optim 71, 193–211 (2018). https://doi.org/10.1007/s10898-017-0589-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-017-0589-7

Keywords

Search

Navigation