Advertisement

Journal of Global Optimization

, Volume 67, Issue 4, pp 851–872 | Cite as

Improving the convergence rate of the DIRECT global optimization algorithm

  • Qunfeng Liu
  • Guang Yang
  • Zhongzhi Zhang
  • Jinping Zeng
Article

Abstract

DIRECT is derivative-free global-search algorithm has been found to perform robustly across a wide variety of low-dimensional test problems. The reason DIRECT’s robustness is its lack of algorithmic parameters that need be “tuned” to make the algorithm perform well. In particular, there is no parameter that determines the relative emphasis on global versus local search. Unfortunately, the same algorithmic features that enable DIRECT to perform so robustly have a negative side effect. In particular, DIRECT is usually quick to get close to the global minimum, but very slow to refine the solution to high accuracy. This is what we call DIRECT’s “eventually inefficient behavior.” In this paper, we outline two root causes for this undesirable behavior and propose modifications to eliminate it. The paper builds upon our previously published “MrDIRECT” algorithm, which we can now show only addressed the first root cause of the “eventually inefficient behavior.” The key contribution of the current paper is a further enhancement that allows MrDIRECT to address the second root cause as well. To demonstrate the effectiveness of the enhanced MrDIRECT, we have identified a set of test functions that highlight different situations in which DIRECT has convergence issues. Extensive numerical work with this test suite demonstrates that the enhanced version of MrDIRECT does indeed improve the convergence rate of DIRECT.

Keywords

Global optimization Multilevel algorithm DIRECT algorithm MrDIRECT algorithm Convergence rate 

Notes

Acknowledgments

We would like to thank two anonymous reviewers for their very helpful suggestions, which improve this paper greatly. We would like to thank Doctor Finkel D.E., Professor Kelley C.T. and Professor Sergeyev Ya. D. for their DIRECT code and the GKLS codes, respectively.

Supplementary material

10898_2016_447_MOESM1_ESM.pdf (195 kb)
Supplementary material 1 (pdf 194 KB)
10898_2016_447_MOESM2_ESM.pdf (24 kb)
Supplementary material 2 (pdf 23 KB)
10898_2016_447_MOESM3_ESM.pdf (32 kb)
Supplementary material 3 (pdf 32 KB)
10898_2016_447_MOESM4_ESM.pdf (75 kb)
Supplementary material 4 (pdf 74 KB)
10898_2016_447_MOESM5_ESM.pdf (284 kb)
Supplementary material 5 (pdf 283 KB)
10898_2016_447_MOESM6_ESM.pdf (808 kb)
Supplementary material 6 (pdf 808 KB)
10898_2016_447_MOESM7_ESM.pdf (59 kb)
Supplementary material 7 (pdf 58 KB)

References

  1. 1.
    Björkman, M., Holmström, K.: Global optimization using the DIRECT algorithm in Matlab. Adv. Model. Optim. 1, 17–37 (1999)zbMATHGoogle Scholar
  2. 2.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Elsakov, S.M., Shiryaev, V.I.: Homogeneous algorithms for multiextremal optimization. Comput. Math. Math. Phys. 50(10), 1642–1654 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Finkel D.E.: Global optimization with the DIRECT algorithm. PHD thesis, North Carolina State University (2005)Google Scholar
  5. 5.
    Finkel, D.E., Kelley, C.T.: Additive scaling and the DIRECT algorithm. J. Global Optim. 36, 597–608 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Global Optim. 45, 3–38 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Gaviano, M., Kvasov, D.E., Lera, D., Sergeyev, Y.D.: Algorithm 829: software for generation of classes of test functions with known local and global minima for global optimization. ACM Trans. Math. Softw. 9(4), 469–480 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Gaviano, M., Lera, D.: Test functions with variable attraction regions for global optimization problems. J. Global Optim. 13(2), 207–223 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Gablonsky, J.M., Kelley, C.T.: A locally-biased form of the DIRECT algorithm. J. Global Optim. 21, 27–37 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
  11. 11.
    He, J., Watson, L.T., et al.: Dynamic data structures for a direct search algorithm. Comput. Optim. Appl. 23(1), 5–25 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Holland, J.H.: Adaption in Nature and Artificial Systems, 2nd edn. MIT Press, Cambrige, MA (1992)Google Scholar
  13. 13.
    Holmstrom, K., Goran A.O., Edvall M.M.: User’s Guide for TOMLAB 7. Tomlab optimization. http://tomopt.com
  14. 14.
    Huyer, W., Neumaier, A.: Global optimization by multilevel coordinate search. J. Global Optim. 14(4), 331–355 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79(1), 157–181 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Jones, D.R.: The DIRECT global optimization algorithm. In: Floudas, C.A., Pardalos, P.M. (eds.) Encyclopedia of optimization. Kluwer Academic, Dordrecht (2001)Google Scholar
  17. 17.
    Kirkpatrick, S., Gelatt Jr., C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Lera, D., Sergeyev, Y.D.: Deterministic global optimization using space-filling curves and multiple estimates of Lipschitz and H\(\ddot{o}\)lder constants. Commun. Nonlinear Sci. Numer. Simul. 23(1–3), 328–342 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Liang J.J.: Novel particle swarm optimizers with hybrid, dynamic & adaptive neighborhood structures. PhD thesis, Nanyang Technological University, Singapore (2008)Google Scholar
  20. 20.
    Liang J.J., Qu B.Y., Suganthan P.N.: Problem definitions and evaluation criteria for the CEC 2013 special session and competition on real-parameter optimization. Technical Report 201212, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou China and Technical Report, Nanyang Technological University, Singapore, January (2013)Google Scholar
  21. 21.
    Liu, Q.F.: Linear scaling and the DIRECT algorithm. J. Global Optim. 56(3), 1233–1245 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Liu, Q.F., Cheng, W.Y.: A modified DIRECT algorithm with bilevel partition. J. Global Optim. 60(3), 483–499 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Liu, Q.F., Zeng, J.P.: Global optimization by multilevel partition. J. Global Optim. 61(1), 47–69 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Liu, Q.F., Zeng, J.P., Yang, G.: MrDIRECT: a multilevel robust DIRECT algorithm for global optimization problems. J. Global Optim. 62(2), 205–227 (2015)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Liuzzi, G., Lucidi, S., Piccialli, V.: A DIRECT-based approach exploiting local minimizations for the solution of large-scale global optimization problems. Comput. Optim. Appl. 45(2), 353–375 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Liuzzi, G., Lucidi, S., Piccialli, V.: A partion-based global optimization algorithm. J. Global Optim. 48, 113–128 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Liuzzi, G., Lucidi, S., Piccialli, V.: Exploiting derivative-free local searches in DIRECT-type algorithms for global optimization. Comput. Optim. Appl. (2015). doi: 10.1007/s10589-015-9741-9
  28. 28.
    Ljunberg, K., Holmgren, S.: Simultaneous search for multiple QTL using the global optimization algorithm DIRECT. Bioinformatics 20(12), 1887–1895 (2004)CrossRefGoogle Scholar
  29. 29.
    Moré, J., Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Pardalos, P.M., Schoen, F.: Recent advances and trends in global optimization: deterministic and stochastic methods. In: Proceedings of the Sixth International Conference on Foundations of Computer-Aided Process Design, DSI, vol. 1, pp. 119–131 (2004)Google Scholar
  31. 31.
    Paulavic̆ius, R., Sergeyev, Y.D., Kvasov, D.E., Z̆ilinskas, J.: Globally-biased Disimpl algorithm for expensive global optimization. J. Global Optim. 59, 545–567 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Global Optim. 56(3), 1247–1293 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Sergeyev, Y.D., Kvasov, D.E.: Global search based on efficient diagonal partitions and a set of Lipschitz constants. SIAM J. Optim. 16(3), 910–937 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Žilinskas, A.: On strong homogeneity of two global optimization algorithms based on statistical models of multimodal objective functions. Appl. Math. Comput. 218(16), 8131–8136 (2012)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Qunfeng Liu
    • 1
  • Guang Yang
    • 1
  • Zhongzhi Zhang
    • 1
  • Jinping Zeng
    • 1
  1. 1.College of Computer Science and TechnologyDongguan University of TechnologyDongguanChina

Personalised recommendations