Journal of Global Optimization

, Volume 46, Issue 2, pp 273–286 | Cite as

Stopping and restarting strategy for stochastic sequential search in global optimization

  • Zelda B. Zabinsky
  • David Bulger
  • Charoenchai Khompatraporn


Two common questions when one uses a stochastic global optimization algorithm, e.g., simulated annealing, are when to stop a single run of the algorithm, and whether to restart with a new run or terminate the entire algorithm. In this paper, we develop a stopping and restarting strategy that considers tradeoffs between the computational effort and the probability of obtaining the global optimum. The analysis is based on a stochastic process called Hesitant Adaptive Search with Power-Law Improvement Distribution (HASPLID). HASPLID models the behavior of stochastic optimization algorithms, and motivates an implementable framework, Dynamic Multistart Sequential Search (DMSS). We demonstrate here the practicality of DMSS by using it to govern the application of a simple local search heuristic on three test problems from the global optimization literature.


Stopping criteria Sequential search Pure adaptive search 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abramowitz M., Stegun I.: Handbook of Mathematical Functions with Formulas, Graphs and Mathematical Tables. Dover, New York (1964)Google Scholar
  2. 2.
    Ali M.M., Khompatraporn C., Zabinsky Z.B.: A numerical evaluation of several global optimization algorithms on selected benchmark test problems. J. Glob. Optim. 31, 635–672 (2005)CrossRefGoogle Scholar
  3. 3.
    Atkinson A.: A segmented algorithm for simulated annealing. Stat. Comput. 2, 221–230 (1992)CrossRefGoogle Scholar
  4. 4.
    Betrò B., Schoen F.: Sequential stopping rules for the multistart algorithm in global optimisation. Math. Program. 38, 271–286 (1987)CrossRefGoogle Scholar
  5. 5.
    Boender C., Rinnooy Kan A.: Bayesian stopping rules for multistart global optimization methods. Math. Program. 37, 59–80 (1987)CrossRefGoogle Scholar
  6. 6.
    Boender C.G.E., Romeijn H.E.: Stochastic methods. In: Horst, R., Pardalos, P. (eds) Handbook of Global Optimization, pp. 829–869. Kluwer, The Netherlands (1995)Google Scholar
  7. 7.
    Bulger D.W., Wood G.R.: Hesitant adaptive search for global optimisation. Math. Program. 81, 89–102 (1998)Google Scholar
  8. 8.
    Dekkers A., Aarts E.: Global optimization and simulated annealing. Math. Program. 50, 367–393 (1991)CrossRefGoogle Scholar
  9. 9.
    Dür M., Khompatraporn C., Zabinsky Z.B.: Solving fractional problems with dynamic multistart improving Hit-and-Run. Ann. Oper. Res. 156, 25–44 (2007)CrossRefGoogle Scholar
  10. 10.
    Glidewell, M., Ng, K., Hensel, E.: A combinatorial optimization approach as a pre-processor for impedance tomography. In: Proceedings of the Annual Conference of the IEEE/Engineering in Medicine and Biology Society, pp. 1–2 (1991)Google Scholar
  11. 11.
    Hajek B.: Cooling schedules for optimal annealing. Math. Oper. Res. 18, 311–329 (1988)CrossRefGoogle Scholar
  12. 12.
    Khompatraporn, C.: Analysis and Development of Stopping Criteria for Stochastic Global Optimization Algorithms. Ph.D. Dissertation, University of Washington, Washington (2004)Google Scholar
  13. 13.
    Li, H., Lim, A.: A Metaheuristic for the Pickup and Delivery Problem with Time Windows. In: Proceedings of the 13th IEEE International Conference on Tools with Artificial Intelligence, pp. 160–167 (2001)Google Scholar
  14. 14.
    Locatelli M.: Convergence of a simulated annealing algorithm for continuous global optimization. J. Glob. Optim. 18, 219–234 (2000)CrossRefGoogle Scholar
  15. 15.
    Lundy M., Mees A.: Convergence of an annealing algorithm. Math. Program. 34, 111–124 (1986)CrossRefGoogle Scholar
  16. 16.
    Muselli M.: A theoretical approach to restart in global optimization. J. Glob. Optim. 10, 1–16 (1997)CrossRefGoogle Scholar
  17. 17.
    Romeijn H.E., Smith R.L.: Simulated annealing and adaptive search in global optimization. Probab. Eng. Inf. Sci. 8, 571–590 (1994)CrossRefGoogle Scholar
  18. 18.
    Treadgold N., Gedeon T.: Simulated annealing and weight decay in adaptive learning: the SARPROP algorithm. IEEE Trans. Neural Netw. 9, 662–668 (1998)CrossRefGoogle Scholar
  19. 19.
    Theodosopoulos T.V.: Some remarks on the optimal level of randomization in global optimization. In: Pardalos, P., Rajasekaran, S., Rolim, J. (eds) Randomization Methods in Algorithm Design, DIMACS Series 43., pp. 303–318. American Mathematical Society, RI (1999)Google Scholar
  20. 20.
    Wales D.J., Doye J.P.K.: Global optimization by basin-hopping and the lowest energy structures of Lennard–Jones clusters containing up to 110 atoms. J. Phys. Chem. A 101, 5111–5116 (1997)CrossRefGoogle Scholar
  21. 21.
    Wood G.R., Zabinsky Z.B., Kristinsdóttir B.P.: Hesitant adaptive search: the distribution of the number of iterations to convergence. Math. Program. A 89, 479–486 (2001)CrossRefGoogle Scholar
  22. 22.
    Zabinsky Z.B.: Stochastic Adaptive Search for Global Optimization. Kluwer, The Netherlands (2003)Google Scholar
  23. 23.
    Zabinsky Z.B., Smith R.L.: Pure adaptive search in global optimization. Math. Program. 53, 323–338 (1992)CrossRefGoogle Scholar
  24. 24.
    Zabinsky Z.B., Smith R.L., McDonald J.F., Romeijn H.E., Kaufman D.E.: Improving Hit-and-Run for global optimization. J. Glob. Optim. 3, 171–192 (1993)CrossRefGoogle Scholar
  25. 25.
    Zielinski R.: A statistical estimate of the structure of multiextremal problems. Math. Program. 21, 348–356 (1981)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2009

Authors and Affiliations

  • Zelda B. Zabinsky
    • 1
  • David Bulger
    • 2
  • Charoenchai Khompatraporn
    • 3
  1. 1.Industrial EngineeringUniversity of WashingtonSeattleUSA
  2. 2.Department of StatisticsMacquarie UniversitySydneyAustralia
  3. 3.Department of Production EngineeringKing Mongkut’s University of Technology ThonburiBangkokThailand

Personalised recommendations