Journal of Global Optimization

, Volume 1, Issue 3, pp 207–228 | Cite as

Stochastic techniques for global optimization: A survey of recent advances

  • Fabio Schoen


In this paper stochastic algorithms for global optimization are reviewed. After a brief introduction on random-search techniques, a more detailed analysis is carried out on the application of simulated annealing to continuous global optimization. The aim of such an analysis is mainly that of presenting recent papers on the subject, which have received only scarce attention in the most recent published surveys. Finally a very brief presentation of clustering techniques is given.

Key words

Stochastic algorithms simulated annealing clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aarts, E. and Korst, J. (1989), Simulated Annealing and Boltzmann Machines, J. Wiley & Sons.Google Scholar
  2. 2.
    Aluffi-Pentini, F., Parisi, V., and Zirilli, F. (1985), Global Optimization and Stochastic Differential Equations, J. O. T. A. 47, 1–16.Google Scholar
  3. 3.
    Archetti, F. and Schoen, F. (1984), A Survey on the Global Optimization Problem: General Theory and Computational Approaches, Annals of Operations Research 1, 87–110.Google Scholar
  4. 4.
    Becker, R. W. and Lago, G. W. (1970), A Global Optimization Algorithm, in Proceedings of the 8th Allerton Conference on Circuits and Systems Theory, Monticello (Illinois), 3–12.Google Scholar
  5. 5.
    Betrò, B. (1990), Bayes Methods in Global Optimization, in B. Betrò, M. Gugiani and F. Schoen, Monte Carlo Methods in Numerical Integration and Optimization, Monografie di Matematica Applicata, CNR.Google Scholar
  6. 6.
    Betrò, B. and Schoen, F. (1987), Sequential Stopping Rules for the Multistart Algorithm in Global Optimisation, Mathematical Programming 38, 271–286.Google Scholar
  7. 7.
    Betrò, B. and Schoen, F. (1988), Optimal and Suboptimal Stopping Rules for the Multistart Method in Global Optimization, report CNR-IAMI 88.11, Milano.Google Scholar
  8. 8.
    Betrò, B. and Schoen, F. (1990), A stochastic Technique for Global Optimization, to appear in Int. J. of Computers and Mathematics with Applications.Google Scholar
  9. 9.
    Boender, C. G. E. and Rinnooy Kan, A. H. G. (1987), Bayesian Stopping Rules for Multistart Optimization Methods, Mathematical Programming 37, 59–80.Google Scholar
  10. 10.
    Boender, C. G. E., Rinnooy Kan, A. H. G., Stougie, L., and Timmer, G. T. (1982), A Stochastic Method for Global Optimization, Mathematical Programming 22, 125–140.Google Scholar
  11. 11.
    Bohachevsky, I. O., Johnson, M. E., and Stein, M. L. (1986), Generalized Simulated Annealing for Function Optimization, Technometrics 28, 209–217.Google Scholar
  12. 12.
    Brooks, D. G. and Verdini, W. A. (1988), Computational Experience with Generalized Simulated Annealing over Continuous Variables, American J. of Mathematical and Management Sciences 8, 425–449.Google Scholar
  13. 13.
    Cerny, V. (1985), Thermodynamical Approach to the Traveling Salesman Problem: An Efficient Simulation Algorithm, J. O. T. A. 45, 41–51.Google Scholar
  14. 14.
    Chiang, T.-S., Hwang, C.-R., and Sheu, S.-J. (1987), Diffusion for Global Optimization in R n, SIAM J. Control and Optimization in 25, 737–753.Google Scholar
  15. 15.
    Collins, N. E., Eglese, R. W., and Golden, B. L. (1988), Simulated Annealing — An Annotated Bibliography, American J. of Mathematical and Management Sciences 8, 209–308.Google Scholar
  16. 16.
    Corana, A., Marchesi, M., Martini, C., and Ridella, S. (1987), Minimizing Multimodal Functions of Continuous Variables with the ‘Simulated Annealing’ Algorithm, ACM Trans. Math. Software 13, 2–280.Google Scholar
  17. 17.
    DeBiase, L. and Frontini, F. (1978), A Stochastic Method for Global Optimization: Its Structure and Numerical Performance, in Towards Global Optimization 2, L. C. W.Dixon and G. P.Szegö (eds.), North-Holland, Amsterdam, 85–102.Google Scholar
  18. 18.
    Dixon, L. C. W. and Szegö, G. P. (eds.) (1978), Towards Global Optimization 2, North-Holland, Amsterdam.Google Scholar
  19. 19.
    Gelfand, S. B. and Mitter, S. K. (1989), Simulated Annealing Type Algorithms for Multivariate Optimization, Technical Report LIDS-P-1845, M.I.T., Cambridge (MA).Google Scholar
  20. 20.
    Geman, S. and Hwang, C.-R. (1986), Diffusions for Global Optimization, SIAM J. Control and Optimization 24, 1031–1043.Google Scholar
  21. 21.
    Gidas, B. (1985), Global Optimization via the Langevin Equation, Proceedings of the 24th IEEE Conference on Decision and Control, Ft. Lauderdale FL.Google Scholar
  22. 22.
    Goldberg, D. E. (1989), Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading (MA).Google Scholar
  23. 23.
    Huang, M. D., Romeo, F., and Sangiovanni-Vincentelli, A. (1986), An Efficient General Cooling Schedule for Simulated Annealing, IEEE Int. Conf. Computer-Aided Design, Santa Clara CA, 381–384.Google Scholar
  24. 24.
    Kirkpatrick, S., Gelatt Jr., C. D., and Vecchi, M. P. (1982), Optimization by Simulated Annealing, IBM Research Report RC 9355.Google Scholar
  25. 25.
    Kushner, H. J. (1987), Asymptotic Global Behavior for Stochastic Approximation and Diffusions with Slowly Decreasing Noise Effects: Global Minimization via Monte Carlo, SIAM J. Appl. Math. 47, 169–185.Google Scholar
  26. 26.
    vanLaarhoven, P. J. M. and Aarts, E. H. L. (1987), Simulated Annealing: Theory and Practice, Reidel, Dordrecht.Google Scholar
  27. 27.
    Levy, A. V. and Montalvo, A. (1985), The Tunneling Algorithm for the Global Minimization of Functions, SIAM J. Sci. Stat. Comput., 6, 15–29.Google Scholar
  28. 28.
    Lucidi, S. and Piccioni, M. (1989), Random Tunneling by Means of Acceptance-Rejection Sampling for Global Optimization, J. O. T. A. 62, 255–277.Google Scholar
  29. 29.
    Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., and Teller, M. (1953), Equation of State Calculations by Fast Computing Machines, Journal of Chemical Physics 21, 1087–1092.Google Scholar
  30. 30.
    Piccioni, M. (1987), A Combined Multistart-Annealing Algorithm for Continuous Global Optimization, Technical Research Report 87–45, Systems and Research Center, The University of Maryland, College Park MD.Google Scholar
  31. 31.
    Rinnooy Kan, A. H. G. and Timmer, G. T. (1987), Stochastic Global Optimization Methods. Part I: Clustering Methods, Mathematical Programming 39, 27–56.Google Scholar
  32. 32.
    Rinnooy Kan, A. H. G. and Timmer, G. T. (1987), Stochastic Global Optimization Methods. Part II: Multi Level Methods, Mathematical Programming 39, 57–78.Google Scholar
  33. 33.
    Rinnooy Kan, A. H. G. and Timmer, G. T. (1986), Global Optimization, report 8612/A, Erasmus University, Rotterdam.Google Scholar
  34. 34.
    Rotondi, R. (1987), A New Method for Global Optimization Based on the k-th Nearest Neighbor, CNR-IAMI report, Milano.Google Scholar
  35. 35.
    Solis, F. J. and Wets, R. J.-B. (1981), Minimization by Random Search Techniques, Mathematics of Operations Research 6, 19–30.Google Scholar
  36. 36.
    Timmer, G. T. (1984), Global Optimization: A Stochastic Approach, Ph.D. thesis, Erasmus University, Rotterdam.Google Scholar
  37. 37.
    Törn, A. (1973), Global Optimization as a Combination of Global and Local Search, Gothenburg Business Adm. Studies 17, 191–206, 1973.Google Scholar
  38. 38.
    Törn, A. (1977), Cluster Analysis Using Seed Points and Density Determined Hyperspheres as an Aid to Global Optimization, IEEE Trans. Syst. Men and Cybernetics 7, 610–616.Google Scholar
  39. 39.
    Törn, A. A. and Zilinskas, A. (1989), Global Optimization, Springer-Verlag.Google Scholar
  40. 40.
    Tsitsiklis, J. N. (1986), A Survey of Large Time Asymptotics of Simulated Annealing Algorithms, Technical Report LIDS-P-1623, M.I.T., Cambridge (MA).Google Scholar
  41. 41.
    Vanderbilt, D. and Louie, S. G. (1984), A Monte Carlo Simulated Annealing Approach to Optimization over Continuous Variables, Journal of Computational Physics 56, 259–271.Google Scholar
  42. 42.
    Zieliński, Global Stochastic Approximation: A Review of Results and Some Open Problems, in Numerical Techniques for Stochastic Systems, North-Holland, 379–386, 1980.Google Scholar
  43. 43.
    Zieliński, R. (1981), A Statistical Estimate of the Structure of Multiextremal Problems, Mathematical Programming 21, 348–356.Google Scholar
  44. 44.
    Zieliński, G. S. A. vs. S. R. A., private communication, 1990.Google Scholar

Copyright information

© Kluwer Academic Publishers 1991

Authors and Affiliations

  • Fabio Schoen
    • 1
  1. 1.Dept. Information SciencesUniversity of MilanoMilanoItaly

Personalised recommendations