Mathematical Programming

, Volume 15, Issue 1, pp 330–342 | Cite as

Progressive global random search of continuous functions

  • Luc P. Devroye


A sequential random search method for the global minimization of a continuous function is proposed. The algorithm gradually concentrates the random search effort on areas neighboring the global minima. A modification is included for the case that the function cannot be exactly evaluated. The global convergence and the asymptotical optimality of the sequential sampling procedure are proved for both the stochastic and deterministic optimization problem.

Key words

Global Optimization Random Search Convergence Sequential Minimization Lipschitz Functions Stochastic Programming 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    R.S. Anderssen and P. Bloomfield, “Properties of the random search in global optimization”,Journal of Optimization Theory and Applications 16 (1975) 383–398.Google Scholar
  2. [2]
    G. Bennett, “Probability inequalities for the sums of independent random variables”,Journal of Proceedings of the Third IFAC Symposium, Ischia, Italy (1973) 298–306.Google Scholar
  3. [3]
    S.H. Brooks, “Discussion of random methods for locating surface maxima”,Operations Research 6 (1958) 244–251.Google Scholar
  4. [4]
    S.H. Brooks, “A comparison of maximum-seeking methods”,Operations Research 7 (1959) 430–457.Google Scholar
  5. [5]
    L.D. Cockrell and K.S. Fu, “On search techniques in adaptive systems”, Tech. Rept. TR-EE-70-1, Purdue University, Lafayette (1970).Google Scholar
  6. [6]
    L.P. Devroye, “On the convergence of statistical search”,IEEE Transactions on Systems, Man and Cybernetics 6 (1976) 46–56.Google Scholar
  7. [7]
    J.K. Hartman, “Some experiments in global optimization”,Naval Research Logistics Quarterly 20 (1973) 569–576.Google Scholar
  8. [8]
    J.D. Hill, “A search technique for multimodal surfaces”,IEEE Transactions on Systems, Science and Cybernetics 5 (1969) 2–8.Google Scholar
  9. [9]
    W. Hoeffding, “Probability inequalities for the sums of bounded random variables”,Journal of the American Statistical Association 58 (1963) 13–30.Google Scholar
  10. [10]
    R.A. Jarvis, “Optimization strategies in adaptive control: a selective survey”,IEEE Transactions on Systems, Man and Cybernetics 5 (1975) 83–94.Google Scholar
  11. [11]
    R.A. Jarvis, “Adaptive global search by the process of competitive evolution”,IEEE Transactions on Systems, Man and Cybernetics 5 (1975) 297–311.Google Scholar
  12. [12]
    R. Konakovsky and Z. Binder, “A multimodal searching technique using a learning controller”, Proceedings of the Third IFAC Symposium, Ischia, Italy (1973) 298–306.Google Scholar
  13. [13]
    M. Loeve,Probability (Van Nostrand, Princeton, NJ, 1963).Google Scholar
  14. [14]
    J. Matyas, “Random optimization”,Automation and Remote Control 26 (1965) 244–251.Google Scholar
  15. [15]
    G.J. McMurtry and K.S. Fu, “A variable structure automaton used as a multimodal searching technique”,IEEE Transactions on Automatic Control 11 (1966) 379–387.Google Scholar
  16. [16]
    G.J. McMurtry, “Adaptive optimization procedures”, in: J.M. Mendel and K.S. Fu, eds.,Adaptive, learning and pattern recognition systems (Academic Press, New York, 1970).Google Scholar
  17. [17]
    B.O. Shubert, “A sequential method seeking the global maximum of a function”,SIAM Journal on Numerical Analysis 9 (1972) 379–388.Google Scholar
  18. [18]
    L.P. Sysoev, “Statistical learning methods based on teacher identification”,Automation and Remote Control 31 (1970) 1733–1741.Google Scholar
  19. [19]
    L.P. Sysoev, “Training procedures which combine stochastic approximation and minimization of the empirical risk”,Automation and Remote Control 34 (1973) 398–411.Google Scholar
  20. [20]
    A. Torn, “Global optimization as a combination of global and local search”, Skriftserie Utgiven Av Handelshogskolan vid Abo Akademi, Abo, Finland (1974).Google Scholar
  21. [21]
    V.N. Vapnik and A.Ya. Chervonenkis, “Ordered risk minimization. I”,Automation and Remote Control 35 (1974) 1226–1235.Google Scholar
  22. [22]
    V.N. Vapnik and A.Ya. Chervonenkis, “Ordered risk minimization. II”,Automation and Remote Control 35 (1974) 1403–1412.Google Scholar
  23. [23]
    E.M. Vaysbord and D.B. Yudin, “Multiextremal stochastic approximation”,Engineering Cybernetics 5 (1968) 1–10.Google Scholar

Copyright information

© The Mathematical Programming Society 1978

Authors and Affiliations

  • Luc P. Devroye
    • 1
  1. 1.McGill UniversityMontrealCanada

Personalised recommendations