Mathematical Programming

, Volume 46, Issue 1–3, pp 1–29

Concurrent stochastic methods for global optimization

  • Richard H. Byrd
  • Cornelius L. Dert
  • Alexander H. G. Rinnooy Kan
  • Robert B. Schnabel
Article

Abstract

The global optimization problem, finding the lowest minimizer of a nonlinear function of several variables that has multiple local minimizers, appears well suited to concurrent computation. This paper presents a new parallel algorithm for the global optimization problem. The algorithm is a stochastic method related to the multi-level single-linkage methods of Rinnooy Kan and Timmer for sequential computers. Concurrency is achieved by partitioning the work of each of the three main parts of the algorithm, sampling, local minimization start point selection, and multiple local minimizations, among the processors. This parallelism is of a coarse grain type and is especially well suited to a local memory multiprocessing environment. The paper presents test results of a distributed implementation of this algorithm on a local area network of computer workstations. It also summarizes the theoretical properties of the algorithm.

Key words

Global optimization concurrent parallel stochastic network of computers 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. R.S. Anderssen, “Global optimization,” in: R.S. Anderssen, L.S. Jennings and D.M. Ryan, eds.Optimization (University of Queensland Press, 1972).Google Scholar
  2. C.G.E. Boender, “The generalized multinormal distribution: a Bayesian analysis and applications,” Ph.D. thesis, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1984).Google Scholar
  3. C.G.E. Boender and A.H.G. Rinnooy Kan, “Bayesian stopping rules for a class of stochastic global optimization methods,” Technical Report, Erasmus University (Rotterdam, The Netherlands, 1983).Google Scholar
  4. C.G.E. Boender, A.H.G. Rinnooy Kan, L. Stougie and G.T. Timmer, “A stochastic method for global optimization,”Mathematical Programming 22 (1982) 125–140.Google Scholar
  5. F.H. Branin, “Widely convergent methods for finding multiple solutions of simultaneous nonlinear equations,”IBM Journal of Research Developments (1972) 504–522.Google Scholar
  6. F.H. Branin and S.K. Hoo, “A method for finding multiple extreme of a function ofn variables,” in: F.A. Lootsma, ed.,Numerical Methods of Nonlinear Optimization (Academic Press, London, 1972).Google Scholar
  7. S.H. Brooks, “A discussion of random methods for seeking maxima,”Operations Research 6 (1958) 244–251.Google Scholar
  8. J.E. Dennis Jr. and R.B. Schnabel,Numerical Methods for Nonlinear Equations and Unconstrained Optimization (Prentice-Hall, Englewood Cliffs, New Jersey, 1983).Google Scholar
  9. C.L. Dert, “A parallel algorithm for global optimization,” Masters thesis, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1986).Google Scholar
  10. L. Devroye, “A bibliography on random search,” Technical Report, McGill University (Montreal, 1979).Google Scholar
  11. L.C.W. Dixon and G.P. Szego, eds.,Towards Global Optimization 2 (North-Holland, Amsterdam, 1978).Google Scholar
  12. E. Eskow and R.B. Schnabel, “Using mathematical modeling to aid in parallel algorithm development,”Proceedings of Third SIAM Conference on Parallel Processing for Scientific Computation (Los Angeles, 1987).Google Scholar
  13. J.B.E. Frenk and A.H.G. Rinnooy Kan, “The asymptotic optimality of the LPT rule,”Mathematics of Operations Research (to appear).Google Scholar
  14. T.J. Gardner, I.M. Gerard, C.R. Mowers, E. Nemeth and R.B. Schnabel, “DPUP: A distributed processing utilities package,” Technical Report CU-CS-337-86, University of Colorado, Department of Computer Science (Boulder, Colorado, 1986).Google Scholar
  15. P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, London, 1981).Google Scholar
  16. A.A. Goldstein and J.F. Price, “On descent from local minima,”Mathematics of Computation 25 (1971) 569–574.Google Scholar
  17. E.R. Hansen, “Global optimization using interval analysis—The multidimensional case,”Numerical Math 34 (1980) 247–270.Google Scholar
  18. E.R. Hansen and S. Sengupta, “Global constrained optimization using interval analysis,” in: K. Nickel, ed.,Interval Mathematics (Academic Press, London, 1980).Google Scholar
  19. A.V. Levy and S. Gomez, “The tunneling method applied to global optimization,” in: P.T. Boggs, R.H. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (SIAM, Philadelphia, 1985) 213–244.Google Scholar
  20. A.V. Levy and A. Montalvo, “The tunneling algorithm for the global minimization of functions,”SIAM Journal on Scientific and Statistical Computing 6 (1985) 15–29.Google Scholar
  21. J.T. Postmus, A.H.G. Rinnooy Kan and G.T. Timmer, “An efficient dynamic selection method,”Communications of the ACM 26 (1983) 878–881.Google Scholar
  22. W.L. Price, “A controlled random search procedure for global optimization,” in: L.C.W. Dixon and G.P. Szego, eds.,Towards Global Optimization 2 (North-Holland, Amsterdam, 1978) 71–84.Google Scholar
  23. A.H.G. Rinnooy Kan and G.T. Timmer, “Stochastic methods for global optimization,”American Journal of Mathematical and Management Sciences 4 (1984) 7–40.Google Scholar
  24. A.H.G. Rinnooy Kan and G.T. Timmer, “A stochastic approach to global optimization,” in: P. Boggs, R. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (SIAM, Philadelphia, 1985a) 245–262.Google Scholar
  25. A.H.G. Rinnooy Kan and G.T. Timmer, “Stochastic global optimization methods—Part I: Clustering methods,” Report 85391 A, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1985b).Google Scholar
  26. A.H.G. Rinnooy Kan and G.T. Timmer, “Stochastic global optimization methods—Part II: Multi-level methods,” Report 85401 A, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1985c).Google Scholar
  27. C.L. Seitz, “The cosmic cube,”Communications of the ACM 28 (1985) 22–33.Google Scholar
  28. B.O. Shubert, “A sequential method seeking the global maximum of function,”SIAM Journal on Numerical Analysis 9 (1972) 379–388.Google Scholar
  29. F.J. Solis and R.J.E. Wets, “Minimization by random search techniques,”Mathematics of Operations Research 6 (1981) 19–30.Google Scholar
  30. G.T. Timmer, “Global optimization: A Bayesian approach,” Ph.D. thesis, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1984).Google Scholar
  31. G.W. Walster, E.R. Hansen and S. Sengupta, “Test results for a global optimization algorithm,” in: P. Boggs, R.H. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (SIAM, Philadelphia, 1985) 272–287.Google Scholar
  32. R. Zielinski, “A stochastic estimate of the structure of multi-external problems,”Mathematical Programming 22 (1981) 104–116.Google Scholar

Copyright information

© North-Holland 1990

Authors and Affiliations

  • Richard H. Byrd
    • 1
  • Cornelius L. Dert
    • 1
    • 2
  • Alexander H. G. Rinnooy Kan
    • 2
  • Robert B. Schnabel
    • 1
  1. 1.Department of Computer ScienceUniversity of ColoradoBoulderUSA
  2. 2.Econometric InstituteErasmus University RotterdamThe Netherlands

Personalised recommendations