Abstract
The global optimization problem, finding the lowest minimizer of a nonlinear function of several variables that has multiple local minimizers, appears well suited to concurrent computation. This paper presents a new parallel algorithm for the global optimization problem. The algorithm is a stochastic method related to the multi-level single-linkage methods of Rinnooy Kan and Timmer for sequential computers. Concurrency is achieved by partitioning the work of each of the three main parts of the algorithm, sampling, local minimization start point selection, and multiple local minimizations, among the processors. This parallelism is of a coarse grain type and is especially well suited to a local memory multiprocessing environment. The paper presents test results of a distributed implementation of this algorithm on a local area network of computer workstations. It also summarizes the theoretical properties of the algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
R.S. Anderssen, “Global optimization,” in: R.S. Anderssen, L.S. Jennings and D.M. Ryan, eds.Optimization (University of Queensland Press, 1972).
C.G.E. Boender, “The generalized multinormal distribution: a Bayesian analysis and applications,” Ph.D. thesis, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1984).
C.G.E. Boender and A.H.G. Rinnooy Kan, “Bayesian stopping rules for a class of stochastic global optimization methods,” Technical Report, Erasmus University (Rotterdam, The Netherlands, 1983).
C.G.E. Boender, A.H.G. Rinnooy Kan, L. Stougie and G.T. Timmer, “A stochastic method for global optimization,”Mathematical Programming 22 (1982) 125–140.
F.H. Branin, “Widely convergent methods for finding multiple solutions of simultaneous nonlinear equations,”IBM Journal of Research Developments (1972) 504–522.
F.H. Branin and S.K. Hoo, “A method for finding multiple extreme of a function ofn variables,” in: F.A. Lootsma, ed.,Numerical Methods of Nonlinear Optimization (Academic Press, London, 1972).
S.H. Brooks, “A discussion of random methods for seeking maxima,”Operations Research 6 (1958) 244–251.
J.E. Dennis Jr. and R.B. Schnabel,Numerical Methods for Nonlinear Equations and Unconstrained Optimization (Prentice-Hall, Englewood Cliffs, New Jersey, 1983).
C.L. Dert, “A parallel algorithm for global optimization,” Masters thesis, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1986).
L. Devroye, “A bibliography on random search,” Technical Report, McGill University (Montreal, 1979).
L.C.W. Dixon and G.P. Szego, eds.,Towards Global Optimization 2 (North-Holland, Amsterdam, 1978).
E. Eskow and R.B. Schnabel, “Using mathematical modeling to aid in parallel algorithm development,”Proceedings of Third SIAM Conference on Parallel Processing for Scientific Computation (Los Angeles, 1987).
J.B.E. Frenk and A.H.G. Rinnooy Kan, “The asymptotic optimality of the LPT rule,”Mathematics of Operations Research (to appear).
T.J. Gardner, I.M. Gerard, C.R. Mowers, E. Nemeth and R.B. Schnabel, “DPUP: A distributed processing utilities package,” Technical Report CU-CS-337-86, University of Colorado, Department of Computer Science (Boulder, Colorado, 1986).
P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, London, 1981).
A.A. Goldstein and J.F. Price, “On descent from local minima,”Mathematics of Computation 25 (1971) 569–574.
E.R. Hansen, “Global optimization using interval analysis—The multidimensional case,”Numerical Math 34 (1980) 247–270.
E.R. Hansen and S. Sengupta, “Global constrained optimization using interval analysis,” in: K. Nickel, ed.,Interval Mathematics (Academic Press, London, 1980).
A.V. Levy and S. Gomez, “The tunneling method applied to global optimization,” in: P.T. Boggs, R.H. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (SIAM, Philadelphia, 1985) 213–244.
A.V. Levy and A. Montalvo, “The tunneling algorithm for the global minimization of functions,”SIAM Journal on Scientific and Statistical Computing 6 (1985) 15–29.
J.T. Postmus, A.H.G. Rinnooy Kan and G.T. Timmer, “An efficient dynamic selection method,”Communications of the ACM 26 (1983) 878–881.
W.L. Price, “A controlled random search procedure for global optimization,” in: L.C.W. Dixon and G.P. Szego, eds.,Towards Global Optimization 2 (North-Holland, Amsterdam, 1978) 71–84.
A.H.G. Rinnooy Kan and G.T. Timmer, “Stochastic methods for global optimization,”American Journal of Mathematical and Management Sciences 4 (1984) 7–40.
A.H.G. Rinnooy Kan and G.T. Timmer, “A stochastic approach to global optimization,” in: P. Boggs, R. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (SIAM, Philadelphia, 1985a) 245–262.
A.H.G. Rinnooy Kan and G.T. Timmer, “Stochastic global optimization methods—Part I: Clustering methods,” Report 85391 A, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1985b).
A.H.G. Rinnooy Kan and G.T. Timmer, “Stochastic global optimization methods—Part II: Multi-level methods,” Report 85401 A, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1985c).
C.L. Seitz, “The cosmic cube,”Communications of the ACM 28 (1985) 22–33.
B.O. Shubert, “A sequential method seeking the global maximum of function,”SIAM Journal on Numerical Analysis 9 (1972) 379–388.
F.J. Solis and R.J.E. Wets, “Minimization by random search techniques,”Mathematics of Operations Research 6 (1981) 19–30.
G.T. Timmer, “Global optimization: A Bayesian approach,” Ph.D. thesis, Econometric Institute, Erasmus University (Rotterdam, The Netherlands, 1984).
G.W. Walster, E.R. Hansen and S. Sengupta, “Test results for a global optimization algorithm,” in: P. Boggs, R.H. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (SIAM, Philadelphia, 1985) 272–287.
R. Zielinski, “A stochastic estimate of the structure of multi-external problems,”Mathematical Programming 22 (1981) 104–116.
Author information
Authors and Affiliations
Additional information
Research supported by AFOSR grant AFOSR-85-0251, ARO contract DAAG 29-84-K-0140, NSF grant DCR-8403483, and NSF cooperative agreement DCR-8420944.
Rights and permissions
About this article
Cite this article
Byrd, R.H., Dert, C.L., Rinnooy Kan, A.H.G. et al. Concurrent stochastic methods for global optimization. Mathematical Programming 46, 1–29 (1990). https://doi.org/10.1007/BF01585724
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01585724