Skip to main content

Application of Bayesian approach to numerical methods of global and stochastic optimization

Abstract

In this paper a review of application of Bayesian approach to global and stochastic optimization of continuous multimodal functions is given. Advantages and disadvantages of Bayesian approach (average case analysis), comparing it with more usual minimax approach (worst case analysis) are discussed. New interactive version of software for global optimization is discussed. Practical multidimensional problems of global optimization are considered

This is a preview of subscription content, access via your institution.

References

  • Al-Khayyal, F.A. and Falk, J.E. (1983), Jointly Constrained Biconvex Programming,Mathematics of Operations Research 8, 273–286.

    Google Scholar 

  • Alufi-Pentini, F., Parisi, V. and Zirilli, F. (1985), Global Optimization and Stochastic Differential Equations,J. of Optimization Theory and Applications 47, 1–16.

    Google Scholar 

  • Archetti, F. and Betro, B. (1979), A Probabilistic Algorithm for Global Optimization,Calcolo 16, 335–343.

    Google Scholar 

  • Baskis, A. and Mockus, L. (1988), Application of Global Optimization Software for the Optimization of Differential Amplifier,Theory of Optimal Decisions, 9–16, Vilnius, Lithuania (in Russian).

    Google Scholar 

  • Belykh, L.N. (1983), On the Computational Methods in Disease Models,Mathematical Modeling in Immunology and Medicine, ed. G.I. Marchuk and L.N. Belykh, North-Holland Publishing Company, New York, 79–84.

    Google Scholar 

  • Benson, H.P. (1982), Algorithms for Parametric Nonconvex Programming,J. of Optimization Theory and Applications 38, 316–340.

    Google Scholar 

  • Boender, G. and Rinnoy Kan, A. (1987), Bayesian Stopping Rules for Multi-Start Global Optimization Methods,Mathematical Programming 37, 59–80.

    Google Scholar 

  • Craven, P. and Wahba, G. (1979), Smoothing Noisy Data with Spline Functions,Numerische Mathematik 31, 377–403.

    Google Scholar 

  • De Groot, M. (1970),Optimal Statistical Decisions, McGraw-Hill, New York.

    Google Scholar 

  • Dixon, L.C.W. and Szego, G.P. (1978),Towards Global Optimization, North Holland, Amsterdam.

    Google Scholar 

  • Donnelly, R.A. and Rogers, J.W. (1988), A Discrete Search Technique for Global Optimization,International Journal of Quantum Chemistry: Quantum Chemistry Symposium 22, 507–513.

    Google Scholar 

  • Ermakov, S.M. and Zigliavski, A.A. (1983), On Random Search of Global Extremum,Probability Theory and Applications 83, 129–136 (in Russian).

    Google Scholar 

  • Evtushenko, Yu. G. (1985),Numerical Optimization Techniques, Optimization Software, Inc., New York.

    Google Scholar 

  • Floudas, C.A. and Pardalos, P.M. (1987),A Collection of Test Problems for Constrained Global Optimization Algorithms, Lecture Notes in Computer Science455, Springer-Verlag.

  • Friedman, J.H., Jacobson, M., and Stuetzle, W. (1980), Projection Pursuit Regression, Technical Report # 146, March, Department of Statistics, Stanford University, 1–27.

  • Galperin, E. and Zheng, Q. (1987), Nonlinear Observation via Global Optimization Methods: Measure Theory Approach,J. of Optimization Theory and Applications 54, 63–92.

    Google Scholar 

  • Hansen, E. (1984), Global Optimization with Data Perturbation,Computational Operations Research 11, 97–104.

    Google Scholar 

  • Hong, Ch.S. and Zheng, Q. (1988),Integral Global Optimization Lecture Notes in Economics and Mathematical Systems298, Springer-Verlag.

  • Horst, R. and Tuy, H. (1990),Global Optimization, Springer-Verlag.

  • Kiefer, J. (1953), Sequential Minimax Search for a Maximum,Proceedings of American Mathematical Society 4, 502–506.

    Google Scholar 

  • Kushner, M.J. (1964), A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise,J. of Basic Engineering 86, 97–106.

    Google Scholar 

  • Levy, A.V, Montalvo, A., Gomez, S. and Calderon, A. (1982),Topics in Global Optimization, Lecture Notes in Mathematics # 909, 18–33.

  • Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H. and Teller, E. (1953), Equations of State Calculation by Fast Computing Machines,Journal of Chemical Physics 21, 1087–1092.

    Google Scholar 

  • Michalevich, V., Supel, A. and Norkin, V. (1978),Methods of Nonconvex Optimization, Nauka, Moscow (in Russian).

    Google Scholar 

  • Mockus, A. and Mockus, L. (1990), Design of Software for Global Optimization,Informatica 1, 71–88.

    Google Scholar 

  • Mockus, J. (1972), On Bayesian Methods of Extremum Search,Automatics and Computer Technics 72, 53–62 (in Russian).

    Google Scholar 

  • Mockus, J. (1989),Bayesian Approach to Global Optimization, Kluwer Academic Publishers, Dordrecht-London-Boston.

    Google Scholar 

  • Mockus, J. and Mockus, L. (1991), Bayesian Approach to Global Optimization and Applications to Multiobjective and Constrained Optimization,J. of Optimization Theory and Applications 70, 155–171.

    Google Scholar 

  • Pardalos, P.M. and Rosen, J.B. (1987),Constrained Global Optimization: Algorithms and Applications, Lecture Notes in Computer Science # 268, Berlin, Springer-Verlag.

    Google Scholar 

  • Pijavskij, S.A. (1972), An Algorithm for Finding the Absolute Extremum of Function,Computational Mathematics and Mathematical Physics, 57–67.

  • Powell, M.J.D. (1971), On the Convergence Rate of the Variable Metric Algorithm,J. Inst. of Mathematics and Applications 7, 21–36.

    Google Scholar 

  • Rastrigin, L.A. (1968),Statistical Methods of Search, Nauka, Moscow.

    Google Scholar 

  • Ratschek, H. and Rokne, J. (1988)New Computer Methods for Global Optimization, John Witey, New York.

    Google Scholar 

  • Saltenis, V. (1971), On One Method of Multiextremal Optimization,Automatics and Computer Technics 71, 33–38.

    Google Scholar 

  • Schittkowski, K. (1985/86), NLPQL: A FORTRAN Subroutine Solving Constrained Nonlinear Programming Problems,Annals of Operations Research 5, 485–500.

    Google Scholar 

  • Schnabel, R.B. (1987), Concurrent Function Evaluations in Local and Global Optimization,Computer Methods in Applied Mechanics and Engineering 64, 537–552.

    Google Scholar 

  • Shubert, B.O. (1972), A Sequential Method Seeking the Global Maximum of Function, 57AMJournal on Numerical Analysis 9, 379–388.

    Google Scholar 

  • Sobolj, I.M. (1967), On a Systematic Search in a Hypercube,SLAM Journal on Numerical Analysis 16, 790–793.

    Google Scholar 

  • Stoyan, Yu.G. and Sokolowskij, V.Z. (1980),Solution of some Multiextremal Problems by the Method of Narrowing Domains, Naukova Dumka, Kiev (in Russian).

    Google Scholar 

  • Strongin, R.G. (1978),Numerical Methods of Multiextremal Optimization, Nauka, Moscow.

    Google Scholar 

  • Subba Rao, T. and Gabr, M.M. (1984),An Introduction to Bispectral Analysis and Bilinear Time Series Models, Lecture Notes in Statistics # 24, Berlin, Springer-Verlag.

    Google Scholar 

  • Sukharev, A.G. (1975),Optimal Search of Extremum, Moscow University Press, Moscow.

    Google Scholar 

  • Torn, A. and Zilinskas, A. (1989),Global Optimization, Lecture Notes in Computer Science # 350, Berlin, Springer-Verlag.

    Google Scholar 

  • Zabinsky, Z.B., Smith, R.L. and McDonald, J.F. (1990),Improving Hit and Run for Global Optimization, Working paper, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI.

    Google Scholar 

  • Zigliavskij, A.A. (1985),Mathematical Theory of Global Random Search, Leningrad University Press, Leningrad (in Russian).

    Google Scholar 

  • Zilinskas, A. (1986),Global Optimization: Axiomatic of Statistical Models, Algorithms and their Applications, Mokslas, Vilnius (in Russian).

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mockus, J. Application of Bayesian approach to numerical methods of global and stochastic optimization. J Glob Optim 4, 347–365 (1994). https://doi.org/10.1007/BF01099263

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01099263

Key words

  • Optimization
  • global
  • Bayesian
  • continuous
  • stochastic