Application of Bayesian approach to numerical methods of global and stochastic optimization

Abstract

In this paper a review of application of Bayesian approach to global and stochastic optimization of continuous multimodal functions is given. Advantages and disadvantages of Bayesian approach (average case analysis), comparing it with more usual minimax approach (worst case analysis) are discussed. New interactive version of software for global optimization is discussed. Practical multidimensional problems of global optimization are considered

This is a preview of subscription content, access via your institution.

References

  1. Al-Khayyal, F.A. and Falk, J.E. (1983), Jointly Constrained Biconvex Programming,Mathematics of Operations Research 8, 273–286.

    Google Scholar 

  2. Alufi-Pentini, F., Parisi, V. and Zirilli, F. (1985), Global Optimization and Stochastic Differential Equations,J. of Optimization Theory and Applications 47, 1–16.

    Google Scholar 

  3. Archetti, F. and Betro, B. (1979), A Probabilistic Algorithm for Global Optimization,Calcolo 16, 335–343.

    Google Scholar 

  4. Baskis, A. and Mockus, L. (1988), Application of Global Optimization Software for the Optimization of Differential Amplifier,Theory of Optimal Decisions, 9–16, Vilnius, Lithuania (in Russian).

    Google Scholar 

  5. Belykh, L.N. (1983), On the Computational Methods in Disease Models,Mathematical Modeling in Immunology and Medicine, ed. G.I. Marchuk and L.N. Belykh, North-Holland Publishing Company, New York, 79–84.

    Google Scholar 

  6. Benson, H.P. (1982), Algorithms for Parametric Nonconvex Programming,J. of Optimization Theory and Applications 38, 316–340.

    Google Scholar 

  7. Boender, G. and Rinnoy Kan, A. (1987), Bayesian Stopping Rules for Multi-Start Global Optimization Methods,Mathematical Programming 37, 59–80.

    Google Scholar 

  8. Craven, P. and Wahba, G. (1979), Smoothing Noisy Data with Spline Functions,Numerische Mathematik 31, 377–403.

    Google Scholar 

  9. De Groot, M. (1970),Optimal Statistical Decisions, McGraw-Hill, New York.

    Google Scholar 

  10. Dixon, L.C.W. and Szego, G.P. (1978),Towards Global Optimization, North Holland, Amsterdam.

    Google Scholar 

  11. Donnelly, R.A. and Rogers, J.W. (1988), A Discrete Search Technique for Global Optimization,International Journal of Quantum Chemistry: Quantum Chemistry Symposium 22, 507–513.

    Google Scholar 

  12. Ermakov, S.M. and Zigliavski, A.A. (1983), On Random Search of Global Extremum,Probability Theory and Applications 83, 129–136 (in Russian).

    Google Scholar 

  13. Evtushenko, Yu. G. (1985),Numerical Optimization Techniques, Optimization Software, Inc., New York.

    Google Scholar 

  14. Floudas, C.A. and Pardalos, P.M. (1987),A Collection of Test Problems for Constrained Global Optimization Algorithms, Lecture Notes in Computer Science455, Springer-Verlag.

  15. Friedman, J.H., Jacobson, M., and Stuetzle, W. (1980), Projection Pursuit Regression, Technical Report # 146, March, Department of Statistics, Stanford University, 1–27.

  16. Galperin, E. and Zheng, Q. (1987), Nonlinear Observation via Global Optimization Methods: Measure Theory Approach,J. of Optimization Theory and Applications 54, 63–92.

    Google Scholar 

  17. Hansen, E. (1984), Global Optimization with Data Perturbation,Computational Operations Research 11, 97–104.

    Google Scholar 

  18. Hong, Ch.S. and Zheng, Q. (1988),Integral Global Optimization Lecture Notes in Economics and Mathematical Systems298, Springer-Verlag.

  19. Horst, R. and Tuy, H. (1990),Global Optimization, Springer-Verlag.

  20. Kiefer, J. (1953), Sequential Minimax Search for a Maximum,Proceedings of American Mathematical Society 4, 502–506.

    Google Scholar 

  21. Kushner, M.J. (1964), A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise,J. of Basic Engineering 86, 97–106.

    Google Scholar 

  22. Levy, A.V, Montalvo, A., Gomez, S. and Calderon, A. (1982),Topics in Global Optimization, Lecture Notes in Mathematics # 909, 18–33.

  23. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H. and Teller, E. (1953), Equations of State Calculation by Fast Computing Machines,Journal of Chemical Physics 21, 1087–1092.

    Google Scholar 

  24. Michalevich, V., Supel, A. and Norkin, V. (1978),Methods of Nonconvex Optimization, Nauka, Moscow (in Russian).

    Google Scholar 

  25. Mockus, A. and Mockus, L. (1990), Design of Software for Global Optimization,Informatica 1, 71–88.

    Google Scholar 

  26. Mockus, J. (1972), On Bayesian Methods of Extremum Search,Automatics and Computer Technics 72, 53–62 (in Russian).

    Google Scholar 

  27. Mockus, J. (1989),Bayesian Approach to Global Optimization, Kluwer Academic Publishers, Dordrecht-London-Boston.

    Google Scholar 

  28. Mockus, J. and Mockus, L. (1991), Bayesian Approach to Global Optimization and Applications to Multiobjective and Constrained Optimization,J. of Optimization Theory and Applications 70, 155–171.

    Google Scholar 

  29. Pardalos, P.M. and Rosen, J.B. (1987),Constrained Global Optimization: Algorithms and Applications, Lecture Notes in Computer Science # 268, Berlin, Springer-Verlag.

    Google Scholar 

  30. Pijavskij, S.A. (1972), An Algorithm for Finding the Absolute Extremum of Function,Computational Mathematics and Mathematical Physics, 57–67.

  31. Powell, M.J.D. (1971), On the Convergence Rate of the Variable Metric Algorithm,J. Inst. of Mathematics and Applications 7, 21–36.

    Google Scholar 

  32. Rastrigin, L.A. (1968),Statistical Methods of Search, Nauka, Moscow.

    Google Scholar 

  33. Ratschek, H. and Rokne, J. (1988)New Computer Methods for Global Optimization, John Witey, New York.

    Google Scholar 

  34. Saltenis, V. (1971), On One Method of Multiextremal Optimization,Automatics and Computer Technics 71, 33–38.

    Google Scholar 

  35. Schittkowski, K. (1985/86), NLPQL: A FORTRAN Subroutine Solving Constrained Nonlinear Programming Problems,Annals of Operations Research 5, 485–500.

    Google Scholar 

  36. Schnabel, R.B. (1987), Concurrent Function Evaluations in Local and Global Optimization,Computer Methods in Applied Mechanics and Engineering 64, 537–552.

    Google Scholar 

  37. Shubert, B.O. (1972), A Sequential Method Seeking the Global Maximum of Function, 57AMJournal on Numerical Analysis 9, 379–388.

    Google Scholar 

  38. Sobolj, I.M. (1967), On a Systematic Search in a Hypercube,SLAM Journal on Numerical Analysis 16, 790–793.

    Google Scholar 

  39. Stoyan, Yu.G. and Sokolowskij, V.Z. (1980),Solution of some Multiextremal Problems by the Method of Narrowing Domains, Naukova Dumka, Kiev (in Russian).

    Google Scholar 

  40. Strongin, R.G. (1978),Numerical Methods of Multiextremal Optimization, Nauka, Moscow.

    Google Scholar 

  41. Subba Rao, T. and Gabr, M.M. (1984),An Introduction to Bispectral Analysis and Bilinear Time Series Models, Lecture Notes in Statistics # 24, Berlin, Springer-Verlag.

    Google Scholar 

  42. Sukharev, A.G. (1975),Optimal Search of Extremum, Moscow University Press, Moscow.

    Google Scholar 

  43. Torn, A. and Zilinskas, A. (1989),Global Optimization, Lecture Notes in Computer Science # 350, Berlin, Springer-Verlag.

    Google Scholar 

  44. Zabinsky, Z.B., Smith, R.L. and McDonald, J.F. (1990),Improving Hit and Run for Global Optimization, Working paper, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI.

    Google Scholar 

  45. Zigliavskij, A.A. (1985),Mathematical Theory of Global Random Search, Leningrad University Press, Leningrad (in Russian).

    Google Scholar 

  46. Zilinskas, A. (1986),Global Optimization: Axiomatic of Statistical Models, Algorithms and their Applications, Mokslas, Vilnius (in Russian).

    Google Scholar 

Download references

Author information

Affiliations

Authors

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mockus, J. Application of Bayesian approach to numerical methods of global and stochastic optimization. J Glob Optim 4, 347–365 (1994). https://doi.org/10.1007/BF01099263

Download citation

Key words

  • Optimization
  • global
  • Bayesian
  • continuous
  • stochastic