Journal of Global Optimization

, Volume 4, Issue 4, pp 347–365 | Cite as

Application of Bayesian approach to numerical methods of global and stochastic optimization

  • Jonas Mockus


In this paper a review of application of Bayesian approach to global and stochastic optimization of continuous multimodal functions is given. Advantages and disadvantages of Bayesian approach (average case analysis), comparing it with more usual minimax approach (worst case analysis) are discussed. New interactive version of software for global optimization is discussed. Practical multidimensional problems of global optimization are considered

Key words

Optimization global Bayesian continuous stochastic 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Al-Khayyal, F.A. and Falk, J.E. (1983), Jointly Constrained Biconvex Programming,Mathematics of Operations Research 8, 273–286.Google Scholar
  2. Alufi-Pentini, F., Parisi, V. and Zirilli, F. (1985), Global Optimization and Stochastic Differential Equations,J. of Optimization Theory and Applications 47, 1–16.Google Scholar
  3. Archetti, F. and Betro, B. (1979), A Probabilistic Algorithm for Global Optimization,Calcolo 16, 335–343.Google Scholar
  4. Baskis, A. and Mockus, L. (1988), Application of Global Optimization Software for the Optimization of Differential Amplifier,Theory of Optimal Decisions, 9–16, Vilnius, Lithuania (in Russian).Google Scholar
  5. Belykh, L.N. (1983), On the Computational Methods in Disease Models,Mathematical Modeling in Immunology and Medicine, ed. G.I. Marchuk and L.N. Belykh, North-Holland Publishing Company, New York, 79–84.Google Scholar
  6. Benson, H.P. (1982), Algorithms for Parametric Nonconvex Programming,J. of Optimization Theory and Applications 38, 316–340.Google Scholar
  7. Boender, G. and Rinnoy Kan, A. (1987), Bayesian Stopping Rules for Multi-Start Global Optimization Methods,Mathematical Programming 37, 59–80.Google Scholar
  8. Craven, P. and Wahba, G. (1979), Smoothing Noisy Data with Spline Functions,Numerische Mathematik 31, 377–403.Google Scholar
  9. De Groot, M. (1970),Optimal Statistical Decisions, McGraw-Hill, New York.Google Scholar
  10. Dixon, L.C.W. and Szego, G.P. (1978),Towards Global Optimization, North Holland, Amsterdam.Google Scholar
  11. Donnelly, R.A. and Rogers, J.W. (1988), A Discrete Search Technique for Global Optimization,International Journal of Quantum Chemistry: Quantum Chemistry Symposium 22, 507–513.Google Scholar
  12. Ermakov, S.M. and Zigliavski, A.A. (1983), On Random Search of Global Extremum,Probability Theory and Applications 83, 129–136 (in Russian).Google Scholar
  13. Evtushenko, Yu. G. (1985),Numerical Optimization Techniques, Optimization Software, Inc., New York.Google Scholar
  14. Floudas, C.A. and Pardalos, P.M. (1987),A Collection of Test Problems for Constrained Global Optimization Algorithms, Lecture Notes in Computer Science455, Springer-Verlag.Google Scholar
  15. Friedman, J.H., Jacobson, M., and Stuetzle, W. (1980), Projection Pursuit Regression, Technical Report # 146, March, Department of Statistics, Stanford University, 1–27.Google Scholar
  16. Galperin, E. and Zheng, Q. (1987), Nonlinear Observation via Global Optimization Methods: Measure Theory Approach,J. of Optimization Theory and Applications 54, 63–92.Google Scholar
  17. Hansen, E. (1984), Global Optimization with Data Perturbation,Computational Operations Research 11, 97–104.Google Scholar
  18. Hong, Ch.S. and Zheng, Q. (1988),Integral Global Optimization Lecture Notes in Economics and Mathematical Systems298, Springer-Verlag.Google Scholar
  19. Horst, R. and Tuy, H. (1990),Global Optimization, Springer-Verlag.Google Scholar
  20. Kiefer, J. (1953), Sequential Minimax Search for a Maximum,Proceedings of American Mathematical Society 4, 502–506.Google Scholar
  21. Kushner, M.J. (1964), A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise,J. of Basic Engineering 86, 97–106.Google Scholar
  22. Levy, A.V, Montalvo, A., Gomez, S. and Calderon, A. (1982),Topics in Global Optimization, Lecture Notes in Mathematics # 909, 18–33.Google Scholar
  23. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H. and Teller, E. (1953), Equations of State Calculation by Fast Computing Machines,Journal of Chemical Physics 21, 1087–1092.Google Scholar
  24. Michalevich, V., Supel, A. and Norkin, V. (1978),Methods of Nonconvex Optimization, Nauka, Moscow (in Russian).Google Scholar
  25. Mockus, A. and Mockus, L. (1990), Design of Software for Global Optimization,Informatica 1, 71–88.Google Scholar
  26. Mockus, J. (1972), On Bayesian Methods of Extremum Search,Automatics and Computer Technics 72, 53–62 (in Russian).Google Scholar
  27. Mockus, J. (1989),Bayesian Approach to Global Optimization, Kluwer Academic Publishers, Dordrecht-London-Boston.Google Scholar
  28. Mockus, J. and Mockus, L. (1991), Bayesian Approach to Global Optimization and Applications to Multiobjective and Constrained Optimization,J. of Optimization Theory and Applications 70, 155–171.Google Scholar
  29. Pardalos, P.M. and Rosen, J.B. (1987),Constrained Global Optimization: Algorithms and Applications, Lecture Notes in Computer Science # 268, Berlin, Springer-Verlag.Google Scholar
  30. Pijavskij, S.A. (1972), An Algorithm for Finding the Absolute Extremum of Function,Computational Mathematics and Mathematical Physics, 57–67.Google Scholar
  31. Powell, M.J.D. (1971), On the Convergence Rate of the Variable Metric Algorithm,J. Inst. of Mathematics and Applications 7, 21–36.Google Scholar
  32. Rastrigin, L.A. (1968),Statistical Methods of Search, Nauka, Moscow.Google Scholar
  33. Ratschek, H. and Rokne, J. (1988)New Computer Methods for Global Optimization, John Witey, New York.Google Scholar
  34. Saltenis, V. (1971), On One Method of Multiextremal Optimization,Automatics and Computer Technics 71, 33–38.Google Scholar
  35. Schittkowski, K. (1985/86), NLPQL: A FORTRAN Subroutine Solving Constrained Nonlinear Programming Problems,Annals of Operations Research 5, 485–500.Google Scholar
  36. Schnabel, R.B. (1987), Concurrent Function Evaluations in Local and Global Optimization,Computer Methods in Applied Mechanics and Engineering 64, 537–552.Google Scholar
  37. Shubert, B.O. (1972), A Sequential Method Seeking the Global Maximum of Function, 57AMJournal on Numerical Analysis 9, 379–388.Google Scholar
  38. Sobolj, I.M. (1967), On a Systematic Search in a Hypercube,SLAM Journal on Numerical Analysis 16, 790–793.Google Scholar
  39. Stoyan, Yu.G. and Sokolowskij, V.Z. (1980),Solution of some Multiextremal Problems by the Method of Narrowing Domains, Naukova Dumka, Kiev (in Russian).Google Scholar
  40. Strongin, R.G. (1978),Numerical Methods of Multiextremal Optimization, Nauka, Moscow.Google Scholar
  41. Subba Rao, T. and Gabr, M.M. (1984),An Introduction to Bispectral Analysis and Bilinear Time Series Models, Lecture Notes in Statistics # 24, Berlin, Springer-Verlag.Google Scholar
  42. Sukharev, A.G. (1975),Optimal Search of Extremum, Moscow University Press, Moscow.Google Scholar
  43. Torn, A. and Zilinskas, A. (1989),Global Optimization, Lecture Notes in Computer Science # 350, Berlin, Springer-Verlag.Google Scholar
  44. Zabinsky, Z.B., Smith, R.L. and McDonald, J.F. (1990),Improving Hit and Run for Global Optimization, Working paper, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI.Google Scholar
  45. Zigliavskij, A.A. (1985),Mathematical Theory of Global Random Search, Leningrad University Press, Leningrad (in Russian).Google Scholar
  46. Zilinskas, A. (1986),Global Optimization: Axiomatic of Statistical Models, Algorithms and their Applications, Mokslas, Vilnius (in Russian).Google Scholar

Copyright information

© Kluwer Academic Publishers 1994

Authors and Affiliations

  • Jonas Mockus
    • 1
  1. 1.Department of Optimal Decisions TheoryInstitute of Mathematics and InformaticsAkademijos 4Lithuania

Personalised recommendations