One-dimensional Global Optimization Based on Statistical Models

  • James M. Calvin
  • Antanas Žilinskas
Part of the Nonconvex Optimization and Its Applications book series (NOIA, volume 59)


This paper presents a review of global optimization methods based on statistical models of multimodal functions. The theoretical and methodological aspects are emphasized.


Optimization statistical models convergence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Kushner, H.: A versatile stochastic model of a function of unknown and time-varying form, J. Math. Anal. Appl. 5 (1962), 150–167.CrossRefMathSciNetzbMATHGoogle Scholar
  2. [2]
    Mockus, J.: On Bayesian methods of search for extremum, Automatics and Computers 3 (1972), 53–62 (in Russian).MathSciNetzbMATHGoogle Scholar
  3. [3]
    Mockus, J.: Bayesian Approach to Global Optimization, Kluwer Acad. Publ., Dordrecht, 1989.zbMATHGoogle Scholar
  4. [4]
    Boender, G. and Romeijn, E.: Stochastic methods, In: R. Horst and P. Pardalos (eds), Handbook of Global Optimization, Kluwer Acad. Publ., Dordrecht, 1995, pp. 829–869.Google Scholar
  5. [5]
    Törn, A. and Žilinskas, A.: Global Optimization, Springer, 1989.Google Scholar
  6. [6]
    Strongin, R.: Numerical Methods in Multiextremal Optimization (in Russian), Nauka, 1978.Google Scholar
  7. [7]
    Horst, R. and Tuy, H.: Global Optimization — Deterministic Approaches, 2nd edn, Springer, 1992.Google Scholar
  8. [8]
    Zhigljavski, A. and Žilinskas, A.: Methods of Search for Global Extremum (in Russian), Nauka, Moscow, 1991.Google Scholar
  9. [9]
    Kushner, H.: A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise, J. Basic Eng. 86 (1964), 97–106.Google Scholar
  10. [10]
    Žilinskas A.: On global one-dimensional optimization, Engrg. Cybernet., Izv. AN USSR 4 (1976), 71–74 (in Russian).Google Scholar
  11. [11]
    Archetti, F. and Betro, B.: A probabilistic algorithm for global optimization, Calcolo 16 (1979), 335–343.MathSciNetzbMATHGoogle Scholar
  12. [12]
    Žilinskas, A.: Optimization of one-dimensional multimodal functions, algorithm 133. Appl. Statist. 23 (1978), 367–385.Google Scholar
  13. [13]
    Žilinskas, A.: Mimun-optimization of one-dimensional multimodal functions in the presence of noise, algoritmus 44, Apl. Mat. 25 (1980), 392–402.Google Scholar
  14. [14]
    Žilinskas A.: Two algorithms for one-dimensional multimodal minimization, Mathematische Operationsforschung und Statistik, ser. Optimization 12 (1981), 53–63.zbMATHGoogle Scholar
  15. [15]
    Locatelli, M.: Baeysian algorithms for one-dimensional global optimization, J. Global Optim. 10 (1997), 57–76.CrossRefMathSciNetzbMATHGoogle Scholar
  16. [16]
    Calvin, J. M. and Glynn, P. W.: Average case behavior of random search for the maximum, J. Appl. Probab. 34 (1997), 631–642.MathSciNetGoogle Scholar
  17. [17]
    Ritter, K.: Approximation and optimization on the Wiener space, J. Complexity 6 (1990), 337–364.CrossRefMathSciNetzbMATHGoogle Scholar
  18. [18]
    Shepp, L. A.: The joint density of the maximum and its location for a Wiener process with drift, J. Appl. Probab. 16 (1976), 423–427.MathSciNetGoogle Scholar
  19. [19]
    Lindgren, G.: Local maxima of Gaussian fields, Ark. Math. 10 (1972), 195–218.MathSciNetzbMATHGoogle Scholar
  20. [20]
    Calvin, J. M. and Žilinskas, A.: On convergence of a p-algorithm based on a statistical model of continuously differentiable functions, J. Global Optim. 19 (2001), 229–245.CrossRefMathSciNetzbMATHGoogle Scholar
  21. [21]
    Calvin, J. M. and Žilinskas, A.: On convergence of the p-algorithm for one-dimensional global optimization of smooth functions, J. Optim. Theory Appl. 102(3) (1999), 479–495.CrossRefMathSciNetzbMATHGoogle Scholar
  22. [22]
    Neimark, J. and Strongin, R.: Information approach to search for minimum of a function, Izv. AN SSSR, Engineering Cybernetics No. 1 (1966), 17–26 (in Russian).Google Scholar
  23. [23]
    Fine, T.: Optimal search for location of the maximum of a unimodal function, IEEE Trans. Inform. Theory (2) (1966), 103–111.Google Scholar
  24. [24]
    Converse, A.: The use of uncertainty in a simultaneos search, Oper. Res. 10 (1967), 1088–1095.Google Scholar
  25. [25]
    Heyman, M.: Optimal simultaneos search for the maximum by the principle of statistical information, Oper. Res. (1968), 1194–1205.Google Scholar
  26. [26]
    Neuman, P.: An asymptotically optimal procedure for searching a zero or an extremum of a function, In: Proceedings of 2nd Prague Symp. Asymp. Statist., 1981, pp. 291–302.Google Scholar
  27. [27]
    Timonov, L.: An algorithm for search of a global extremum, Engrg. Cybernet. 15 (1977), 38–44.MathSciNetGoogle Scholar
  28. [28]
    Shaltenis, V.: Structure Analysis of Optimization Problems (in Russian), Mokslas, Vilnius, 1989.Google Scholar
  29. [29]
    Žilinskas, A.: On statistical models for multimodal optimization, Mathematische Operationsforschung und Statistik, ser. Statistics 9 (1978), 255–266.zbMATHGoogle Scholar
  30. [30]
    Žilinskas, A.: Axiomatic approach to statistical models and their use in multimodal optimization theory, Math. Programming 22 (1982), 104–116.MathSciNetzbMATHGoogle Scholar
  31. [31]
    Žilinskas, A.: A reviewo f statistical models for global optimization, J. Global Optim. (1992), 145–153.Google Scholar
  32. [32]
    Luce, D. and Suppes, P.: Preference, utility and subjective probability, In: R. Bush, D. Luce and E. Galanter (eds), Handbook of Global Optimization, Wiley, 1965, pp. 249–410.Google Scholar
  33. [33]
    Žilinskas, A.: Axiomatic approach to extrapolation problem under uncertainty, Automatics and Remote Control, No. 12 (1979), 66–70 (in Russian).Google Scholar
  34. [34]
    Gutman, H.-M.: A radial basis function method for global optimization, J. Global Optim. 19 (2001), 201–227.MathSciNetGoogle Scholar
  35. [35]
    Žilinskas, A.: Axiomatic characterization of a global optimization algorithm and investigation of its search strategies, Oper. Res. Lett. 4 (1985), 35–39.MathSciNetzbMATHGoogle Scholar
  36. [36]
    Shagen, I.: Internal modelling of objective functions for global optimization, J. Optim. Theory Appl. 51 (1986), 345–353.MathSciNetGoogle Scholar
  37. [37]
    Žilinskas A.: Statistical models for global optimization by means of select and clone, Optimization 48 (2000), 117–135.MathSciNetzbMATHGoogle Scholar
  38. [38]
    Žilinskas, A.: One-step Bayesian method for the search of the optimium of one-variable functions, Cybernetics, No. 1 (1975), 139–144 (in Russian).Google Scholar
  39. [39]
    Calvin, J.M. and Žilinskas, A.: On the choice of statistical model for one-dimensional p-algorithm, Control and Cybernet. 29(2) (2000), 555–565.MathSciNetzbMATHGoogle Scholar
  40. [40]
    Calvin, J. M.: Convergence rate of the p-algorithm for optimization of continuous functions, In: P. Pardalos (ed.), Approximation and Complexity in Numerical Optimization: Continuous and Discrete Problems, Kluwer Acad. Publ., Boston, 1999, pp. 116–129.Google Scholar
  41. [41]
    Calvin, J.M. and Žilinskas, A.: A one-dimensional p-algorithm with convergence rate o(n −3+δ) for smooth functions, J. Optim. Theory Appl. 106 (2000), 297–307.CrossRefMathSciNetzbMATHGoogle Scholar
  42. [42]
    Törn, A. and Žilinskas, A.: Parallel global optimization algorithms in optimal design, Lecture Notes in Control and Inform. Sci. 143, Springer, 1990, pp. 951–960.Google Scholar
  43. [43]
    Locatelli, M. and Schoen, F.: An adaptive stochastic global optimization algorithm for one-dimensional functions, Ann. Oper. Res. 58 (1995), 263–278.CrossRefMathSciNetzbMATHGoogle Scholar
  44. [44]
    Sergeyev, Y.: An information global optimization algorithm with local tuning, SIAM J. Optim. 5 (1995), 858–870.CrossRefMathSciNetzbMATHGoogle Scholar
  45. [45]
    Žilinskas, A.: Note on Pinter’s paper, Optimization 19 (1988), 195.MathSciNetGoogle Scholar
  46. [46]
    Žilinskas A.: A note on extended univariate algorithms by J. Pinter, Computing 41 (1989), 275–276.MathSciNetzbMATHGoogle Scholar
  47. [47]
    Gergel, V. and Sergeev, Y.: Sequential and parallel algorithms for global minimizing functions with Lipshitz derivatives, Internat. J. Comput. Math. Appl. 37 (1999), 163–179.zbMATHGoogle Scholar
  48. [48]
    Sergeyev, Y.: Parallel information algorithm with local tuning for solving multidimensional global optimization problems, J. Global Optim. 15 (1999), 157–167.CrossRefMathSciNetzbMATHGoogle Scholar
  49. [49]
    Hansen, P. and Jaumard, B.: Lipshitz optimization, In: R. Horst and P. Pardalos (eds), Handbook of Global Optimization, Kluwer Acad. Publ., Dordrecht, 1995, pp. 404–493.Google Scholar
  50. [50]
    Pinter, J.: Extended univariate algorithms for n-dimensional global optimization, Computing 36 (1986), 91–103.MathSciNetzbMATHGoogle Scholar
  51. [51]
    Hansen, P. and Jaumard, B.: On Timonov’s algorithm for global optimization of univariate Lipshitz functions, J. Global Optim. 1 (1991), 37–46.MathSciNetzbMATHGoogle Scholar
  52. [52]
    Al-Mharmah, H. and Calvin, J. M.: Optimal random non-adaptive algorithm for global optimization of brownian motion, J. Global Optim. 8 (1996), 81–90.CrossRefMathSciNetzbMATHGoogle Scholar
  53. [53]
    Ritter, K.: Average-Case Analysis of Numerical Problems, Lecture Notes in Math. 1733, 2000.Google Scholar
  54. [54]
    Calvin, J. M.: Average performance of passive algorithms for global optimization, J. Math. Anal. Appl. 191 (1995), 608–617.CrossRefMathSciNetzbMATHGoogle Scholar
  55. [55]
    Calvin, J. M. and Glynn, P. M.: Complexity of non-adaptive optimization algorithms for a class of diffusions. Comm. Statist. Stochastic Models 12 (1996), 343–365.MathSciNetCrossRefzbMATHGoogle Scholar
  56. [56]
    Novak, E.: Deterministic and Stochastic Error Bounds in Numerical Analysis, Lecture Notes in Math. 1349, Springer, 1988.Google Scholar
  57. [57]
    Calvin, J. M.: A one-dimensional optimization algorithm and its convergence rate under the wiener measure, J. Complexity (2001), accepted.Google Scholar
  58. [58]
    Calvin, J. M.: Average performance of a class of adaptive algorithms for global optimization, Ann. Appl. Probab. 7 (1997), 711–730.MathSciNetzbMATHGoogle Scholar
  59. [59]
    Törn, A., Viitanen, S. and Ali, M.: Stochastic global optimization: Problem classes and solution techniques, J. Global Optim. 14 (1999), 437–447.MathSciNetzbMATHGoogle Scholar
  60. [60]
    Floudas, C. and Pardalos, P.: Handbook of Test Problems in Local and Global Optimization, Kluwer Acad. Publ.. Dordrecht, 1999.zbMATHGoogle Scholar
  61. [61]
    Wingo, D.: Fitting three parameter lognormal model by numerical global optimization—an improved algorithm. Computational Statistics and Data Analysis, No. 2 (1984), 13–25.Google Scholar
  62. [62]
    Orsier, B. and Pellegrini, C.: Using global line searches for finding global minima of mlp error functions. In: International Conference on Neural Networks and their Applications, Marseilles, 1997, pp. 229–235.Google Scholar
  63. [63]
    Groch, A., Vidigal, L. and Director, S.: A new global optimization method for electronic circuit design, IEEE Trans. on Circuits and Systems 32 (1985), 160–170.CrossRefGoogle Scholar
  64. [64]
    Perttunen, C., Jones, D. and Stuckman, B.: Lipshitzian optimization without the Lipshitz constant, J. Optim. Theory Appl. 79 (1993), 157–181.MathSciNetzbMATHGoogle Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • James M. Calvin
    • 1
  • Antanas Žilinskas
    • 2
  1. 1.Department of Computer and Information ScienceNew Jersey Institute of TechnologyNewarkUSA
  2. 2.Institute of Mathematics and InformaticsVMUVilniusLithuania

Personalised recommendations