Skip to main content

Bi-objective Decisions and Partition-Based Methods in Bayesian Global Optimization

Part of the SpringerBriefs in Optimization book series (BRIEFSOPTI)

Abstract

We present in this chapter our recent work in Bayesian approach to continuous non-convex optimization. A brief review precedes the main results to have our work presented in the context of challenges of the approach.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-64712-4_2
  • Chapter length: 48 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-64712-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   69.99
Price excludes VAT (USA)
Fig. 2.1
Fig. 2.2
Fig. 2.3
Fig. 2.4
Fig. 2.5
Fig. 2.6

Notes

  1. 1.

    About 24,000,000 results (0.56 s), accessed September 28, 2020.

References

  1. S. Agrawal, N. Goyal, Analysis of Thompson sampling for the Multi-armed Bandit problem, in Proceedings of 25 Conference on Learning Theory, pp. 39.1—39.26 (2012)

    Google Scholar 

  2. A. Aprem, A Bayesian optimization approach to compute the Nash equilibria of potential games using bandit feedback (2018). arXiv:1811.06503v1

    Google Scholar 

  3. F. Archetti, A. Candelieri, Bayesian Optimization and Data Science (Springer, 2019)

    Google Scholar 

  4. P. Auer, Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3, 586–594 (2002)

    MathSciNet  Google Scholar 

  5. F. Bachoc, C. Helbert, V. Picheny, Gaussian process optimization with failures: classification and convergence proof (2020) HAL Id: hal-02100819

    Google Scholar 

  6. R. Bardenet, B. Kegl, Surrogating the surrogate: accelerating Gaussian-process-based global optimization with a mixture cross-entropy algorithm, in Proceedings of 26 International Conference on Learning Theory, pp. 1–8 (2010)

    Google Scholar 

  7. A. Basudhar, C. Dribusch, S. Lacaze, S. Missoum, Constrained efficient global optimization with support vector machines. Struct. Multidiscip. Optim. 46, 201–221 (2012)

    MATH  CrossRef  Google Scholar 

  8. J. Berk, V. Sunil, G. Santu, R. Venkatesh, Exploration enhanced expected improvement for Bayesian optimization. in Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 621–637 (2018)

    Google Scholar 

  9. H. Bijl, T. Schon, J.-W. van Wingerden, M. Verhaegen, A sequential Monte Carlo approach to Thompson sampling for Bayesian optimization (2017). arXiv:1604.00169v3

    Google Scholar 

  10. E. Brochu, V. Cora, and N. de Freitas, A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning (2010). arXiv:1012.2599v1

    Google Scholar 

  11. A. Bull, Convergence rates of efficient global optimization algorithms. J. Mach. Learn. Res. 12, 2879–2904 (2011)

    MathSciNet  MATH  Google Scholar 

  12. J. Calvin, Consistency of a myopic Bayesian algorithm for one-dimensional global optimization. J. Glob. Optim. 3, 223–232 (1993)

    MathSciNet  MATH  CrossRef  Google Scholar 

  13. J. Calvin, A lower bound on complexity of optimization on the Wiener space. Theor. Comput. Sci. 383, 132–139 (2007)

    MathSciNet  MATH  CrossRef  Google Scholar 

  14. J. Calvin, An adaptive univariate global optimization algorithm and its convergence rate under the wiener measure. Informatica 22(4), 471–488 (2011)

    MathSciNet  MATH  CrossRef  Google Scholar 

  15. J. Calvin, Probability models in global optimization. Informatica 27(2), 323–334 (2016)

    MATH  CrossRef  Google Scholar 

  16. J. Calvin, A. Žilinskas, A one-dimensional P-algorithm with convergence rate O(n −3+δ) for smooth functions. JOTA 106, 297–307 (2000)

    MathSciNet  MATH  CrossRef  Google Scholar 

  17. J. Calvin, A. Žilinskas, On convergence of a P-algorithm based on a statistical model of continuosly differentiable functions functions. J. Glob. Optim. 19, 229–245 (2001)

    MATH  CrossRef  Google Scholar 

  18. J. Calvin, A. Žilinskas, A one-dimensional global optimization for observations with noise. Comp. Math. Appl. 50, 157–169 (2005)

    MathSciNet  MATH  CrossRef  Google Scholar 

  19. J. Calvin, A. Žilinskas, On a global optimization algorithm for bivariate smooth functions. JOTA 163(2), 528–547 (2014)

    MathSciNet  MATH  CrossRef  Google Scholar 

  20. J.M. Calvin, M. Hefter, A. Herzwurm, Adaptive approximation of the minimum of Brownian motion. J. Complexity 39, 17–37 (2017)

    MathSciNet  MATH  CrossRef  Google Scholar 

  21. J.M. Calvin, A. Z̆ilinskas, On the convergence of the p-algorithm for one-dimensional global optimization of smooth functions. JOTA 102, 479–495 (1999)

    Google Scholar 

  22. J. Calvin, et al., On convergence rate of a rectangular partition based global optimization algorithm. J. Glob. Optim. 71, 165–191 (2018)

    MathSciNet  MATH  CrossRef  Google Scholar 

  23. A. Candelieri, Sequential model based optimization of partially defined functions under unknown constraints. J. Glob. Optim. Published online:1–23 (2019)

    Google Scholar 

  24. J. Cashore, L. Kumarga, P. Frazier, Multi-step Bayesian optimization for one-dimensional feasibility determination (2016). arXiv:1607.03195

    Google Scholar 

  25. L. Chan, G. Hutchison, G. Morris, Bayesian optimization for conformer generation. J. Cheminformatics 11(32), 1–11 (2020)

    Google Scholar 

  26. B. Chen, R. Castro, A. Krause, Joint optimization and variable selection of high-dimensional Gaussian processes, in 29th International Conference on Machine Learning (Omnipress, 2012), pp. 1379–1386

    Google Scholar 

  27. T. Cormen, C. Leiserson, R. Rivest, C. Stein, Introduction to Algorithms (MIT Press, 1989)

    Google Scholar 

  28. J. Cui, B. Yang, Survey on Bayesian optimization methodology and applications. J. Softw. 29(10), 3068–3090 (2007)

    MathSciNet  MATH  Google Scholar 

  29. N. Dolatnia, A. Fern, X. Fern, Bayesian optimization with resource constraints and production, in Proceedings of 26 International Conference on Automated Planning and Scheduling, pp. 115–123 (AAAI, 2016)

    Google Scholar 

  30. K. Dong et al., Scalable log determinants for Gaussian process kernel learning. Adv. Neural Inf. Proces. Syst. 30, 6327–6337 (2017)

    Google Scholar 

  31. Z. Dou, Bayesian global optimization approach to the oil well placement problem with quantified uncertainties, Dissertation. Purdue University (2015)

    Google Scholar 

  32. D. Eriksson et al., Scaling Gaussian process regression with derivatives. Adv. Neural Inf. Proces. Syst. 31, 6867–6877 (2018)

    Google Scholar 

  33. D. Eriksson et al., Scalable global optimization via local Bayesian optimization. Adv. Neural Inf. Proces. Syst. 32, 5496–5507 (2019)

    Google Scholar 

  34. Z. Feng et al., A multiobjective optimization based framework to balance the global exploration and local exploitation in expensive optimization. J. Glob. Optim. 61, 677–694 (2015)

    MathSciNet  MATH  CrossRef  Google Scholar 

  35. P. Frazier, W. Powell, S. Dayanik, The knowledge-gradient policy for correlated normal beliefs. INFORMS J. Comput. 21(4), 599—613 (2009)

    MathSciNet  MATH  CrossRef  Google Scholar 

  36. J. Gardner, M. Kusner, Z. Xu, K. Weinberger, J. Cunningham, Bayesian optimization with inequality constraints, in Proceedings of the 31st International Conference on Machine Learning, pp. II–937–II–945 (2014)

    Google Scholar 

  37. R. Garnett, H. Osborne, S. Roberts, Bayesian optimization for sensor set selection, in Proceedings of International Conference on Information Proceedings Sensor Networks, pp. 209–219 (2010)

    Google Scholar 

  38. M. Gelbart, Constrained Bayesian Optimization and Applicationss. Doctoral dissertation, Harvard University (2015)

    Google Scholar 

  39. M. Gelbart, J. Snoek, R. Adams, Bayesian optimization with unknown constraints, in Proceedings of 30 conference on Uncertainty in AI, pp. 250–259 (2014)

    Google Scholar 

  40. E. Gilboa, Y. Saatci, J. Cunningham, Scaling multidimensional Gaussian processes using projected additive approximations, in Proceedings of the 30 International Conference on Machine Learning, vol. 28 (2013), pp. I–454–I–461

    Google Scholar 

  41. D. Ginsbourger, J. Janusevskis, R. Le Riche, Dealing with asynchronicity in parallel Gaussian process based global optimization (2011). HAL Id: hal-00507632

    Google Scholar 

  42. D. Ginsbourger, R. Le Riche, Towards GP-based optimization with finite time horizon (2009). https://hal.archives-ouvertes.fr/hal-00424309/en/

  43. R. Gramacy, H. Lee, Optimization under unknown constraints. Bayesian Stat. 9, 1–18 (2011)

    Google Scholar 

  44. R.B. Gramacy, J. Niemi, R.M. Weiss, Massively parallel approximate Gaussian process regression. SIAM/ASA J. Uncertain. Quantif. 2(1), 564–584 (2014)

    MathSciNet  MATH  CrossRef  Google Scholar 

  45. R. Griffiths, Constrained Bayesian Optimization for Automatic Chemical Design. Dissertion, University of Cambridge (2017)

    Google Scholar 

  46. R. Griffiths, J. Hernandez-Lobato, Constrained Bayesian optimization for automatic chemical design (2019). arXiv:1709.05501v6

    Google Scholar 

  47. Z. Han, M. Abu-Zurayk, S. G¨örtz, C. Ilic, Surrogate-based aerodynamic shape optimization of awing-body transport aircraft configuration, in Notes on Numerical Fluid Mechanics and Multidisciplinary Design, vol. 138 (Springer, 2018), pp. 257–282

    Google Scholar 

  48. F. Hase et al., Phoenics: A Bayesian optimizer for chemistry. ACS Cent. Sci. 4, 1134–1145 (2020)

    CrossRef  Google Scholar 

  49. P. Hennig, C. Schuler, Entropy search for information-efficient global optimization. J. Mach. Learn. Res. 13, 1809–1837 (2012)

    MathSciNet  MATH  Google Scholar 

  50. J. Hernandez-Lobato, M. Gelbart, R. Adams, M. Hofman, Z. Ghahramani, A general framework for constrained Bayesian optimization using information-based search. J. Mach. Learn. Res. 17, 1–53 (2016)

    MathSciNet  MATH  Google Scholar 

  51. J. Hernandez-Lobato, M. Gelbart, M. Hofman, R. Adams, Z. Ghahramani, Predictive entropy search for Bayesian optimization with unknown constraints (2015). arXiv:1502.05312v2

    Google Scholar 

  52. J. Hernandez-Lobato, J. Requeima, E. Pyzer-Knapp, A. Aspuru-Guzik, Parallel and distributed Thompson sampling for large-scale accelerated exploration of chemical space (2017). arXiv:1706.01825v1

    Google Scholar 

  53. D. Huang, T. Allen, W. Notz, R. Miller, Sequential kriging optimization using multiple-fidelity evaluations. Struct. Multidiscip. Optim. 32, 369—382 (2006)

    CrossRef  Google Scholar 

  54. H. Jalali, I. Nieuwenhuyse, V. Picheny, Comparison of kriging-based algorithms for simulation optimization with heterogeneous noise. EJOR 261(1), 279–301 (2017)

    MathSciNet  MATH  CrossRef  Google Scholar 

  55. S. Jeong, M. Murayama, K. Yamamoto, Efficient optimization design method using kriging model. J. Aircraft 42(2), 413–422 (2005)

    CrossRef  Google Scholar 

  56. D.R. Jones, C.D. Perttunen, C.D. Stuckman, Lipschitzian optimization without the Lipschitz constant. JOTA 79, 157–181 (1993)

    MathSciNet  MATH  CrossRef  Google Scholar 

  57. D.R. Jones, M. Schonlau, W. Welch, Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13, 455–492 (1998)

    MathSciNet  MATH  CrossRef  Google Scholar 

  58. K. Kandasamy, A. Krishnamurthy, J. Schneider, B. Poczos, Parallelised Bayesian optimisation via Thompson sampling, in Proceedings of 21 International Conference on Artificial Intelligence and Statistics, pp. 1–10 (2018)

    Google Scholar 

  59. J. Kim, S. Choi, Clustering-guided GP-UCB for Bayesian optimization, in IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 2461–2465 (2018)

    Google Scholar 

  60. T. Kim, J. Lee, Y. Choe, Bayesian optimization-based global optimal rank selection for compression of convolutional neural networks. IEEE Access 8, 17605–17618 (2020)

    CrossRef  Google Scholar 

  61. J. Kleijnen, W. van Beers, I. van Nieuwenhuyse, Expected improvement in efficient global optimization through bootstrapped kriging. J. Glob. Optim. 54, 59–73 (2012)

    MathSciNet  MATH  CrossRef  Google Scholar 

  62. J. Knowles, D. Corne, A. Reynolds, Noisy multiobjective optimization on a budget of 250 evaluations, in Lecture Notes in Computer Science, ed. by M. Ehrgott et al. vol. 5467 (Springer, 2009), pp. 36–50

    Google Scholar 

  63. H. Kushner, A versatile stochastic model of a function of unknown and time-varying form. J. Math. Anal. Appl. 5, 150–167 (1962)

    MathSciNet  MATH  CrossRef  Google Scholar 

  64. H. Kushner, A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. J. Basic Eng. 86, 97–106 (1964)

    CrossRef  Google Scholar 

  65. R. Lam, M. Poloczeky, P. Frazier, K. Willcox, Advances in Bayesian optimization with applications in aerospace engineering, in AIAA Non-Deterministic Approaches Conference, pp. 1–10 (2018)

    Google Scholar 

  66. R. Lam, K. Willcox, Lookahead Bayesian optimization with inequality constraints, in 31st Conference on Neural Information Processing Systems, pp. 4–5 (2017)

    Google Scholar 

  67. L. Cornejo-Buenoa, E.C. Garrido-Merchánb, D. Hernández-Lobatob, S. Salcedo-Sanza, Bayesian optimization of a hybrid system for robust ocean wave features prediction. Neurocomputing 275, 818–828 (2018)

    CrossRef  Google Scholar 

  68. B. Letham, B. Karrery, G. Ottoniz, E. Bakshy, Constrained Bayesian optimization with noisy experiments (2018) arXiv:1706.07094v2

    Google Scholar 

  69. C. Li, S. Gupta, S. Rana, V. Nguyen, S. Venkatesh, A. Shilton, High dimensional Bayesian optimization using dropout, in Proceedings of 26 International Conference on AI, pp. 2096–2102 (2017)

    Google Scholar 

  70. C. Li, K. Kandasamy, B. Poczos, J. Schneider, High dimensional Bayesian optimization via restricted projection pursuit models, in Proceedings of 19 International Conference on Artificial Intelligence and Statistics (Springer, 2016), pp. 884–892

    Google Scholar 

  71. D. Lindberg, H.K. Lee, Optimization under constraints by applying an asymmetric entropy measure. J. Comput. Graph. Stat. 24(2), 379–393 (2015)

    MathSciNet  CrossRef  Google Scholar 

  72. W.-L. Loh, T.-K. Lam, Estimating structured correlation matrices in smooth Gaussian random field models. Ann. Stat. 28, 880–904 (2000)

    MathSciNet  MATH  CrossRef  Google Scholar 

  73. M. Maier, A. Rupenyan1, C. Bobst, K. Wegener, Self-optimizing grinding machines using Gaussian process models and constrained Bayesian optimization (2020). arXiv:2006.05360v1

    Google Scholar 

  74. A. Makauskas, On a possibility to use gradients in statistical models of global optimization of objective functions. Informatica 2, 248–254 (1991)

    MathSciNet  MATH  Google Scholar 

  75. G. Malkomes, R. Garnett, Automating Bayesian optimization with Bayesian optimization, in 32 Conference on Neural Information Processing Systems, pp. 1–11 (2018)

    Google Scholar 

  76. M. McLeod, M. Osborne, S. Roberts, Optimization, fast and slow: optimally switching between local and Bayesian optimization (2018). arXiv:1805.08610v1

    Google Scholar 

  77. A. Mittal, S. Aggarwal, Hyperparameter optimization using sustainable proof of work in blockchain. Front. Blockchain 3(23), 1–13 (2020)

    Google Scholar 

  78. J. Mockus, On Bayes methods for seeking an extremum. Avtomatika i Vychislitelnaja Technika (3), 53–62 (1972) in Russian

    Google Scholar 

  79. J. Mockus, Bayesian Approach to Global Optimization (Kluwer Academic Publishers, 1988)

    Google Scholar 

  80. J. Mockus, V. Tiešis, A. Žilinskas, The application of Bayesian methods for seeking the extremum, in Towards Global Optimization 2, ed. by L.C.W. Dixon, G.P. Szego (North Holland, 1978), pp. 117–129

    Google Scholar 

  81. J. Mockus et al., Bayesian Heuristic Approach to Discrete and Global Optimization (Kluwer Academic Publishers, Dodrecht, 1997)

    MATH  CrossRef  Google Scholar 

  82. M. Morrar, J. Knowles, S. Sampaio, Initialization of Bayesian optimization viewed as part of a larger algorithm portfolio, in CEC2017 and CPAIOR 2017, pp. 1–6 (2017)

    Google Scholar 

  83. M. Mutny, A. Krause, Efficient high dimensional Bayesian optimization with additivity and quadrature Fourier features, in 32 Conference on Neural Information Processing Systems, pp. 1–12 (2018)

    Google Scholar 

  84. V. Nguyen et al., Regret for expected improvement over the best-observed value and stopping condition, in Proceedings of 9 Asian Conference on Machine Learning, vol. 77 (PMLR, 2017), pp. 279–294

    Google Scholar 

  85. E. Novak, Deterministic and Stochastic Error Bounds in Numerical Analysis, volume 1349 of Lecture Notes in Mathematics (Springer, Berlin, 1988)

    Google Scholar 

  86. E. Novak, H. Woźniakowski, Tractability of Multivariate Problems, volume II of Tracts in Mathematics (European Mathematical Society, Zürich, 2010)

    Google Scholar 

  87. S. Olofsson et al., Bayesian multiobjective optimisation with mixed analytical and black-box functions: Application to tissue engineering. IEEE Trans. Biomed. Eng. 66(3), 727–739 (2019)

    CrossRef  Google Scholar 

  88. M. Osborne, R. Garnett, S. Roberts, Gaussian processes for global optimization (2009). http://www.robots.ox.ac.uk

  89. Y. Ozaki et al., Automated crystal structure analysis based on blackbox optimisation. Comput. Mat. 6(75), 1–7 (2020)

    Google Scholar 

  90. R. Paulavičius et al., Globally-biased Disimpl algorithm for expensive global optimization. J. Glob. Optim. 59, 545–567 (2014)

    MathSciNet  MATH  CrossRef  Google Scholar 

  91. V. Picheny, Multiobjective optimization using gaussian process emulators via stepwise uncertainty reduction. Stat. Comput. 25, 1265–1280 (2015)

    MathSciNet  MATH  CrossRef  Google Scholar 

  92. V. Picheny, D. Ginsbourger, Y. Richet, Noisy expected improvement and on-line computation time allocation for the optimization of simulators with tunable fidelity, in Proceedings of 2nd International Conference on Engineering Opt (2010)

    Google Scholar 

  93. V. Picheny, R. Gramacy, S. Wild, S. Le Digabel, Bayesian optimization under mixed constraints with a slack-variable augmented lagrangian (2016). arXiv:1605.09466v1

    Google Scholar 

  94. V. Picheny, T. Wagner, D. Ginsbourger, A benchmark of kriging-based infill criteria for noisy optimization. Struct. Multidiscip. Optim. 48(3), 607—626 (2013)

    CrossRef  Google Scholar 

  95. J. Pinter, Global Optimization in Action (Kluwer Academic Publisher, 1996)

    Google Scholar 

  96. R. Preuss, U. von Toussaint, Global optimization employing Gaussian process-based Bayesian surrogates. Entropy 20, 201–214 (2018)

    MATH  CrossRef  Google Scholar 

  97. R. Priem et al., An adaptive feasibility approach for constrained Bayesian optimization with application in aircraft design, in 6 International Conference on Engineering Optimization (EngOpt2018) (2018)

    Google Scholar 

  98. H. Prosper, Deep learning and Bayesian methods. EPJ Web Conf. 137, 11007 (2018)

    CrossRef  Google Scholar 

  99. S. Rana, C. Li, S. Gupta, V. Nguyen, S. Venkatesh, High dimensional Bayesian optimization with elastic Gaussian process, in Proceedings of 34th International Conference on Machine Learning, pp. 2883–2891 (2017)

    Google Scholar 

  100. C.E. Rasmussen, C. Williams, Gaussian Processes for Machine Learning (MIT Press, 2006)

    Google Scholar 

  101. B. Rezaeianjouybari, M. Sheikholeslami, A. Shafee, H. Babazadeh, A novel Bayesian optimization for flow condensation enhancement using nanorefrigerant: A combined analytical and experimental study. Chem. Eng. Sci. 215, 115465 (2020)

    CrossRef  Google Scholar 

  102. K. Ritter, Approximation and optimization on the Wiener space. J. Complexity 6, 337—364 (1990)

    MathSciNet  MATH  CrossRef  Google Scholar 

  103. M. Sacher et al., A classification approach to efficient global optimization in presence of non-computable domains. Struct. Multidiscip. Optim. 58(4), 1537–1557 (2018)

    MathSciNet  CrossRef  Google Scholar 

  104. J. Sacks, S.B. Schiller, W.J. Welch, Designs for computer experiments. Technometrics 31(1), 41–47 (1989)

    MathSciNet  CrossRef  Google Scholar 

  105. J. Sacks, W.J. Welch, T.J. Mitchell, H.P. Wynn, Design and analysis of computer experiments. Stat. Sci. 4, 409–423 (1989)

    MathSciNet  MATH  CrossRef  Google Scholar 

  106. M. Sasena, Dissertation: Flexibility and Efficiency Enhancements for Constrained Global Design Optimization with Kriging Approximations. Michigan University (2002)

    Google Scholar 

  107. M. Schonlau, W. Welch, D. Jones, Global versus local search in constrained optimization of computer models, Technical Report Number 83. National Institute of Statistical Sciences (1998)

    Google Scholar 

  108. Y. Sergeyev, An efficient strategy for adaptive partition of n-dimensional intervals in the framework of diagonal algorithms. JOTA 107, 145–168 (2000)

    MathSciNet  MATH  CrossRef  Google Scholar 

  109. Y. Sergeyev, D. Kvasov, Global search based on efficient diagonal partitions and a set of Lipschitz constants. SIAM J. Optim. 16, 910–937 (2006)

    MathSciNet  MATH  CrossRef  Google Scholar 

  110. Y. Sergeyev, Numerical infinities and infinitesimals: Methodology, applications, and repercussions on two Hilbert problems. EMS Surv. Math. Sci. 4, 219–320 (2017)

    MathSciNet  MATH  CrossRef  Google Scholar 

  111. B. Shahriari, K. Swersky, Z. Wang, R. Adams, N. de Freitas, Taking the human out of the loop: A review of Bayesian optimization. Proc. IEEE 104(1), 148–175 (2016)

    CrossRef  Google Scholar 

  112. B. Shahriari, Z. Wang, M. Hoffman, An entropy search portfolio for Bayesian optimization (2015). arXiv:1406.4625v4

    Google Scholar 

  113. V. Shaltenis, On a method of multiextremal optimization. Avtomatika i Vychislitelnaja Technika 3, 33–38 in Russian (1971)

    Google Scholar 

  114. D. Silveret et al., Mastering the game of Go with deep neural networks and tree search. Nature 529, 484—489 (2016)

    CrossRef  Google Scholar 

  115. T. Simpson, J. Korte, F. Mistree, Kriging models for global approximation in simulation-based multidisciplinary design optimization. AIAA J. 39(12), 2233–2242 (2001)

    CrossRef  Google Scholar 

  116. N. Srinivas, A. Krause, S. Kakade, M. Seeger, Gaussian process optimization in the Bandit setting: No regret and experimental design, in Proceedings of 27th International Conference on Machine Learning, pp. 1015–1022 (Omnipress, 2010)

    Google Scholar 

  117. M.L. Stein, Interpolation of Spatial Data: Some Theory of Kriging (Springer, 1999)

    Google Scholar 

  118. L. Stripinis, R. Paulavičius, J. Žilinskas, Improved scheme for selection of potentially optimal hyper-rectangles in DIRECT. Optim. Lett. 12, 1699–1712 (2018)

    MathSciNet  MATH  CrossRef  Google Scholar 

  119. R. Strongin, Information method of global minimization in the presence of noise. Eng. Cybern. 6, 118–126 (1969) in Russian

    MathSciNet  Google Scholar 

  120. R.G. Strongin, Numerical Methods of Multiextremal Minimization. Nauka, (1978) in Russian

    Google Scholar 

  121. R.G. Strongin, Y.D. Sergeyev, Global Optimization with Non-convex Constraints: Sequential and Parallel Algorithms (Kluwer Academic Publishers, 2000)

    Google Scholar 

  122. A. Sukharev, Minimax Models in the Theory of Numerical Methods (Springer, 2012)

    Google Scholar 

  123. A.G. Sukharev, Optimal strategies of search for an extremum. USSR Comput. Math. Math. Phys. 11(4), 910–924 (1971)

    MathSciNet  CrossRef  Google Scholar 

  124. R. Tamura, K. Hukushima, Bayesian optimization for computationally extensive probability distributions. PLoS ONE 13, e0193785 (2018)

    CrossRef  Google Scholar 

  125. A. Törn, A. Žilinskas, Global Optimization (Springer, 1989)

    Google Scholar 

  126. E. Vazquez, J. Bect, Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. J. Stat. Plan. Infer. 140(11), 3088–3095 (2010)

    MathSciNet  MATH  CrossRef  Google Scholar 

  127. N. Vien, H. Zimmermann, M. Toussaint, Bayesian functional optimization, in 32 AAAI Conference on AI, pp. 4171–4178 (AAAI, 2018)

    Google Scholar 

  128. J. Villemonteix, E. Vazquez, E. Walter, An informational approach to the global optimization of expensive to evaluate functions. J. Glob Optim. 44(4), 509–534 (2009)

    MathSciNet  MATH  CrossRef  Google Scholar 

  129. K. Wabersich, Gaussian processes and Bayesian optimization (2016)

    Google Scholar 

  130. K. Wabersich, M. Toussaint, Advancing Bayesian optimization: The mixed-global-local kernel and length-scale cool down (2016). arXiv:1612.03117v1

    Google Scholar 

  131. J. Wang, S. Clark, E. Liu, P. Frazier, Parallel Bayesian global optimization of expensive functions (2019). arXiv:1602.05149v4

    Google Scholar 

  132. Z. Wang, S. Jagelka, Max-value entropy search for efficient Bayesian optimization (2018). arXiv:1703.01968v3

    Google Scholar 

  133. K. Wang et al., Exact Gaussian processes on a million data points, in 33rd Conference on Neural Information Processing Systems, pp. 1—13 (2019). arXiv:1903.08114v2

    Google Scholar 

  134. Z. Wang et al., Bayesian optimization in a billion dimensions via random embeddings. J. AI Res. 55, 361–387 (2016)

    MathSciNet  MATH  Google Scholar 

  135. Z. Wang et al., Bayesian optimization in high dimensions via random embeddings, in Proceedings of 23 International Conference on AI, pp. 1778–1784 (2017)

    Google Scholar 

  136. J. Wilson, V. Borovitskiy, A. Terenin, P. Mostowsky, M. Deisenroth, Efficiently sampling functions from gaussian process posteriors (2020). arXiv:2002.09309v1

    Google Scholar 

  137. J. Wilson, F. Hutter, M. Deisenroth, Maximizing acquisition functions for Bayesian optimization (2018). arXiv:1805.10196v2

    Google Scholar 

  138. A. Wu, M. Aoi, J. Pillow, Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature (2018). arXiv:1704.00060v2

    Google Scholar 

  139. J. Wu, P. Frazier, Discretization-free Knowledge Gradient methods for Bayesian optimization (2017). arXiv:1707.06541v1

    Google Scholar 

  140. J. Wu, M. Poloczek, A. Wilson, P. Frazier, Bayesian optimization with gradients, in Proceedings of 31st International Conference on Neural Information Processing Systems, pp. 5273–5284 (IEEE, 2017)

    Google Scholar 

  141. J. Wu et al., Hyperparameter optimization for machine learning models based on Bayesian optimization. J. Electron. Sci. Technol. 17(1), 26–40 (2019)

    Google Scholar 

  142. W. Xu, M.L. Stein, Maximum likelihood estimation for smooth Gaussian random field model. SIAM/ASA Uncertain. Quantif. 5, 138–175 (2017)

    MathSciNet  MATH  CrossRef  Google Scholar 

  143. A.M. Yaglom, Correlation Theory of Stationary and Related Random Functions, vol. 1 (Springer, 1987)

    Google Scholar 

  144. J. Yim, B. Lee, C. Kim, Exploring multi-stage shape optimization strategy of multi-body geometries using kriging-based model and adjoint method. Comput. Fluids 68, 71–87 (2012)

    MathSciNet  MATH  CrossRef  Google Scholar 

  145. A. Zhigljavsky, A. Z̆ilinskas. Methods of Search for Global Extremum (Nauka, Moscow, 1991), in Russian

    Google Scholar 

  146. A. Zhigljavsky, A. Žilinskas, Stochastic Global Optimization (Springer, 2008)

    Google Scholar 

  147. A. Zhigljavsky, A. Žilinskas, Selection of a covariance function for a Gaussian random field aimed for modeling global optimization problems. Opt. Lett. 13(2), 249—259 (2019)

    MathSciNet  MATH  CrossRef  Google Scholar 

  148. A. Žilinskas, One-step Bayesian method for the search of the optimum of one-variable functions. Cybernetics 1, 139–144 (1975) in Russian

    Google Scholar 

  149. A. Žilinskas, On global one-dimensional optimization. Izv. Acad. Nauk USSR Eng. Cybern. 4, 71–74 (1976) in Russian

    Google Scholar 

  150. A. Žilinskas, Optimization of one-dimensional multimodal functions, algorithm 133. J. Roy. Stat. Soc. Ser C 23, 367–385 (1978)

    MATH  Google Scholar 

  151. A. Žilinskas, MIMUN-optimization of one-dimensional multimodal functions in the presence of noise. Aplikace Matematiky 25, 392–402 (1980)

    MATH  Google Scholar 

  152. A. Žilinskas, Two algorithms for one-dimensional multimodal minimization. Math. Oper. Stat. Ser. Optim. 12, 53–63 (1981)

    MathSciNet  MATH  Google Scholar 

  153. A. Žilinskas, Axiomatic approach to statistical models and their use in multimodal optimization theory. Math. Program. 22, 104–116 (1982)

    MathSciNet  MATH  CrossRef  Google Scholar 

  154. A. Žilinskas, Axiomatic characterization of a global optimization algorithm and investigation of its search strategies. Oper. Res. Lett. 4, 35–39 (1985)

    MathSciNet  MATH  CrossRef  Google Scholar 

  155. A. Žilinskas, Global Optimization: Axiomatic of Statistical Models, Algorithms, Applications (Mokslas, Vilnius, 1986) in Russian

    Google Scholar 

  156. A. Žilinskas, Statistical models for global optimization by means of select and clone. Optimization 48, 117–135 (2000)

    MathSciNet  MATH  CrossRef  Google Scholar 

  157. A. Žilinskas, On the worst-case optimal multi-objective global optimization. Optim. Lett. 7(8), 1921–1928 (2013)

    MathSciNet  MATH  CrossRef  Google Scholar 

  158. A. Žilinskas, Including the derivative information into statistical models used in global optimization. AIP Conf. Proc. 2070(020020), 1–4 (2019)

    Google Scholar 

  159. A. Žilinskas, A. Makauskas, On possibility of use of derivatives in statistical models of multimodal functions, in Teorija Optimaljnych Reshenij, vol. 14, pp. 63–77. Inst. Math. Cybern. Lith. Acad. Sci. (1990) in Russian

    Google Scholar 

  160. A. Žilinskas, J.M. Calvin, Bi-objective decision making in global optimization based on statistical models. J. Glob. Optim. 74, 599–609 (2019)

    MathSciNet  MATH  CrossRef  Google Scholar 

  161. A. Žilinskas, G. Gimbutienė, On asymptotic property of a simplicial statistical model of global optimization, in Springer Proceedings in Mathematics and Statistics, vol. 130 (2015), pp. 383–392

    Google Scholar 

  162. A. Žilinskas, G. Gimbutienė, A hybrid of Bayesian approach based global search with clustering aided local refinement. Commun. Nonlinear Sci. Numer. Simul. 78, 104857 (2019)

    MathSciNet  MATH  CrossRef  Google Scholar 

  163. A. Žilinskas, L. Litvinas, A hybrid of the simplicial partition-based bayesian global search with the local descent. Soft Comput. 24, 17601–17608 (2020)

    CrossRef  Google Scholar 

  164. A. Žilinskas, J. Mockus, On a Bayesian method for seeking the minimum. Avtomatika i Vychislitelnaja Technika 4, 42–44 (1972) in Russian

    Google Scholar 

  165. A. Žilinskas, E. Senkiene, On estimating the parameter of Wiener process. Lith. Math. J. 3, 59–62 (1978) in Russian

    MATH  Google Scholar 

  166. A. Žilinskas, J. Žilinskas, Global optimization based on a statistical model and simplicial partitioning. Comput. Math. Appl. 44(7), 957–967 (2002)

    MathSciNet  MATH  CrossRef  Google Scholar 

  167. A. Žilinskas, J. Žilinskas, P-algorithm based on a simplicial statistical model of multimodal functions. TOP 18, 396–412 (2010)

    MathSciNet  MATH  CrossRef  Google Scholar 

  168. A. Žilinskas et al., Multi-objective optimization and decision visualization of batch stirred tank reactor based on spherical catalyst particles. Nonlinear Anal. Model. Control 24(6), 1019–1036 (2019)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 The Author(s)

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Zhigljavsky, A., Žilinskas, A. (2021). Bi-objective Decisions and Partition-Based Methods in Bayesian Global Optimization. In: Bayesian and High-Dimensional Global Optimization. SpringerBriefs in Optimization. Springer, Cham. https://doi.org/10.1007/978-3-030-64712-4_2

Download citation