MOI-MBO: Multiobjective Infill for Parallel Model-Based Optimization

  • Bernd BischlEmail author
  • Simon Wessing
  • Nadja Bauer
  • Klaus Friedrichs
  • Claus Weihs
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8426)


The aim of this work is to compare different approaches for parallelization in model-based optimization. As another alternative aside from the existing methods, we propose using a multi-objective infill criterion that rewards both the diversity and the expected improvement of the proposed points. This criterion can be applied more universally than the existing ones because it has less requirements. Internally, an evolutionary algorithm is used to optimize this criterion. We verify the usefulness of the approach on a large set of established benchmark problems for black-box optimization. The experiments indicate that the new method’s performance is competitive with other batch techniques and single-step EGO.


Multiobjective Optimization Kriging Model Multimodal Function Expect Improvement Local Uncertainty 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This paper is based on investigations of the projects B3 and C2 of the Collaborative Research Center SFB 823, which are kindly supported by Deutsche Forschungsgemeinschaft (DFG). It is also partly supported by the French national research agency (ANR) within the Modeles Numeriques project NumBBO. The authors also thank Tobias Wagner for fruitful discussions of multiobjective infill criteria.


  1. 1.
    Koch, P., Bischl, B., Flasch, O., Bartz-Beielstein, T., Weihs, C., Konen, W.: Tuning and evolution of support vector kernels. Evol. Intel. 5(3), 153–170 (2012)CrossRefGoogle Scholar
  2. 2.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 5. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  3. 3.
    Hess, S., Wagner, T., Bischl, B.: PROGRESS: progressive reinforcement-learning-based surrogate selection. In: Nicosia, G., Pardalos, P. (eds.) LION 7. LNCS, vol. 7997, pp. 110–124. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  4. 4.
    Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Krige, D.: A statistical approach to some basic mine valuation problems on the witwatersrand. J. Chem. Metall. Min. Soc. S. Afr. 52(6), 119–139 (1951)Google Scholar
  6. 6.
    Locatelli, M.: Bayesian algorithms for one-dimensional global optimization. J. Global Optim. 10(1), 57–76 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21(4), 345–383 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Vazquez, E., Bect, J.: Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. J. Stat. Plann. Infer. 140(11), 3088–3095 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Ginsbourger, D., Le Riche, R., Carraro, L.: Kriging is well-suited to parallelize optimization. In: Tenne, Y., Goh, C.-K. (eds.) Computational Intel. in Expensive Opti. Prob. ALO, vol. 2, pp. 131–162. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  10. 10.
    Janusevskis, J., Le Riche, R., Ginsbourger, D., Girdziusas, R.: Expected improvements for the asynchronous parallel global optimization of expensive functions: potentials and challenges. In: Hamadi, Y., Schoenauer, M. (eds.) LION 6. LNCS, vol. 7219, pp. 413–418. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  11. 11.
    Chevalier, C., Ginsbourger, D.: Fast computation of the multi-points expected improvement with applications in batch selection. In: Nicosia, G., Pardalos, P. (eds.) LION 7. LNCS, vol. 7997, pp. 59–69. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  12. 12.
    Hutter, F., Hoos, H.H., Leyton-Brown, K.: Parallel algorithm configuration. In: Hamadi, Y., Schoenauer, M. (eds.) LION 6. LNCS, vol. 7219, pp. 55–70. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  13. 13.
    Coello, C.A.C., Lamont, G.B., Van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd edn. Springer, New York (2007)zbMATHGoogle Scholar
  14. 14.
    Brockhoff, D., Friedrich, T., Hebbinghaus, N., Klein, C., Neumann, F., Zitzler, E.: Do additional objectives make a problem harder? In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO ’07, pp. 765–772. ACM (2007)Google Scholar
  15. 15.
    Tran, T.D., Brockhoff, D., Derbel, B.: Multiobjectivization with NSGA-II on the Noiseless BBOB Testbed. In: GECCO (Companion), Workshop on Black-Box Optimization Benchmarking (BBOB’2013), Amsterdam, Pays-Bas, July 2013Google Scholar
  16. 16.
    Ulrich, T., Thiele, L.: Maximizing population diversity in single-objective optimization. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GECCO ’11, pp. 641–648. ACM, New York (2011)Google Scholar
  17. 17.
    Wessing, S., Preuss, M., Rudolph, G.: Niching by multiobjectivization with neighbor information: Trade-offs and benefits. In: IEEE Congress on Evolutionary Computation (CEC), pp. 103–110 (2013)Google Scholar
  18. 18.
    Beume, N., Naujoks, B., Emmerich, M.: SMS-EMOA: Multiobjective selection based on dominated hypervolume. Eur. J. Oper. Res. 181(3), 1653–1669 (2007)CrossRefzbMATHGoogle Scholar
  19. 19.
    Hansen, N., Finck, S., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2009: noiseless functions definitions. Technical report RR-6829, INRIA (2009).
  20. 20.
    Mersmann, O., Bischl, B., Bossek, J., Judt, L.: soobench: Single Objective Optimization Benchmark Functions. R package version 1.1-164Google Scholar
  21. 21.
    Bischl, B.: mlr: Machine Learning in R. R package version 1.2Google Scholar
  22. 22.
    Conover, W.: Practical Nonparametric Statistics, 2nd edn. Wiley, New York (1980)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Bernd Bischl
    • 1
    Email author
  • Simon Wessing
    • 2
  • Nadja Bauer
    • 1
  • Klaus Friedrichs
    • 1
  • Claus Weihs
    • 1
  1. 1.Department of StatisticsTU DortmundDortmundGermany
  2. 2.Department of Computer ScienceTU DortmundDortmundGermany

Personalised recommendations