Faster Exact Algorithms for Computing Expected Hypervolume Improvement

  • Iris Hupkens
  • André Deutz
  • Kaifeng Yang
  • Michael Emmerich
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9019)

Abstract

This paper is about computing the expected improvement of the hypervolume indicator given a Pareto front approximation and a predictive multivariate Gaussian distribution of a new candidate point. It is frequently used as an infill or prescreening criterion in multiobjective optimization with expensive function evaluations where predictions are provided by Kriging or Gaussian process surrogate models. The expected hypervolume improvement has good properties as an infill criterion, but exact algorithms for its computation have so far been very time consuming even for the two and three objective case. This paper introduces faster exact algorithms for computing the expected hypervolume improvement for independent Gaussian distributions. A new general computation scheme is introduced and a lower bound for the time complexity. By providing new algorithms, upper bounds for the time complexity for problems with two as well as three objectives are improved. For the 2-D case the time complexity bound is reduced from previously \(O(n^3 \log n)\) to \(O(n^2)\). For the 3-D case the new upper bound of \(O(n^3)\) is established; previously \(O(n^4 \log n)\). It is also shown how an efficient implementation of these new algorithms can lead to a further reduction of running time. Moreover it is shown how to process batches of multiple predictive distributions efficiently. The theoretical analysis is complemented by empirical speed comparisons of C++ implementations of the new algorithms to existing implementations of other exact algorithms.

Keywords

Expected improvement Time complexity Global multiobjective optimization Hypervolume indicator Kriging surrogate models 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hupkens, I.: Complexity Reduction and Validation of Computing the Expected Hypervolume Improvement, Master’s Thesis, published as LIACS, Internal Report Nr. 2013–12, August 2013. http://www.liacs.nl/assets/Masterscripties/2013-12IHupkens.pdf
  2. 2.
    Shir, O.M., Emmerich, M., Bäck, T., Vrakking, M.J.: The application of evolutionary multi-criteria optimization to dynamic molecular alignment. In: Proc. of IEEE CEC 2007, pp. 4108–4115 (2007)Google Scholar
  3. 3.
    Zaefferer, M., Bartz-Beielstein, T., Naujoks, B., Wagner, T., Emmerich, M.: A case study on multi-criteria optimization of an event detection software under limited budgets. In: Purshouse, R.C., Fleming, P.J., Fonseca, C.M., Greco, S., Shaw, J. (eds.) EMO 2013. LNCS, vol. 7811, pp. 756–770. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  4. 4.
    Shimoyama, K., Sato, K., Jeong, S., Obayashi, S.: Comparison of the criteria for updating Kriging response surface models in multi-objective optimization. In: Proc. of IEEE CEC 2012, pp. 1–8 (2012)Google Scholar
  5. 5.
    Shimoyama, K., Jeong, S., Obayashi, S.: Kriging-surrogate-based optimization considering expected hypervolume improvement in non-constrained many-objective test problems. In: Proc. of IEEE CEC 2013, pp. 658–665 (2013)Google Scholar
  6. 6.
    Couckuyt, I., Dirk, D., Tom, D.: Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. Journal of Global Optimization 60, 575–594 (2013)CrossRefGoogle Scholar
  7. 7.
    Wagner, T., Emmerich, M., Deutz, A., Ponweiser, W.: On expected-improvement criteria for model-based multi-objective optimization. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 718–727. Springer, Heidelberg (2010) Google Scholar
  8. 8.
    Emmerich, M., Deutz, A.H., Klinkenberg, J.W: Hypervolume-based expected improvement: Monotonicity properties and exact computation. In: Proc. of IEEE CEC 2011, pp. 2147–2154 (2011)Google Scholar
  9. 9.
    Beume, N., Fonseca, C.M., Lo’pez-Iba’ez, M., Paquete, L.: On the complexity of computing the hypervolume indicator. IEEE Transactions on Evolutionary Computation 13(5), 1075–1082 (2009)CrossRefGoogle Scholar
  10. 10.
    Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. In: Dixon, L., Szego, G. (eds.) Towards Global Optimization (1978). vol. 2, pp. 117–129 (1978)Google Scholar
  11. 11.
    Donald, R.J., Matthias, S., William, J.W.: Efficient Global Optimization of Expensive Black-Box Functions. Journal of Global Optimization 13(4), 455–492 (1998)CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Emmanuel, V., Julien, B.: Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. Journal of Statistical Planning and Inference 140, 3088–3095 (2010)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Fubini, G.: Sugli integrali multipli. Opere scelte, vol. 2. Cremonese, pp. 243–249Google Scholar
  14. 14.
    Emmerich, M. (2005). Single-and multi-objective evolutionary design optimization assisted by gaussian random field metamodels. Dissertation, TU Dortmund, Informatik, Eldorado. http://hdl.handle.net/2003/21807
  15. 15.
    Zitzler, E., Thiele, L.: Multiobjective optimization using evolutionary algorithms - A comparative case study. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H.-P. (eds.) PPSN 1998. LNCS, vol. 1498, pp. 292–301. Springer, Heidelberg (1998) CrossRefGoogle Scholar
  16. 16.
    Laniewski-Wollk, P., Obayashi S., Jeong, S.: Development of expected improvement for multi-objective problems, In: Proceedings of 42nd Fluid Dynamics Conference/Aerospace Numerical, Simulation Symposium (CD ROM), June 2010Google Scholar
  17. 17.
    Koch, P., Wagner T., Emmerich M., Bäck Th., Konen W.: Efficient Multi-criteria Optimization on Noisy Machine Learning Problems, Applied Soft Computing, (in print) (2014)Google Scholar
  18. 18.
    Luo, C., Shimoyama, K., Obayashi, S.: Kriging model based many-objective optimization with efficient calculation of expected hypervolume improvement. In: Proc. of IEEE CEC 2014, pp. 1187–1194 (2014)Google Scholar
  19. 19.
    Hupkens, I., Emmerich, M. and Deutz, A.: Faster Computation of Expected Hypervolume Improvement. arXiv preprint arXiv: 1408.7114 (2014)
  20. 20.
    Emmerich, M.T.M., Fonseca, C.M.: Computing hypervolume contributions in low dimensions: Asymptotically optimal algorithm and complexity results. In: Takahashi, R.H.C., Deb, K., Wanner, E.F., Greco, S. (eds.) EMO 2011. LNCS, vol. 6576, pp. 121–135. Springer, Heidelberg (2011) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Iris Hupkens
    • 1
  • André Deutz
    • 1
  • Kaifeng Yang
    • 1
  • Michael Emmerich
    • 1
  1. 1.LIACSLeiden University LeidenThe Netherlands

Personalised recommendations