Computing 3-D Expected Hypervolume Improvement and Related Integrals in Asymptotically Optimal Time

Abstract

The Expected Hypervolume Improvement (EHVI) is a frequently used infill criterion in surrogate-assisted multi-criterion optimization. It needs to be frequently called during the execution of such algorithms. Despite recent advances in improving computational efficiency, its running time for three or more objectives has remained in $O(n^d)$ for $d\ge 3$, where d is the number of objective functions and n is the size of the incumbent Pareto-front approximation. This paper proposes a new integration scheme, which makes it possible to compute the EHVI in $\varTheta (n \log n)$ optimal time for the important three-objective case ($d=3$). The new scheme allows for a generalization to higher dimensions and for computing the Probability of Improvement (PoI) integral efficiently. It is shown, both theoretically and empirically, that the hidden constant in the asymptotic notation is small. Empirical speed comparisons were designed between the C++ implementations of the new algorithm (which will be in the public domain) and those recently published by competitors, on randomly-generated non-dominated fronts of size 10, 100, and 1000. The experiments include the analysis of batch computations, in which only the parameters of the probability distribution change but the incumbent Pareto-front approximation stays the same. Experimental results show that the new algorithm is always faster than the other algorithms, sometimes over $10^4$ times faster.

Appendix

Definition 4

( ${\varPsi _{\infty }}$ function (see also [14])). Let $\phi (s)= 1/\sqrt{2\pi }e^{-\frac{1}{2}s^2} (s\in \mathbb {R})$ denote the probability density function (PDF) of the standard normal distribution. Moreover, let $\varPhi (s)= \frac{1}{2}\left( 1 + \text {erf}\left( \frac{s}{\sqrt{2}}\right) \right)$ denote its cumulative probability distribution function (CDF), and $\text {erf}$ is Gaussian error function. The general normal distribution with mean $\mu$ and standard deviation $\sigma$ has as PDF, $\xi _{\mu ,\sigma }(s)=\phi _{\mu , \sigma }(s) = \frac{1}{\sigma }\phi (\frac{s-\mu }{\sigma })$ and its CDF is $\varPhi _{\mu , \sigma }(s) = \varPhi (\frac{s-\mu }{\sigma })$. Then the function $\varPsi _{\infty }(a,b,\mu ,\sigma )$ is defined as:

$$\begin{aligned} \varPsi _{\infty }(a,b,\mu ,\sigma )&= \mathop {\int }_{b}^{\infty }(z-a)\dfrac{1}{\sigma }\phi \left( \dfrac{z-\mu }{\sigma }\right) dz \\&=\sigma \phi \left( \dfrac{b-\mu }{\sigma }\right) + (\mu -a)\left[ 1-\varPhi \left( \dfrac{b-\mu }{\sigma }\right) \right] \end{aligned}$$