Hesitant adaptive search with estimation and quantile adaptive search for global optimization with noise

Abstract

Adaptive random search approaches have been shown to be effective for global optimization problems, where under certain conditions, the expected performance time increases only linearly with dimension. However, previous analyses assume that the objective function can be observed directly. We consider the case where the objective function must be estimated, often using a noisy function, as in simulation. We present a finite-time analysis of algorithm performance that combines estimation with a sampling distribution. We present a framework called Hesitant Adaptive Search with Estimation, and derive an upper bound on function evaluations that is cubic in dimension, under certain conditions. We extend the framework to Quantile Adaptive Search with Estimation, which focuses sampling points from a series of nested quantile level sets. The analyses suggest that computational effort is better expended on sampling improving points than refining estimates of objective function values during the progress of an adaptive search algorithm.

Appendices

Appendix A Proofs of Theorems for HAS-E Analysis

1.1 A.1 Proof of Theorem 1

Proof of Theorem 1

For any value $y_k$ such that $y_* + \epsilon < y_k \le y^*$, we start by defining an n-ball ${\mathcal {B}}_{y_k}$ as the largest n-ball centered at $x_*$ such that ${\mathcal {B}}_{y_k} \subseteq S_{y_k}$ and let $r_{y_k}$ be its radius. We note that $0 < \nu ({\mathcal {B}}_{y_k}) \le \nu (S_{y_k})$. For any value ${\hat{y}}_{k}^{high}$, we define ${\mathcal {B}}_{{\hat{y}}_{k}^{high}}$ as the smallest n-ball centered at $x_*$ such that $S_{{\hat{y}}_{k}^{high} } \subseteq {\mathcal {B}}_{{\hat{y}}_{k}^{high} }$ and let $r_{{\hat{y}}_{k}^{high}}$ be the radius of ${\mathcal {B}}_{{\hat{y}}_{k}^{high} }$.

We examine two cases. First, if ${\hat{y}}_k^{high} - y_{k} \le \kappa _{q}$ then $\frac{\nu (S_{y_{k}})}{\nu ( S_{{\hat{y}}_k^{high}})} > q$ by definition in (11), and the theorem is proved.

Second, consider ${\hat{y}}_k^{high} - y_{k} > \kappa _{q}$. We define ${\mathcal {K}}_{cone} = \frac{{\hat{y}}_{k}^{high} - y_k}{ r_{{\hat{y}}_{k}^{high}} - r_{y_k} }$, which can be interpreted as the slope that connects the two balls, see Fig. 5. We also write

$$\begin{aligned} r_{{\hat{y}}_{k}^{high}} = r_{y_k} + ({\hat{y}}_{k}^{high} - y_k) / {\mathcal {K}}_{cone}. \end{aligned}$$

Since ${\hat{y}}_k^{high} - y_{k} > \kappa _{q}$, the numerator of ${\mathcal {K}}_{cone}$ is greater than the numerator of ${\mathcal {K}}_{q}$ as in (11), and, since $d >r_{{\hat{y}}_{k}^{high}} - r_{y_k}$ by definition of the diameter, we have ${\mathcal {K}}_{cone} > {\mathcal {K}}_{q}$. Note that ${\mathcal {K}}_{q}$ is independent of the value $y_k$.

We define ${\mathcal {B}}^{large}$ as an n-ball centered at $x_*$ with radius $r_{large}$, where $r_{large} = r_{y_k} + ({\hat{y}}_{k}^{high} - y_k) / {\mathcal {K}}_{q}$. Here we see that $r_{{\hat{y}}_{k}^{high}} \le r_{large}$ since ${\mathcal {K}}_{q} \le {\mathcal {K}}_{cone}$. Therefore $S_{{\hat{y}}_{k}^{high} } \subset {\mathcal {B}}_{{\hat{y}}_{k}^{high} } \subset {\mathcal {B}}^{large}$, as illustrated in Fig. 5.

Fig. 5

An illustration of the largest n-ball inscribed in $S_{y_k}$ $({\mathcal {B}}_{y_k})$, the smallest n-ball inscribing $S_{{\hat{y}}_{k}^{high} }$ $({\mathcal {B}}_{{\hat{y}}_{k}^{high}})$, and a larger ball defined by the slope ${\mathcal {K}}_{q}$ (${\mathcal {B}}^{large}$)

A lower bound on the ratios of volumes is constructed in terms of the dimension n, using multi-dimensional geometry theorems [10],

$$\begin{aligned} \frac{\nu (S_{y_{k}})}{\nu ( S_{{\hat{y}}_k^{high}})} \ge \frac{\nu ({\mathcal {B}}_{y_k})}{\nu ({\mathcal {B}}_{{\hat{y}}_{k}^{high} })} \ge \frac{\nu ({\mathcal {B}}_{y_k})}{\nu ({\mathcal {B}}^{large})} = \left( \frac{r_{y_k} }{r_{y_k} + \frac{{\hat{y}}_k^{high} - y_k}{{\mathcal {K}}_{q}} } \right) ^n. \end{aligned}$$

Since ${\hat{y}}_k^{high} - y_k \le \frac{ 2 \cdot \sigma \cdot z_{\alpha /2}}{\sqrt{R}}$, as given in the theorem statement, we have the following lower bound,

$$\begin{aligned} \frac{\nu (S_{y_{k}})}{\nu ( S_{{\hat{y}}_k^{high}})} \ge \left( \frac{r_{y_k}}{r_{y_k} + \frac{ 2 \cdot \sigma \cdot z_{\alpha /2}}{{\mathcal {K}}_{q} \sqrt{R}} } \right) ^n. \end{aligned}$$

We want to determine R, such that

$$\begin{aligned} \left( \frac{r_{y_k}}{r_{y_k} + \frac{ 2 \cdot \sigma \cdot z_{\alpha /2}}{{\mathcal {K}}_{q} \sqrt{R}} } \right) ^n \ge q. \end{aligned}$$

Through several algebraic manipulations to isolate R, we determine that (13) holds if $R \ge \left( \frac{ \root n \of {q} \cdot 2 \cdot \sigma \cdot z_{\alpha /2} }{(1 - \root n \of {q}) \cdot r_{y_k} \cdot {\mathcal {K}}_{q} } \right) ^2$.

Finally, since $y_k \ge y_*+\epsilon$, then $r_{y_k} \ge r_{y_*+\epsilon }$, and therefore (13) holds if

$$\begin{aligned} R \ge \left( \frac{ \root n \of {q} \cdot 2 \cdot \sigma \cdot z_{\alpha /2} }{(1 - \root n \of {q}) \cdot r_{y_*+\epsilon } \cdot {\mathcal {K}}_{q} } \right) ^2 \end{aligned}$$

which proves Theorem 1. $\blacksquare$

1.2 A.2 Lemmas

The bound on replications, which is quadratic in dimension given in (14), uses a bound stated in Lemma 1. Theorems 2 and 4 make use of Lemma 30 from [23], which is repeated here for convenience as Lemma 2.

Lemma 1

For a given constant a such that $0< a < 1$, and a variable $n \ge 1$, then the function $f(n)= \frac{a^{1/n}}{1-a^{1/n}}$ is bounded by a linear function of n, that is,

$$\begin{aligned} f(n) = \frac{a^{1/n}}{1-a^{1/n}} \le \frac{a}{1-a} - (n-1) \frac{ln(a) }{\left( 1-a\right) ^2}. \end{aligned}$$

(29)

A proof of Lemma 1 is omitted. Other bounds are possible that are still linear in n.

The following lemma is used in the proof of Theorem 3.

Lemma 2

(cf. [23]) Let ${{\bar{Y}}_k^A, k = 0, 1, 2, \ldots }$ and ${{\bar{Y}}_k^B, k = 0, 1, 2,\ldots }$ be two sequences of objective function values generated by algorithms A and B respectively for solving a minimization problem, such that ${\bar{Y}}_{k+1}^A \le {\bar{Y}}_k^A$ and ${\bar{Y}}_{k+1}^B \le {\bar{Y}}_k^B$ for $k = 0, 1, \ldots$. For $y_* < y,z \le y^*$ and $k = 0,1, \ldots$, if

• 1. $P({\bar{Y}}_{k+1}^A \le y | {\bar{Y}}_k^A =z ) \ge P({\bar{Y}}_{k+1}^B\le y | {\bar{Y}}_k^B =z )$

• 2. $P({\bar{Y}}_{k+1}^A \le y | {\bar{Y}}_k^A =z )$ is non-increasing in z, and

• 3. $P({\bar{Y}}_{0}^A \le y ) \ge P({\bar{Y}}_{0}^B \le y )$

then $P(Y_{k}^A \le y ) \ge P(Y_{k}^B \le y)$ for $k=0,1, \ldots$ and $y_* < y \le y^*$.

A proof of Lemma 2 can be found in [23].

1.3 A.3 Proof of Theorem 2

Proof of Theorem 2:

Using the notation in HAS-E on the kth iteration, and based on Lemma 2, as in [23], if the following conditions hold for $y_* < y, {\bar{y}}_k \le y^*$ and $k = 0,1, \ldots$,

• (I) $P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_k^{HASE} ={\bar{y}}_k ) \ge P({\bar{Y}}_{k+1}^{HAS1} \le y | {\bar{Y}}_k^{HAS1} ={\bar{y}}_k )$

• (II) $P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_k^{HASE} ={\bar{y}}_k )$ is non-increasing in ${\bar{y}}_k$, and

• (III) $P({\bar{Y}}_{0}^{HASE} \le y) \ge P({\bar{Y}}_{0}^{HAS1} \le y )$

then $P({\bar{Y}}_k^{HASE} \le y ) \ge P({\bar{Y}}_k^{HAS1} \le y ) \text { for } k = 0,1, \ldots$ and for $y_*< y \le y^*$.

The first step is to prove (I). When $y \ge {\bar{y}}_k$, (I) is true trivially (since the conditional probability equals one on both sides). Now, when $y < {\bar{y}}_k$, we bound the left-hand side of the expression in (I), as,

$$\begin{aligned} \begin{aligned}&P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k ) \ge \gamma \cdot P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high}, {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k ) \end{aligned} \end{aligned}$$

(30)

where we condition on the event that HASE “betters”, that is, that HASE samples from the normalized restriction of $\zeta$ on $S_{{\bar{y}}_k^{high}}$, and consequently ${\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high}$, which occurs with probability at least $\gamma$ by the bound on the bettering probability in Assumption 1 (ii).

We next consider the event $\{ {\bar{y}}_k \le {\bar{y}}_k^{high} \}$, which occurs with probability at least $1-\alpha$ by (8). We rewrite (30) as,

$$\begin{aligned} \begin{aligned}&P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k )\\&\quad \ge \gamma \cdot (1-\alpha ) \cdot P({\bar{Y}}_{k+1}^{HASE} \le y | \{ {\bar{y}}_k \le {\bar{y}}_k^{high} \}, {\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high}) \end{aligned} \end{aligned}$$

(31)

dropping the condition ${\bar{Y}}_{k}^{HASE} = {\bar{y}}_k$ because it is captured in the other conditions.

To further develop the bound in terms of HAS1, we note that $P({\bar{Y}}_{k+1}^{HASE} \le y) \ge P({\bar{Y}}_{0}^{HASE} \le y) \ge P({\bar{Y}}_{0}^{HAS1} \le y)$ by Assumption 1 (i), and since $P({\bar{Y}}_{k+1}^{HASE} \le y) = P({\bar{Y}}_{k+1}^{HASE} \le y \bigcap {\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high})$ and $P({\bar{Y}}_{0}^{HAS1} \le y) = P({\bar{Y}}_{0}^{HAS1} \le y \bigcap {\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})$ we can write

$$\begin{aligned}&P({\bar{Y}}_{k+1}^{HASE} \le y \bigcap {\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high})\left( \frac{P({\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high})}{P({\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high}) } \right) \\&\qquad \ge P({\bar{Y}}_{0}^{HAS1} \le y \bigcap {\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})\left( \frac{ P({\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})}{P({\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})}\right) \end{aligned}$$

implying

$$\begin{aligned} \frac{P({\bar{Y}}_{k+1}^{HASE} \le y \bigcap {\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high})}{P({\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high}) } \ge \frac{ P({\bar{Y}}_{0}^{HAS1} \le y \bigcap {\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})}{P({\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})} \end{aligned}$$

and

$$\begin{aligned} P({\bar{Y}}_{k+1}^{HASE} \le y | \{ {\bar{y}}_k \le {\bar{y}}_k^{high} \}, {\bar{Y}}_{k+1}^{HASE} \le {\bar{y}}_k^{high}) \ge P({\bar{Y}}_{0}^{HAS1} \le y | \{ {\bar{y}}_k \le {\bar{y}}_k^{high} \}, {\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high}). \end{aligned}$$

We now rewrite (31) as

$$\begin{aligned} \begin{aligned}&P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k )\\&\quad \ge \gamma \cdot (1-\alpha ) \cdot P({\bar{Y}}_{0}^{HAS1} \le y | \{ {\bar{y}}_k \le {\bar{y}}_k^{high} \}, {\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high}). \end{aligned} \end{aligned}$$

(32)

Therefore, we can create a lower bound for (32):

$$\begin{aligned} P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k )&\ge \gamma \cdot (1-\alpha ) \cdot P({\bar{Y}}_{0}^{HAS1} \le y | {\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high}, \{ {\bar{y}}_k \le {\bar{y}}_k^{high} \}) \nonumber \\&= \gamma \cdot (1-\alpha ) \cdot \frac{P({\bar{Y}}_{0}^{HAS1} \le y)}{P({\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})} \nonumber \\&= \gamma \cdot (1-\alpha ) \cdot \frac{P({\bar{Y}}_{0}^{HAS1} \le y)}{P({\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k)} \cdot \frac{P({\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k)}{P({\bar{Y}}_{0}^{HAS1} \le {\bar{y}}_k^{high})} \nonumber \\&\ge \gamma \cdot (1-\alpha ) \cdot \frac{\nu (S_{y})}{\nu (S_{{\bar{y}}_k} )} \cdot \frac{\nu (S_{{\bar{y}}_k})}{\nu (S_{{\bar{y}}_k^{high}} )} \nonumber \\&\ge \gamma \cdot (1-\alpha ) \cdot \frac{\nu (S_{y})}{\nu (S_{{\bar{y}}_k} )} \cdot q . \end{aligned}$$

(33)

The last inequality makes use of the lower bound developed in Theorem 1, $\frac{\nu (S_{{\bar{y}}_k})}{ \nu (S_{{\bar{y}}_k^{high}} ) } \ge q$.

We similarly expand the expression for HAS1 in the right-hand side of (I), noting that HAS1 either improves or stays where it is, yielding,

$$\begin{aligned} P({\bar{Y}}_{k+1}^{HAS1} \le y | {\bar{Y}}_{k}^{HAS1} = {\bar{y}}_k ) = b^{HAS1}({\bar{y}}_k) P \left( {\bar{Y}}_{k+1}^{HAS1} \le y | {\bar{Y}}_{k}^{HAS1} = {\bar{y}}_k \right) \end{aligned}$$

where ${X}_{k+1}^{HAS1}$ is sampled according to the normalized restriction of the uniform distribution on the improving level set. Combining this with the bettering probability of HAS1, $b(y)=\gamma \cdot (1-\alpha ) \cdot q$, and when HAS1 “betters”, we have,

$$\begin{aligned}&P({\bar{Y}}_{k+1}^{HAS1} \le y | {\bar{Y}}_{k}^{HAS1} = {\bar{y}}_k ) = \gamma \cdot (1-\alpha ) \cdot q \cdot \frac{\nu (S_{y})}{\nu (S_{{\bar{y}}_k})} . \end{aligned}$$

(34)

Combining (33) and (34) proves condition (I).

We go on to prove (II), that $P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_k^{HASE} = {\bar{y}}_k )$ is non-increasing in ${\bar{y}}_k$. Suppose that ${\bar{y}}_k$ and ${\bar{y}}_k'$ are such that ${\bar{y}}_k < {\bar{y}}_k'$. To show (II) we want to show that:

$$\begin{aligned} P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k) \ge P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k'). \end{aligned}$$

The approach is to condition on the value of ${\bar{y}}_k^{high}$, and since HAS-E samples on $S_{{\bar{y}}_k^{high}}$ in Step 2 of the algorithm, we know that $P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_k^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u )$ is non-increasing, therefore, we have,

$$\begin{aligned}&P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k) \nonumber \\&\quad = \int _{ - \infty }^{\infty } P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = z ) \cdot dP({\bar{y}}_k^{high} \le z | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k ) \end{aligned}$$

and because $\int _{-\infty }^{z} dP\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u \right) =$ $P\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = z \right) - P\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = - \infty \right)$, and since $P\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = -\infty \right) = 1$ (trivially), we substitute $P \left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = z \right)$ as follows,

$$\begin{aligned}&\quad = \int _{-\infty }^{\infty } \left( 1 + \int _{-\infty }^{z} dP\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u \right) \right) \cdot dP\left( {\bar{y}}_k^{high} \le z | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k \right) \end{aligned}$$

and reversing the order of integration, we get

$$\begin{aligned}&\quad = 1 + \int _{-\infty }^{\infty } \int _{u}^{\infty } dP\left( {\bar{y}}_k^{high} \le z | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k \right) \cdot dP\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u \right) \\&\quad = 1 + \int _{-\infty }^{\infty } \left( 1 - P\left( {\bar{y}}_k^{high} \le u | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k \right) \right) \cdot dP({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u ). \end{aligned}$$

Now to show a lower bound in terms of ${\bar{y}}_k'$. However, since $dP\left( {\bar{Y}}_{k+1}^{HASE}\right.$$\left. \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u \right) \le 0$, and since $P\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u \right)$ is non-increasing in ${\bar{y}}_k^{high}$, and since, $P\left( {\bar{y}}_k^{high} \le u | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k \right) \ge P\left( {\bar{y}}_k^{high} \le u | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k' \right)$, the probability that ${\bar{y}}_k^{high}$ is lower than u is always greater for ${\bar{y}}_k < {\bar{y}}_k'$, then

$$\begin{aligned}{} & {} \quad 1 + \int _{-\infty }^{\infty }\left( 1 - P\left( {\bar{y}}_k^{high} \le u | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k \right) \right) \cdot dP\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u\right) \\{} & {} \quad \ge 1 + \int _{-\infty }^{\infty } \left( 1 - P\left( {\bar{y}}_k^{high} \le u | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k' \right) \right) \cdot dP\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u \right) \end{aligned}$$

which is equivalent to

$$\begin{aligned} = 1 + \int _{-\infty }^{\infty } \int _{u}^{\infty } \left( dP \left( {\bar{y}}_k^{high} \le z | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k' \right) \right) \cdot dP({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u ) \end{aligned}$$

and reversing the order of integration:

$$\begin{aligned}= & {} 1 + \int _{-\infty }^{\infty } \int _{-\infty }^{z} dP\left( {\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = u \right) \cdot \left( dP\left( {\bar{y}}_k^{high} \le z | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k' \right) \right) \\= & {} \int _{-\infty }^{\infty } P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = z ) \cdot dP({\bar{y}}_k^{high} \le z | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k' ) \end{aligned}$$

therefore, since $P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k, {\bar{y}}_k^{high} = z ) = P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k', {\bar{y}}_k^{high} = z )$, we write:

$$\begin{aligned}= & {} \int _{-\infty }^{\infty } P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k', {\bar{y}}_k^{high} = z ) \cdot dP({\bar{y}}_k^{high} \le z | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k' )\\= & {} P({\bar{Y}}_{k+1}^{HASE} \le y | {\bar{Y}}_{k}^{HASE} = {\bar{y}}_k') \end{aligned}$$

which proves (II).

Lastly, condition (III) from Lemma 2 is true by Assumption 1 (i) that $P({\bar{Y}}_0^{HASE}\le y) \ge P({\bar{Y}}_0^{HAS1} \le y)$. This proves the theorem through reference to Lemma 2. $\blacksquare$

1.4 A.4 Proof of Theorem 3

Proof of Theorem 3:

By stochastic dominance in Theorem 2, the expected number of iterations to achieve a value within $S_{y_* + \epsilon }$ for HAS-E is less than or equal to the number for HAS1. Since the bettering probability for HAS1 is $b(y) = \gamma \cdot ( 1- \alpha ) \cdot q$ for all $y_* < y \le y^*$, using (4), we have

$$\begin{aligned}&E[N^{HASE}_I(y_* +\epsilon )] \le 1 + \int _{y_* + \epsilon }^{\infty } \frac{d\rho (t)}{\gamma \cdot ( 1 - \alpha ) \cdot q \cdot p(t)} \end{aligned}$$

and since HAS1 uses uniform sampling, i.e., $p(y) = \frac{\nu (S_y)}{\nu (S)}$, we have

$$\begin{aligned}&\quad = 1 + \frac{1}{\gamma \cdot (1-\alpha ) \cdot q } \cdot ln \left( \frac{\nu (S)}{\nu (S_{y_* + \epsilon })} \right) . \end{aligned}$$

Using a constant number of replications R for each iteration, yields

$$\begin{aligned} E[N^{HASE}_R(y_* +\epsilon )] = R \cdot E[N^{HASE}_I(y_* +\epsilon )]. \end{aligned}$$

Setting $R_k=R$ as in (16) yields the result in (18). $\blacksquare$

Appendix B Proofs of Theorems for QAS-E Analysis

The proofs of the theorems for QAS-E are similar to the proofs for HAS-E. The proof of Theorem 4 for QAS-E is provided for completeness.

Proof of Theorem 4

Similar to the proof of Theorem 2, if the three conditions listed in Lemma 2 hold, then $P({\bar{Y}}_k^{QASE} \le y ) \ge P({\bar{Y}}_k^{HAS2} \le y ) \text { for } k = 0,1, \ldots$ and for $y_*< y \le y^*$.

We start by proving the first condition in Lemma 2, that is, we show that

$$\begin{aligned} P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_k^{QASE} ={\bar{y}}_k ) \ge P({\bar{Y}}_{k+1}^{HAS2} \le y | {\bar{Y}}_k^{HAS2} ={\bar{y}}_k ) \end{aligned}$$

for $y_* < y, {\bar{y}}_k \le y^*$ and $k = 0,1, \ldots$. When $y \ge {\bar{y}}_k$, $P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_k^{QASE} ={\bar{y}}_k ) = P({\bar{Y}}_{k+1}^{HAS2} \le y | {\bar{Y}}_k^{QASE} ={\bar{y}}_k ) =1$, and the first condition holds.

Now, when $y < {\bar{y}}_k$, we bound the left-hand side of the expression in (I) by conditioning on the event that $X_{k+1}^{QASE}$ “betters”, that is, the event that ${\bar{Y}}_{k+1}^{QASE} \le {\bar{y}}_k^{high}$, yielding

$$\begin{aligned} P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_k^{QASE} ={\bar{y}}_k )&\ge \gamma \cdot P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_{k+1}^{QASE} \le {\bar{y}}_k^{high}, {\bar{Y}}_k^{QASE} ={\bar{y}}_k ) \end{aligned}$$

(35)

by Assumption 2 (ii).

We next consider the event $\{{\bar{y}}_k \le {\bar{y}}_k^{high}\}$, which occurs with probability at least $1-\alpha$, by (8). We rewrite (35) as,

$$\begin{aligned} \begin{aligned}&P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_{k}^{QASE} = {\bar{y}}_k )\\&\quad \ge \gamma \cdot (1-\alpha ) \cdot P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_{k+1}^{QASE} \le {\bar{y}}_k^{high}, \{{\bar{y}}_k \le {\bar{y}}_k^{high}\}, {\bar{Y}}_k^{QASE} ={\bar{y}}_k ). \end{aligned} \end{aligned}$$

(36)

From Assumption 2 (i), we have

$$\begin{aligned} P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_{k}^{QASE} = {\bar{y}}_k )&\ge \gamma \cdot (1-\alpha ) \cdot P({\bar{Y}}_{0}^{HAS2} \le y | {\bar{Y}}_{0}^{HAS2} \le {\bar{y}}_k^{high}). \end{aligned}$$

Making use of the lower bound developed in Theorem 1, $\frac{\nu (S_{{\bar{y}}_k})}{ \nu (S_{{\bar{y}}_k^{high}} ) } \ge q$, we have,

$$\begin{aligned} P({\bar{Y}}_{k+1}^{QASE} \le y | {\bar{Y}}_{k}^{QASE} = {\bar{y}}_k )&\ge \gamma \cdot (1-\alpha ) \cdot P({\bar{Y}}_{0}^{HAS2} \le y | {\bar{Y}}_{0}^{HAS2} \le {\bar{y}}_k^{high}) \nonumber \\&= \gamma \cdot (1-\alpha ) \cdot \frac{P({\bar{Y}}_{0}^{HAS2} \le y)}{P({\bar{Y}}_{0}^{HAS2} \le {\bar{y}}_k^{high})} \nonumber \\&= \gamma \cdot (1-\alpha ) \cdot \frac{P({\bar{Y}}_{0}^{HAS2} \le y)}{P({\bar{Y}}_{0}^{HAS2} \le {\bar{y}}_k)} \cdot \frac{P({\bar{Y}}_{0}^{HAS2} \le {\bar{y}}_k)}{P({\bar{Y}}_{0}^{HAS2} \le {\bar{y}}_k^{high})} \nonumber \\&\ge \gamma \cdot (1-\alpha ) \cdot \frac{\nu (S_{y})}{\nu (S_{{\bar{y}}_k} )} \cdot \frac{\nu (S_{{\bar{y}}_k})}{\nu (S_{{\bar{y}}_k^{high}} )} \nonumber \\&\ge \gamma \cdot (1-\alpha ) \cdot \frac{\nu (S_{y})}{\nu (S_{{\bar{y}}_k} )} \cdot q . \end{aligned}$$

(37)

We similarly expand the expression for HAS2 in the right-hand side of (I), noting that HAS2 either improves or stays where it is, yielding,

$$\begin{aligned} P({\bar{Y}}_{k+1}^{HAS2} \le y | {\bar{Y}}_{k}^{HAS2} = {\bar{y}}_k ) = b^{HAS2}({\bar{y}}_k) P \left( {\bar{Y}}_{k+1}^{HAS2} \le y | {\bar{Y}}_{k}^{HAS2} = {\bar{y}}_k \right) \end{aligned}$$

and since the bettering probability of HAS2 equals $\gamma \cdot (1-\alpha ) \cdot q$, and when HAS2 “betters”, it samples uniformly on the improving level set, we have,

$$\begin{aligned}&P({\bar{Y}}_{k+1}^{HAS2} \le y | {\bar{Y}}_{k}^{HAS2} = {\bar{y}}_k ) = \gamma \cdot (1-\alpha ) \cdot q \cdot \frac{\nu (S_{y})}{\nu (S_{{\bar{y}}_k})} . \end{aligned}$$

(38)

Combining (37) and (38) proves condition (I).

The second condition in Lemma 2 is satisfied directly by Assumption 2 (iii). The third condition in Lemma 2 is satisfied by Assumption 2 (i). This proves the theorem by Lemma 2. $\blacksquare$

Appendix C Details for Sample Problem

The one-dimensional sample problem f(x), illustrated in Fig. 3, is defined for $x\in [-4, 4]$. The calculations use the following parameter settings: $\sigma = 1$, $\alpha =0.05$, $\gamma =0.5$, and $\epsilon =0.3$. The values for ${\mathcal {K}}_{q}$ are calculated numerically for a range of values of q, and are shown in Fig. 6.

Fig. 6

Values for ${\mathcal {K}}_{q}$ for the one-dimensional sample problem for $q \in [0.3, 0.7]$