Analysis of Surrogate-Assisted Information-Geometric Optimization Algorithms

Abstract

Surrogate functions are often employed to reduce the number of objective function evaluations in a continuous optimization. However, their effects have seldom been investigated theoretically. This paper analyzes the effect of a surrogate function in the information-geometric optimization (IGO) framework, which includes as an algorithm instance a variant of the covariance matrix adaptation evolution strategy—a widely used solver for black-box continuous optimization. We derive a sufficient condition on the surrogate function for the parameter update in the IGO algorithms to point to a descent direction of the objective function expected over the search distribution. The condition is expressed in terms of three measures of correlation between the objective function and the surrogate function. Our result constitutes a partial justification for the use of a surrogate function in IGO algorithms.

A Proofs

1.1 A.1 Proof of Proposition 18

Proof

Let $\Delta ^{(t)}_g = (\Delta _m, {{\,\textrm{vec}\,}}(\Delta _\Sigma ))$. We have

$$\begin{aligned} \psi (\alpha \Delta ^{(t)}_g{;} \theta ^{(t)}) = J(\theta ^{(t)}+ \alpha \Delta ^{(t)}_g) - J(\theta ^{(t)}) - \alpha \nabla J(\theta ^{(t)})^\textrm{T}\Delta ^{(t)}_g = \frac{\alpha ^2}{2} \Delta _m^\textrm{T}A \Delta _{m}. \end{aligned}$$

(77)

Noting that $\mathbb {E}[g(X_\theta )] = {\tilde{\nu }}^\textrm{T}\mathbb {E}[s(\theta {;} X_\theta )] = 0$, we have

$$\begin{aligned} {{\,\textrm{Var}\,}}[g(X_\theta )]\mathbb {E}[ \Delta _m^\textrm{T}A \Delta _{m}]= & {} \frac{1}{\lambda } \mathbb {E}\left[ g(X_\theta )^2 (X_\theta - m)^\textrm{T}A (X_\theta - m) \right] \nonumber \\{} & {} + \frac{\lambda - 1}{\lambda } \mathbb {E}\left[ g(X_\theta ) (X_\theta - m) \right] ^\textrm{T}A \mathbb {E}\left[ g(X_\theta ) (X_\theta - m) \right] ,\nonumber \\ \end{aligned}$$

(78)

where

$$\begin{aligned} \mathbb {E}\left[ g(X_\theta ) (X_\theta - m) \right] = \mathbb {E}[(X_\theta -m)(X_\theta -m)^\textrm{T}\Sigma ^{-1} {\tilde{\nu }}_m] = {\tilde{\nu }}_m. \end{aligned}$$

(79)

Let $H_1 = \Sigma ^{-1/2} {\tilde{\nu }}_m {\tilde{\nu }}_m^\textrm{T}\Sigma ^{-1/2}$, $H_2 = \Sigma ^{-1/2} {\tilde{\nu }}_\Sigma \Sigma ^{-1/2}$, and $H_3 = \Sigma ^{1/2} A \Sigma ^{1/2}$. Let $Z = \Sigma ^{-1/2}(X_\theta - m)$. Then Z follows a standard normal distribution, and we have

$$\begin{aligned}&\mathbb {E}\left[ g(X_\theta )^2 (X_\theta - m)^\textrm{T}A (X_\theta - m) \right] \end{aligned}$$

(80a)

$$\begin{aligned}&\quad =\ \mathbb {E}[(Z^\textrm{T}H_1 Z) (Z^\textrm{T}H_3 Z)] \end{aligned}$$

(80b)

$$\begin{aligned}&\qquad + \frac{1}{4}\mathbb {E}[(Z^\textrm{T}H_2 Z)^2 (Z^\textrm{T}H_3 Z)] \end{aligned}$$

(80c)

$$\begin{aligned}&\qquad - \frac{1}{2}{{\,\textrm{Tr}\,}}({\tilde{\nu }}_\Sigma \Sigma ^{-1})\mathbb {E}[(Z^\textrm{T}H_2 Z) (Z^\textrm{T}H_3 Z)] \end{aligned}$$

(80d)

$$\begin{aligned}&\qquad + \frac{1}{4}{{\,\textrm{Tr}\,}}({\tilde{\nu }}_\Sigma \Sigma ^{-1})^2\mathbb {E}[(Z^\textrm{T}H_3 Z)]. \end{aligned}$$

(80e)

By using the formula for the expectation of the product of quadratic forms of Gaussian random vectors [48, Theorem 5.1], we obtain

$$\begin{aligned} (80b)&= {{\,\textrm{Tr}\,}}(H_1){{\,\textrm{Tr}\,}}(H_3) + 2{{\,\textrm{Tr}\,}}(H_1 H_3), \end{aligned}$$

(81a)

$$\begin{aligned} (80c)&= \frac{1}{4} ({{\,\textrm{Tr}\,}}(H_2)^2 {{\,\textrm{Tr}\,}}(H_3) + 2{{\,\textrm{Tr}\,}}(H_2^2){{\,\textrm{Tr}\,}}(H_3) \nonumber \\&\quad + 4{{\,\textrm{Tr}\,}}(H_2 H_3) {{\,\textrm{Tr}\,}}(H_2) + 8 {{\,\textrm{Tr}\,}}(H_2^2 H_3) ), \end{aligned}$$

(81b)

$$\begin{aligned} (80d)&= - \frac{1}{2}{{\,\textrm{Tr}\,}}(H_2)({{\,\textrm{Tr}\,}}(H_2){{\,\textrm{Tr}\,}}(H_3) + 2{{\,\textrm{Tr}\,}}(H_2 H_3)), \end{aligned}$$

(81c)

$$\begin{aligned} (80e)&= \frac{1}{4}{{\,\textrm{Tr}\,}}(H_2)^2 {{\,\textrm{Tr}\,}}(H_3). \end{aligned}$$

(81d)

Rearranging these terms, we obtain

$$\begin{aligned}&\mathbb {E}\left[ g(X_\theta )^2 (X_\theta - m)^\textrm{T}A (X_\theta - m) \right] \end{aligned}$$

(82a)

$$\begin{aligned}&= {{\,\textrm{Tr}\,}}(H_3) ({{\,\textrm{Tr}\,}}(H_1) + \frac{1}{2}{{\,\textrm{Tr}\,}}(H_2^2)) + 2 {{\,\textrm{Tr}\,}}(H_1 H_3) + 2 {{\,\textrm{Tr}\,}}(H_2^2 H_3) \end{aligned}$$

(82b)

$$\begin{aligned}&= {{\,\textrm{Var}\,}}[g(X_\theta )] {{\,\textrm{Tr}\,}}(A \Sigma ) + 2{\tilde{\nu }}_m^\textrm{T}A {\tilde{\nu }}_m + 2 {{\,\textrm{Tr}\,}}({\tilde{\nu }}_\Sigma \Sigma ^{-1} {\tilde{\nu }}_\Sigma A), \end{aligned}$$

(82c)

for which we used the fact that ${{\,\textrm{Var}\,}}[g(X_\theta )] = {{\,\textrm{Tr}\,}}(H_1) + \frac{1}{2}{{\,\textrm{Tr}\,}}(H_2^2)$. Finally, from (78), we obtain

$$\begin{aligned} \mathbb {E}[ \Delta _m^\textrm{T}A \Delta _{m}] = \frac{1}{\lambda } {{\,\textrm{Tr}\,}}(A \Sigma ) + \frac{1}{\lambda }\frac{(\lambda + 1){\tilde{\nu }}_m^\textrm{T}A {\tilde{\nu }}_m + 2 {{\,\textrm{Tr}\,}}({\tilde{\nu }}_\Sigma \Sigma ^{-1} {\tilde{\nu }}_\Sigma A)}{{\tilde{\nu }}_m^\textrm{T}\Sigma ^{-1} {\tilde{\nu }}_m + \frac{1}{2} {{\,\textrm{Tr}\,}}({\tilde{\nu }}_\Sigma \Sigma ^{-1} {\tilde{\nu }}_\Sigma \Sigma ^{-1})}. \end{aligned}$$

(83)

In light of Proposition 5 and the fact that $\nabla J(\theta )^\textrm{T}{\mathcal {I}}(\theta )^{-1} \nabla J(\theta ) = {{\,\textrm{Var}\,}}[f(X_\theta )] = \nu (\theta )^\textrm{T}{\mathcal {I}}(\theta ) \nu (\theta ) = (m - x^*)^\textrm{T}A \Sigma A (m - x^*) + \frac{1}{2} {{\,\textrm{Tr}\,}}(A\Sigma A\Sigma )$, we have

$$\begin{aligned} \nabla J(\theta ^{(t)})^\textrm{T}\mathbb {E}_t[\Delta ^{(t)}_f] = - \left( (m - x^*)^\textrm{T}A \Sigma A (m - x^*) + \frac{1}{2} {{\,\textrm{Tr}\,}}(A\Sigma A\Sigma )\right) ^{1/2}. \end{aligned}$$

(84)

From (83) and (84), we obtain (71a).

Inequality (71b) is derived as follows. Using the inequalities $\frac{{\tilde{\nu }}_m^\textrm{T}A {\tilde{\nu }}_m}{{\tilde{\nu }}_m^\textrm{T}\Sigma ^{-1} {\tilde{\nu }}_m} \le \sigma _{\max }(A \Sigma )$ and $\frac{ {{\,\textrm{Tr}\,}}({\tilde{\nu }}_\Sigma \Sigma ^{-1} {\tilde{\nu }}_\Sigma A) }{ {{\,\textrm{Tr}\,}}({\tilde{\nu }}_\Sigma \Sigma ^{-1} {\tilde{\nu }}_\Sigma \Sigma ^{-1}) } \le \sigma _{\max }(A \Sigma )$, where $\sigma _{\max }(A \Sigma )$ denotes the greatest singular value of $A \Sigma$, we obtain

$$\begin{aligned} \frac{{{\bar{\alpha }}}(\theta {;} \alpha )}{\alpha / 2} \le \frac{\frac{1}{\lambda } {{\,\textrm{Tr}\,}}(A \Sigma ) + \frac{\lambda + 5}{\lambda } \sigma _{\max }(A \Sigma ) }{ ( (m - x^*)^\textrm{T}A \Sigma A (m - x^*) + \frac{1}{2} {{\,\textrm{Tr}\,}}(A\Sigma A\Sigma ))^{1/2} }. \end{aligned}$$

(85)

Moreover, by using $\frac{{{\,\textrm{Tr}\,}}(A \Sigma )}{{{\,\textrm{Tr}\,}}(A \Sigma A \Sigma )^{1/2}} \le d^{1/2}$ and $\frac{\sigma _{\max }(A \Sigma )}{{{\,\textrm{Tr}\,}}(A \Sigma A \Sigma )^{1/2}} \le 1$, we have

$$\begin{aligned} \frac{{{\bar{\alpha }}}(\theta {;} \alpha )}{\alpha / 2} \le 2^{1/2} \left( \frac{2 d^{1/2}}{\lambda } + \frac{\lambda + 5}{\lambda } \right) . \end{aligned}$$

(86)

This completes the proof. $\square$

1.2 A.2 Proof of Lemma 20

Proof

Note that $\frac{\mathbb {E}[ (\nu (\theta ) s(\theta {;} X_\theta ))^4 ]}{ (\nu (\theta )^\textrm{T}{\mathcal {I}}(\theta ) \nu (\theta ))^2 } = \frac{\mathbb {E}[(f(X_\theta ) - \mathbb {E}[f(X_\theta )])^4]}{\mathbb {E}[(f(X_\theta ) - \mathbb {E}[f(X_\theta )])^2]}$. We bound this ratio as follows. Let the eigenvalue decomposition of $\sqrt{\Sigma } A \sqrt{\Sigma }$ be denoted by $E D E^\textrm{T}$, where D is the diagonal matrix composed of the eigenvalues of $\sqrt{\Sigma } A \sqrt{\Sigma }$, and E is the orthogonal matrix composed of the unit eigenvectors of $\sqrt{\Sigma } A \sqrt{\Sigma }$. Let $Z = E^\textrm{T}\sqrt{\Sigma }^{-1}(X_\theta - m)$ and $v = E^\textrm{T}\sqrt{\Sigma } A (m - x^*)$. Then by a simple derivation, we have

$$\begin{aligned}&f(Z_\theta ) - E[f(X_\theta )] \end{aligned}$$

(87a)

$$\begin{aligned}&\quad = \frac{1}{2} (X_\theta - m)^\textrm{T}A (X_\theta - m) - \frac{1}{2} {{\,\textrm{Tr}\,}}(A \Sigma ) + (m - x^*)^\textrm{T}A (X_\theta - m) \end{aligned}$$

(87b)

$$\begin{aligned}&\quad = \frac{1}{2} Z^\textrm{T}E^\textrm{T}\sqrt{\Sigma } A \sqrt{\Sigma } E Z - \frac{1}{2} {{\,\textrm{Tr}\,}}( \sqrt{\Sigma } A \sqrt{\Sigma }) + v^\textrm{T}E^\textrm{T}\sqrt{\Sigma }^{-1} A^{-1} A \sqrt{\Sigma } E Z \end{aligned}$$

(87c)

$$\begin{aligned}&\quad = \frac{1}{2} Z^\textrm{T}D Z - \frac{1}{2} {{\,\textrm{Tr}\,}}( D ) + v^\textrm{T}Z \end{aligned}$$

(87d)

$$\begin{aligned}&\quad = \frac{1}{2} \sum _{i=1}^{d} \left( d_i (Z_i^2 - 1) + 2 v_i Z_i \right) . \end{aligned}$$

(87e)

Let

$$\begin{aligned} \mu _{i,2} = \mathbb {E}\left[ \left( d_i (Z_i^2 - 1) + 2 v_i Z_i \right) ^2 \right]&= 2 d_i^2 + 4 v_i^2, \end{aligned}$$

(88a)

$$\begin{aligned} \mu _{i,4} = \mathbb {E}\left[ \left( d_i (Z_i^2 - 1) + 2 v_i Z_i \right) ^4 \right]&= 60 d_i^4 + 240 d_i^2 v_i^2 + 48 v_i^4. \end{aligned}$$

(88b)

Note that the $Z_i$ values are independent and follow a standard normal distribution. A simple derivation leads to

$$\begin{aligned} \mathbb {E}[(f(X_\theta ) - \mathbb {E}[f(X_\theta )])^2]&= \frac{1}{4} \sum _{i=1}^{d} \mu _{i,2}, \end{aligned}$$

(89a)

$$\begin{aligned} \mathbb {E}[(f(X_\theta ) - \mathbb {E}[f(X_\theta )])^4]&= \frac{1}{16}\sum _{i=1}^{d}\left( \mu _{i,4} - 3 \mu _{i,2}^2 \right) + \frac{3}{16}\left( \sum _{i=1}^d \mu _{i,2} \right) ^2 . \end{aligned}$$

(89b)

Using $\mu _{i,4} - 3 \mu _{i,2}^2 \le 12 \mu _{i,2}^2$, we obtain

$$\begin{aligned} \frac{\mathbb {E}[(f(X_\theta ) - \mathbb {E}[f(X_\theta )])^4]}{\mathbb {E}[(f(X_\theta ) - \mathbb {E}[f(X_\theta )])^2]^2}&= \frac{\sum _{i=1}^{d}\left( \mu _{i,4} - 3 \mu _{i,2}^2 \right) }{ \left( \sum _{i=1}^d \mu _{i,2} \right) ^2 } + 3 \end{aligned}$$

(90a)

$$\begin{aligned}&\le \frac{12 \sum _{i=1}^{d} \mu _{i,2}^2}{ \left( \sum _{i=1}^d \mu _{i,2} \right) ^2 } + 3 \le 15. \end{aligned}$$

(90b)

Therefore, $N_s \le 15$ for all $\theta \in \Theta$. This completes the proof. $\square$

1.3 A.3 Proof of Lemma 21

Proof

Note that $\lambda \left| {\hat{W}}_i\right| \le N_w$ with probability one for all $i = 1, \dots , \lambda$. Then

$$\begin{aligned} \mathbb {E}_t\left[ \Delta _m^\textrm{T}A \Delta _m \right]&= \sum _{i=1}^{\lambda }\sum _{j=1}^{\lambda } \mathbb {E}\left[ {\hat{W}}_i{\hat{W}}_j (X_i - m)^\textrm{T}A (X_j - m) \right] \end{aligned}$$

(91a)

$$\begin{aligned}&\le \sum _{i=1}^{\lambda }\sum _{j=1}^{\lambda } \mathbb {E}\left[ \left| {\hat{W}}_i\right| \left| {\hat{W}}_j\right| \left| (X_i - m)^\textrm{T}A (X_j - m)\right| \right] \end{aligned}$$

(91b)

$$\begin{aligned}&\le \frac{N_w^2}{\lambda ^2} \sum _{i=1}^{\lambda }\sum _{j=1}^{\lambda } \mathbb {E}\left[ \left| (X_i - m)^\textrm{T}A (X_j - m)\right| \right] \end{aligned}$$

(91c)

$$\begin{aligned}&= \frac{N_w^2}{\lambda ^2} \sum _{i=1}^{\lambda }\sum _{j \ne i} \mathbb {E}\left[ \left| (X_i - m)^\textrm{T}A (X_j - m)\right| \right] \end{aligned}$$

(91d)

$$\begin{aligned}&\quad + \frac{N_w^2}{\lambda ^2} \sum _{i=1}^{\lambda } \mathbb {E}\left[ (X_i - m)^\textrm{T}A (X_i - m) \right] \end{aligned}$$

(91e)

$$\begin{aligned}&\le \left( 1 - \frac{1}{\lambda }\right) N_w^2 {{\,\textrm{Tr}\,}}((A \Sigma )^2)^{1/2} + \frac{1}{\lambda }N_w^2 {{\,\textrm{Tr}\,}}\left( A \Sigma \right) . \end{aligned}$$

(91f)

To obtain the last inequality, we applied the Cauchy–Schwarz inequality to the first term, and we used the fact that $\mathbb {E}\left[ (X_\theta - m)^\textrm{T}A (X_\theta - m) \right] = {{\,\textrm{Tr}\,}}(\Sigma A )$. In light of (77), we obtain (73).

In light of Proposition 16, Lemma 20, and the fact that $\nu (\theta )^\textrm{T}{\mathcal {I}}(\theta ) \nu (\theta ) = (m - x^*)^\textrm{T}A \Sigma A (m - x^*) + \frac{1}{2} {{\,\textrm{Tr}\,}}(A\Sigma A\Sigma )$, we have

$$\begin{aligned}{} & {} \nabla J(\theta ^{(t)})^\textrm{T}\mathbb {E}_t[\Delta ^{(t)}_f] \nonumber \\{} & {} \quad \le - \frac{M_w}{3\cdot 2^{1/2}} \left( (m - x^*)^\textrm{T}A \Sigma A (m - x^*) + \frac{1}{2} {{\,\textrm{Tr}\,}}(A\Sigma A\Sigma )\right) ^{1/2}. \end{aligned}$$

(92)

From (73) and (92), we obtain

$$\begin{aligned} \frac{{{\bar{\alpha }}}(\theta {;} \alpha )}{\alpha / 2} \le \frac{6 N_w^2}{M_w} \bigg (\left( 1 - \frac{1}{\lambda }\right) + \frac{d^{1/2}}{\lambda } \bigg ), \end{aligned}$$

(93)

for which we used the fact that $\frac{{{\,\textrm{Tr}\,}}(A \Sigma )}{{{\,\textrm{Tr}\,}}(A \Sigma A \Sigma )^{1/2}} \le d^{1/2}$. This completes the proof. $\square$