An algorithm for the computation of optimum designs under a given covariance structure

Summary

Regression models with correlated errors lead to nonadditivity of the information matrix. This makes the usual approach of design optimization (approximation by a design measure, application of an equivalence theorem, numerical calculations by a gradient algorithm) impossible. Therefore extended information matrices depending upon design measures have been proposed recently and herein we present a first order iterative design optimization algorithm based upon them. A heuristic is formulated to circumvent the nonconvexity of the problem and the method is applied to typical examples from the literature.

Appendices

Appendix A: The first order derivative of the extended information matrix

For λ > 0 we have from (6)

$$\begin{array}{*{20}c}{{{\partial J_\kappa ^{(\gamma )}\left( {{\xi _\lambda }} \right)} \over {\partial \lambda }} = - \rho \sum\limits_{x,z,u \in {\cal X}} f (x)\left[ {C({\cal X}) + \rho W_\kappa ^{(\gamma )}\left( {{\xi _\lambda }} \right)} \right]_{x,u}^{ - 1}} \hfill \\ { \times \left[ {{\partial \over {\partial \lambda }}\ln {{\left[ {{{_\gamma {\xi _\lambda }} \over {_\gamma {\xi _\lambda }(u)}}} \right]}^\gamma }} \right]\left[ {C({\cal X}) + \rho W_\kappa ^{(\gamma )}\left( {{\xi _\lambda }} \right)} \right]_{u,z}^{ - 1}{f^T}(z),} \hfill \\ \end{array} $$

where ξ_λ = (1 − λ)μ + λη, and μ is supported on the whole χ.

So the derivative of the criterion function based on the information matrix $J_\kappa ^{\left( \gamma \right)}\left( \xi \right)$ is

$${\partial \over {\partial \lambda }}{\rm{\Phi }}\left[ {J_\kappa ^{\left( \gamma \right)}\left( {{\xi _\lambda }} \right)} \right] = - \rho \sum\limits_{u \in {\cal X}} {{d_\gamma }\left( {{\xi _\lambda },u} \right){\partial \over {\partial \lambda }}\ln {{\left[ {{{_\gamma {\xi _\lambda }} \over {_\gamma {\xi _\lambda }\left( u \right)}}} \right]}^\gamma },} $$

where

$${d_\gamma }\left( {\xi ,u} \right) = a_\gamma ^T\left( u \right)\nabla {\rm{\Phi }}\left[ {J_\kappa ^{\left( \gamma \right)}\left( {{\xi _\lambda }} \right)} \right]{a_\gamma }\left( u \right)$$

((A.9))

and

$${a_\gamma }\left( u \right) = \sum\limits_{x \in {\cal X}} {\left[ {C\left( {\cal X} \right) + \rho W_\kappa ^{\left( \gamma \right)}\left( {{\xi _\lambda }} \right)} \right]_{u,x}^{ - 1}f\left( x \right).} $$

By taking limits we obtain

$$\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\Phi \left[ {J_\kappa ^{(\gamma )}\left( {{\xi _\lambda }} \right)} \right] = - \rho \sum\limits_{x \in {\cal X}} {\mathop {\lim }\limits_{\gamma \to 0} } \;{d_\gamma }(\mu ,x)\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} \left[ {{\partial \over {\partial \lambda }}\gamma {{\ln }_\gamma }{\xi _\lambda } - {\partial \over {\partial \lambda }}\gamma {{\ln }_\gamma }{\xi _\lambda }(x)} \right].$$

((A.10))

We have

$$\mathop {\lim }\limits_{\lambda \to 0} \ln \left( {{{_\gamma {\xi _\lambda }} \over {_\gamma {\xi _\lambda }\left( x \right)}}} \right) = \ln {{_\gamma \mu } \over {_\gamma \mu \left( x \right)}}.$$

Hence, we obtain, like in the proof of the Proposition in Appendix B

$$\mathop {\lim }\limits_{\lambda \to 0} {\left[ {C\left( {\cal X} \right) + \rho W_\kappa ^{\left( \gamma \right)}\left( {{\xi _\lambda }} \right)} \right]^{ - 1}} = {\left[ {C\left( {\cal X} \right) + \rho W_\kappa ^{\left( \gamma \right)}\left( \mu \right)} \right]^{ - 1}},$$

and

$$\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} {\left[ {C\left( {\cal X} \right) + \rho W_\kappa ^{\left( \gamma \right)}\left( {{\xi _\lambda }} \right)} \right]^{ - 1}} = {\left[ {C\left( {\cal X} \right) + \rho V_\kappa ^{\left( \gamma \right)}\left( \mu \right)} \right]^{ - 1}}.$$

The limit derivative of ln(_γξ_λ) and of ln_γξλ(u) are given in Lemma 2 in Appendix B.

Note that in the case that μ(x) < κ or μ_max < κ, the logarithm in Lemma 2 is multiplied by the number γ, which in the limit tends to zero. So in this case the limit derivative in (A.10) is infinitely larger than for the case μ(x) > κ, μ_max > κ. Instead of (A.10) we then compute

$$\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} {1 \over \gamma }{\partial \over {\partial \lambda }}{\rm{\Phi }}\left[ {J_\kappa ^{\left( \gamma \right)}\left( {{\xi _\lambda }} \right)} \right]$$

This influences the algorithm which is described in Section 3.

Appendix B: Further Properties

Proposition: We have

$$\mathop {\lim }\limits_{\gamma \to 0} W_\kappa ^{\left( \gamma \right)}\left( \xi \right) = {V_\kappa }\left( \xi \right),$$

and

$$\mathop {\lim }\limits_{\gamma \to 0} J_\kappa ^{\left( \gamma \right)}\left( \xi \right) = \sum\limits_{x,z \in X} {f\left( x \right){{\left[ {C\left( {\cal X} \right) + \rho {V_\kappa }\left( \xi \right)} \right]}^{ - 1}}{f^T}\left( z \right)} ,$$

where V_κ(ξ) is a diagonal matrix with entries

$${\left[ {{V_\kappa }(\xi )} \right]_{x,x}} = \left\{ {\begin{array}{*{20}c}{\ln {\kappa \over {\xi (x)}};} & {{\rm{ if }}\xi (x) < \kappa } \\ {0;} & {{\rm{ if }}\xi (x) > \kappa .} \\ \end{array} } \right.$$

The proof follows directly from Lemma 1 given in the Appendix B. We note that [V_κ(ξ)]_x,x can be continuously extended to the cases κ = ξ(x) and κ = ξ_max, which are not considered in the Proposition.

Lemma 1:

If ξ(x) > κ, then

$$\mathop {\lim }\limits_{\gamma \to 0} \ln {\left( {{{_\gamma \xi } \over {_\gamma \xi \left( x \right)}}} \right)^\gamma } = 0.$$

If ξ(x) < κ, then

$$\mathop {\lim }\limits_{\gamma \to 0} \ln {\left( {{{_\gamma \xi } \over {_\gamma \xi \left( x \right)}}} \right)^\gamma } = \ln {\kappa \over {\xi \left( x \right)}} > 0.$$

Proof. We shall consider the terms in

$$\ln {\left( {{{\gamma \xi } \over {\gamma \xi (x)}}} \right)^\gamma } = \gamma \ln {\rm{[}}\gamma \xi {\rm{]}} - \gamma \ln {\rm{[}}\gamma \xi (x){\rm{]}}{\rm{.}}$$

• i) If ξ(x) > κ, then (4) implies lim_γ→0[_γξ(x)] = ξ(x)−κ, hence lim_γ→0γ ln[_γξ(x)] = 0.

• ii) Similarly, ξ_max > κ ⇒ lim_γ→0 γln[γξ] = 0.

• iii) If ξ(x) < κ then from (4) we obtain

  $$\gamma \ln \left[ {_\gamma \xi \left( x \right)} \right] = \gamma \ln \kappa + \gamma \ln \left\{ {{{\left[ {1 + {t^{{1 \over \gamma }}}\left( x \right)} \right]}^\gamma } - 1} \right\},$$

  ((A.11))

  with $t\left( x \right) = {{\xi \left( x \right)} \over \kappa } < 1$. By the Taylor formula of the function z → (1+z)^γ in the neighborhood of z = 0, we have

  $${\left( {1 + z} \right)^\gamma } = 1 + \gamma z + {1 \over 2}\gamma \left( {\gamma - 1} \right){z^2} + o\left( {{z^2}} \right).$$

Hence from (A.11) we obtain

$$\begin{array}{*{20}c}{\gamma \ln {[_\gamma }\xi (x)] = \gamma \ln \kappa + \gamma \ln \left\{ {\gamma {t^{{1 \over \gamma }}}(x) + {1 \over 2}\gamma (\gamma - 1){t^{{2 \over \gamma }}}(x) + o\left( {{t^{{2 \over \gamma }}}(x)} \right)} \right\}} \\ { = \gamma \ln \kappa + \gamma \ln \left\{ {\gamma {t^{{1 \over \gamma }}}(x)} \right\} + \gamma \ln \left\{ {1 + {1 \over 2}(\gamma - 1){t^{{1 \over \gamma }}}(x) + o\left( {{t^{{1 \over 2}}}(x)} \right)} \right\}} \\ {{ \to _{\gamma \to 0}}\ln t(x) = \ln {{\xi (x)} \over \kappa }.} \\ \end{array} $$

□

Lemma 2:

For ξ_λ(x) = (1 − λ)μ(x) + λη(x), we have

$$\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\ln \left[ {_\gamma {\xi _\lambda }\left( x \right)} \right] = {{\eta \left( x \right) - \mu \left( x \right)} \over {\mu \left( x \right) - \kappa }}\quad {\rm{if}}\;\mu \left( x \right) > \kappa ,$$

$$\begin{array}{*{20}c}{\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} \gamma {\partial \over {\partial \lambda }}\ln {[_\gamma }{\xi _\lambda }(x){\rm{]}} = {{\eta (x)} \over {\mu (x)}} - 1\quad {\rm{if }}\mu (x) < \kappa ,} \\ {\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\ln \left[ {_\gamma {\xi _\lambda }} \right] = {{{\mu _{\max }}} \over {{\mu _{\max }} - \kappa }}\left\{ {{E_{{B_\mu }}}\left[ {{{\eta (.)} \over {\mu (.)}}} \right] - 1} \right\}{\rm{ }}\quad {\rm{if }}{\mu _{\max }} > \kappa ,} \\ {\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} \gamma {\partial \over {\partial \lambda }}\ln \left[ {_\gamma {\xi _\lambda }} \right] = {E_{{B_\mu }}}\left[ {{{\eta (.)} \over {\mu (.)}}} \right] - 1\quad {\rm{ if }}{\mu _{\max }} < \kappa ,} \\ \end{array} $$

where

$${E_{{B_\mu }}}\left[ {{{\eta \left( . \right)} \over {\mu \left( . \right)}}} \right] = {1 \over {{N_{{B_\mu }}}}}\sum\limits_{x \in {B_\mu }} {{{\eta \left( x \right)} \over {\mu \left( x \right)}}} .$$

Proof. According to (4) we have

$${\partial \over {\partial \lambda }}\ln \left[ {_\gamma {\xi _\lambda }} \right] = {\partial \over {\partial \lambda }}\ln \left\{ {{{\left[ {{h_\lambda }\left( {{\xi _\lambda }} \right)} \right]}^\gamma } - \kappa } \right\},$$

where ${h_\gamma }\left( \xi \right) = {\kappa ^{{1 \over \gamma }}} + \sum\nolimits_{x \in {\cal X}} {\xi {{\left( x \right)}^{{1 \over \gamma }}}}$

By direct differentiation we obtain

$$\begin{array}{*{20}c}{{\partial \over {\partial \lambda }}\ln {{\rm{[}}_\gamma }{\xi _\lambda }{\rm{]}} = {1 \over {{{\left[ {{h_\gamma }\left( {{\xi _\lambda }} \right)} \right]}^\gamma } - \kappa }}{{\left[ {{h_\gamma }\left( {{\xi _\lambda }} \right)} \right]}^{\gamma - 1}}\sum\limits_{x \in {\cal X}} {\xi _\lambda ^{{1 \over \gamma } - 1}} (x){{d{\xi _\lambda }(x)} \over {d\lambda }}} \\ {{ \to _{\lambda \to 0}}{{{{\left[ {{h_\gamma }(\mu )} \right]}^\gamma }} \over {{{\left[ {{h_\gamma }(\mu )} \right]}^\gamma } - \kappa }}{{\sum\nolimits_{x \in {\cal X}} {{\mu ^{{1 \over \gamma }}}} (x)} \over {{h_\gamma }(\mu )}}{\rm{E}}_\mu ^{\left( {{1 \over \gamma }} \right)}\left[ {{{\eta (.)} \over {\mu (.)}} - 1} \right]{\rm{,}}} \\ \end{array} $$

((A.12))

where $E_\mu ^{\left( {{1 \over \gamma }} \right)}$ denotes the weighted mean with weights equal to ${{{\mu ^{{1 \over \gamma }}}\left( x \right)} \over {\sum\nolimits_{u \in {\cal X}} {{\mu ^{{1 \over \gamma }}}\left( u \right)} }}$.

Similarly

$$\mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\ln {{\rm{[}}_\gamma }{\xi _\lambda }(x){\rm{]}} = {{{{\left[ {{l_\gamma }(x)} \right]}^\gamma }} \over {{{\left[ {{l_\gamma }(x)} \right]}^\gamma } - \kappa }}\quad {{{\mu ^{{1 \over \gamma }}}(x)} \over {{l_\gamma }(x)}}\left[ {{{\eta (x)} \over {\mu (x)}} - 1} \right],$$

((A.13))

where ${l_\gamma }\left( x \right) = {\kappa ^{{1 \over \gamma }}} + {\mu ^{{1 \over \gamma }}}\left( x \right)$

• i) Let μ_max > κ. Then we obtain directly from (A.12)

  $$\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\ln {{\rm{[}}_\gamma }{\xi _\lambda }] = {{{\mu _{\max }}} \over {{\mu _{\max }} - \kappa }}{E_{{B_\mu }}}\left[ {{{\eta (.)} \over {\mu (.)}} - 1} \right].$$

• ii) Similarly, if μ(x) > κ we obtain from (A.13)

  $$\mathop {\lim }\limits_{\gamma \to 0} \mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\ln {{\rm{[}}_\gamma }{\xi _\lambda }(x){\rm{]}} = {{\eta (x) - \mu (x)} \over {\mu (x) - \kappa }}$$

• iii) Suppose that μ_max < κ, and denote $s\left( x \right) = {{\mu \left( x \right)} \over \kappa } < 1$. From (A.12) we obtain

  $$\begin{array}{*{20}c}{\mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\gamma \ln {{\rm{[}}_\gamma }{\xi _\lambda }{\rm{]}} = {{{{\left[ {1 + \sum\nolimits_{x \in {\cal X}} {{s^{{1 \over \gamma }}}} (x)} \right]}^\gamma }} \over {{{\left[ {1 + \sum\nolimits_{x \in {\cal X}} {{s^{{1 \over \gamma }}}} (x)} \right]}^\gamma } - 1}}{{\gamma \sum\nolimits_{x \in {\cal X}} {{s^{{1 \over \gamma }}}} (x)} \over {\left[ {1 + \sum\nolimits_{x \in {\cal X}} {{s^{{1 \over \gamma }}}} (x)} \right]}}{\rm{E}}_\mu ^{\left( {{1 \over \gamma }} \right)}\left[ {{{\eta (.)} \over {\mu (.)}} - 1} \right]} \\ {{ \to _{\gamma \to 0}}{E_{{B_\mu }}}\left[ {{{\eta (.)} \over {\mu (.)}} - 1} \right]\mathop {\lim }\limits_{\gamma \to 0} {{\gamma \sum\nolimits_{x \in {\cal X}} {{s^{{1 \over \gamma }}}} (x)} \over {{{\left[ {1 + \sum\nolimits_{x \in {\cal X}} {{s^{{1 \over \gamma }}}} (x)} \right]}^\gamma } - 1}}{\rm{.}}} \\ \end{array} $$

  Using the Taylor formula, like in the proof of Lemma 1, we obtain that the last limit is equal to 1.

• iv) Suppose now, that μ(x) < κ. Then from (A.13) we obtain

  $$\begin{array}{*{20}c}{\mathop {\lim }\limits_{\lambda \to 0} {\partial \over {\partial \lambda }}\gamma \ln {{\rm{[}}_\gamma }{\xi _\lambda }(x){\rm{]}} = \gamma {{{{\left[ {1 + {s^{{1 \over \gamma }}}(x)} \right]}^\gamma }} \over {{{\left[ {1 + {s^{{1 \over \gamma }}}(x)} \right]}^\gamma } - 1}} \times {{{s^{{1 \over \gamma }}}(x)} \over {1 + {s^{{1 \over \gamma }}}(x)}}\left[ {{{\eta (x)} \over {\mu (x)}} - 1} \right]} \\ {{\rm{ }}{ \to _{\gamma \to 0}}\left[ {{{\eta (x)} \over {\mu (x)}} - 1} \right]\mathop {\lim }\limits_{\gamma \to 0} {{\gamma {s^{{1 \over \gamma }}}(x)} \over {{{\left[ {1 + {s^{{1 \over \gamma }}}(x)} \right]}^\gamma } - 1}},} \\ \end{array} $$

  and the last limit is equal to 1, which can be proved in the same way as in iii). □