A Convergent Algorithm for Finding KL-Optimum Designs and Related Properties

Abstract

Among optimality criteria adopted to select best experimental designs to discriminate between different models, the KL-optimality criterion is very general. A KL-optimum design is obtained from a minimax optimization problem on an infinite-dimensional space. In this paper some important properties of the KL-optimality criterion function are highlighted and an algorithm to construct a KL-optimum design is proposed. It is analytically proved that a sequence of designs obtained by iteratively applying this algorithm converges to the set of KL-optimum designs, provided that the designs are regular. Furthermore a regularization procedure is discussed.

Appendix

The convergence of the algorithm is studied by means of the property of closeness of point-to-set maps (Luenberger and Ye 2008), which is a generalization of the classical concept of continuity.

Lemma 1

is continuous in (ξ,β ₂).

Proof

Take (ξ _n ,β _n )→(ξ,β). We have

From the definition of weak convergence, it follows that A→0 as ξ _n →ξ, since is continuous in x and is compact. To prove that as ξ _n →ξ, take a converging sequence β _n →β and define the function . Let $\hat{x}_{n}$ be a maximum point: . Since is compact, from any subsequence of $(\hat{x}_{n} )_{n}$, we can extract a converging subsequence $\hat{x}_{n_{k}}\to\hat{x}$. Hence

The continuity of with respect to both the variables concludes the proof. □

Corollary 1

The map Map ₁ is closed.

Proof

Let ξ _n →ξ, β _n ∈Ω ₂(ξ _n ) and β _n →β. We must prove that β∈Ω ₂(ξ). By Lemma 1, we have that, for n sufficiently large,

Moreover, since I _2,1 is a continuous function, then I _2,1(ξ)≤ε+I _2,1(ξ _n ) (again for n sufficiently large). Therefore, since , we get

The arbitrary choice of ε ensures that . □

Lemma 2

The map is closed.

Proof

First note that for any β, since is compact and is continuous. Now, let β _n →β, and x _n →x. By definition, for any n and s. The desired result is a consequence of the continuity of . □

The following lemma extends the closedness of line search algorithms in an infinite-dimensional space.

Lemma 3

The map Map _ξ is closed.

Proof

Let (ξ _n ,x _n )→(ξ,x), $\xi'_{n}\in\mathbf{Map}_{\boldsymbol{\xi}}(\xi_{n},x_{n})$ and $\xi'_{n}\to\xi '$. We need to prove that ξ′∈Map _ξ (ξ,x). For any n, define

$$K_n = \bigl\{(1-\alpha) \xi_n + \alpha \delta_{x_n} \text{ for some }0\leq \alpha \leq1 \bigr\}. $$

Since

$$d \bigl[ (1-\alpha) \xi_n + \alpha\delta_{x_n} , (1- \alpha) \xi+ \alpha \delta_{x} \bigr] \leq(1-\alpha) d( \xi_n,\xi) + \alpha|{x_n}-x|, $$

we have that d(K _n ,K)→0, where K={(1−α)ξ+αδ _x for some 0≤α≤1}.

Since $\xi'_{n}\in K_{n}$, it follows that

$$d \bigl(\xi',K \bigr) \leq d \bigl(\xi', \xi'_n \bigr) + d \bigl(\xi'_n,K_n \bigr) + d(K_n,K)\to0, $$

which implies ξ′∈K, that is, ξ′=(1−α′)ξ+α′δ _x for some α′∈[0,1].

By the definition of $\xi_{n}'$, we have that ${I_{2,1}}(\xi_{n}') \geq{I_{2,1}} [(1-\alpha) \xi_{n} + \alpha\delta_{x_{n}}]$ for any α∈[0,1]. Letting n→∞, we get

$${I_{2,1}} \bigl(\xi' \bigr) \geq{I_{2,1}} \bigl[(1- \alpha) \xi+ \alpha\delta_{x} \bigr] . $$

Thus I _2,1(ξ′)≥max _α∈[0,1] I _2,1[(1−α)ξ+αδ _x ], and hence ξ′∈Map _ξ (ξ,x). □

Corollary 2

The map Map ₂ is closed.

Proof

By Lemmas 2 and 3, the maps and are closed. Since is compact, the composition of the closed point-to-set mappings

is closed (see Luenberger and Ye 2008, p. 205, Cor. 1). □

Proof of Theorem 1

From Lemma 1, Lemma 2 and Luenberger and Ye (2008, Cor. 2, p. 205), it follows that Alg _KL is closed. Moreover, as a consequence of Theorem 1 of López-Fidalgo et al. (2007), it is simple to prove that I _2,1(ξ) is an ascent function for the set of KL-optimal designs and Alg _KL . Finally, it is sufficient to apply the Global Convergence Theorem for ascendant algorithms in Luenberger and Ye (2008, p. 206). □