Optimal designs for prediction in random coefficient regression with one observation per individual

The subject of this work is random coefficient regression models with only one observation per observational unit (individual). An analytical solution in form of optimality conditions is proposed for optimal designs for the prediction of individual random effect for a group of selected individuals. The behavior of optimal designs is illustrated by the example of linear regression models.


Introduction
The subject of this work is random coefficient regression (RCR) models in which only one observation per observational unit (individual) is possible. These models are popular e.g. in psychology (see Freund and Holling (2008)) and pharmacokinetics (see Patan and Bogacka (2007)). The main purpose of the present paper is to obtain an analytical solution for the designs that are optimal for the prediction of random effects. Optimal designs for prediction in RCR models have been discussed e.g. in Gladitz and Pilz (1982), Fedorov and Hackl (1997), Prus and Schwabe (2016) and Prus (2023). In Gladitz and Pilz (1982) and Prus and Schwabe (2016) the number of observations per individual is required to be not smaller than the number of unknown model parameters, which excludes the case of only one observation. Also the same design for all individuals has been assumed in the both papers. Fedorov and Hackl (1997) investigated models with specific regression functions. The models considered in the present work may be seen as a particular case (with one individual per group) of the multiple-group models discussed in Prus (2023). However, the solution developed in that paper is based on the assumption of sufficient number of observations per individual, i.e. not smaller than the number of parameters. Models with only one observation per individual were discussed e.g. in Patan and Bogacka (2007) and Graßhoff et al. (2012). Patan and Bogacka (2007) investigated non-linear mixed-effects models. Graßhoff et al. (2012) proposed a solution for optimal designs in form of an optimality condition for RCR models. In these papers optimal design were determined for estimation of fixed effects. However, not much has been done for prediction of random effects.
In the present work we determine optimal designs for prediction of random effects for a group of selected individuals in RCR models with one observation per individual. The obtained design criterion (linear criterion) results in a multiple-design problem, for which the standard approach for design optimization proposed in Kiefer (1974) cannot be applied. We make use of the optimality conditions for multiple-design problems proposed in Prus (2022). The obtained analytical results are illustrated by examples of linear regression models with particular covariance matrices.
The paper has the following structure: In the second section the model is specified and the mean squared error (MSE) matrix for the best linear unbiased prediction (BLUP) of the random effects is determined. In Sect. 3 the linear design criterion for the prediction and the resulting optimality conditions are formulated. The analytical results are illustrated by examples in Sect. 4. The paper is concluded by a short discussion in Sect. 5.

Model specification
In this work we the consider RCR models with only one observation per observational unit (individual) of the following form: where Y i is an observation at the i-th individual, n is the number of individuals, f = ( f 1 , . . . , f p ) is a vector of known regression functions, experimental settings x i come from an experimental region X . The observational errors ε i are assumed to have zero mean and common variance σ 2 > 0. The individual parameters β i = (β i1 , . . . , β i p ) have unknown expected value (population mean) E (β i ) = β and known covariance matrix Cov (β i ) = σ 2 D. All individual parameters β i and all observational errors ε i are assumed to be uncorrelated. According to Graßhoff et al. (2012) the covariance matrix of the best linear unbiased estimator (BLUE)β for the population parameter (fixed effects) β is given by where In this work we focus on random effects, in particular on the individual deviations from the mean: b i = β i − β (see e.g. Prus and Schwabe (2013)). Namely, we consider the situation in which our main interest is in some selected individuals. We determine optimal experimental settings for prediction of the individual effects for the k selected individuals: . We assume n ≥ 2 p, otherwise [ p, n − p] = ∅. Note that the order of the individuals does not matter in this case. Therefore, we can consider k first individuals without loss of generality.
Further we search for experimental settings that minimize the MSE matrix of the BLUPˆ for the individual deviations .

Lemma 1 The MSE matrix of the BLUPˆ is given by
where The proof of Lemma 1 is deferred to Appendix A.

Experimental design
We consider the following two groups of individuals: the k selected individuals build the first group (Group 1), and the second group (Group 2) consists of the n − k remaining individuals. We also allow for group-specific restrictions with respect to experimental settings, i.e. the experimental regions for the two groups of individuals may differ from each other. In practice it can be useful in case of some particular restrictions in two centers/clinics/etc. Further X denotes the experimental region for group , = 1, 2, and X 1 ∪ X 2 = X . The particular case X 1 = X 2 = X will be later considered more detailed. We add the group index to experimental settings for clear notation and we define an exact design in group as where x j ∈ X are the support point of ξ ,e , m j > 0 is the number of observations at x j with N 1 j=1 m 1 j = k and N 2 j=1 m 2 j = n − k and N = |{x 1 , . . . , x N }|. Note that m j can be larger than one in case if observations for more than one individual are taken at point x j . Note also that the set {x 11 , . . . , For analytical purposes we also introduce approximate designs: where w j > 0 denotes the weight of observations at x j , Ñ j=1 w j = 1, = 1, 2, andÑ = |{x 1 , . . . , x Ñ }|. Note thatÑ = N for exact designs. Further we use the notation ξ = (ξ 1 , ξ 2 ) for the pair of group designs.
We also define the information (or moment) matrices for the first and second design as respectively.
For exact designs we obtain In this work we focus on the linear criterion for the prediction of individual deviations for the selected individuals, which is defined for exact designs as where L denotes the transformation matrix.
Neglecting the constants that have no influence on designs, we obtain for approximate designs the following results.

Theorem 1 The linear criterion for the prediction of individual effects is given by
whereL = DLD and L = L L.
The proof of this result is deferred to Appendix B. Note that we assume matrices M 1,ξ and M 2,ξ and consequently M k non-singular, which requires k ∈ [p, n − p]. Otherwise linear criterion (7) cannot be written in form (8).
As we can see by formula (8), the linear criterion depends on two designs simultaneously. In this case the general equivalence theorem (see Kiefer (1974)) cannot be directly applied. Instead we use the extended version for multiple-design problems proposed in Prus (2022). To make use of the equivalence theorems presented in that work, we have to verify convexity of the criterion.

Lemma 2 The linear criterion for the prediction of individual effects is convex with
For the proof of Lemma 2 see Appendix C. As the linear criterion for the prediction of the individual deviations is differentiable and convex with respect to both moment matrices, optimality conditions can be formulated.
Theorem 2 Approximate designs ξ * = (ξ * 1 , ξ * 2 ) are L-optimal for the prediction of the individual effects iff for all x ∈ X 1 , and for all x ∈ X 2 . For support points of ξ * 1 and ξ * 2 equalities hold in (9) and (10), respectively. The proof is deferred to Appendix D. Note that optimal designs depend on the dispersion matrix of random effects D, the total number of individuals n and the number of selected individuals k.
Note also that in case where the design region is the same for all individuals X 1 = X 2 = X , optimal designs may be also the same for both groups, i.e. M 1,ξ * = M 2,ξ * . In this situation design criterion (8) simplifies to which is linear in design, i.e. singular designs (that result in a singular moment matrix) or all permissible designs may be optimal. The optimality conditions simplify to the following inequality:

M. Prus
Behavior of such designs will be illustrated by Example 4.1.

Examples
We consider the linear regression model which is the particular case of model (1) with f(x) = (1, x) , and we assume the diagonal structure of the covariance matrix of random effects: D = diag(d 1 , d 2 ). Further we distinguish between the two different cases: the same design region for all individuals (Example 4.1) and different design regions for different groups (Example 4.2). We focus on the A-optimality criterion, i.e. L = I p , where I p denotes the p × p identity matrix.

Example 1
In this example we consider the situation where there are no particular restrictions for the designs for the selected individuals (that are of our main interest) or for all other individuals, i.e. the design region is the same for both groups: X 1 = X 2 = X . In particular, we consider symmetric design regions: X = [−a, a], for a > 0. Further we focus on the designs that are the same for both groups: ξ 1 = ξ 2 with all observations at the endpoints: where w is the weight of observations at point a. The total number of individuals n and the number of selected individuals k have no influence on designs in the present case and do not need to be specified. If optimal designs of form (14) exist, they assign w = 0 or w = 1 (and make the moment matrix singular) or all values of w are equally good. For the present model we obtain and The simplified linear criterion (11) is given by which is independent of the designs, i.e. all designs (all w ∈ (0, 1)) are optimal or there is no optimal designs of form (14) at all. The optimality condition (12) is given by which is satisfied only in case Hence, under condition (15) all design are equally good. Otherwise, there is no solution with ξ 1 = ξ 2 , where ξ 1 is of form (14).

Example 2
In this example we assume different design regions X 1 and X 2 . In particular, we concentrate on symmetric design regions of different lengths: X = [−1, 1] and X = [−a, a], for a > 0. We consider the endpoints-designs and where w 1 and w 2 are the weights of observations at points 1 and a for the first and the second group, respectively. We simplified the notations w 12 and w 22 to w 1 and w 2 , as there are only two support points for each design. Further we consider two different cases for the covariance matrix of random effects: random intercept and random slope. Case 1 d 1 = d and d 2 = 0 In the case of random intercept we have (1, x) , Linear criterion (8) for this model will not be presented here because of its complexity. We use software Maple2020 for further computations. For the following points the first derivative of the criterion function is zero: All designs with property (18) turn out to be optimal, i.e. they satisfy optimality conditions (9) and (10) for all possible values of a. Note that the designs are independent of the variance parameter d.
Case 2 d 1 = 0 and d 2 = d For the random slope we obtain Using the same approach as for the random intercept case, it can be verified that all designs with are optimal for all a > 0. Note that in both cases: random intercept and random slope, the obtained designs are optimal for all values of the total number of individuals n and the number of selected individuals k ∈ [p, n − p]. Note also that a = 1 would lead to w 1 = w 2 , which is in accordance with Example 4.1.

Discussion
We considered RCR models in which only one observation per individual is possible. We focused on individual random effects for a group of selected individuals, in particular on the mean random effect in the group. The solution for optimal designs is proposed in form of optimality conditions. The number of selected individuals k is assumed to be between the number of parameters p and n − p, where n is the total number of individuals, which is possible only for models with n ≥ 2 p. If the main interest is in a very small, i.e. k < p, or a very large number of selected individuals, another approach is, however, needed. Optimal designs are determined for the BLUP of the random effects, which depends on the variance parameters (covariance matrices). The variance parameters are assumed to be known, which is in general not the case in practice. In a practical situation, where the covariance matrices have to be estimated, we deal with estimated BLUP (EBLUP). The obtained optimal designs in general depend on the variance parameters and are, therefore, locally optimal. The problem of local optimality may be solved for particular models by considering robust design criteria, for example minimax-criterion. For specific covariance structure, optimal designs may be independent of variance parameters, which has been illustrated by the example of the linear regression models with random intercept and random slope. All observational errors were assumed to have the same variance. Heteroscedastic errors, especially depending on the experimental settings (as considered by Graßhoff et al. (2012)), are analytically more challenging. However, an extension may be considered in a future research. Moreover optimal designs have been obtained for individual effects (individual deviations from the population parameter) via their arithmetic mean. Design optimization for the prediction of the individual deviations or individual parameters themselves turned out to be more challenging. It may also be a subject of future investigations, especially for particular models.
Funding Open Access funding enabled and organized by Projekt DEAL.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

A Proof of Lemma 1
Model (1) can be rewritten in vector form as follows with x 1 ) , . . . f(x n ) ) and block-diag (A 1 , . . . , A n ) is the block-diagonal matrix with blocks A 1 , . . . , A n . This model is the classical linear mixed model considered e.g. in Henderson (1975).
Then the MSE matrix for the BLUPb for random effects b = (b 1 , . . . , b n ) can be computed as follows: Then from = 1 k 1 k , 0 n−k ⊗ I p b and consequently follows Eq. (6).

B Proof of Theorem 1
To determine the linear criterion for approximate designs, we use the basic formula (7) and we replace matrices M k and M by kM 1,ξ and kM 1,ξ + (n − k)M 2,ξ , respectively. Then we suppress the first term ( 1 k tr(D)), which is independent of designs, and the multiplicator σ 2 /k 2 , and we obtain L (ξ ) = tr LD kM 1,ξ kM 1,ξ + (n − k)M 2,ξ −1 kM 1,ξ − kM 1,ξ D , which can be easily simplified to (8) using the properties of the trace and the standard formula for the inverse of sum of two non-singular matrices.