# Variability Attribution for Automated Model Building

- 167 Downloads

## Abstract

We investigated the possible advantages of using linearization to evaluate models of residual unexplained variability (RUV) for automated model building in a similar fashion to the recently developed method “residual modeling.” Residual modeling, although fast and easy to automate, cannot identify the impact of implementing the needed RUV model on the imprecision of the rest of model parameters. We used six RUV models to be tested with 12 real data examples. Each example was first linearized; then, we assessed the agreement in improvement of fit between the base model and its extended models for linearization and conventional analysis, in comparison to residual modeling performance. Afterward, we compared the estimates of parameters’ variabilities and their uncertainties obtained by linearization to conventional analysis. Linearization accurately identified and quantified the nature and magnitude of RUV model misspecification similar to residual modeling. In addition, linearization identified the direction of change and quantified the magnitude of this change in variability parameters and their uncertainties. This method is implemented in the software package PsN for automated model building/evaluation with continuous data.

## KEY WORDS

automated model building linearization model evaluation nonlinear mixed effects models stochastic model## INTRODUCTION

Nonlinear mixed effect (NLME) modeling, commonly known as the population approach, is increasingly used to describe longitudinal data from preclinical/clinical experiments, either to improve the efficiency of the drug development process and subsequent dosing, or increase the understanding of the studied underlying pathophysiological system (1). In contrast to naive pooling approach, which ignores individual differences, and two-stage approach, which does not distinguish between subject and observation variability, NLME models allow pooling of sparse data from different subjects while simultaneously quantifying multiple levels of variability, thanks to its mixed effects nature. In mixed-effects analysis, population parameters are included in a model as fixed effects, and the variability within this population as random effects. Random effects can incorporate variability on both the subject and observation levels, as inter-individual variability (IIV), between occasion variability, between study variability, and residual unexplained variability (RUV). This ability to identify different sources of variability is particularly critical to many clinical applications, e.g., therapeutic drug monitoring.

For highly nonlinear models, extending the structural base model to include covariates or test different models for random effects can be tedious and interrupted by numerical difficulties. These problems increase exponentially with increasing the complexity of the structure, covariate, and variability models. To overcome such computational and time-intensive burden, linear approximation of first-order conditional estimation (FOCE) method was proposed and applied as a diagnostic tool for testing covariates and random effects (2,3). When successfully implemented, linearization substantially reduced runtimes compared to standard NLME models as the fixed effects are not estimated in the linearized models, but fixed to their estimates from the fit of the NLME model. Linearized models were shown to result in similar objective function values (OFVs) to the NLME models, and accurately identify significant covariate relations and stochastic components similar to conventional analysis. Hence, linearization output models in a standardized coding format, linearization was also recommended for automated model building by coupling to other covariate modeling algorithms as stepwise covariate method (SCM) or full random effects covariate modeling (FREM) (4). However, linearized models still need to be estimated given the original observations similar to NLME models, so it might be sensitive to local minima or other estimation-related issues, especially in presence of interactions between empirical Bayes estimates and RUV models. Major deviations between the OFV of the linearized structure base model and its corresponding NLME model should be interpreted as a failure of implementation of linearization and must be solved prior to further investigations using the linearized model. It has not been shown previously that random effects estimated in linearized models or their uncertainties’ have similar values if estimated in the corresponding NLME models, which if true, will support the role of linearization in automated model building to predict changes in random variability assigned to model parameters upon the inclusion of a potential covariate or adoption of a new RUV model.

Meanwhile, a new method “residual modeling” was proposed as a fast and robust diagnostic tool for assessing RUV models for NLME analysis with continuous outcomes (5). Residual modeling treats the outputted residuals from a NLME model execution as a dependent variable to model its distribution’s mean and variance by a linear base model, then this base model is extended to assess different RUV extensions. The improvement in the fit between the residuals base model and its extended versions can accurately identify the nature and magnitude of potential RUV model improvements/misspecifications, and hence, residual modeling has been already implemented for automated model building. Residual modeling uses a built-in library of six RUV extensions to model the variance of the residuals’ distribution from a NLME model execution. The built-in library includes autoregressive (AR1), dynamic transform both sides (dTBS), residuals’ IIV, power, t-distribution, and time-varying RUV models (6, 7, 8, 9, 10). The investigated residuals were conditional weighted residuals (CWRES), conditional weighted residuals with interaction (CWRESI), individual weighted residuals (IWRES), and normalized prediction distribution errors (NPDE); CWRES outperformed the rest, as CWRES modeling correctly identified the type of the needed RUV model and accurately predicted both the estimates of parameters governing this RUV model and the magnitude of improvement of fit after implementing such RUV model. Residual modeling does not suffer from local minima problems or estimation related issues, as it is using residuals data, not the original observations. This is an advantage for its purpose in fast and robust selection of the best RUV model, but then by definition, it cannot predict the impact of implementing a new RUV model on random variability assigned to the rest of model parameters or their uncertainties.

Here, we investigated if linearization can predict variability attribution for automated model building on the inclusion of a new RUV extension. We used the same six RUV models from our previous work for residual modeling. (5) First, we compared the performance of linearization to residual modeling in selecting the best RUV extension; then, we compared random effects’ estimates and uncertainties on linearized models with the different RUV extensions to their corresponding NLME models.

## METHODS

### Linearization

*y*

_{ij}be the observation

*j*for individual

*i*,

*θ*is the vector of population parameters,

*η*

_{i}is the vector of unexplained deviation of individual parameters

*θ*

_{i}from the population parameters

*θ*,

*x*

_{ij}is the vector of individual

*i*design components as dose and sampling times, and

*ε*

_{ij}is the residual error of observation

*j*for individual

*i*, then the NLME model describing the observations:

*f*is model prediction, and

*h*is the RUV model to be function of

*ε*

_{ij}. Such model can be extended further for multivariate outcome, baseline or time varying covariates. Both random effects

*η*

_{i}and

*ε*

_{ij}are assumed to follow normal distribution with mean 0 and covariance matrix Ω and Σ, respectively, and the unknown model parameters are estimated by maximum likelihood. According to the way of the dependence of

*h*on

*f*, this NLME model can be linearized based on first-order Taylor expansion around

*ε*

_{ij}= 0 and the empirical Bayes estimate \( {\widehat{\eta}}_i \):

where *y*_{ij}^{∗} is the linearized model, *f*_{ij} is the approximated individual predictions, and *h*_{ij} is the approximated individual residual errors.

The NLME model is first evaluated to calculate the different partial derivatives and \( {\widehat{\eta}}_i \) needed, then *y*_{ij}^{∗} is estimated on the same dataset of the NLME model to obtain *η*_{i} and *ε*_{ij}, as these are the only unknown parameters in *y*_{ij}^{∗}. With estimating only random effects alongside its standard coding format, *y*_{ij}^{∗} can be easily and quickly used as a base model for further explaining *η*_{i} with covariates or using different *ε*_{ij} models (3). Here, we extended (Eq. 5) to test six RUV models, and compare their goodness of fit, parameters’ variability estimates, and uncertainties to conventional testing by NLME models as follow and shown in the Supplementary material.

### RUV extensions

*ε*

_{ij}at time point

*j*on

*ε*

_{ik}at time point k,

*autoregression*(AR1)

*error model*with one extra parameter can be implemented:

where *ρ* is the correlation between these errors and *t*_{1/2} is the half-life of *ρ*. The improvement of fit after implementing AR1 error model in the linearized model (∆OFV_{lin, AR1}) is calculated as the difference in OFV of the linearized base model (Eq. 5) and OFV of the linearized model with AR1 error model (Eqs. 5 and 6). ∆OFV_{lin, AR1} is comparable to the improvement of fit on implementing AR1 error model in the NLME model (∆OFV_{NLME, AR1}) between the base NLME model (Eq. 1) and its AR1 error model extension.

*dynamic transform both sides*(dTBS) approach is useful through the estimation of a Box–Cox shape parameter

*λ*and a power term

*ζ*that also address possible scedasticity in residual magnitudes (8,9). Linearized models with dTBS approach follows (Eq. 7) if

*λ*was estimated to 0, and (Eq. 8) otherwise. Improvement of fit on dTBS implementation (∆OFV

_{lin, dTBS}) is the difference in OFVs of the dTBS linearized model with

*λ*and

*ζ*fixed to 1 and 0, respectively, and the dTBS linearized model with both

*λ*and

*ζ*estimated.

*ε*

_{ij}is being identically distributed, this assumption can be relaxed by adding

*inter-individual variability η*

_{i, RUV}

*on the residuals*to allow different RUV magnitudes. Improvement of such extension in the fit of the linearized model (∆OFV

_{lin, IIV}) is the difference in OFVs of (Eq. 5) and (Eq. 9).

*ζ*alone in what is known as

*power RUV model*. Improvement of fit on applying the power RUV model to the linearized models (∆OFV

_{lin, ζ}) is the difference in OFVs of (Eq. 5) and (Eq. 10).

*t-distributed residuals*. The Laplacian method with user-defined conditional likelihood (

*L*) had to be used for a Laplace linearized base model (Eq. 11) and linearized model with t-distributed residuals (Eq. 12), where

*σ*is the square root of

*h*

_{ij}, and

*υ*is the degrees of freedom; the difference of these models’ OFVs is ∆OFV

_{lin, υ}.

*time-varying errors*allow different error magnitudes for different time points. A typical example is that the absorption phase in a pharmacokinetic model can have larger errors than the elimination phase. This is implemented by allowing the change of the standard deviation of residuals to be a step function of the time or time after dose, at selected cutoff time point

*X*.

where θ_{1} is the standard deviation of residuals before the cutoff time point *X*, θ_{2} is the standard deviation of residuals after this cutoff time point, and Σ is fixed to 1, as multiplying a random variable by constant (ω ∙ *ε*_{ij}) increase the variance by the square of this constant (ω^{2}). We used three cutoff points to divide the data into four equal sized groups, and the improvement of fit (∆OFV_{lin, time}) after extending the linearized base model (Eqs. 4 and 5) to the linearized model with time varying residuals (Eqs. 5 and 13) is the difference between their respective OFVs.

### Evaluations

_{lin}, as well as Ωs’ estimates and uncertainties. We compared the performance of ∆OFV

_{lin}in predicting ∆OFV

_{NLME}to that of ∆OFV

_{Diagnostic}obtained by residual modeling, where

*diagnostic*refers to the used residual. Afterwards, we compared Ωs’ estimates and uncertainties of linearized models to their respective NLME models as shown in Fig. 1. We used 12 real data examples for our evaluation (Table I). Only when the linearized base model and the NLME base model had similar OFV were RUV extensions added and further estimated. All real data examples were treated as continuous. Asenapine effects were assessed using PANSS, which is a composite score, where items of positive, negative, and general nature are scored and combined into one assessment. Despite this, the asenapine data was treated as continuous data in the model. Also, the asenapine model was implemented with residuals’ IIV model from the start. Models varied in structure components from simple pharmacokinetic one compartment model as moxonodine, to complex description of nonlinear system of interacting multi-dependent variables as the integrated glucose-insulin (IGI) model. Seven models used log-transformed data. Two models used a combined error model, two models used a proportional error model and the remaining models used additive error models. NONMEM version 7.4.3 (ICON Development Solutions, Hanover, MD, USA) was used for the analysis (22), with the aid of the

*linearize*tool in PsN (3,23), and graphs were generated in R (24). To obtain the improvement of fit by residual modeling (∆OFV

_{Diagnostic}) when testing the different RUV extensions on the real data examples, we used the

*resmod*tool in PsN (5).

Summary of real data examples used for investigation

Model | Data type | RUV model | Transformation | No. of observations | No. of subjects | No. of THETAs | No. of OMEGAs including covariances | No. of SIGMAs |
---|---|---|---|---|---|---|---|---|

Asenapine | PD | Additive with IIV | – | 7728 | 1328 | 16 | 5 | 1 |

Clomethiazole (12) | PK | Additive | Log | 2177 | 772 | 10 | 5 | 1 |

Daunorubicin | PD | Additive | Log | 112 | 41 | 7 | 3 | 1 |

Digoxin | PK/PD | PD: proportional PK: additive | – | 941 | 225 | 6 | 3 | 1 |

Disufenton sodium | PK | Additive | Log | 1196 | 175 | 7 | 3 | 1 |

Ethambutol | PK | Combined | Log | 1869 | 189 | 8 | 3 | 1 |

IGI | PD | Additive | Log | 6382 | 72 | 26 | 15 | 1 |

Miltefosine | PK | Proportional | – | 350 | 31 | 7 | 4 | 1 |

Moxonodine | PK | Additive | Log | 1021 | 74 | 5 | 6 | 1 |

Paclitaxel | PD | Combined | – | 530 | 46 | 6 | 3 | 1 |

Pefloxacin | PK | Additive | Log | 337 | 74 | 4 | 6 | 1 |

r-Hfsh (21) | PK | Additive | – | 314 | 60 | 7 | 2 | 1 |

## RESULTS

_{lin}and ∆OFV

_{NLME}was good as shown in Fig. 2. Comparing to the performance of residual modeling in predicting ∆OFV

_{NLME}, linearization surpassed CWRESI, IWRES, and NPDE over all different ranges of ∆OFV

_{NLME}, and performed better than CWRES at most ranges of ∆OFV

_{NLME}except at low ranges of ∆OFV

_{NLME}(~ 10) where CWRES was slightly better. Linearization identified accurately the most important RUV extension to all examples similar to conventional analysis, surpassing CWRES modeling that reversed the order of 1st and 2nd most important extensions with two examples, Ethambutol and Disufenton sodium models. Also, linearization identified the RUV extensions resulting in significant improvement of fit in all examples similar to conventional analysis, while CWRES modeling missed only t-distribution error model with Asenapine model, shown in Fig. 2. Asenapine model is the only model with residuals’ IIV model as the base model, which may be sufficient in explaining outliers and would turn the t-distributed error model rather less important. The median ratio of ΔOFV

_{lin}/ΔOFV

_{NLME}was 0.95 among models with significant improvement, compared to 0.8 for the median ratio of ΔOFV

_{CWRES}/ΔOFV

_{NLME}.

_{lin}) and their respective estimates on NLME models (Ω

_{NLME}), they showed good agreement with only one outlier: AUC50’s variability in asenapine model. A plot of log (Ω

_{NLME}) versus log (Ω

_{lin}) across all examples with base models and their RUV extensions is shown in Fig. 3, with estimates less than − 4 on log scale excluded from the graph, given that these estimates are low and would not be considered in further model development. In total, nine estimates were excluded based on this, e.g., the variability assigned to the intercompartmental clearance in Clomethiazole model under IIV on RUV and dTBS extensions. Standard errors (SEs) of each Ω

_{lin}and its respective Ω

_{NLME}showed a good agreement in the commonly expected range of SEs for a well identifiable continuous data variability parameter (0–1), and bad agreement at the extreme estimates of SE(Ω

_{NLME}), for instance, the SE of PAN0’s variability in asenapine model was > 1000 with both dTBS and power RUV extensions, which is unacceptable. This may be related to the scores used to measure asenapine effect, i.e., PANSS. A plot of the log-transformed estimates of SE(Ω

_{NLME}) and SE(Ω

_{lin}) is presented in Fig. 4, with estimates less than − 4 on log scale excluded from the graph. Lastly, relative standard errors (RSEs) for each Ω

_{lin}and its corresponding Ω

_{NLME}were calculated on the standard deviation scale as (Eq. 14), and their log-transformed estimates are presented in Fig. 5, that in addition to showing the same trends as Fig. 4, showed that standard errors after implementing t-distribution extensions are less predictable by linearization than the other RUV extensions.

## DISCUSSION

In this paper, we explored if the use of linearization to identify and quantify RUV model misspecifications, similar to residual modeling (5), can provide additional advantages. Residual modeling assesses whether RUV extensions are required to address an RUV misspecification. It is done in an extremely fast and robust way, thanks to the simple nature of models for residuals data. In case of multiple dependent variables, residual modeling evaluates the RUV extensions separately for each dependent variable, identifying which variable need which extension, and so reducing the risk of ending up with an over-parameterized NLME model. However, being estimated on residual data has shortcomings, as residual modeling cannot inform on the rest of the NLME model parameters. Implementation of a needed RUV extension in a NLME model would be expected to improve the uncertainties of Ω and θ subsequently, as the latter is a function of the former. Linearization, in contrast to residual modeling, uses the calculated parameters’ partial derivatives with respect to \( {\widehat{\eta}}_i \) from the fit of the NLME model. It estimates the RUV model incorporating any extension and the random effects components given the same data as the NLME model. Thus, linearization can estimate explicitly the random effects and their uncertainties in the base and the extended model, and implicitly the magnitude and the direction of change in the random effects and their uncertainties, and that is what we had shown here.

We successfully implemented six RUV extensions to the standardized linearization framework and linearized all real data examples. However, estimation difficulties were present when applying AR1 and t-distribution RUV extensions to the NLME/linearized models of Clomethiazole and the IGI, but not in their respective residual modeling. The agreement between ΔOFV_{lin} and ΔOFV_{NLME} when improvement of fit is > 10 was nearly perfect, indicating that only the estimates of random effects were changing on implementing the different RUV extensions in the NLME models. Deviations would be expected if estimates of fixed effects were also changing. The overall prediction performance of ΔOFV_{NLME} by ΔOFV_{lin} was better than ΔOFV_{CWRES}, however not by much.

Linearization identified and quantified the nature and the magnitude of RUV model misspecifications in these real data examples more accurately than CWRES modeling, the latter reversed the order of the most two important extensions in Ethambutol and Disufenton sodium models and could not identify t-distribution as a significant extension in Asenapine model. Even though it is a minor difference, it illustrated the high sensitivity of linearization to detect differences between the RUV extensions that introduce similar flexibility in the model, e.g., IIV and t-distribution RUV models both offer outlier robustness in the NLME model. This showed that conclusions drawn from the results of automated testing of RUV extensions will remain the same on replacement of residual modeling with linearization, and the only expected difference would be an increased run time for linearization of structure models with large random effects models.

Regarding the prediction of the impact of RUV extensions on Ω_{NLME}, linearization showed a good ability with only one outlier (the Ω assigned to AUC50 in Asenapine model). Interestingly, linearization underestimated this Ω with all RUV extensions except t-distribution. This underestimation issue will escalate when it comes to predicting SE(Ω_{NLME}). Linearization did well in assessing the expected ranges of uncertainties of variability assigned to model parameters describing continuous data; more deviations occurred as uncertainties’ estimates moved away from that range, with the main problem being Asenapine model. This might point out that violation of assumptions regarding the nature of data will be a problematic in this automated testing procedure as Asenapine effects are measured using PANSS, which is a composite score, but treated as continuous data in the model. Among the RUV extensions, t-distribution was the most associated with deviations, mainly underestimation. That can be easily tracked down to the use of LAPLACE method commonly known for minimization-related problems, for instance, SE(Ω_{NLME, υ}) of AUC50 in Asenapine model was 1.66 × 10^{−4} which is too close to 0, and an unreasonable estimate for uncertainty, given that the estimate of Ω_{NLME, υ} of AUC50 is 2.9. This problem of unreasonable estimates in SE(Ω_{NLME}) explain all the extreme deviations seen in Fig. 4. One of these is PAN0 parameter in asenapine model which Ω_{NLME, dTBS} estimate was 168, but its SE(Ω_{NLME, dTBS}) was 4.15 × 10^{4}. With these deviations being justified, it is safe to claim that linearization itself or its predictive performance of SE(Ω_{NLME}) showed no built-in drawbacks. The same issues go for RSE(Ω_{NLME}) as not respecting the nature of the data, t-distribution extension, and the unrealistic estimates of Ω_{NLME} and their uncertainties propagated to most of the outliers in Fig. 5.

In conclusion, we investigated the possible merits of linearization if used to evaluate RUV models for continuous data. Linearization accurately identified the nature of RUV extension if needed and predicted the improvement of fit on its inclusion similar to residual modeling. In addition, linearization can predict the impact of including such RUV extension on the variability assigned to model parameters and their uncertainties, allowing its utilization for variability attribution with automated model building procedures.

## Notes

### Acknowledgments

MMA Ibrahim acknowledges the Egyptian Ministry of Higher Education for financial support.

### Compliance with Ethical Standards

### Conflict of Interest/Disclosure

There is no involvement, financial or otherwise, that might potentially bias this work.

## Supplementary material

## References

- 1.Gobburu JV. Pharmacometrics 2020. J Clin Pharmacol. 2010;50:151S–7S.CrossRefGoogle Scholar
- 2.Khandelwal A, Harling K, Jonsson EN, Hooker AC, Karlsson MO. A fast method for testing covariates in population PK/PD models. AAPS J. 2011;13:464–72.CrossRefGoogle Scholar
- 3.Svensson EM, Karlsson MO. Use of a linearization approximation facilitating stochastic model building. J Pharmacokinet Pharmacodyn. 2014;41(2):153–8.CrossRefGoogle Scholar
- 4.Yngman G, Nordgren R, Freiberga S, Karlsson MO. Linearization of full random effects modeling (FREM) for time-efficient automatic covariate assessment. 2018; PAGE 27 Abstr 8750.Google Scholar
- 5.Ibrahim MMA, Nordgren R, Kjellsson MC, Karlsson MO. Model-based residual post-processing for residual model identification. AAPS J. 2018;20(5):81.CrossRefGoogle Scholar
- 6.Chi EM, Reinsel GC. Models for longitudinal data with random effects and AR(1) errors. J Am Stat Assoc. 1989;84:452–9.CrossRefGoogle Scholar
- 7.Karlsson MO, Beal SL, Sheiner LB. Three new residual error models for population PK/PD analyses. J Pharmacokinet Biopharm. 1995;23(6):651–72.CrossRefGoogle Scholar
- 8.Dosne A, Bergstrand M, Karlsson MO. A strategy for residual error modeling incorporating scedasticity of variance and distribution shape. J Pharmacokinet Pharmacodyn. 2015;43(2):137–51.CrossRefGoogle Scholar
- 9.Box GEP, Cox DR. An analysis of transformations. J R Stat Soc B is journal of the Royal Statistical Society, Series B 1964;26(2):211–52.Google Scholar
- 10.Karlsson MO, Jonsson EN, Wiltse CG, Wade JR. Assumption testing in population pharmacokinetic models: illustrated with an analysis of moxonidine data from congestive heart failure patients. J Pharmacokinet Biopharm. 1998;26(2):207–46.CrossRefGoogle Scholar
- 11.Friberg L, Greef RD, Kerbusch T, Karlsson MO. Modeling and simulation of the time course of asenapine exposure response and dropout patterns in acute schizophrenia. Clin Pharmacol Ther. 2009;86(1):84–91.CrossRefGoogle Scholar
- 12.Zingmark P, Ekblom M, Odergren T, Ashwood T, Lyden P, Karlsson MO, et al. Population pharmacokinetics of clomethiazole and its effect on the natural course of sedation in acute stroke patients. Br J Clin Pharmacol. 2003;56(2):173–83.CrossRefGoogle Scholar
- 13.Bogason A, Quartino AL, Lafolie P, Masquelier M, Karlsson MO, Paul C, et al. Inverse relationship between leukaemic cell burden and plasma concentrations of daunorubicin in patients with acute myeloid leukaemia. Br J Clin Pharmacol. 2011;71(4):514–21.CrossRefGoogle Scholar
- 14.Hornestam B, Jerling M, Karlsson MO, Held P. Intravenously administered digoxin in patients with acute atrial fibrillation: a population pharmacokinetic/pharmacodynamic analysis based on the Digitalis in Acute Atrial Fibrillation trial. Eur J Clin Pharmacol. 2003;58(11):747–55.CrossRefGoogle Scholar
- 15.Jonsson S, Cheng Y, Edenius C, Lees KR, Odergren T, Karlsson MO. Population pharmacokinetic modeling and estimation of dosing strategy for NXY-059, a nitrone being developed for stroke. Clin Pharmacokinet. 2005;44(8):863–78.CrossRefGoogle Scholar
- 16.Jonsson S, Davidse A, Wilkins J, Walt JV, Simonsson US, Karlsson MO, et al. Population pharmacokinetics of ethambutol in South African tuberculosis patients. Antimicrob Agents Chemother. 2011;55(9):4230–7.CrossRefGoogle Scholar
- 17.Silber HE, Jauslin PM, Frey N, Gieschke R, Simonsson US, Karlsson MO. An integrated model for glucose and insulin regulation in healthy volunteers and type 2 diabetic patients following intravenous glucose provocations. J Clin Pharmacol. 2007;47(9):1159–71.CrossRefGoogle Scholar
- 18.Dorlo TP, Van Thiel PPAM, Huitema AD, Keizer RJ, Vries HJ, Beijnen JH, et al. Pharmacokinetics of miltefosine in Old World cutaneous leishmaniasis patients. Antimicrob Agents Chemother. is journal of Antimicrobial Agents and Chemotherapy 2008;52(8):2855–60.Google Scholar
- 19.Friberg LE, Henningsson A, Maas H, Nguyen L, Karlsson MO. Model of chemotherapy-induced myelosuppression with parameter consistency across drugs. J Clin Oncol. 2002;20(24):4713–21.CrossRefGoogle Scholar
- 20.Wahlby U, Thomson AH, Milligan PA, Karlsson MO. Models for time-varying covariates in population pharmacokinetic-pharmacodynamic analysis. Br J Clin Pharmacol. 2004;58(4):367–77.CrossRefGoogle Scholar
- 21.Karlsson MO, Wade JR, Loumaye E, Munafo A. The population pharmacokinetics of recombinant- and urinary-human follicle stimulating hormone in women. Br J Clin Pharmacol. 2002;45(1):13–20.CrossRefGoogle Scholar
- 22.Beal S, Sheiner LB, Boeckmann A, Bauer RJ, NONMEM user’s guides.
*Icon Development Solutions*, Ellicott City, MD, USA, 1989-2009; 2009.Google Scholar - 23.Lindbom L, Pihlgren P, Jonsson EN. PsN-toolkit—a collection of computer intensive statistical methods for non-linear mixed effect modeling using NONMEM. Comput Methods Prog Biomed. 2005;79(3):241–57.CrossRefGoogle Scholar
- 24.Team RC. R: a language and environment for statistical computing. Vienna, Austria: 2014. Available from: http://www.R-project.org.

## Copyright information

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.