In pharmacometrics, understanding a covariate effect on an interested outcome is essential for assessing the importance of the covariate. Variance-based global sensitivity analysis (GSA) can simultaneously quantify contribution of each covariate effect to the variability for the interested outcome considering with random effects. The aim of this study was to apply GSA to pharmacometric models to assess covariate effects. Simulations were conducted with pharmacokinetic models to characterize the GSA for assessment of covariate effects and with an example of quantitative systems pharmacology (QSP) models to apply the GSA to a complex model. In the simulations, covariate and random variables were generated to simulate the outcomes using the models. Ratios of variance explained by each factor (each covariate and random effect) over the overall variance of the outcome were used as sensitivity indices. The sensitivity indices were consistent with the effect size of covariate. The sensitivity indices identified the importance of creatinine clearance on the pharmacokinetic exposure for a renally-excreted drug. These sensitivity indices could be applied to plasma concentrations over time (repeated measurable outcomes over time) as interested outcomes. Using the GSA, each contribution of all of the covariate effects could be efficiently identified even in the complex QSP model. Variance-based GSA can provide insight when considering the importance of covariate effects by simultaneously and quantitatively assessing all covariate and random effects on interested outcomes in pharmacometrics.
In pharmacometrics, a covariate effect is usually incorporated in a model as a relationship between a covariate and a model parameter . Understanding the covariate effect on an interested outcome [e.g., maximum concentration (Cmax), a pharmacodynamic (PD) marker, an efficacy/safety endpoint] is essential for assessing the importance of the covariate. In non-linear mixed effects models, inter-individual variability (IIV) for a model parameter is incorporated in a model as a random effect. Covariate effects are considered to be strongly associated with IIV for the outcome since it is derived from the sum of covariate and random effects.
In covariate modeling, the inferential test can be used to assess the impact of covariates on model parameters . In this test, the effect size of the covariate is assessed as a ratio of a parameter estimate with a high or low value of the covariate (e.g., a 5- or 95-percentile value) relative to the parameter estimate with a reference value of the covariate (e.g., a median value). The clinical relevance of covariate effect is determined based on the ratio for pharmacokinetic (PK) parameters, eg, if the ratio is within 0.8–1.25, no effect of the covariate can be established. However, the inferential test allows us to evaluate the change of only one covariate for one parameter at a time, with the random effect not being evaluated in this approach. In addition, the inferential test would not efficiently and simultaneously evaluate interaction or contribution of multiple covariates (factors) especially in complex models, such as quantitative system pharmacology (QSP) models consisting of multi-components.
As a full model approach for detecting the contribution of covariates, the full random effects modeling (FREM) was reported . In the modeling with FREM, the importance of covariates is determined based on the inferential test or the contribution on unexplained variability for the model parameters, but not for the interested outcome. Also, Wald’s approximation method is a full model approach to evaluate covariate effects based on statistical significance with the approximate likelihood ratio . However, the statistical test does not quantify clinically importance of the covariate effect.
Variance-based global sensitivity analysis (GSA) is a simulation-based approach to simultaneously assess the impact of each factor on an interested outcome . It has been applied to QSP models . As a covariate or random effect can be considered as a factor, this analysis can be applied to assessments of the importance of a covariate effect in consideration with random effects in non-linear mixed effects modeling in pharmacometrics. Unlike the inferential test, the variance-based GSA should be an efficient and comprehensive evaluation approach for considering all covariate effects with all random effects. It can even be applied to complex models with contribution of multiple factors.
In this study, variance-based GSA was applied to the evaluation of covariate effects in pharmacokinetic (PK) models and a QSP model to assess its applicability and implementation for pharmacometrics.
Variance-based global sensitivity analysis for assessing a covariate effect
In variance-based GSA, variance ratios (also known as the Sobol index ) were used as a sensitivity index. The variance ratio of the variance explained by each factor (covariate or random variable) over the overall variance of the outcome is calculated as given in Eq. 1.
The scheme of the algorithm to evaluate the sensitivity index is shown in Supplemental Figure S1. Monte-Carlo simulations were performed to simulate two sets of factors composed of covariates and inter-individual random variables, which are defined as ETA in NONMEM , for N subjects. The two factor sets for N subjects were defined as A and B for this explanation. A new factor set, AB(i), is generated, where the ith factors for N subjects come from B and all other factors for N subjects come from A. Using A, B, and AB(i), variances of the model-predicted outcome (Y) derived from the 1st-order or total-order effect of the ith factor are calculated based on the following equations which was modified  from the formula reported by Jansen  and Saltelli et al. .
Variance of the outcome derived from the 1st-order effect of the ith factor (Vi)
where V(Y) is the variance for the model-predicted outcome using the factors of A, N is the sample size, f(B)j is the model-predicted outcome from the factors of B for the jth subject, and f(AB(i))j is the model-predicted outcome from the factors of AB(i) for the jth subject.
Variance of the outcome derived from the total-order effect of the ith factor (Vitot)
where N is the sample size, f(A)j is the model-predicted outcome from the factors of A for the jth subject, and f(AB(i))j is the model-predicted outcome from the factors of AB(i) for the jth subject.
Vi is the variance of the outcome derived from a single effect of each covariate or ETA, while Vitot is the variance of the outcome derived from the overall effect of each covariate or ETA including interactions with other covariates and ETAs. The difference between the total-order and 1st-order effects arises due to the interaction of the factors. The sensitivity indices (variance ratios) were calculated based on variances from the 1st-order and total-order effects as follows.
Sensitivity index of the ith variable for the 1st order effect (Si)
Sensitivity index of the ith variable for the total order effect (Sitot)
An example of estimation of the sensitivity index for a covariate effect using a linear model for CL with a covariate effect and a random effect (specified below as Model 1) is presented in Supplemental Information 1. The overall variances, variances explained by each covariate and random variable, and sensitivity indices (variance ratios) were calculated using the soboljansen in the R library “sensitivity” . An example code in R is provided in Supplemental Information 2.
Evaluation of the sensitivity index using model simulations
The example models tested in this study are shown in Table 1 with the equations, key parameters, and outcomes. Each model is explained in more detail below.
A linear model for total clearance (CL) with a covariate effect and a random effect (Model 1) was assessed as a model with a simple structure and simple outcome to characterize the variance-based GSA. An exponential model was used for both the covariate and random effects. Two coefficients (0.822 and 1.40) were tested for the covariate effect, which corresponded to the inferential CL ratios of 1.5 and 2, respectively, as ratios of the CL value with the 95th percentile covariate relative to that with the median covariate. Three coefficients of variation for IIV were assumed to be 30%, 40%, and 60% (0.09, 0.16, and 0.36 of variances for ETAs) for a random effect. The covariates and ETAs were simulated to calculate CL. A logarithmic transformed CL was used as the outcome to calculate sensitivity indices.
An oral 1-compartment PK model with three covariates, one covariate on each of the three PK parameters, and no random effect for IIV for any PK parameter (Model 2) was assessed to test the effects of covariates on exposure indices. The covariates were generated to simulate plasma concentrations at every 0.1 h. Logarithms of Cmax, minimum concentrations (Cmin), and plasma concentrations at steady state were used as interested outcomes to calculate sensitivity indices.
A population PK model (3-compartment PK model; Model 3) of cefiderocol, a parenteral siderophore cephalosporin , was assessed with the sensitivity index as a model with influential covariates developed based on clinical data. As cefiderocol is mainly excreted unchanged via the kidneys, the renal function is an influential factor in the PK of cefiderocol. The PK model included the effects of creatinine clearance (CLcr) on CL and body weight on volume of distribution in the central and peripheral compartments (V1 and V2) with the power models. IIV was considered for CL, V1, and V2. Covariates (CLcr and body weight) were simulated from the distribution based on the original dataset used to construct the model, and ETAs for PK parameters were simulated according to the parameter estimates. Based on the parameter sets, plasma concentrations of cefiderocol were simulated. Logarithmic transformed PK parameters of cefiderocol (CL, V1, and V2) were used as PK outcomes to calculate sensitivity indices, which was compared with the inferential test. In addition, the logarithmic transformed steady-state plasma concentrations of cefiderocol were used as a time-varying outcome to calculate sensitivity indices. The plasma concentrations were simulated at every 0.1 h over the dosing interval (i.e., 8 h).
As a complex model, the QSP model for thrombopoiesis and platelet life-cycle  (Model 4) was assessed with the sensitivity index. The QSP model included mechanism-based system models for platelet life-cycle and the same PK model as the lusutrombopag PK/PD model . The QSP model can describe platelet count profiles including drug intervention (ie, the dose of lusutrombopag). Effects of hypothetical covariates that explained 50% of IIV for certain parameters (CL/F, EC50, PLT0, TPO, and PP) were incorporated in the QSP model. Body weight, a covariate on lusutrombopag PK, was simulated according to the distribution of body weight in the dataset used to develop the lusutrombopag model, and ETAs for the parameters were simulated according to the parameter estimates . Based on the parameter sets, platelet counts were simulated at every 2 days to estimate peak platelet counts. Logarithmic transformed peak platelet counts were used as an interested PD outcome to calculate sensitivity indices.
R version 3.6.3  was used for the simulations, calculations, and graphics.
For the linear model for CL with a covariate (Model 1), the variances of CL explained by a covariate or ETA are shown in Supplemental Figure S2, and the sensitivity indices are shown in Fig. 1. As an inferential test, CL simulated with the 95th percentile value of a covariate was compared to that simulated with the median value of a covariate (Fig. 2). The sensitivity indices for a larger covariate effect (the CL ratio of 2) were higher than those for a smaller covariate effect (the CL ratio of 1.5) in each case of ETA variances of 0.09, 0.16, and 0.36. The inferential test noted clear differences between the mean and 95th percentile in cases of a CL ratio of 2 in the inferential test (right panels in Fig. 2), in which the sensitivity indices for a covariate effect were 0.33–0.67 for the total-order effect (right panels in Fig. 1). The inferential test also noted clear differences between the mean and 95th percentile in cases where the variances of ETAs were 0.09 and 0.16 for the CL ratio of 1.5 (left panels in Fig. 2), in which the sensitivity indices were 0.41 and 0.28, respectively, for the total-order effect (left panels in Fig. 1). On the other hand, the CL ranges in the inferential test showed much overlap in cases where the variance of ETA was 0.36, in which the sensitivity index was a lower value of 0.15 (bottom left panels in Figs. 1 and 2). The sensitivity indices for covariate effects were dependent on the effect size as well as the ETA variances (i.e., the sizes of random effects), indicating that the sensitivity index represented the importance of a covariate in consideration with random effects.
For the oral 1-compartment PK model with three covariates (one on each of CL/F, V/F, and Ka) (Model 2), the sensitivity indices (variance ratios) for Cmax and Cmin are shown in Fig. 3. The sensitivity indices indicated that the covariate effect on V/F was the main factor in Cmax, and the covariate effects on CL/F and V/F were the main factors for Cmin. These findings agreed with the general understanding that Cmax is highly dependent on a covariate on V/F and that Cmin is highly dependent on a covariate on CL/F with a partial contribution of the covariate on V/F. The sensitivity indices for plasma concentrations at every 0.1 h are shown in Fig. 4. At around the time to maximum concentration (Tmax, 0.46 h as the mean value), the covariate effect on V/F was the main factor based on the sensitivity index. At the end of the time interval (i.e., 8 h), the covariate effects on CL/F and V/F were the main factors. The covariate effect on CL/F greatly contributed to the variability of plasma concentrations between the Tmax and the end of the dosing interval. The contribution of the covariate effect on V/F to the variability of plasma concentrations peaked at Tmax, and then decreased followed by an increase between the Tmax and the end of the dosing interval. The covariate effect on KA contributed to the variability of plasma concentrations just after the dose, and the contribution rapidly decreased to become negligible before achieving the peak concentration. The time course of the sensitivity indices was useful for understanding the time-varying contribution of each factor on the interested outcomes.
For the population PK model of cefiderocol (an antibiotic mainly excreted via the kidneys) (Model 3), the effect of CLcr was the main factor on the variability for CL, while the effects of body weight were relatively low compared with ETA (Fig. 5) on the variability for V1 and V2. The inferential test for covariate effects based on the point estimates and standard errors indicated that both CLcr and body weight were clinically significant covariates (Supplementary Figure S3). The difference between the sensitivity analysis and the inferential test was due to simultaneous consideration of random effects on the variability of the parameters in the GSA. The sensitivity indices for the plasma concentrations at every 0.1 h (Fig. 6) indicated that the effect of CLcr (a covariate on CL) was the main factor for the dosing interval, while the effect of body weight (covariates on V1 and V2) had a modest impact on the variability in the plasma concentrations. The ETA of CL was another meaningful factor for the variability of the plasma concentrations.
For the QSP model of thrombopoiesis and platelet life-cycle (Model 4), the sensitivity indices for the covariate effects and random effects on EC50 and PLT0 were higher than those for the other factors (Fig. 7), suggesting that the covariate effects on EC50 and PLT0 influenced the increase of platelets. Both sensitivity indices for the covariate effects on EC50 and PLT0 for the total-order effect were 0.23, while those for the other covariate effects were less than 0.05. The variance-based GSA can allow us to efficiently understand the contribution of each covariate effect even in a complex model.
The sensitivity index as variance ratios enabled the quantitative assessment of each covariate and random effect on overall variability for an interested outcome, and was useful for understanding the importance of a covariate effect on the tested models (PK models and a QSP model).
In the evaluation of the linear model for CL (Model 1), the sensitivity index of a minimal covariate effect (CL ratio = 1.5, ETA variance = 0.36) was 0.15 for the total-order effect, and those for the cases of larger covariate effects were 0.28–0.67 (Fig. 1). In the PK model of cefiderocol, the sensitivity index was 0.56 for the effect of CLcr on CL, which was previously identified as a clinically meaningful covariate . For the effects of body weight on the PK of cefiderocol (Model 3) and the platelet response of lusutrombopag (Model 4), which were identified as a statistically significant but not meaningful covariate in previous reports [11, 13], the sensitivity indices for the total-order effect were less than 0.04 over the dosing interval for Model 3 (Fig. 6) and were 0.02 for Model 4 (Fig. 7).
The sensitivity index of 0.2 may serve as a cut-off value to determine whether each of the covariate effects is meaningful or not in comparison with the overall variability. The cutoff was similar with the finding of Cannavo that a sensitivity index of less than 0.3 indicated the parameter was empirically irrelevant to the output . However, since the number of tested models was limited in this research, further investigation is required to identify the cutoff value of a sensitivity index.
Time courses of sensitivity indices are useful for understanding the change of importance of each covariate effect over time. For the oral 1-compartment PK model with three covariates (Model 2, Fig. 4), it was noted that (i) the covariate effect on KA contributed to plasma concentrations in the absorption phase, (ii) the effect of the covariate on V/F contributed to those around Tmax and in the elimination phase, and (iii) the covariate effect on CL/F contributed to those in the elimination phase.
In the evaluation of the population PK model of cefiderocol (Model 3), the effect of CLcr was the main factor for CL, while the effects of body weight for V1 and V2 were relatively low compared with ETA (Fig. 5). This was consistent with the visual inspection from the box plots that showed clear differences in CL among the renal function groups and the minimal difference in V1 among the body weight groups , although both were meaningful covariates identified in the inferential test (Supplementary Figure S3). These suggested that the effect of body weight partially explained the variability of V1 and V2, although the effect of body weight was expected to be a meaningful covariate. Similar sensitivity indices for the effect of CLcr and random variable (ie, ETA for CL) suggested that half of the variability of CL was explained by CLcr. In addition, it is because plasma concentration data used for model development were highly variable since most of data were from infectious patients with sparse PK samplings. On the other hand, the sensitivity index suggested that the variability for CL was explained mainly by CLcr. The time courses of sensitivity indices (Fig. 6) indicated that the effect of CLcr (a covariate on CL) was only the main factor for steady-state plasma concentrations of cefiderocol over the dosing interval, whereas the effects of body weight (covariates on V1 and V2) were modest in spite of being statistically significant covariates in the population PK analysis . The modest effect of body weight was probably because the effect of CLcr was more influential as a renally-excreted drug and the effect of body weight partially explained the variability of V1 and V2 (Fig. 5). The sensitivity indices can be applied to repeated measurable outcomes over time, such as plasma concentrations, for which the inferential test would not practically be applicable. The sensitivity index provided additional insight into the importance of covariate effects.
While the inferential test allows us to evaluate the effect of a change in one covariate on one parameter at a time, the variance-based GSA can simultaneously quantify the contributions of each covariate and random effects on overall variability for an interested outcome. Therefore, the variance-based GSA can efficiently quantify each covariate effect even in complex models, such as QSP models consisting of multi-components, for which the inferential test could not efficiently evaluate all of the covariate effects with consideration of the random effects. In addition, the GSA approach could quantify clinically importance of the covariate effects by evaluating the contribution of all covariate effects to the variability in the interested output. This is an advantage over the reported full model approaches evaluating all covariate effects simultaneously (eg, the contribution to unexplained variability in the parameters in FREM , the approximate likelihood ratio in Wald’s approximation method ). The GSA approach would support quantitative evaluation of individual covariate effects in the developed models.
In this study, only one example of QSP model was tested to evaluate effects of covariates using the proposed approach. There are various QSP models, in which model structures and complexity are different. The approach based on GSA should further be evaluated with a wide variety of QSP models.
Variance-based GSA offers an approach to evaluating each covariate effect by simultaneously and quantitatively assessing all covariate and random effects on interested outcomes in pharmacometrics.
Dartois C, Brendel K, Comets E, Laffont CM, Laveille C, Tranchand B, Mentre F, Lemenuel-Diot A, Girard P (2007) Overview of model-building strategies in population PK/PD analyses: 2002–2004 literature survey. Br J Clin Pharmacol 64:603–612
Gastonguay MR (2004) A full model estimation approach for covariate effects: inference based on clinical importance and estimation precision. AAPS J 6(S1):W4354
Karlsson M (2012) A full model approach based on the covariance matrix of parameters and covariates. PAGE 2455:21
Kowalski KG, Hutmacher MM (2001) Efficient screening of covariates in population models using Wald’s approximation to the likelihood ratio test. J Pharmacokinet Pharmacodyn 28:253–275
Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, Tarantola S (2010) Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comput Phys Commun 18:259–270
Zhang XY, Trame MN, Lesko LJ, Schmidt S (2015) Sobol sensitivity analysis: a tool to guide the development and evaluation of systems pharmacology models. CPT Pharmacometrics Syst Pharmacol 4:69–79
Sobol IM (1993) Sensitivity estimates for nonlinear mathematical models. Math Model Comp Exp 1:407–414
Beal SL, Sheiner LB, Boeckmann AJ (1989–2006) NONMEM users guide. Icon development solutions, Ellicott city
Iooss B, Da Veiga S, Janon A, Pujol G (2021) R package ‘sensitivity’ version 1.24.0. https://CRAN.R-project.org/package=sensitivity
Jansen MJW (1999) Analysis of variance designs for model output. Comput Phys Commun 117:35–43
Kawaguchi N, Katsube T, Echols R, Wajima T (2018) Population pharmacokinetic analysis of cefiderocol, a parenteral siderophore cephalosporin, in healthy subjects, subjects with various degrees of renal function, and patients with complicated urinary tract infection or acute uncomplicated pyelonephritis. Antimicrob Agents Chemother 62:e01391-e1417
Shimizu R, Katsube T, Wajima T (2021) Quantitative systems pharmacology model of thrombopoiesis and platelet life-cycle, and its application to thrombocytopenia based on chronic liver disease. CPT Pharmacomet Syst Pharmacol 10:489–499
Katsube T, Ishibashi T, Kano T, Wajima T (2016) Population pharmacokinetic and pharmacodynamic modeling of lusutrombopag, a newly developed oral thrombopoietin receptor agonist, in healthy subjects. Clin Pharmacokinet 55:1423–1433
R Core Team (2013) R: a language and environment for statistical computing [computer program]. R Foundation for Statistical Computing, Vienna
Cannavo F (2012) Sensitivity analysis for volcanic source modeling quality assessment and model selection. Comput Geosci 44:52–59
This study was supported by Shionogi & Co., Ltd. Takayuki Katsube is an employee of Shionogi & Co. Ltd. and Toshihiro Wajima was an employee of Shionogi & Co., Ltd at the time of this research.
Conflict of interest
This study was supported by Shionogi & Co., Ltd. Takayuki Katsube is an employee of Shionogi & Co. Ltd. and Toshihiro Wajima was an employee of Shionogi & Co., Ltd at the time of this research.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Below is the link to the electronic supplementary material.
About this article
Cite this article
Katsube, T., Wajima, T. Evaluation of covariate effects using variance-based global sensitivity analysis in pharmacometrics. J Pharmacokinet Pharmacodyn (2021). https://doi.org/10.1007/s10928-021-09775-8
- Variance-based global sensitivity analysis
- Covariate effect
- Non-linear mixed effects model