Abstract
The posterior predictive check (PPC) is a model evaluation tool. It assigns a value (pPPC) to the probability that the value of a given statistic computed from data arising under an analysis model is as or more extreme than the value computed from the real data themselves. If this probability is too small, the analysis model is regarded as invalid for the given statistic. Properties of the PPC for pharmacokinetic (PK) and pharmacodynamic (PD) model evaluation are examined herein for a particularly simple simulation setting: extensive sampling of a single individual's data arising from simple PK/PD and error models. To test the performance characteristics of the PPC, repeatedly, “real” data are simulated and for a variety of statistics, the PPC is applied to an analysis model, which may (null hypothesis) or may not (alternative hypothesis) be identical to the simulation model. Five models are used here: (PK1) mono-exponential with proportional error, (PK2) biexponential with proportional error, (PK2ε) biexponential with additive error, (PD1) E max model with additive error under the logit transform, and (PD2) sigmoid E max model with additive error under the logit transform. Six simulation/analysis settings are studied. The first three, (PK1/PK1), (PK2/PK2), and (PD1/PD1) evaluate whether the PPC has appropriate type-I error level, whereas the second three (PK2/PK1), (PK2ε/PK2), and (PD2/PD1) evaluate whether the PPC has adequate power. For a set of 100 data sets simulated/analyzed under each model pair according to a stipulated extensive sampling design, the pPPC is computed for a number of statistics in three different ways (each way uses a different approximation to the posterior distribution on the model parameters). We find that in general; (i) The PPC is conservative under the null in the sense that for many statistics, prob(pPPC≤α)<α for small α. With respect to such statistics, this means that useful models will rarely be regarded incorrectly as invalid. A high correlation of a statistic with the parameter estimates obtained from the same data used to compute the statistic (a measure of statistical “sufficiency”) tends to identify the most conservative statistics. (ii) Power is not very great, at least for the alternative models we tested, and it is especially poor with “statistics” that are in part a function of parameters as well as data. Although there is a tendency for nonsufficient statistics (as we have measured this) to have greater power, this is by no means an infallible diagnostic. (iii) No clear advantage for one or another method of approximating the posterior distribution on model parameters is found.
Similar content being viewed by others
REFERENCES
L. B. Sheiner and J.-L. Steimer. Pharmacokinetic pharmacodynamic modeling in drug development, Ann. Reυ. Pharmacol. Toxicol. 40:67–96 (2000).
Guidance for Industry—Population Pharmacokinetics (available at http:__www.fda.gov_ cder_guidance_index.htm), CDER, FDA, 1999.
F. Mentré and M.-E. Ebelin. Validation of population pharmacokinetic/pharmacodyn-amic analyses: Review of proposed approaches. In The Population Approach: Measuring and Managing Variability in Response Concentration and Dose (COST B1), Office for official publications of the European Communities, Brussels, 1997, pp. 149–160.
R. Bruno, N. Vivier, J. C. Vergniol, S. L. De Phillips, G. Montay, and L. B. Sheiner. A population pharmacokinetic model for docetaxel (Taxotere): model building and validation. J. Pharmacokin. Biopharm. 24:153–172 (1996).
E. B. Roecker. Prediction error and its estimation for subset-selected models. Techno-metrics 33:459–468 (1991).
D. B. Rubin. Bayesian justifiable and relevant frequency calculations for the applied statistician. Ann. Statist. 12:1151–1172 (1984).
A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian Data Analysis, Chapman and Hall, London, 1995.
A. Gelman, F. Bios, and J. Jiang. Physiological pharmacokinetic analysis using population modeling and informative prior distribution, J. Am. Stat. Assoc. 91:1400–1412 (1996).
T. R. Belin and D. B. Rubin. The analysis of repeated-measures data on schizophrenic reaction times using mixture models. Statist. Med. 14:747–768 (1995).
A. Gelman, X.-L. Meng, and H. Stern. Posterior predictive assessment of model fitness via realized discrepancies, Statist. Sinica 6:733–807 (1996).
P. Girard, T. F. Blaschke, H. Kastrissios, and L. B. Sheiner. A Markov mixed effect regression model for drug compliance. Statist. Med. 17:2313–2333 (1998).
E. H. Cox, C. V.-Follet, A. E. Fuseau, S. Kenkare, S. L. Beal, and L. B. Sheiner. A population pharmacokinetic–pharmacodynamic analysis of repeated measures time-to-event pharmacodynamic responses: The Antiemetic Effect of Ondansetron. J. Pharmaco-kin. Biopharmaceut. 27:625–644 (1999).
E. O. George and G. S. Mudholkar. P values for two-sided tests. Biometrical J. 32:747–751 (1990).
S-plus Users Manual, Statistical Sciences, Inc., Seattle, WA (1993).
G. E. P. Box and M. E. Muller. A note on the generation of random normal deviates. Ann. Math. Statist. 29:160–161 (1958).
P. Lewis, A. Goodman, and J. Miller. A pseudonormal number generator for the system 360. IBM Systems J. 8:135–146 (1969).
N. M. Laird and T. A. Louis. Empirical Bayes confidence intervals for a series of related experiments. Biometrics 45:481–495 (1989).
S. L. Beal and L. B. Sheiner. NONMEM Users' Guides, NONMEM Project Group, University of California, San Francisco, CA, 1996.
L. Breiman and J. H. Friedman. Estimating optimal transformation for multiple regression and correlation. J. Am. Statist. Assoc. 80:580–619 (1985).
W.-Y. Loh. Calibrating confidence coefficients. J. Am. Statist. Assoc. 82:155–162 (1987).
M. A. Newton and C. J. Geyer. Bootstrap recycling: A Monte Carlo alternative to the nested bootstrap. J. Am. Statist. Assoc. 89:905–912 (1994).
R. Beran. Prepivoting test statistics: A bootstrap view of asymptotic refinements. J. Am. Statist. Assoc. 83:687–697 (1988).
P. Hall and M. A. Martin. On bootstrap resampling and iteration. Biometrika 75:661–771 (1988).
D. V. Hinkley and S. Shi. Importance sampling and the nested bootstrap. Biometrika 76:435–446 (1989).
Rights and permissions
About this article
Cite this article
Yano, Y., Beal, S.L. & Sheiner, L.B. Evaluating Pharmacokinetic/Pharmacodynamic Models Using the Posterior Predictive Check. J Pharmacokinet Pharmacodyn 28, 171–192 (2001). https://doi.org/10.1023/A:1011555016423
Issue Date:
DOI: https://doi.org/10.1023/A:1011555016423