Pharmaceutical Research

, Volume 33, Issue 12, pp 2979–2988 | Cite as

Enhanced Method for Diagnosing Pharmacometric Models: Random Sampling from Conditional Distributions

  • Marc Lavielle
  • Benjamin RibbaEmail author
Research Paper



For nonlinear mixed-effects pharmacometric models, diagnostic approaches often rely on individual parameters, also called empirical Bayes estimates (EBEs), estimated through maximizing conditional distributions. When individual data are sparse, the distribution of EBEs can “shrink” towards the same population value, and as a direct consequence, resulting diagnostics can be misleading.


Instead of maximizing each individual conditional distribution of individual parameters, we propose to randomly sample them in order to obtain values better spread out over the marginal distribution of individual parameters.


We evaluated, through diagnostic plots and statistical tests, hypothesis related to the distribution of the individual parameters and show that the proposed method leads to more reliable results than using the EBEs. In particular, diagnostic plots are more meaningful, the rate of type I error is correctly controlled and its power increases when the degree of misspecification increases. An application to the warfarin pharmacokinetic data confirms the interest of the approach for practical applications.


The proposed method should be implemented to complement EBEs-based approach for increasing the performance of model diagnosis.


model diagnostics modeling and simulation pharmacokinetics and pharmacodynamics 



Empirical Bayes estimates


Maximum a posteriori


Markov Chain Monte Carlo






Posterior predictive checks


Visual predictive checks

Supplementary material

11095_2016_2020_Fig6_ESM.gif (30 kb)
Supplemental Figure S1

Warfarin PK data. (GIF 30 kb)

11095_2016_2020_MOESM1_ESM.eps (162 kb)
High resolution image (EPS 161 kb)
11095_2016_2020_Fig7_ESM.gif (41 kb)
Supplemental Figure S2

Empirical distribution of the individual parameters. The estimated pdf’s are displayed in solid line. Top: EBEs, bottom: sampled from the conditional distributions. (GIF 40 kb)

11095_2016_2020_MOESM2_ESM.eps (94 kb)
High resolution image (EPS 94.4 kb)
11095_2016_2020_Fig8_ESM.gif (37 kb)
Supplemental Figure S3

Sampled individual parameters versus weight. The correlation coefficient and the p-value of the test r = 0 are displayed for each parameters. (GIF 37 kb)

11095_2016_2020_MOESM3_ESM.eps (358 kb)
High resolution image (EPS 358 kb)
11095_2016_2020_Fig9_ESM.gif (54 kb)
Supplemental Figure S4

Joint distributions of the sampled random effects. The correlation coefficient and the p-value of the test r = 0 are displayed for each pair of parameters. (GIF 53 kb)

11095_2016_2020_MOESM4_ESM.eps (505 kb)
High resolution image (EPS 505 kb)


  1. 1.
    Bonate PL. Pharmacokinetic-pharmacodynamic modeling and simulation. Springer. 2011.Google Scholar
  2. 2.
    Lavielle M. Mixed effects models for the population approach: models, tasks, methods and tools. Chapman and Hall/CRC. 2014.Google Scholar
  3. 3.
    Comets E, Brendel K, Mentré F. Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: the npde add-on package for R. Comput Methods Prog Biomed. 2008;90(2):154–66.CrossRefGoogle Scholar
  4. 4.
    Comets E, Brendel K. Model evaluation in nonlinear mixed effect models, with applications to pharmacokinetics. Journal de la Société Française de Statistique. 2010;151(1):106–28.Google Scholar
  5. 5.
    Karlsson M, Savic R. Diagnosing model diagnostics. Clin Pharmacol Therapeut. 2007;82(1):17–20.CrossRefGoogle Scholar
  6. 6.
    Lavielle M, Bleakley K. Automatic data binning for improved visual diagnosis of pharmacometric models. J Pharmacokinet Pharmacodyn. 2011;38(6):861–71.CrossRefPubMedGoogle Scholar
  7. 7.
    Yano Y, Beal SL, Sheiner LB. Evaluating pharmacokinetic/pharmacodynamic models using the posterior predictive check. J Pharmacokinet Pharmacodyn. 2001;28(2):171–92.CrossRefPubMedGoogle Scholar
  8. 8.
    Combes F, Retout S, Frey N, Mentré F. Powers of the likelihood ratio test and the correlation test using empirical bayes estimates for various shrinkages in population pharmacokinetics. CPT: Pharmacom Syst Pharmacol. 2014;3(4):1–9.Google Scholar
  9. 9.
    Savic R, Karlsson M. Importance of shrinkage in empirical Bayes estimates for diagnostics: problems and solutions. AAPS J. 2009;11(3):558–69.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Drikvandi R, Verbeke G, Khodadadi A, Nia VP. Testing multiple variance components in linear mixed-effects models. Biostatistics. 2013;14(1):144–59.CrossRefPubMedGoogle Scholar
  11. 11.
    Li Z, Zhu L. A new test for random effects in linear mixed models with longitudinal data. J Stat Plan Infer. 2013;143(1):82–95.CrossRefGoogle Scholar
  12. 12.
    Mun J, Lindstrom MJ. Diagnostics for repeated measurements in linear mixed effects models. Stat Med. 2013;32(8):1361–75.CrossRefPubMedGoogle Scholar
  13. 13.
    Alonso A, Litière S, Molenberghs G. A family of tests to detect misspecifications in the random-effects structure of generalized linear mixed models. Comput Stat Data Anal. 2008;52(9):4474–86.CrossRefGoogle Scholar
  14. 14.
    Huang X. Diagnosis of random-effect model misspecification in generalized linear mixed models for binary response. Biometrics. 2009;65(2):361–8.CrossRefPubMedGoogle Scholar
  15. 15.
    Vonesh EF, Chinchilli VM, Pu K. Goodness-of-fit in generalized nonlinear mixed-effects models. Biometrics. 1996;52(2):572–87.CrossRefPubMedGoogle Scholar
  16. 16.
    Claeskens G, Hart J. Goodness-of-fit tests in mixed models. Test. 2009;18(2):213–39.CrossRefGoogle Scholar
  17. 17.
    Ritz C. Goodness-of-fit tests for mixed models. Scand J Stat. 2004;31(3):443–58.CrossRefGoogle Scholar
  18. 18.
    Meintanis SG, Portnoy S. Specification tests in mixed effects models. J Stat Plan Infer. 2011;141(8):2545–55.CrossRefGoogle Scholar
  19. 19.
    Parke J. A procedure for generating bootstrap samples for the validation of nonlinear mixed-effects population models. Comput Methods Prog Biomed. 1999;59(1):19–29.CrossRefGoogle Scholar
  20. 20.
    Laffont CM, Concordet D. A new exact test for the evaluation of population pharmacokinetic and/or pharmacodynamic models using random projections. Pharm Res. 2011;28(8):1948–62.CrossRefPubMedGoogle Scholar
  21. 21.
    Robert CP, Casella G. Monte Carlo statistical methods. Springer texts in statistics. 2004.Google Scholar
  22. 22.
    Holford N. Clinical pharmacokinetics and pharmacodynamics of warfarin. understanding the dose-effect relationship. Clin Pharmacokinet. 1986;11(6):483–504.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Inria Saclay & CMAP, Ecole PolytechniqueUniversity Paris-SaclaySaint-AubinFrance
  2. 2.Roche Pharma Research and Early DevelopmentRoche Innovation Center BaselBaselSwitzerland

Personalised recommendations