The AAPS Journal

, Volume 13, Issue 2, pp 143–151 | Cite as

Prediction-Corrected Visual Predictive Checks for Diagnosing Nonlinear Mixed-Effects Models

  • Martin Bergstrand
  • Andrew C. Hooker
  • Johan E. Wallin
  • Mats O. Karlsson
Research Article


Informative diagnostic tools are vital to the development of useful mixed-effects models. The Visual Predictive Check (VPC) is a popular tool for evaluating the performance of population PK and PKPD models. Ideally, a VPC will diagnose both the fixed and random effects in a mixed-effects model. In many cases, this can be done by comparing different percentiles of the observed data to percentiles of simulated data, generally grouped together within bins of an independent variable. However, the diagnostic value of a VPC can be hampered by binning across a large variability in dose and/or influential covariates. VPCs can also be misleading if applied to data following adaptive designs such as dose adjustments. The prediction-corrected VPC (pcVPC) offers a solution to these problems while retaining the visual interpretation of the traditional VPC. In a pcVPC, the variability coming from binning across independent variables is removed by normalizing the observed and simulated dependent variable based on the typical population prediction for the median independent variable in the bin. The principal benefit with the pcVPC has been explored by application to both simulated and real examples of PK and PKPD models. The investigated examples demonstrate that pcVPCs have an enhanced ability to diagnose model misspecification especially with respect to random effects models in a range of situations. The pcVPC was in contrast to traditional VPCs shown to be readily applicable to data from studies with a priori and/or a posteriori dose adaptations.


mixed-effects modeling model diagnostics pcVPC prediction correction VPC 

Supplementary material

12248_2011_9255_MOESM1_ESM.pdf (197 kb)
ESM 1 (PDF 403 kb)


  1. 1.
    Orloff J, Douglas F, Pinheiro J, Levinson S, Branson M, Chaturvedi P, et al. The future of drug development: advancing clinical trial design. Nat Rev Drug Discov. 2009;8(12):949–57.PubMedGoogle Scholar
  2. 2.
    Karlsson MO, Savic RM. Diagnosing model diagnostics. Clin Pharmacol Ther. 2007;82(1):17–20.PubMedCrossRefGoogle Scholar
  3. 3.
    Sheiner LB, Beal SL, Dunne A. Analysis of nonrandomly censored ordered categorical longitudinal data from analgesic trials. J Am Stat Assoc. 1997;92(440):1235–44.CrossRefGoogle Scholar
  4. 4.
    Holford N. The visual predictive check—superiority to standard diagnostic (Rorschach) plots. PAGE 14 (2005) Abstr 738 [].
  5. 5.
    Karlsson MO, Holford N. A tutorial on visual predictive checks. PAGE 17 (2008) Abstr 1434 [].
  6. 6.
    Post TM, Freijer JI, Ploeger BA, Danhof M. Extensions to the visual predictive check to facilitate model performance evaluation. J Pharmacokinet Pharmacodyn. 2008;35(2):185–202.PubMedCrossRefGoogle Scholar
  7. 7.
    Wang DD, Zhang S. Standardized visual predictive check—how and when to used it in model evaluation. PAGE 18 (2009) Abstr 1501 [].
  8. 8.
    Wallin JE, Friberg LE, Fasth A, Staatz CE. Population pharmacokinetics of tacrolimus in pediatric hematopoietic stem cell transplant recipients: new initial dosage suggestions and a model-based dosage adjustment tool. Ther Drug Monit. 2009 Jun 11.Google Scholar
  9. 9.
    Gibiansky L. Parameter estimates of population models: comparison of nonmem versions and estimation methods. PAGE 17 (2008) Abstr 1268 [].
  10. 10.
    Brynne L, McNay JL, Schaefer HG, Swedberg K, Wiltse CG, Karlsson MO. Pharmacodynamic models for the cardiovascular effects of moxonidine in patients with congestive heart failure. Br J Clin Pharmacol. 2001;51(1):35–43.PubMedCrossRefGoogle Scholar
  11. 11.
    Beal SL, Sheiner LB, Boeckmann AJ. NONMEM user’s guides. Ellicot City: MD: Icon Development Solutions; 1989–2006.Google Scholar
  12. 12.
    Lindbom L, Pihlgren P, Jonsson EN. PsN-Toolkit—a collection of computer intensive statistical methods for non-linear mixed effect modeling using NONMEM. Comput Method Programs Biomed. 2005;79(3):241–57.CrossRefGoogle Scholar
  13. 13.
    Lindbom L, Ribbing J, Jonsson EN. Perl-speaks-NONMEM (PsN)—a Perl module for NONMEM related programming. Comput Method Programs Biomed. 2004;75(2):85–94.CrossRefGoogle Scholar
  14. 14.
    Harling K, Ueckert S, Hooker AC, Jonsson EN, Karlsson MO. Xpose and Perl speaks NONMEM (PsN). PAGE 19 (2010) Abstr 1842 [].
  15. 15.
    Jonsson EN, Karlsson MO. Xpose—an S-PLUS based population pharmacokinetic/pharmacodynamic model building aid for NONMEM. Comput Method Programs Biomed. 1999;58(1):51–64.CrossRefGoogle Scholar
  16. 16.
    R: a language and environment for statistical computing. R Foundation for Statistical Computing; 2009 [2009-12-30]; Available from:
  17. 17.
    Box GEP, Draper NR. Empirical model-building and response surfaces: Wiley; 1987.Google Scholar
  18. 18.
    Bergstrand M, Hooker AC, Wallin JE, Karlsson MO. Prediction corrected visual predictive checks. ACoP (2009) Abstr F7
  19. 19.
    Ma G, Olsson BL, Rosenborg J, Karlsson MO. Quantifying lung function progression in asthma. PAGE 18 (2009) Abstr 1562 [].
  20. 20.
    Dansirikul C, Silber HE, Karlsson MO. Approaches to handling pharmacodynamic baseline responses. J Pharmacokinet Pharmacodyn. 2008;35(3):269–83.PubMedCrossRefGoogle Scholar
  21. 21.
    Bergstrand M, Karlsson MO. Handling data below the limit of quantification in mixed effect models. AAPS J. 2009;11(2):371–80.PubMedCrossRefGoogle Scholar
  22. 22.
    Friberg LE, de Greef R, Kerbusch T, Karlsson MO. Modeling and simulation of the time course of asenapine exposure response and dropout patterns in acute schizophrenia. Clin Pharmacol Ther. 2009;86(1):84–91.PubMedCrossRefGoogle Scholar
  23. 23.
    Kuhn E, Lavielle M. Maximum likelihood estimation in nonlinear mixed effects models. Comput Stat Data Anal. 2005;49(4):1020–38.CrossRefGoogle Scholar
  24. 24.
    Khachman D, Laffont CM, Concordet D. You have problems to interpret VPC? Try VIPER! PAGE 19 (2010) Abstr 1892 [].
  25. 25.
    Laffont CM, Concordet D. A new exact test to globally assess a population PK and/or PD model. PAGE 18 (2009) Abstr 1633 [].
  26. 26.
    Comets E, Brendel K, Mentré F. Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: the npde add-on package for R. Comput Meth Programs Biomed. 2008;90(2):154–66.CrossRefGoogle Scholar
  27. 27.
    Comets E, Brendel K, Mentré F. Model evaluation in nonlinear mixed effect models, with applications to pharmacokinetics. J Soc Fr Stat. 2010;151(1):106–28.Google Scholar

Copyright information

© American Association of Pharmaceutical Scientists 2011

Authors and Affiliations

  • Martin Bergstrand
    • 1
  • Andrew C. Hooker
    • 1
  • Johan E. Wallin
    • 1
    • 2
  • Mats O. Karlsson
    • 1
  1. 1.Department of Pharmaceutical BiosciencesUppsala UniversityUppsalaSweden
  2. 2.Lilly PKPD&TSWindlesham SurreyUK

Personalised recommendations