Skip to main content
Log in

Use of normalized prediction distribution errors for assessing population physiologically-based pharmacokinetic model adequacy

  • Original Paper
  • Published:
Journal of Pharmacokinetics and Pharmacodynamics Aims and scope Submit manuscript


Currently employed methods for qualifying population physiologically-based pharmacokinetic (Pop-PBPK) model predictions of continuous outcomes (e.g., concentration–time data) fail to account for within-subject correlations and the presence of residual error. In this study, we propose a new method for evaluating Pop-PBPK model predictions that account for such features. The approach focuses on deriving Pop-PBPK-specific normalized prediction distribution errors (NPDE), a metric that is commonly used for population pharmacokinetic model validation. We describe specific methodological steps for computing NPDE for Pop-PBPK models and define three measures for evaluating model performance: mean of NPDE, goodness-of-fit plots, and the magnitude of residual error. Utility of the proposed evaluation approach was demonstrated using two simulation-based study designs (positive and negative control studies) as well as pharmacokinetic data from a real-world clinical trial. For the positive-control simulation study, where observations and model simulations were generated under the same Pop-PBPK model, the NPDE-based approach denoted a congruency between model predictions and observed data (mean of NPDE =  − 0.01). In contrast, for the negative-control simulation study, where model simulations and observed data were generated under different Pop-PBPK models, the NPDE-based method asserted that model simulations and observed data were incongruent (mean of NPDE =  − 0.29). When employed to evaluate a previously developed clindamycin PBPK model against prospectively collected plasma concentration data from 29 children, the NPDE-based method qualified the model predictions as successful (mean of NPDE = 0). However, when pediatric subpopulations (e.g., infants) were evaluated, the approach revealed potential biases that should be explored.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others


  1. Grimstein M, Yang Y, Zhang X, Grillo J, Huang SM, Zineh I, Wang Y (2019) Physiologically based pharmacokinetic modeling in regulatory science: an update from the U.S. Food and Drug Administration's Office of Clinical Pharmacology. J Pharm Sci 108(1):21–25.

    Article  CAS  PubMed  Google Scholar 

  2. Guideline on the Qualification and Reporting of Physiologically Based Pharmacokinetic (PBPK) Modelling and Simulation (2016) Eurpoean Medicines Agency. Accessed 23 July 2018

  3. Physiologically Based Pharmacokinetic Analyses — Format and Content Guidance for Industry (Draft Guidance) (2016) U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research Accessed 23 July 2018

  4. Emoto C, Fukuda T, Johnson TN, Neuhoff S, Sadhasivam S, Vinks AA (2017) Characterization of contributing factors to variability in morphine clearance through PBPK modeling implemented with OCT1 transporter. CPT Pharmacomet Syst Pharmacol 6(2):110–119.

    Article  CAS  Google Scholar 

  5. Maharaj AR, Barrett JS, Edginton AN (2013) A workflow example of PBPK modeling to support pediatric research and development: case study with lorazepam. AAPS J 15(2):455–464.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Zhou W, Johnson TN, Xu H, Cheung S, Bui KH, Li J, Al-Huniti N, Zhou D (2016) Predictive performance of physiologically based pharmacokinetic and population pharmacokinetic modeling of renally cleared drugs in children. CPT Pharmacomet Syst Pharmacol 5(9):475–483.

    Article  CAS  Google Scholar 

  7. Diestelhorst C, Boos J, McCune JS, Russell J, Kangarloo SB, Hempel G (2013) Physiologically based pharmacokinetic modelling of Busulfan: a new approach to describe and predict the pharmacokinetics in adults. Cancer Chemother Pharmacol 72(5):991–1000.

    Article  CAS  PubMed  Google Scholar 

  8. Laughon MM, Benjamin DK Jr, Capparelli EV, Kearns GL, Berezny K, Paul IM, Wade K, Barrett J, Smith PB, Cohen-Wolkowiez M (2011) Innovative clinical trial design for pediatric therapeutics. Expert Rev Clin Pharmacol 4(5):643–652.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Hornik CP, Wu H, Edginton AN, Watt K, Cohen-Wolkowiez M, Gonzalez D (2017) Development of a pediatric physiologically-based pharmacokinetic model of clindamycin using opportunistic pharmacokinetic data. Clin Pharmacokinet 56(11):1343–1353.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Willmann S, Hohn K, Edginton A, Sevestre M, Solodenko J, Weiss W, Lippert J, Schmitt W (2007) Development of a physiology-based whole-body population model for assessing the influence of individual variability on the pharmacokinetics of drugs. J Pharmacokinet Pharmacodyn 34(3):401–431.

    Article  PubMed  Google Scholar 

  11. Samant TS, Lukacova V, Schmidt S (2017) Development and qualification of physiologically based pharmacokinetic models for drugs with atypical distribution behavior: a desipramine case study. CPT Pharmacomet Syst Pharmacol 6(5):315–321.

    Article  CAS  Google Scholar 

  12. Maharaj AR, Wu H, Hornik CP, Cohen-Wolkowiez M (2019) Pitfalls of using numerical predictive checks for population physiologically-based pharmacokinetic model evaluation. J Pharmacokinet Pharmacodyn.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Salerno SN, Edginton A, Cohen-Wolkowiez M, Hornik CP, Watt KM, Jamieson BD, Gonzalez D (2017) Development of an adult physiologically based pharmacokinetic model of solithromycin in plasma and epithelial lining fluid. CPT Pharmacomet Syst Pharmacol 6(12):814–822.

    Article  CAS  Google Scholar 

  14. Brendel K, Comets E, Laffont C, Laveille C, Mentre F (2006) Metrics for external model evaluation with an application to the population pharmacokinetics of gliclazide. Pharm Res 23(9):2036–2049.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Comets E, Brendel K, Mentre F (2008) Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: the npde add-on package for R. Comput Methods Progr Biomed 90(2):154–166.

    Article  Google Scholar 

  16. Brendel K, Comets E, Laffont C, Mentre F (2010) Evaluation of different tests based on observations for external model evaluation of population analyses. J Pharmacokinet Pharmacodyn 37(1):49–65.

    Article  PubMed  Google Scholar 

  17. Keizer R (2019) vpc: create visual predictive checks. R package version 1.1.9000.

  18. Nguyen TH, Mouksassi MS, Holford N, Al-Huniti N, Freedman I, Hooker AC, John J, Karlsson MO, Mould DR, Perez Ruixo JJ, Plan EL, Savic R, van Hasselt JG, Weber B, Zhou C, Comets E, Mentre F, Model Evaluation Group of the International Society of Pharmacometrics Best Practice C (2017) Model evaluation of continuous data pharmacometric models: metrics and graphics. CPT Pharmacometrics Syst Pharmacol 6(2):87–109.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. PK-Sim® Ontogeny Database (version 7.1) (2017)

  20. Comets E, Brendel K, Nguyen TH, Mentre F (2012) User guide for npde 2.0.

  21. Rodgers T, Leahy D, Rowland M (2005) Physiologically based pharmacokinetic modeling 1: predicting the tissue distribution of moderate-to-strong bases. J Pharm Sci 94(6):1259–1276.

    Article  CAS  PubMed  Google Scholar 

  22. Rodgers T, Leahy D, Rowland M (2005) Tissue distribution of basic drugs: accounting for enantiomeric, compound and regional differences amongst beta-blocking drugs in rat. J Pharm Sci 94(6):1237–1248.

    Article  CAS  PubMed  Google Scholar 

  23. Rodgers T, Rowland M (2006) Physiologically based pharmacokinetic modelling 2: predicting the tissue distribution of acids, very weak bases, neutrals and zwitterions. J Pharm Sci 95(6):1238–1257.

    Article  CAS  PubMed  Google Scholar 

  24. Bergstrand M, Hooker AC, Wallin JE, Karlsson MO (2011) Prediction-corrected visual predictive checks for diagnosing nonlinear mixed-effects models. AAPS J 13(2):143–151.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Knoppert D, Reed M, Benavides S, Totton J, Hoff D, Moffett B, Norris K, Vaillancourt R, Aucoin R, Worthington M (2007) Position paper: paediatric age categories to be used in differentiating between listing on a model essential medicines list for children. Accessed 2 Aug 2018

  26. Anderson BJ, Holford NH (2013) Understanding dosing: children are small adults, neonates are immature children. Arch Dis Child 98(9):737–744.

    Article  PubMed  Google Scholar 

  27. Edginton A, Willmann S (2006) Physiology-based versus allometric scaling of clearance in children; an eliminating process based comparison. Paediatr Perinat Drug Ther 7(3):146–153.

    Article  CAS  Google Scholar 

  28. Karlsson MO, Sheiner LB (1993) The importance of modeling interoccasion variability in population pharmacokinetic analyses. J Pharmacokinet Biopharm 21(6):735–750

    Article  CAS  PubMed  Google Scholar 

  29. Bouazza N, Pestre V, Jullien V, Curis E, Urien S, Salmon D, Treluyer JM (2012) Population pharmacokinetics of clindamycin orally and intravenously administered in patients with osteomyelitis. Br J Clin Pharmacol 74(6):971–977.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Bloomfield C, Staatz CE, Unwin S, Hennig S (2016) Assessing predictive performance of published population pharmacokinetic models of intravenous tobramycin in pediatric patients. Antimicrob Agents Chemother 60(6):3407–3414.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Zhao W, Kaguelidou F, Biran V, Zhang D, Allegaert K, Capparelli EV, Holford N, Kimura T, Lo YL, Peris JE, Thomson A, van den Anker JN, Fakhoury M, Jacqz-Aigrain E (2013) External evaluation of population pharmacokinetic models of vancomycin in neonates: the transferability of published models to different clinical settings. Br J Clin Pharmacol 75(4):1068–1080.

    Article  CAS  PubMed  Google Scholar 

  32. Khalil F, Laer S (2014) Physiologically based pharmacokinetic models in the prediction of oral drug exposure over the entire pediatric age range-sotalol as a model drug. AAPS J 16(2):226–239.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Poulin P, Jones HM, Jones RD, Yates JW, Gibson CR, Chien JY, Ring BJ, Adkison KK, He H, Vuppugalla R, Marathe P, Fischer V, Dutta S, Sinha VK, Bjornsson T, Lave T, Ku MS (2011) PhRMA CPCDC initiative on predictive models of human pharmacokinetics, part 1: goals, properties of the PhRMA dataset, and comparison with literature datasets. J Pharm Sci 100(10):4050–4073.

    Article  CAS  PubMed  Google Scholar 

  34. Mould DR, Upton RN (2013) Basic concepts in population modeling, simulation, and model-based drug development-part 2: introduction to pharmacokinetic modeling methods. CPT Pharmacomet Syst Pharmacol 2:e38.

    Article  CAS  Google Scholar 

  35. Sager JE, Yu J, Ragueneau-Majlessi I, Isoherranen N (2015) Physiologically based pharmacokinetic (PBPK) modeling and simulation approaches: a systematic review of published models, applications, and model verification. Drug Metab Dispos 43(11):1823–1837.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Gonzalez D, Delmore P, Bloom BT, Cotten CM, Poindexter BB, McGowan E, Shattuck K, Bradford KK, Smith PB, Cohen-Wolkowiez M, Morris M, Yin W, Benjamin DK Jr, Laughon MM (2016) Clindamycin pharmacokinetics and safety in preterm and term infants. Antimicrob Agents Chemother 60(5):2888–2894.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Gonzalez D, Melloni C, Yogev R, Poindexter BB, Mendley SR, Delmore P, Sullivan JE, Autmizguine J, Lewandowski A, Harper B, Watt KM, Lewis KC, Capparelli EV, Benjamin DK Jr, Cohen-Wolkowiez M, Best Pharmaceuticals for Children Act - Pediatric Trials Network Administrative Core C (2014) Use of opportunistic clinical data and a population pharmacokinetic model to support dosing of clindamycin for premature infants to adolescents. Clin Pharmacol Ther 96(4):429–437.

    Article  CAS  PubMed  Google Scholar 

  38. Nguyen TH, Comets E, Mentre F (2012) Extension of NPDE for evaluation of nonlinear mixed effect models in presence of data below the quantification limit with applications to HIV dynamic model. J Pharmacokinet Pharmacodyn 39(5):499–518.

    Article  PubMed  Google Scholar 

  39. Maharaj AR, Edginton AN (2014) Physiologically based pharmacokinetic modeling and simulation in pediatric drug development. CPT Pharmacomet Syst Pharmacol 3:e150.

    Article  CAS  Google Scholar 

Download references


This study was funded by the National Institutes of Health (1R01-HD076676-01A1; M.C.W.).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Michael Cohen-Wolkowiez.

Ethics declarations

Conflict of interest

M.C.W. receives support for research from the NIH (5R01-HD076676), NIH (HHSN275201000003I), NIAID/NIH (HHSN272201500006I), FDA (1U18-FD006298), the Biomedical Advanced Research and Development Authority (HHSO1201300009C), and from the industry for the drug development in adults and children ( C.P.H. receives salary support for research from National Institute for Child Health and Human Development (NICHD) (K23HD090239), the U.S. government for his work in pediatric and neonatal clinical pharmacology (Government Contract HHSN267200700051C, PI: Benjamin, under the Best Pharmaceuticals for Children Act), and industry for drug development in children. KJD has received research support from Merck, Inc. and Pfizer, Inc. and is supported by NICHD K23HD091365.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 1295 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maharaj, A.R., Wu, H., Hornik, C.P. et al. Use of normalized prediction distribution errors for assessing population physiologically-based pharmacokinetic model adequacy. J Pharmacokinet Pharmacodyn 47, 199–218 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: