Abstract
Currently employed methods for qualifying population physiologically-based pharmacokinetic (Pop-PBPK) model predictions of continuous outcomes (e.g., concentration–time data) fail to account for within-subject correlations and the presence of residual error. In this study, we propose a new method for evaluating Pop-PBPK model predictions that account for such features. The approach focuses on deriving Pop-PBPK-specific normalized prediction distribution errors (NPDE), a metric that is commonly used for population pharmacokinetic model validation. We describe specific methodological steps for computing NPDE for Pop-PBPK models and define three measures for evaluating model performance: mean of NPDE, goodness-of-fit plots, and the magnitude of residual error. Utility of the proposed evaluation approach was demonstrated using two simulation-based study designs (positive and negative control studies) as well as pharmacokinetic data from a real-world clinical trial. For the positive-control simulation study, where observations and model simulations were generated under the same Pop-PBPK model, the NPDE-based approach denoted a congruency between model predictions and observed data (mean of NPDE = − 0.01). In contrast, for the negative-control simulation study, where model simulations and observed data were generated under different Pop-PBPK models, the NPDE-based method asserted that model simulations and observed data were incongruent (mean of NPDE = − 0.29). When employed to evaluate a previously developed clindamycin PBPK model against prospectively collected plasma concentration data from 29 children, the NPDE-based method qualified the model predictions as successful (mean of NPDE = 0). However, when pediatric subpopulations (e.g., infants) were evaluated, the approach revealed potential biases that should be explored.
Similar content being viewed by others
References
Grimstein M, Yang Y, Zhang X, Grillo J, Huang SM, Zineh I, Wang Y (2019) Physiologically based pharmacokinetic modeling in regulatory science: an update from the U.S. Food and Drug Administration's Office of Clinical Pharmacology. J Pharm Sci 108(1):21–25. https://doi.org/10.1016/j.xphs.2018.10.033
Guideline on the Qualification and Reporting of Physiologically Based Pharmacokinetic (PBPK) Modelling and Simulation (2016) Eurpoean Medicines Agency. https://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2016/07/WC500211315.pdf. Accessed 23 July 2018
Physiologically Based Pharmacokinetic Analyses — Format and Content Guidance for Industry (Draft Guidance) (2016) U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM531207.pdf. Accessed 23 July 2018
Emoto C, Fukuda T, Johnson TN, Neuhoff S, Sadhasivam S, Vinks AA (2017) Characterization of contributing factors to variability in morphine clearance through PBPK modeling implemented with OCT1 transporter. CPT Pharmacomet Syst Pharmacol 6(2):110–119. https://doi.org/10.1002/psp4.12144
Maharaj AR, Barrett JS, Edginton AN (2013) A workflow example of PBPK modeling to support pediatric research and development: case study with lorazepam. AAPS J 15(2):455–464. https://doi.org/10.1208/s12248-013-9451-0
Zhou W, Johnson TN, Xu H, Cheung S, Bui KH, Li J, Al-Huniti N, Zhou D (2016) Predictive performance of physiologically based pharmacokinetic and population pharmacokinetic modeling of renally cleared drugs in children. CPT Pharmacomet Syst Pharmacol 5(9):475–483. https://doi.org/10.1002/psp4.12101
Diestelhorst C, Boos J, McCune JS, Russell J, Kangarloo SB, Hempel G (2013) Physiologically based pharmacokinetic modelling of Busulfan: a new approach to describe and predict the pharmacokinetics in adults. Cancer Chemother Pharmacol 72(5):991–1000. https://doi.org/10.1007/s00280-013-2275-x
Laughon MM, Benjamin DK Jr, Capparelli EV, Kearns GL, Berezny K, Paul IM, Wade K, Barrett J, Smith PB, Cohen-Wolkowiez M (2011) Innovative clinical trial design for pediatric therapeutics. Expert Rev Clin Pharmacol 4(5):643–652. https://doi.org/10.1586/ecp.11.43
Hornik CP, Wu H, Edginton AN, Watt K, Cohen-Wolkowiez M, Gonzalez D (2017) Development of a pediatric physiologically-based pharmacokinetic model of clindamycin using opportunistic pharmacokinetic data. Clin Pharmacokinet 56(11):1343–1353. https://doi.org/10.1007/s40262-017-0525-5
Willmann S, Hohn K, Edginton A, Sevestre M, Solodenko J, Weiss W, Lippert J, Schmitt W (2007) Development of a physiology-based whole-body population model for assessing the influence of individual variability on the pharmacokinetics of drugs. J Pharmacokinet Pharmacodyn 34(3):401–431. https://doi.org/10.1007/s10928-007-9053-5
Samant TS, Lukacova V, Schmidt S (2017) Development and qualification of physiologically based pharmacokinetic models for drugs with atypical distribution behavior: a desipramine case study. CPT Pharmacomet Syst Pharmacol 6(5):315–321. https://doi.org/10.1002/psp4.12180
Maharaj AR, Wu H, Hornik CP, Cohen-Wolkowiez M (2019) Pitfalls of using numerical predictive checks for population physiologically-based pharmacokinetic model evaluation. J Pharmacokinet Pharmacodyn. https://doi.org/10.1007/s10928-019-09636-5
Salerno SN, Edginton A, Cohen-Wolkowiez M, Hornik CP, Watt KM, Jamieson BD, Gonzalez D (2017) Development of an adult physiologically based pharmacokinetic model of solithromycin in plasma and epithelial lining fluid. CPT Pharmacomet Syst Pharmacol 6(12):814–822. https://doi.org/10.1002/psp4.12252
Brendel K, Comets E, Laffont C, Laveille C, Mentre F (2006) Metrics for external model evaluation with an application to the population pharmacokinetics of gliclazide. Pharm Res 23(9):2036–2049. https://doi.org/10.1007/s11095-006-9067-5
Comets E, Brendel K, Mentre F (2008) Computing normalised prediction distribution errors to evaluate nonlinear mixed-effect models: the npde add-on package for R. Comput Methods Progr Biomed 90(2):154–166. https://doi.org/10.1016/j.cmpb.2007.12.002
Brendel K, Comets E, Laffont C, Mentre F (2010) Evaluation of different tests based on observations for external model evaluation of population analyses. J Pharmacokinet Pharmacodyn 37(1):49–65. https://doi.org/10.1007/s10928-009-9143-7
Keizer R (2019) vpc: create visual predictive checks. R package version 1.1.9000. https://github.com/ronkeizer/vpc
Nguyen TH, Mouksassi MS, Holford N, Al-Huniti N, Freedman I, Hooker AC, John J, Karlsson MO, Mould DR, Perez Ruixo JJ, Plan EL, Savic R, van Hasselt JG, Weber B, Zhou C, Comets E, Mentre F, Model Evaluation Group of the International Society of Pharmacometrics Best Practice C (2017) Model evaluation of continuous data pharmacometric models: metrics and graphics. CPT Pharmacometrics Syst Pharmacol 6(2):87–109. https://doi.org/10.1002/psp4.12161
PK-Sim® Ontogeny Database (version 7.1) (2017) https://github.com/Open-Systems-Pharmacology/OSPSuite.Documentation/raw/32fd65c26118fdaa271ac95c896230888c20b3b1/PK-Sim%20Ontogeny%20Database%20Version%207.1.pdf
Comets E, Brendel K, Nguyen TH, Mentre F (2012) User guide for npde 2.0. https://www.npde.biostat.fr/userguide_npde2.0.pdf
Rodgers T, Leahy D, Rowland M (2005) Physiologically based pharmacokinetic modeling 1: predicting the tissue distribution of moderate-to-strong bases. J Pharm Sci 94(6):1259–1276. https://doi.org/10.1002/jps.20322
Rodgers T, Leahy D, Rowland M (2005) Tissue distribution of basic drugs: accounting for enantiomeric, compound and regional differences amongst beta-blocking drugs in rat. J Pharm Sci 94(6):1237–1248. https://doi.org/10.1002/jps.20323
Rodgers T, Rowland M (2006) Physiologically based pharmacokinetic modelling 2: predicting the tissue distribution of acids, very weak bases, neutrals and zwitterions. J Pharm Sci 95(6):1238–1257. https://doi.org/10.1002/jps.20502
Bergstrand M, Hooker AC, Wallin JE, Karlsson MO (2011) Prediction-corrected visual predictive checks for diagnosing nonlinear mixed-effects models. AAPS J 13(2):143–151. https://doi.org/10.1208/s12248-011-9255-z
Knoppert D, Reed M, Benavides S, Totton J, Hoff D, Moffett B, Norris K, Vaillancourt R, Aucoin R, Worthington M (2007) Position paper: paediatric age categories to be used in differentiating between listing on a model essential medicines list for children. https://archives.who.int/eml/expcom/children/Items/PositionPaperAgeGroups.pdf. Accessed 2 Aug 2018
Anderson BJ, Holford NH (2013) Understanding dosing: children are small adults, neonates are immature children. Arch Dis Child 98(9):737–744. https://doi.org/10.1136/archdischild-2013-303720
Edginton A, Willmann S (2006) Physiology-based versus allometric scaling of clearance in children; an eliminating process based comparison. Paediatr Perinat Drug Ther 7(3):146–153. https://doi.org/10.1185/146300906x148530
Karlsson MO, Sheiner LB (1993) The importance of modeling interoccasion variability in population pharmacokinetic analyses. J Pharmacokinet Biopharm 21(6):735–750
Bouazza N, Pestre V, Jullien V, Curis E, Urien S, Salmon D, Treluyer JM (2012) Population pharmacokinetics of clindamycin orally and intravenously administered in patients with osteomyelitis. Br J Clin Pharmacol 74(6):971–977. https://doi.org/10.1111/j.1365-2125.2012.04292.x
Bloomfield C, Staatz CE, Unwin S, Hennig S (2016) Assessing predictive performance of published population pharmacokinetic models of intravenous tobramycin in pediatric patients. Antimicrob Agents Chemother 60(6):3407–3414. https://doi.org/10.1128/AAC.02654-15
Zhao W, Kaguelidou F, Biran V, Zhang D, Allegaert K, Capparelli EV, Holford N, Kimura T, Lo YL, Peris JE, Thomson A, van den Anker JN, Fakhoury M, Jacqz-Aigrain E (2013) External evaluation of population pharmacokinetic models of vancomycin in neonates: the transferability of published models to different clinical settings. Br J Clin Pharmacol 75(4):1068–1080. https://doi.org/10.1111/j.1365-2125.2012.04406.x
Khalil F, Laer S (2014) Physiologically based pharmacokinetic models in the prediction of oral drug exposure over the entire pediatric age range-sotalol as a model drug. AAPS J 16(2):226–239. https://doi.org/10.1208/s12248-013-9555-6
Poulin P, Jones HM, Jones RD, Yates JW, Gibson CR, Chien JY, Ring BJ, Adkison KK, He H, Vuppugalla R, Marathe P, Fischer V, Dutta S, Sinha VK, Bjornsson T, Lave T, Ku MS (2011) PhRMA CPCDC initiative on predictive models of human pharmacokinetics, part 1: goals, properties of the PhRMA dataset, and comparison with literature datasets. J Pharm Sci 100(10):4050–4073. https://doi.org/10.1002/jps.22554
Mould DR, Upton RN (2013) Basic concepts in population modeling, simulation, and model-based drug development-part 2: introduction to pharmacokinetic modeling methods. CPT Pharmacomet Syst Pharmacol 2:e38. https://doi.org/10.1038/psp.2013.14
Sager JE, Yu J, Ragueneau-Majlessi I, Isoherranen N (2015) Physiologically based pharmacokinetic (PBPK) modeling and simulation approaches: a systematic review of published models, applications, and model verification. Drug Metab Dispos 43(11):1823–1837. https://doi.org/10.1124/dmd.115.065920
Gonzalez D, Delmore P, Bloom BT, Cotten CM, Poindexter BB, McGowan E, Shattuck K, Bradford KK, Smith PB, Cohen-Wolkowiez M, Morris M, Yin W, Benjamin DK Jr, Laughon MM (2016) Clindamycin pharmacokinetics and safety in preterm and term infants. Antimicrob Agents Chemother 60(5):2888–2894. https://doi.org/10.1128/AAC.03086-15
Gonzalez D, Melloni C, Yogev R, Poindexter BB, Mendley SR, Delmore P, Sullivan JE, Autmizguine J, Lewandowski A, Harper B, Watt KM, Lewis KC, Capparelli EV, Benjamin DK Jr, Cohen-Wolkowiez M, Best Pharmaceuticals for Children Act - Pediatric Trials Network Administrative Core C (2014) Use of opportunistic clinical data and a population pharmacokinetic model to support dosing of clindamycin for premature infants to adolescents. Clin Pharmacol Ther 96(4):429–437. https://doi.org/10.1038/clpt.2014.134
Nguyen TH, Comets E, Mentre F (2012) Extension of NPDE for evaluation of nonlinear mixed effect models in presence of data below the quantification limit with applications to HIV dynamic model. J Pharmacokinet Pharmacodyn 39(5):499–518. https://doi.org/10.1007/s10928-012-9264-2
Maharaj AR, Edginton AN (2014) Physiologically based pharmacokinetic modeling and simulation in pediatric drug development. CPT Pharmacomet Syst Pharmacol 3:e150. https://doi.org/10.1038/psp.2014.45
Funding
This study was funded by the National Institutes of Health (1R01-HD076676-01A1; M.C.W.).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
M.C.W. receives support for research from the NIH (5R01-HD076676), NIH (HHSN275201000003I), NIAID/NIH (HHSN272201500006I), FDA (1U18-FD006298), the Biomedical Advanced Research and Development Authority (HHSO1201300009C), and from the industry for the drug development in adults and children (www.dcri.duke.edu/research/coi.jsp). C.P.H. receives salary support for research from National Institute for Child Health and Human Development (NICHD) (K23HD090239), the U.S. government for his work in pediatric and neonatal clinical pharmacology (Government Contract HHSN267200700051C, PI: Benjamin, under the Best Pharmaceuticals for Children Act), and industry for drug development in children. KJD has received research support from Merck, Inc. and Pfizer, Inc. and is supported by NICHD K23HD091365.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Maharaj, A.R., Wu, H., Hornik, C.P. et al. Use of normalized prediction distribution errors for assessing population physiologically-based pharmacokinetic model adequacy. J Pharmacokinet Pharmacodyn 47, 199–218 (2020). https://doi.org/10.1007/s10928-020-09684-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10928-020-09684-2