Skip to main content

Advertisement

SpringerLink
Log in

Model Averaging in Viral Dynamic Models

  • Research Article
  • Published:
The AAPS Journal Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The paucity of experimental data makes both inference and prediction particularly challenging in viral dynamic models. In the presence of several candidate models, a common strategy is model selection (MS), in which models are fitted to the data but only results obtained with the “best model” are presented. However, this approach ignores model uncertainty, which may lead to inaccurate predictions. When several models provide a good fit to the data, another approach is model averaging (MA) that weights the predictions of each model according to its consistency to the data. Here, we evaluated by simulations in a nonlinear mixed-effect model framework the performances of MS and MA in two realistic cases of acute viral infection, i.e., (1) inference in the presence of poorly identifiable parameters, namely, initial viral inoculum and eclipse phase duration, (2) uncertainty on the mechanisms of action of the immune response. MS was associated in some scenarios with a large rate of false selection. This led to a coverage rate lower than the nominal coverage rate of 0.95 in the majority of cases and below 0.50 in some scenarios. In contrast, MA provided better estimation of parameter uncertainty, with coverage rates ranging from 0.72 to 0.98 and mostly comprised within the nominal coverage rate. Finally, MA provided similar predictions than those obtained with MS. In conclusion, parameter estimates obtained with MS should be taken with caution, especially when several models well describe the data. In this situation, MA has better performances and could be performed to account for model uncertainty.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ho DD, Neumann AU, Perelson AS, Chen W, Leonard JM, Markowitz M. Rapid turnover of plasma virions and CD4 lymphocytes in HIV-1 infection. Nature. 1995;373(6510):123–6.

    Article  CAS  Google Scholar 

  2. Wei X, Ghosh SK, Taylor ME, Johnson VA, Emini EA, Deutsch P, et al. Viral dynamics in human immunodeficiency virus type 1 infection. Nature. 1995;373(6510):117–22.

    Article  CAS  Google Scholar 

  3. Perelson AS, Ribeiro RM. Introduction to modeling viral infections and immunity. Immunol Rev. 2018;285(1):5–8.

    Article  CAS  Google Scholar 

  4. Perelson AS. Modelling viral and immune system dynamics. Nat Rev Immunol. 2002;2(1):28–36.

    Article  CAS  Google Scholar 

  5. Best K, Perelson AS. Mathematical modeling of within-host Zika virus dynamics. Immunol Rev. 2018;285(1):81–96.

    Article  CAS  Google Scholar 

  6. Ciupe SM. Modeling the dynamics of hepatitis B infection, immunity, and drug therapy. Immunol Rev. 2018;285(1):38–54.

    Article  CAS  Google Scholar 

  7. Lavielle M, Mentré F. Estimation of population pharmacokinetic parameters of saquinavir in HIV patients with the MONOLIX software. J Pharmacokinet Pharmacodyn. 2007;34(2):229–49.

    Article  CAS  Google Scholar 

  8. Guedj J, Thiébaut R, Commenges D. Practical identifiability of HIV dynamics models. Bull Math Biol. 2007;69(8):2493–513.

    Article  CAS  Google Scholar 

  9. Snoeck E, Chanu P, Lavielle M, Jacqmin P, Jonsson EN, Jorga K, et al. A comprehensive hepatitis C viral kinetic model explaining cure. Clin Pharmacol Ther. 2010;87(6):706–13.

    Article  CAS  Google Scholar 

  10. Nguyen T, Guedj J. HCV kinetic models and their implications in drug development: HCV kinetic models and their implications. CPT Pharmacometrics Syst Pharmacol. 2015;4(4):231–42.

    Article  CAS  Google Scholar 

  11. Handel A, Longini IM, Antia R. Towards a quantitative understanding of the within-host dynamics of influenza A infections. J R Soc Interface. 2010;7(42):35–47.

    Article  Google Scholar 

  12. Smith AM, Adler FR, Ribeiro RM, Gutenkunst RN, McAuley JL, McCullers JA, et al Kinetics of coinfection with influenza A virus and Streptococcus pneumoniae. Grenfell BT, editor. PLoS Pathogens. 2013;9(3):e1003238.

  13. Ganusov VV. Strong inference in mathematical modeling: a method for robust science in the twenty-first century. Front Microbiol. 2016;7(1137):1–10.

  14. Buckland ST, Burnham KP, Augustin NH. Model selection: an integral part of inference. Biometrics. 1997;53(2):603.

    Article  Google Scholar 

  15. Boulesteix A-L. Ten simple rules for reducing overoptimistic reporting in methodological computational research. Lewitter F, editor. PLoS Comput Biol. 2015;11(4):e1004191.

  16. Kirk PDW, Babtie AC, Stumpf MPH. Systems biology (un)certainties. Science. 2015;350(6259):386–8.

    Article  CAS  Google Scholar 

  17. Burnham KP, Anderson DR. Model selection and multimodel inference: a practical information-theoretic approach. 2nd ed, [4. Printing]. New York: Springer; 2010. p. 488.

    Google Scholar 

  18. Claeskens G, Hjort NL. Model selection and model averaging [internet]. Cambridge: Cambridge University Press; 2008. p. 332

  19. Best K, Guedj J, Madelain V, de Lamballerie X, Lim S-Y, Osuna CE, et al. Zika plasma viral dynamics in nonhuman primates provides insights into early infection and antiviral strategies. Proc Natl Acad Sci. 2017;114(33):8847–52.

    Article  CAS  Google Scholar 

  20. Madelain V, Baize S, Jacquot F, Reynard S, Fizet A, Barron S, et al. Ebola viral dynamics in nonhuman primates provides insights into virus immuno-pathogenesis and antiviral strategies. Nat Commun. 2018;9(1):4013.

  21. Bertrand J, Comets E, Mentré F. Comparison of model-based tests and selection strategies to detect genetic polymorphisms influencing pharmacokinetic parameters. J Biopharm Stat. 2008;18(6):1084–102.

    Article  Google Scholar 

  22. Bozdogan H. Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions. Psychometrika. 1987;52(3):345–70.

    Article  Google Scholar 

  23. Anderson DR, Burnham KP. Understanding information criteria for selection among capture-recapture or ring recovery models. Bird Study. 1999;46(sup1):S14–21.

    Article  Google Scholar 

  24. Neath AA, Cavanaugh JE. The Bayesian information criterion: background, derivation, and applications: the Bayesian information criterion. WIREs Comp Stat. 2012;4(2):199–203.

    Article  Google Scholar 

  25. Buatois S, Ueckert S, Frey N, Retout S, Mentré F. Comparison of model averaging and model selection in dose finding trials analyzed by nonlinear mixed effect models. AAPS J. 2018;20(3):56.

    Article  Google Scholar 

  26. Aoki Y, Röshammar D, Hamrén B, Hooker AC. Model selection and averaging of nonlinear mixed-effect models for robust phase III dose selection. J Pharmacokinet Pharmacodyn. 2017;44(6):581–97.

    Article  CAS  Google Scholar 

  27. Kakizoe Y, Nakaoka S, Beauchemin CAA, Morita S, Mori H, Igarashi T, et al. A method to determine the duration of the eclipse phase for in vitro infection with a highly pathogenic SHIV strain. Sci Rep. 2015;5(1):10371.

    Article  Google Scholar 

  28. Xia X, Moog CH. Identifiability of nonlinear systems with application to HIV/AIDS models. IEEE Trans Autom Control. 2003;48(2):330–6.

    Article  Google Scholar 

  29. Wu H, Zhu H, Miao H, Perelson AS. Parameter identifiability and estimation of HIV/AIDS dynamic models. Bull Math Biol. 2008;70(3):785–99.

    Article  Google Scholar 

  30. Miao H, Dykes C, Demeter LM, Cavenaugh J, Park SY, Perelson AS, et al. Modeling and estimation of kinetic parameters and replicative fitness of HIV-1 from flow-cytometry-based growth competition experiments. Bull Math Biol. 2008;70(6):1749–71.

    Article  CAS  Google Scholar 

  31. Dumont C, Lestini G, Le Nagard H, Mentré F, Comets E, Nguyen TT, et al. PFIM 4.0, an extended R program for design evaluation and optimization in nonlinear mixed-effect models. Comput Methods Prog Biomed. 2018;156:217–29.

    Article  Google Scholar 

  32. Baccam P, Beauchemin C, Macken CA, Hayden FG, Perelson AS. Kinetics of influenza A virus infection in humans. J Virol. 2006;80(15):7590–9.

    Article  CAS  Google Scholar 

  33. Pawelek KA, Huynh GT, Quinlivan M, Cullinane A, Rong L, Perelson AS. Modeling within-host dynamics of influenza virus infection including immune responses. PLoS Comput Biol. 2012;8(6):e1002588

  34. Pinheiro J, Bornkamp B, Glimm E, Bretz F. Model-based dose finding under model uncertainty using general parametric models. Statist Med. 2014;33(10):1646–61.

    Article  Google Scholar 

  35. Schorning K, Bornkamp B, Bretz F, Dette H. Model selection versus model averaging in dose finding studies: K. SCHORNING ET AL Stat Med 2016;35(22):4021–40.

  36. Saenz RA, Quinlivan M, Elton D, MacRae S, Blunden AS, Mumford JA, et al. Dynamics of influenza virus infection and pathology. J Virol. 2010;84(8):3974–83.

    Article  CAS  Google Scholar 

  37. Hoeting JA, Raftery AE, Madigan D. Bayesian model averaging: a tutorial. Stat Sci. 1999;14(4):382–417.

    Article  Google Scholar 

  38. Thai H-T, Mentré F, Holford NHG, Veyrat-Follet C, Comets E. Evaluation of bootstrap methods for estimating uncertainty of parameters in nonlinear mixed-effects models: a simulation study in population pharmacokinetics. J Pharmacokinet Pharmacodyn. 2014;41(1):15–33.

  39. Dosne A-G, Bergstrand M, Harling K, Karlsson MO. Improving the estimation of parameter uncertainty distributions in nonlinear mixed effects models using sampling importance resampling. J Pharmacokinet Pharmacodyn. 2016 Dec;43(6):583–96.

  40. Ueckert S, Riviere M-K, Mentré F. Alternative to resampling methods in maximum likelihood estimation for NLMEMs by borrowing from bayesian methodology. www.page-meeting.org/?abstract=3632.

  41. Lloyd AL. The dependence of viral parameter estimates on the assumed viral life cycle: limitations of studies of viral load data. Proc Biol Sci. 2001;268(1469):847–54.

    Article  CAS  Google Scholar 

  42. Ribeiro RM, Qin L, Chavez LL, Li D, Self SG, Perelson AS. Estimation of the initial viral growth rate and basic reproductive number during acute HIV-1 infection. J Virol. 2010;84(12):6096–102.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

The authors also would like to acknowledge Hervé Le Nagard and Lionel de la Tribouille for the use of CATIBioMed calculus facility.

Funding

Antonio Gonçalves was funded by a grant from Roche Pharmaceutical Research and Early Development.

Author information

Authors and Affiliations

  1. Université de Paris, IAME, INSERM, Henri Huchard, F-75018, Paris, France

    Antonio Gonçalves, France Mentré & Jérémie Guedj

  2. Roche Pharmaceutical Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center, Basel, Switzerland

    Annabelle Lemenuel-Diot

Authors
  1. Antonio Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. France Mentré
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Annabelle Lemenuel-Diot
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jérémie Guedj
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Gonçalves.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 8.24 mb)

ESM 2

(ZIP 2.43 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gonçalves, A., Mentré, F., Lemenuel-Diot, A. et al. Model Averaging in Viral Dynamic Models. AAPS J 22, 48 (2020). https://doi.org/10.1208/s12248-020-0426-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1208/s12248-020-0426-7

Key Words

Search

Navigation