The Value of Further Research: The Added Value of Individual-Participant Level Data

  • Pedro SaramagoEmail author
  • Manuel A. Espinoza
  • Alex J. Sutton
  • Andrea Manca
  • Karl Claxton
Practical Application


Judgements based on average cost-effectiveness estimates may disguise significant heterogeneity in net health outcomes. Decisions about coverage of new interventions are often more efficient when they consider between-patient heterogeneity, which is usually operationalized as different selections for different subgroups. While most model-based cost-effectiveness studies are populated with aggregated-level sub-group estimates, individual-level data are recognized as the best source of evidence to produce unbiased and efficient estimates to explore this heterogeneity. This paper extends a previously published framework to assesses the added value of having access to individual-level data, compared to using aggregate-level data only, in the absence/presence of mutually exclusive population subgroups. Supported by a case study on the cost-effectiveness of interventions to increase uptake of smoke-alarms, the extended framework provided a quantification of the benefits foregone of not using individual-level data, pointed to the optimal number of subgroups and where further research should be undertaken. Although not indicating changes in reimbursement decisions, results showed that irrespective of using aggregate or individual-level data, no substantial additional gains are obtained if more than two subgroups are taken into account. However, depending on the evidence type used, different subgroups are revealed as warranting larger research funds. The use of individual-level data, rather than aggregate, may however influence not only the extent to which an appropriate understanding of existing heterogeneity is attained, but, more importantly, it may shape approval decisions for particular population subgroups and judgements of future research.



We would like to thank members of the Keeping Children Safe study [26] for providing the case study data. In addition, we acknowledge Professor Mark Sculpher and Cynthia Iglesias for useful comments and suggestions as Thesis Advisory Group members of Dr. Pedro Saramago’s PhD.

Author contributions

PS, MAE and KC conceptually developed the framework. PS and MAE were responsible for the analysis, interpretation of the findings and writing of the manuscript. AJS and AM contributed to the interpretation of results and commented on drafts of the manuscript. KC oversaw the work and commented on drafts of the manuscript. All authors contributed to the editing of this manuscript. All authors read and approved the final manuscript.

Compliance with Ethical Standards

Pedro Saramago was funded through a Medical Research Council Capacity Building Grant (Grant number G0800139) and the Portuguese Fundacao para a Ciencia e a Tecnologia (Grant number SFRH/BD/61448/2009).

Conflict of interest

Pedro Saramago, Manuel A. Espinoza, Alex J Sutton, Andrea Manca and Karl Claxton: None to declare.


  1. 1.
    Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ. 1999;18(3):341–64.CrossRefGoogle Scholar
  2. 2.
    Claxton KP, Sculpher MJ. Using value of information analysis to prioritise health research: some lessons from recent UK experience. Pharmacoeconomics. 2006;24(11):1055–68.CrossRefGoogle Scholar
  3. 3.
    Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD, et al. Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Mak. 2012;32(5):722–32.CrossRefGoogle Scholar
  4. 4.
    Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Econ. 1996;5(6):513–24.;2-9.CrossRefGoogle Scholar
  5. 5.
    Sculpher M. Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics. 2008;26(9):799–806.CrossRefGoogle Scholar
  6. 6.
    NICE. Guide to the methods of thechnology appraisal. London: NICE, NHS2013.Google Scholar
  7. 7.
    Grutters JP, Sculpher M, Briggs AH, Severens JL, Candel MJ, Stahl JE, et al. Acknowledging patient heterogeneity in economic evaluation: a systematic literature review. Pharmacoeconomics. 2013;31(2):111–23. Scholar
  8. 8.
    Ramaekers BL, Joore MA, Grutters JP. How should we deal with patient heterogeneity in economic evaluation: a systematic review of national pharmacoeconomic guidelines. Value Health. 2013;16(5):855–62. Scholar
  9. 9.
    Zaric GS. The impact of ignoring population heterogeneity when Markov models are used in cost-effectiveness analysis. Med Decis Mak. 2003;23(5):379–96.CrossRefGoogle Scholar
  10. 10.
    Coyle D, Buxton MJ, O’Brien BJ. Stratified cost-effectiveness analysis: a framework for establishing efficient limited use criteria. Health Econ. 2003;12(5):421–7. Scholar
  11. 11.
    Espinoza MA, Sculpher MJ, Manca A, Basu A. Analysing heterogeneity to support decision making—A2. In: Culyer AJ, editor. Encyclopedia of health economics. San Diego: Elsevier; 2014. p. 71–6.CrossRefGoogle Scholar
  12. 12.
    Espinoza MA, Manca A, Claxton K, Sculpher MJ. The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application. Med Decis Mak. 2014;34(8):951–64.CrossRefGoogle Scholar
  13. 13.
    Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof. 2002;25(1):76–97.CrossRefGoogle Scholar
  14. 14.
    Smith CT, Williamson PR, Marson AG. An overview of methods and empirical comparison of aggregate data and individual patient data results for investigating heterogeneity in meta-analysis of time-to-event outcomes. J Eval Clin Pract. 2005;11(5):468–78.CrossRefGoogle Scholar
  15. 15.
    Simmonds MC, Higgins JP, Stewart LA, Tierney JF, Clarke MJ, Thompson SG. Meta-analysis of individual patient data from randomized trials: a review of methods used in practice. Clin Trials. 2005;2(3):209–17.CrossRefGoogle Scholar
  16. 16.
    Saramago P, Manca A, Sutton AJ. Deriving input parameters for cost-effectiveness modeling: taxonomy of data types and approaches to their statistical synthesis. Value Health. 2012;15(5):639–49.CrossRefGoogle Scholar
  17. 17.
    Dias S, Sutton AJ, Welton NJ, Ades AE. Evidence synthesis for decision making 3: heterogeneity—subgroups, meta-regression, bias, and bias-adjustment. Med Decis Mak. 2013;33(5):618–40.CrossRefGoogle Scholar
  18. 18.
    Debray TP, Schuit E, Efthimiou O, Reitsma JB, Ioannidis JP, Salanti G, et al. An overview of methods for network meta-analysis using individual participant data: when do benefits arise? Stat Methods Med Res. 2016;1:1. Scholar
  19. 19.
    van Walraven C. Individual patient meta-analysis—rewards and challenges. J Clin Epidemiol. 2010;63(3):235–7. Scholar
  20. 20.
    Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;340:c221. Scholar
  21. 21.
    Basu A, Meltzer D. Quantitative methods for valuing comparative effectiveness information. Biopharm Rep. 2010;17(2):2–10.Google Scholar
  22. 22.
    Basu A, Meltzer D. Value of information on preference heterogeneity and individualized care. Med Decis Mak. 2007;27(2):112–27. Scholar
  23. 23.
    Saramago P, Sutton AJ, Cooper NJ, Manca A. Mixed treatment comparisons using aggregate and individual participant level data. Stat Med. 2012;31(28):3516–36.CrossRefGoogle Scholar
  24. 24.
    Cooper N, Kendrick D, Achana F, Dhiman P, He Z, Wynn P, et al. Network meta-analysis to evaluate the effectiveness of interventions to increase the uptake of smoke alarms. Epidemiol Rev. 2012;34(1):32–45. Scholar
  25. 25.
    Saramago P, Cooper NJ, Sutton AJ, Hayes M, Dunn K, Manca A, et al. Cost-effectiveness of interventions for increasing the possession of functioning smoke alarms in households with pre-school children: a modelling study. BMC Public Health. 2014;14:459.CrossRefGoogle Scholar
  26. 26.
    Kendrick D, Ablewhite J, Achana F, Benford P, Clacy R, Coffey F, et al. Keeping Children Safe: a multicentre programme of research to increase the evidence base for preventing unintentional injuries in the home in the under-fives. Southampton: Programme Grants for Applied Research; 2017.Google Scholar
  27. 27.
    Claxton K, Sculpher M, Drummond M. A rational framework for decision making by the National Institute for Clinical Excellence (NICE). Lancet. 2002;360(9334):711–5. Scholar
  28. 28.
    Philips Z, Claxton K, Palmer S. The half-life of truth: what are appropriate time horizons for research decisions? Med Decis Mak. 2008;28(3):287–99. Scholar
  29. 29.
    ONS. Office for national statistics, household projections, 2008 to 2033. Office for National Statistics—Communities and Local Government, Newport; 2010.Google Scholar
  30. 30.
    Fenwick E, O’Brien BJ, Briggs A. Cost-effectiveness acceptability curves—facts, fallacies and frequently asked questions. Health Econ. 2004;13(5):405–15. Scholar
  31. 31.
    Claxton K, editor. Heterogeneity in the cost-effectiveness of medical interventions: the challenge of matching patients to appropriate care. ISPOR, Madrid; 2011.Google Scholar
  32. 32.
    Lambert PC, Sutton AJ, Abrams KR, Jones DR. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol. 2002;55(1):86–94.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Centre for Health EconomicsThe University of YorkYorkUK
  2. 2.Departamento de Salud PúblicaPontificia Universidad Católica de ChileSantiagoChile
  3. 3.Department of Health SciencesUniversity of LeicesterLeicesterUK

Personalised recommendations