Skip to main content
SpringerLink
Log in

Conducting Economic Evaluations Alongside Randomised Trials: Current Methodological Issues and Novel Approaches

  • Review Article
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Trial-based economic evaluations are an important aspect of health technology assessment. The availability of patient-level data coupled with unbiased estimates of clinical outcomes means that randomised controlled trials are effective vehicles for the generation of economic data. However there are methodological challenges to trial-based evaluations, including the collection of reliable data on resource use and cost, choice of health outcome measure, calculating minimally important differences, dealing with missing data, extrapolating outcomes and costs over time and the analysis of multinational trials. This review focuses on the state of the art of selective elements regarding the design, conduct, analysis and reporting of trial-based economic evaluations. The limitations of existing approaches are detailed and novel methods introduced. The review is internationally relevant but with a focus towards practice in the UK.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Sculpher MJ, Claxton K, Drummond M, McCabe C. Whither trial-based economic evaluation for health care decision making? Health Econ. 2006;15(7):677–87.

    Article  PubMed  Google Scholar 

  2. Glick H, Doshi J, Sonad S, Polsky D. Economic evaluation in clinical trials. 1st ed. Oxford: Oxford University Press; 2007.

    Google Scholar 

  3. Petrou S, Gray A. Economic evaluation alongside randomised controlled trials: design, conduct, analysis, and reporting. BMJ. 2011;342:d1548.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Ramsey SD, Willke RJ, Glick H, Reed SD, Augustovski F, Jonsson B, et al. Cost-effectiveness analysis alongside clinical trials II. An ISPOR Good Research Practices Task Force report. Value Health. 2015;18(2):161–72.

    Article  PubMed  Google Scholar 

  5. Ridyard CH, Hughes DA. Methods for the collection of resource use data within clinical trials: a systematic review of studies funded by the UK Health Technology Assessment program. Value Health. 2010;13(8):867–72.

    Article  PubMed  Google Scholar 

  6. Ahonkallio S, Santala M, Valtonen H, Martikainen H. Cost-minimisation analysis of endometrial thermal ablation in a day case or outpatient setting under different anaesthesia regimens. Eur J Obstet Gynecol Reprod Biol. 2012;162(1):102–4.

    Article  PubMed  Google Scholar 

  7. National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. Available at: http://publications.nice.org.uk/pmg9. Accessed 15 June 2015.

  8. Evans WK, Coyle D, Gafni A, Walker H, National Cancer Institute of Canada Clinical Trials Group-Working Group on Economic Analysis. Which cancer clinical trials should be considered for economic evaluation? Selection criteria from the National Cancer Institute of Canada’s Working Group on Economic Analysis. Chronic Dis Can. 2003;24(4):102–7.

    PubMed  Google Scholar 

  9. Revicki DA, Paramore C, Rentz AM. Incorporating pharmacoeconomic and health outcomes into randomized clinical trials. Expert Rev Pharmacoecon Outcomes Res. 2005;5(6):695–703.

    Article  PubMed  Google Scholar 

  10. Marshall DA, Hux M. Design and analysis issues for economic analysis alongside clinical trials. Med Care. 2009;47(7 Suppl 1):S14–20.

    Article  PubMed  Google Scholar 

  11. Ramsey SD, McIntosh M, Sullivan SD. Design issues for conducting cost-effectiveness analyses alongside clinical trials. Annu Rev Public Health. 2001;22:129–41.

    Article  CAS  PubMed  Google Scholar 

  12. Medical Research Council. Developing and evaluating complex interventions: new guidance. 2008. Available at: http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/. Accessed 15 June 2015.

  13. Gomes M, Grieve R, Nixon R, Edmunds WJ. Statistical methods for cost-effectiveness analyses that use data from cluster randomized trials: a systematic review and checklist for critical appraisal. Med Decis Mak. 2012;32(1):209–20.

    Article  Google Scholar 

  14. Edwards RT, Charles JM, Lloyd-Williams H. Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. BMC Public Health. 2013;13:1001.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kelly MP, McDaid D, Ludbrook A, Powell J. Economic appraisal of public health interventions. NHS Health Development Agency briefing document. 2005. Available at: http://www.cawt.com/Site/11/Documents/Publications/Population%20Health/Economics%20of%20Health%20Improvement/Economic_appraisal_of_public_health_interventions.pdf. Accessed 5 Jan 2016.

  16. Payne K, McAllister M, Davies LM. Valuing the economic benefits of complex interventions: when maximising health is not sufficient. Health Econ. 2013;22(3):258–71.

    Article  PubMed  Google Scholar 

  17. Weatherly H, Drummond M, Claxton K, Cookson R, Ferguson B, Godfrey C, et al. Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy. 2009;93(2–3):85–92.

    Article  PubMed  Google Scholar 

  18. Fischer AJ, Threlfall A, Meah S, Cookson R, Rutter H, Kelly MP. The appraisal of public health interventions: an overview. J Public Health (Oxf). 2013;35(4):488–94.

    Article  CAS  Google Scholar 

  19. Martin A, Jones A, Mugford M, Shemilt I, Hancock R, Wittenberg R. Methods used to identify and measure resource use in economic evaluations: a systematic review of questionnaires for older people. Health Econ. 2012;21(8):1017–22.

    Article  PubMed  Google Scholar 

  20. Beecham J, Knapp M. Costing psychiatric interventions. Discussion Paper 1536. Personal Social Services Research Unit, University of Kent at Canterbury. 1999. Available at: http://www.pssru.ac.uk/pdf/dp1536.pdf. Accessed 28 Oct 2013.

  21. Chisholm D, Knapp MR, Knudsen HC, Amaddeo F, Gaite L, van Wijngaarden B. Client Socio-Demographic and Service Receipt Inventory-European Version: development of an instrument for international research. EPSILON Study 5. European Psychiatric Services: inputs linked to outcome domains and needs. Br J Psychiatry Suppl. 2000;177(39):s28–33.

    Article  Google Scholar 

  22. Clarke PM, Fiebig DG, Gerdtham UG. Optimal recall length in survey design. J Health Econ. 2008;27(5):1275–84.

    Article  PubMed  Google Scholar 

  23. Brown JB, Adams ME. Patients as reliable reporters of medical care process. Recall of ambulatory encounter events. Med Care. 1992;30(5):400–11.

    Article  CAS  PubMed  Google Scholar 

  24. Evans CJ, Crawford B. Data collection methods in prospective economic evaluations: how accurate are the results? Value Health. 2000;3(4):277–86.

    Article  CAS  PubMed  Google Scholar 

  25. Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev. 2006;63(2):217–35.

    Article  PubMed  Google Scholar 

  26. Richards SH, Coast J, Peters TJ. Patient-reported use of health service resources compared with information from health providers. Health Soc Care Community. 2003;11(6):510–8.

    Article  PubMed  Google Scholar 

  27. Ridyard CH, Hughes DA, DIRUM Team. Development of a database of instruments for resource-use measurement: purpose, feasibility, and design. Value Health. 2012;15(5):650–5.

    Article  PubMed  Google Scholar 

  28. McPherson CJ, Addington-Hall JM. Judging the quality of care at the end of life: can proxies provide reliable information? Soc Sci Med. 2003;56(1):95–109.

    Article  CAS  PubMed  Google Scholar 

  29. Department of Health Payment by Results Team. A simple guide to payment by results. November 2012. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/213150/PbR-Simple-Guide-FINAL.pdf. Accessed 5 Jan 2016.

  30. Lyons RA, Jones KH, John G, Brooks CJ, Verplancke JP, Ford DV, et al. The SAIL databank: linking multiple health and social care datasets. BMC Med Inform Decis Mak. 2009;9:3.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hounsome N, Orrell M, Edwards RT. EQ-5D as a quality of life measure in people with dementia and their carers: evidence and key issues. Value Health. 2011;14(2):390–9.

    Article  PubMed  Google Scholar 

  32. Noyes JP, Williams A, Allen D, Brocklehurst P, Carter C, Gregory JW, et al. Evidence into practice: evaluating a child-centred intervention for diabetes medicine management. The EPIC Project. BMC Pediatr. 2010;10:70.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Noyes JP, Lowes L, Whitaker R, Allen D, Carter C, Edwards RT, et al. Developing and evaluating a child-centred intervention for diabetes medicine management using mixed methods and a multicentre randomised controlled trial. Health Serv Deliv Res. 2014;2(8). doi:10.3310/hsdr02080.

  34. Thorrington D, Eames K. Measuring health utilities in children and adolescents: a systematic review of the literature. PLoS One. 2015;10(8):e0135672.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Drummond MF, Sculpher M, Torrance GW, O’Brien BJ, Stoddart GL. Methods for the economic evaluation of health care programmes. 3rd ed. Oxford: Oxford University Press; 2005.

    Google Scholar 

  36. Al-Janabi H, Flynn TN, Coast J. QALYs and carers. Pharmacoeconomics. 2011;29(12):1015–23.

    Article  PubMed  Google Scholar 

  37. Mulhern B, Rowen D, Jacoby A, Marson T, Snape D, Hughes D, et al. The development of a QALY measure for epilepsy: NEWQOL-6D. Epilepsy Behav. 2012;24(1):36–43.

    Article  PubMed  Google Scholar 

  38. Marson AG, Appleton R, Baker GA, Chadwick DW, Doughty J, Eaton B, et al. A randomised controlled trial examining the longer-term outcomes of standard versus new antiepileptic drugs. The SANAD Trial. Health Technol Assess. 2007;11(37):iii–iv, ix–x, 1–134.

  39. Versteegh MM, Leunis A, Uyl-de Groot CA, Stolk EA. Condition-specific preference-based measures: benefit or burden? Value Health. 2012;15(3):504–13.

    Article  PubMed  Google Scholar 

  40. Clarke P, Gray A, Holman R. Estimating utility values for health states of type 2 diabetic patients using the EQ-5D (UKPDS 62). Med Decis Mak. 2002;22(4):340–9.

    Article  Google Scholar 

  41. Briggs AH, Bousquet J, Wallace MV, Busse WW, Clark TJ, Pedersen SE, et al. Cost-effectiveness of asthma control: an economic appraisal of the GOAL study. Allergy. 2006;61(5):531–6.

    Article  CAS  PubMed  Google Scholar 

  42. Ryan M, Gerard K, Amaya-Amaya M. Using discrete choice experiments to value health and health care, vol. 11. Dordrecht: Springer; 2008.

    Google Scholar 

  43. Latimer NR. Survival analysis for economic evaluations alongside clinical trials–extrapolation with patient-level data: inconsistencies, limitations, and a practical guide. Med Decis Mak. 2013;33(6):743–54.

    Article  Google Scholar 

  44. Grieve R, Hawkins N, Pennington M. Extrapolation of survival data in cost-effectiveness analyses: improving the current state of play. Med Decis Mak. 2013;33(6):740–2.

    Article  Google Scholar 

  45. Bagust A, Beale S. Survival analysis and extrapolation modeling of time-to-event clinical trial data for economic evaluation: an alternative approach. Med Decis Mak. 2014;34(3):343–51.

    Article  Google Scholar 

  46. Bilcke J, Beutels P, Brisson M, Jit M. Accounting for methodological, structural, and parameter uncertainty in decision-analytic models: a practical guide. Med Decis Mak. 2011;31(4):675–92.

    Article  Google Scholar 

  47. Jackson CH, Bojke L, Thompson SG, Claxton K, Sharples LD. A framework for addressing structural uncertainty in decision models. Med Decis Mak. 2011;31(4):662–74.

    Article  Google Scholar 

  48. Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–7. Value Health. 2012;15(6):843–50.

    Article  PubMed  Google Scholar 

  49. Noble SM, Hollingworth W, Tilling K. Missing data in trial-based cost-effectiveness analysis: the current state of play. Health Econ. 2012;21(2):187–200.

    Article  PubMed  Google Scholar 

  50. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99.

    Article  PubMed  Google Scholar 

  51. White IR, Thompson SG. Adjusting for partially missing baseline measurements in randomized trials. Stat Med. 2005;24(7):993–1007.

    Article  PubMed  Google Scholar 

  52. Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Ramsey S, Willke R, Briggs A, Brown R, Buxton M, Chawla A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force report. Value Health. 2005;8(5):521–33.

    Article  PubMed  Google Scholar 

  54. Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Sylvestre Y. CONSORT: missing missing data, guidelines, the effects on HTA monograph reporting. Trials. 2011;12(Suppl 1):A61.

    Article  PubMed Central  Google Scholar 

  56. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Pharmacoeconomics. 2013;31(5):361–7.

    Article  PubMed  Google Scholar 

  57. Wood AM, White IR, Thompson SG. Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals. Clin Trials. 2004;1(4):368–76.

    Article  PubMed  Google Scholar 

  58. Little RJA, Rubin DB. Statistical analysis with missing data. 2nd ed. Hoboken: Wiley; 2002.

    Google Scholar 

  59. Little RJA. Missing data adjustments in large surveys. J Bus Econ Stat. 1988;6(3):287–96.

    Google Scholar 

  60. Sinharay S, Stern HS, Russell D. The use of multiple imputation for the analysis of missing data. Psychol Methods. 2001;6(4):317–29.

    Article  CAS  PubMed  Google Scholar 

  61. Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol. 2009;9:57.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Wayman J. Multiple imputation for missing data: what is it and how can I use it? Paper presented at the proceedings of the annual meeting of the American educational research association. Chicago; 2003.

  63. Heckman J. Sample selection bias as a specification error. Econometrica. 1979;47(1):153–61.

    Article  Google Scholar 

  64. Little R. Pattern-mixture models for multivariate incomplete data. J Am Stat Assoc. 1993;88(421):125–34.

    Google Scholar 

  65. Carpenter JR, Kenward MG, White IR. Sensitivity analysis after multiple imputation under missing at random: a weighting approach. Stat Methods Med Res. 2007;16(3):259–75.

    Article  PubMed  Google Scholar 

  66. Rubin DB, Stern HS, Vehovar V. Handling, “don’t know” survey responses: the case of the Slovenian plebiscite. J Am Stat Assoc. 1995;90:822–8.

    Google Scholar 

  67. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7(2):147–77.

    Article  PubMed  Google Scholar 

  68. Shroufi A, Powles JW. Adherence and chemoprevention in major cardiovascular disease: a simulation study of the benefits of additional use of statins. J Epidemiol Community Health. 2010;64(2):109–13.

    Article  CAS  PubMed  Google Scholar 

  69. Kadambi A, Leipold RJ, Kansal AR, Sorensen S, Getsios D. Inclusion of compliance and persistence in economic models: past, present and future. Appl Health Econ Health Policy. 2012;10(6):365–79.

    Article  PubMed  Google Scholar 

  70. Hughes DA, Bagust A, Haycox A, Walley T. Accounting for noncompliance in pharmacoeconomic evaluations. Pharmacoeconomics. 2001;19(12):1185–97.

    Article  CAS  PubMed  Google Scholar 

  71. Hughes D, Cowell W, Koncz T, Cramer J, International Society for Pharmacoeconomics & Outcomes Research Economics of Medication Compliance Working Group. Methods for integrating medication compliance and persistence in pharmacoeconomic evaluations. Value Health. 2007;10(6):498–509.

    Article  PubMed  Google Scholar 

  72. Fischer K, Goetghebeur E, Vrijens B, White IR. A structural mean model to allow for noncompliance in a randomized trial comparing 2 active treatments. Biostatistics. 2011;12(2):247–57.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Hughes DA, Walley T. Predicting, “real world” effectiveness by integrating adherence with pharmacodynamic modeling. Clin Pharmacol Ther. 2003;74(1):1–8.

    Article  CAS  PubMed  Google Scholar 

  74. Reed SD, Anstrom KJ, Bakhai A, Briggs AH, Califf RM, Cohen DJ, et al. Conducting economic evaluations alongside multinational clinical trials: toward a research consensus. Am Heart J. 2005;149(3):434–43.

    Article  PubMed  Google Scholar 

  75. Sculpher MJ, Pang FS, Manca A, Drummond MF, Golder S, Urdahl H, et al. Generalisability in economic evaluation studies in healthcare: a review and case studies. Health Technol Assess. 2004;8(49):iii–iv, 1–192.

  76. Drummond M, Barbieri M, Cook J, Glick HA, Lis J, Malik F, et al. Transferability of economic evaluations across jurisdictions: ISPOR Good Research Practices Task Force report. Value Health. 2009;12(4):409–18.

    Article  PubMed  Google Scholar 

  77. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94–104.

    Article  PubMed  Google Scholar 

  78. Manca A, Sculpher MJ, Goeree R. The analysis of multinational cost-effectiveness data for reimbursement decisions: a critical appraisal of recent methodological developments. Pharmacoeconomics. 2010;28(12):1079–96.

    Article  PubMed  Google Scholar 

  79. Keeling MJ, Rohani P. Modeling infectious diseases in humans and animals. 1st ed. Princeton: Princeton University Press; 2007.

    Google Scholar 

Download references

Author contributions

Dyfrig Hughes conceived this review and, with Eifiona Wood, revised the manuscript critically for important intellectual content. Dyfrig Hughes, Joanna Charles, Dalia Dawoud, Rhiannon Tudor Edwards, Emily Holmes, Carys Jones, Paul Parham, Catrin Plumpton, Colin Ridyard, Huw Lloyd-Williams, Eifiona Wood and Seow Tien Yeo made substantial contributions to the design and drafting of the work, approved the version to be published, and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Dyfrig Hughes will act as guarantor of the work presented in this paper.

Author information

Authors and Affiliations

  1. Centre for Health Economics and Medicines Evaluation, Ardudwy, Bangor University, Holyhead Road, Wales, LL57 2PZ, UK

    Dyfrig Hughes, Joanna Charles, Rhiannon Tudor Edwards, Emily Holmes, Carys Jones, Catrin Plumpton, Colin Ridyard, Huw Lloyd-Williams, Eifiona Wood & Seow Tien Yeo

  2. Clinical Pharmacy Department, Faculty of Pharmacy, Cairo University, Giza, Egypt

    Dalia Dawoud

  3. Department of Public Health and Policy, Faculty of Health and Life Sciences, University of Liverpool, Liverpool, UK

    Paul Parham

Authors
  1. Dyfrig Hughes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joanna Charles
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dalia Dawoud
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rhiannon Tudor Edwards
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emily Holmes
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carys Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Paul Parham
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Catrin Plumpton
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Colin Ridyard
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Huw Lloyd-Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Eifiona Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Seow Tien Yeo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dyfrig Hughes.

Ethics declarations

The preparation of this review was funded by the Medical Research Council North-West Hub in Trials Methodology Research, Reference Number G0800792.

Conflict of interest

Dyfrig Hughes, Joanna Charles, Dalia Dawoud, Rhiannon Tudor Edwards, Emily Holmes, Carys Jones, Paul Parham, Catrin Plumpton, Colin Ridyard, Huw Lloyd-Williams, Eifiona Wood and Seow Tien Yeo declare they have no conflicts of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hughes, D., Charles, J., Dawoud, D. et al. Conducting Economic Evaluations Alongside Randomised Trials: Current Methodological Issues and Novel Approaches. PharmacoEconomics 34, 447–461 (2016). https://doi.org/10.1007/s40273-015-0371-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-015-0371-y

Keywords

Search

Navigation