Skip to main content
SpringerLink
Log in

Potential Bias Associated with Modeling the Effectiveness of Healthcare Interventions in Reducing Mortality Using an Overall Hazard Ratio

  • Original Research Article
  • Published:
PharmacoEconomics Aims and scope Submit manuscript

Abstract

Background

Clinical trials often report intervention efficacy in terms of the reduction in all-cause mortality between the treatment and control arms (i.e., an overall hazard ratio [oHR]) instead of the reduction in disease-specific mortality (i.e., a disease-specific hazard ratio [dsHR]). Using oHR to reduce all-cause mortality beyond the time horizon of the trial may introduce bias if the relative proportion of other-cause mortality increases with age. We sought to quantify this oHR extrapolation bias and propose a new approach to overcome this bias.

Methods

We simulated a hypothetical cohort of patients with a generic disease that increased background mortality by a constant additive disease-specific rate. We quantified the bias in terms of the percentage change in life expectancy gains with the intervention under an oHR compared with a dsHR approach as a function of the cohort start age, the disease-specific mortality rate, dsHR, and the duration of the intervention’s effect. We then quantified the bias in a cost-effectiveness analysis (CEA) of implantable cardioverter-defibrillators based on efficacy estimates from a clinical trial.

Results

For a cohort of 50-year-old patients with a disease-specific mortality of 0.05, a dsHR of 0.5, a calculated oHR of 0.55, and a lifetime duration of effect, the bias was 28%. We varied these key parameters over wide ranges and the resulting bias ranged between 3 and 140%. In the CEA, the use of oHR as the intervention’s effectiveness overestimated quality-adjusted life expectancy by 9% and costs by 3%, biasing the incremental cost-effectiveness ratio by − 6%.

Conclusions

The use of an oHR approach to model the intervention’s effectiveness beyond the time horizon of the trial overestimates its benefits. In CEAs, this bias could decrease the cost of a QALY, overestimating interventions’ cost effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Kuntz KM, Russell LB, Owens DK, Sanders GD, Trikalinos TA, Salomon JA. Decision models in cost-effectiveness analysis. In: Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG, editors. Cost-effectiveness Heal Med. Second. New York: Oxford University Press; 2017. p. 105–36.

    Chapter  Google Scholar 

  2. Kuntz KM, Weinstein MC. Life expectancy biases in clinical decision modeling. Med Decis Mak. 1995;15:158–69.

    Article  CAS  Google Scholar 

  3. Bentley TGK, Weinstein MC, Kuntz KM. Effects of categorizing continuous variables in decision-analytic models. Med Decis Mak. 2009;29:549–56.

    Article  Google Scholar 

  4. Bentley TGK, Kuntz KM, Ringel JS. Bias associated with failing to incorporate dependence on event history in Markov models. Med Decis Mak. 2010;30:651–60.

    Article  Google Scholar 

  5. Goldie SJ, Kuntz KM. A potential error in evaluating cancer screening: a comparison of 2 approaches for modeling unerlying disease progression. Med Decis Mak. 2010;23:232–41.

    Article  Google Scholar 

  6. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.

    Google Scholar 

  7. Weinstein MC, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices-Modeling Studies. Value Health. 2003;6:9–17.

    Article  Google Scholar 

  8. National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013 [Internet]. London, UK; 2013. p. 1–93. Available from: http://www.nice.org.uk/article/pmg9/resources/non-guidance-guide-to-the-methods-of-technology-appraisal-2013-pdf.

  9. Blagoev KB, Wilkerson J, Fojo T. Hazard ratios in cancer clinical trials—a primer. Nat Rev Clin Oncol. 2012;9:178–83.

    Article  CAS  Google Scholar 

  10. Cox DR. Models and life-tables regression. J R Stat Soc. 1972;34:187–220.

    Google Scholar 

  11. Spruance SL, Reid JE, Grace M, Samore M. Hazard ratio in clinical trials. Antimicrob Agents Chemother. 2004;48:2787–92.

    Article  CAS  Google Scholar 

  12. Ramsey SD, Willke R, Briggs AH, Brown RE, Buxton M, Chawla A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force Report. Value Health. 2005;8:521–33.

    Article  Google Scholar 

  13. Ramsey SD, Willke RJ, Glick H, Reed SD, Augustovski F, Jonsson B, et al. Cost-effectiveness analysis alongside clinical trials II—an ISPOR Good Research Practices Task Force Report. Value Health. 2015;18:161–72.

    Article  Google Scholar 

  14. Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG, editors. Cost-effectiveness in health and medicine. Second. New York: Oxford University Press, Incorporated; 2017.

    Google Scholar 

  15. Briggs A, Sculpher M, Claxton K. Decision modelling for health economic evaluation. New York: Oxford University Press; 2006.

    Google Scholar 

  16. Canadian Agency for Drugs and Technologies in Health (CADTH). Guidelines for the Economic Evaluation of Health Technologies. 3rd ed. 2006.

  17. Latimer NR. Survival analysis for economic evaluations alongside clinical trials–extrapolation with patient-level data: Inconsistencies, limitations, and a practical guide. Med Decis Mak. 2013;33:743–54.

    Article  Google Scholar 

  18. Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Med Decis Mak. 2012;32:667–77.

    Article  Google Scholar 

  19. Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ, Stoddart GL. Methods for the economic evaluation of health care programmes. 3rd ed. New York: Oxford University Press; 2005.

    Google Scholar 

  20. O’Brien B. Economic evaluation of pharmaceuticals: Frankenstein’s monster or vampire of trials? Med Care. 1996;34(12 Suppl):DS99–108.

    PubMed  Google Scholar 

  21. Benaglia T, Jackson CH, Sharples LD. Survival extrapolation in the presence of cause specific hazards. Stat Med. 2014;34:796–811.

    Article  Google Scholar 

  22. Jackson C, Stevens J, Ren S, Latimer N, Bojke L, Manca A, et al. Extrapolating survival from randomized trials using external data: a review of methods. Med Decis Mak. 2017;37:377–90.

    Article  Google Scholar 

  23. Williams C, Lewsey JD, Briggs AH, Mackay DF. Estimation of survival probabilities for use in cost-effectiveness analysis: a comparison of a multi-state modelling survival analysis approach with partitioned survival and Markov decision-analytic modelling. Med Decis Mak. 2017;37:427–39.

    Article  Google Scholar 

  24. Kuntz KM, Weinstein MC. Modelling in economic evaluation. In: Drummond MF, McGuire A, editors. Econ eval heal care merging theory with pract. 2nd ed. New York: Oxford University Press; 2001. p. 141–71.

    Google Scholar 

  25. Arias E. United States Life Tables, 2009. Nat Vital Stat Rep. 2014;62(7):1–63.

    Google Scholar 

  26. Kuntz KM, Goldie SJ. Assessing the sensitivity of decision-analytic results to unobserved markers of risk: defining the effects of heterogeneity bias. Med Decis Mak. 2002;22:218–27.

    Article  Google Scholar 

  27. Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Med Decis Mak. 1993;13:322–38.

    Article  CAS  Google Scholar 

  28. Beck JR, Pauker SG. The Markov process in medical prognosis. Med Decis Mak. 1983;3:419–58.

    Article  CAS  Google Scholar 

  29. Harwell M, Kohli N, Peralta Y. Experimental design and data analysis in computer simulation studies in the behavioral sciences. J Mod Appl Stat Methods. 2017;16:3–28.

    Article  Google Scholar 

  30. Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38:2074–102.

    Article  Google Scholar 

  31. Mirowski M, Reid PR, Mower MM, Watkins L, Gott VL, Schauble JF, et al. Termination of malignant ventricular arrhythmias with an implanted automatic defibrillator in human beings. N Engl J Med. 1980;303:322–4.

    Article  CAS  Google Scholar 

  32. Moss AJ, Zareba W, Hall WJ, Klein H, Wilber DJ, Cannom DS, et al. Prophylactic implantation of a defibrillator in patients with myocardial infarction and reduced ejection fraction. N Engl J Med. 2002;346:877–83.

    Article  Google Scholar 

  33. Sanders GD, Hlatky MA, Owens DK. Cost-effectiveness of implantable cardioverter-defibrillators. N Engl J Med. 2005;353:1471–80.

    Article  CAS  Google Scholar 

  34. Technology Evaluation Center. Special report: cost-effectiveness of implantable cardioverter-defibrillators in a MADIT-II population. Assess Progr. 2004;19:1–2.

    Google Scholar 

  35. Fryback DG, Dasbach EJ, Klein R, Klein BE, Dorn N, Peterson K, et al. The Beaver Dam Health Outcomes Study: initial catalog of health-state quality factors. Med Decis Mak. 1993;13:89–102.

    Article  CAS  Google Scholar 

  36. Owens DK, Sanders GD, Harris RA, McDonald KM, Heidenreich PA, Dembitzer AD, et al. Cost-effectiveness of implantable cardioverter defibrillators relative to amiodarone for prevention of sudden cardiac death. Ann Intern Med. 1997;126:1–12.

    Article  CAS  Google Scholar 

  37. O’Brien BJ, Connolly SJ, Goeree R, Blackhouse G, Willan A, Yee R, et al. Cost-effectiveness of the implantable cardioverter-defibrillator: results from the Canadian Implantable Defibrillator Study (CIDS). Circulation. 2001;103:1416–21.

    Article  Google Scholar 

  38. Jalal H, Pechlivanoglou P, Krijkamp E, Alarid-Escudero F, Enns EA, Hunink MGM. An overview of R in health decision sciences. Med Decis Mak. 2017;37:735–46.

    Article  Google Scholar 

  39. Sanders GD, Owens DK, Hlatky MA. Potential cost-effectiveness of wearable cardioverter-defibrillator early post myocardial infarction. Innov Card Rhythm Manag. 2015;6:1929–40.

    Google Scholar 

  40. Theuns DAMJ, Smith T, Hunink MGM, Bardy GH, Jordaens L. Effectiveness of prophylactic implantation of cardioverter-defibrillators without cardiac resynchronization therapy in patients with ischaemic or non-ischaemic heart disease: a systematic review and meta-analysis. Europace. 2010;12:1564–70.

    Article  Google Scholar 

  41. Smith T, Jordaens L, Theuns DAMJ, Van Dessel PF, Wilde AA, Myriam Hunink MG. The cost-effectiveness of primary prophylactic implantable defibrillator therapy in patients with ischaemic or non-ischaemic heart disease: a European analysis. Eur Heart J. 2013;34:211–9.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Drug Policy Program, Center for Research and Teaching in Economics (CIDE)-CONACyT, Circuito Tecnopolo Norte 117, Col. Tecnopolo Pocitos II, 20313, Aguascalientes, AGS, Mexico

    Fernando Alarid-Escudero

  2. Division of Health Policy and Management, School of Public Health, University of Minnesota, Minneapolis, Minnesota, USA

    Karen M. Kuntz

Authors
  1. Fernando Alarid-Escudero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karen M. Kuntz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

FA-E and KMK: study design and analysis. All authors participated in the interpretation of the data, drafting of the manuscript, critical revision of the manuscript, and approval of the final manuscript.

Corresponding author

Correspondence to Fernando Alarid-Escudero.

Ethics declarations

Funding/support

Dr Alarid-Escudero was supported by a grant from Fulbright-García Robles and the National Council of Science and Technology of Mexico (CONACYT) as part of Dr Alarid-Escudero’s doctoral program. Drs Kuntz and Alarid-Escudero were supported by a grant from the National Cancer Institute (U01-CA-199335) as part of the Cancer Intervention and Surveillance Modeling Network (CISNET). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funding agencies had no role in the design of the study, interpretation of results, or writing of the manuscript. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Conflict of interest

FA-E reports no conflicts of interest. KMK reports no conflicts of interest.

Data availability statement

Data and statistical code are provided in the dsHR R package hosted in the GitHub repository https://github.com/feralaes/dshr. The version of dsHR released in this article is available at https://doi.org/10.5281/zenodo.3546663.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 168 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alarid-Escudero, F., Kuntz, K.M. Potential Bias Associated with Modeling the Effectiveness of Healthcare Interventions in Reducing Mortality Using an Overall Hazard Ratio. PharmacoEconomics 38, 285–296 (2020). https://doi.org/10.1007/s40273-019-00859-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-019-00859-5

Search

Navigation