Do Research Groups Align on an Intervention’s Value? Concordance of Cost-Effectiveness Findings Between the Institute for Clinical and Economic Review and Other Health System Stakeholders

  • Matthew SussmanEmail author
  • Jeffrey C. Yu
  • Joseph Menzin
Review Article


The Institute for Clinical and Economic Review (ICER) employs fixed cost-effectiveness (CE) thresholds that guide their appraisal of an intervention’s long-term economic value. Given ICER’s rising influence in the healthcare field, we undertook an assessment of the concordance of ICER’s CE findings to the published CE findings from other research groups (i.e., “non-ICER” researchers including life science manufacturers, academics, and government institutions). Disease areas and pharmaceutical interventions for comparison were determined based on ICER evaluations conducted from 1 January 2015 to 31 December 2017. A targeted literature search was conducted for non-ICER CE publications using PubMed. Studies had to be conducted from the US setting, include the same disease characteristics (e.g., disease severity; treatment history), incorporate the same pharmaceutical interventions and comparison groups, and present incremental costs per quality-adjusted life-year (QALY) gained from the healthcare sector or payer perspective. Discordance was measured as the proportion of unique interventions that would have had more favorable valuations (i.e., low, intermediate, high value-for-money) if the CE findings from other research groups had been used for decision making instead of ICER’s findings. More favorable valuations were defined as transitioning from low value (as determined by ICER) to intermediate or high value (as determined by other researchers) and from intermediate value (as determined by ICER) to high value (as determined by other researchers). Among the 13 non-ICER studies meeting inclusion criteria, six disease areas and 14 interventions were assessed. Of the 14 interventions, a more favorable valuation would have been recommended for ten therapies if the CE ratios from other research groups had been used for decision making instead of ICER’s findings, representing a 71.4% (10/14) discordance rate. Moreover, these discrepancies were found in each of the evaluated disease areas, with the largest number of discordant valuations found in rheumatoid arthritis (five out of six interventions were discordant) followed by one valuation each in multiple sclerosis (one out of three), non-small cell lung cancer (one out of two), multiple myeloma (one out of one), high cholesterol (one out of one), and congestive heart failure (one out of one). Our findings indicate high discordance when comparing ICER’s appraisals to the CE findings of non-ICER researchers. To understand the value of new interventions, the totality of evidence on the CE of an intervention—including results from ICER and non-ICER modeling efforts—should be considered when making coverage and reimbursement decisions.


Author Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by Matthew Sussman, Jeffrey C. Yu, and Joseph Menzin. The first draft of the manuscript was written by Matthew Sussman and Jeffrey C. Yu and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Compliance with Ethical Standards


This study received no external funding.

Conflict of interest

Matthew Sussman, Jeffrey C. Yu, and Joseph Menzin declare no conflicts of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. 1.
    CVS Health. Current and new approaches to making drugs more affordable. Updated August 2018. Accessed 16 Feb 2019.
  2. 2.
    Institute for Clinical and Economic Review. The Institute for Clinical and Economic Review to Collaborate with the Department of Veteran Affairs’ Pharmacy Benefits Management Services Office. Updated June 27, 2017. Accessed 16 Feb 2019.
  3. 3.
    Robinson JC. Amgen Cuts Repatha’s Price By 60% Will Value-based Pricing Support Value-based Patient Access? Health Affairs. Updated November 28, 2018. Accessed 16 Feb 2019.
  4. 4.
    Institute for Clinical and Economic Review. PCSK9 inhibitors for treatment of high cholesterol: effectiveness, value, and value-based price benchmarks. Accessed 20 Nov 2019.
  5. 5.
    Institute for Clinical and Economic Review. About. Accessed 16 Feb 2019.
  6. 6.
    Institute for Clinical and Economic Review. Final value assessment framework for 2017–2019. Accessed 16 Feb 2019.
  7. 7.
    Dubois, RW. From the Editorial Board: Robert W. Dubois, MD, PhD. Am J Manag Care. 2019;25(7):302. Accessed 15 Aug 2019.
  8. 8.
    Center for the Evaluation of Value and Risk in Health. CEA Registry. Accessed 25 Mar 2019.
  9. 9.
    Institute for Clinical and Economic Review. Calcitonin gene-related peptide (CGRP) inhibitors as preventive treatments for patients with episodic or chronic migraine: effectiveness and value. Accessed 20 Nov 2019.
  10. 10.
    Sussman M, Benner J, Neumann P, Menzin J. Cost-effectiveness analysis of erenumab for the preventive treatment of episodic and chronic migraine: results from the US societal and payer perspectives. Cephalalgia. 2018;38(10):1644–57.CrossRefGoogle Scholar
  11. 11.
    Institute for Clinical and Economic Review. Targeted immune modulators for rheumatoid arthritis: effectiveness & value. Accessed 20 Nov 2019.
  12. 12.
    Institute for Clinical and Economic Review. Disease-modifying therapies for relapsing-remitting and primary-progressive multiple sclerosis: effectiveness & value. Accessed 20 Nov 2019.
  13. 13.
    Institute for Clinical and Economic Review. Treatment options for advanced non-small cell lung cancer: effectiveness, value, and value-based price benchmarks. Accessed 20 Nov 2019.
  14. 14.
    Institute for Clinical and Economic Review. Treatment options for relapsed or refractory multiple myeloma: effectiveness, value, and value-based price benchmarks. Accessed 20 Nov 2019.
  15. 15.
    Institute for Clinical and Economic Review. CardioMEMS HF System (St. Jude Medical, Inc.) and sacubitril/valsartan (Entresto, Novartis AG) for management of congestive heart failure: effectiveness, value, and value-based price benchmarks. Accessed 20 Nov 2019.
  16. 16.
    Spalding JR, Hay J. Cost effectiveness of tumour necrosis factor-α inhibitors as first-line agents in rheumatoid arthritis. Pharmacoeconomics. 2006;24(12):1221–32.CrossRefGoogle Scholar
  17. 17.
    Wong JB, Singh G, Kavanaugh A. Estimating the cost-effectiveness of 54 weeks of infliximab for rheumatoid arthritis. Am J Med. 2002;113(5):400–8.CrossRefGoogle Scholar
  18. 18.
    Yuan Y, Trivedi D, Maclean R, et al. Indirect cost-effectiveness analyses of abatacept and rituximab in patients with moderate-to-severe rheumatoid arthritis in the United States. J Med Econ. 2010;13(1):33–41.CrossRefGoogle Scholar
  19. 19.
    Vera-Llonch M, Massarotti E, Wolfe F, et al. Cost-effectiveness of abatacept in patients with moderately to severely active rheumatoid arthritis and inadequate response to tumor necrosis factor-α antagonists. J Rheumatol. 2008;35(9):1745–53.PubMedGoogle Scholar
  20. 20.
    Bell C, Graham J, Earnshaw S. Cost-effectiveness of four immunomodulatory therapies for relapsing-remitting multiple sclerosis: a Markov model based on long-term clinical data. J Manag Care Pharm. 2007;13(3):245–61.PubMedGoogle Scholar
  21. 21.
    Pan F, Goh JW, Cutter G, et al. Long-term cost-effectiveness model of interferon beta-1b in the early treatment of multiple sclerosis in the United States. Clin Ther. 2012;34(9):1966–76.CrossRefGoogle Scholar
  22. 22.
    Earnshaw SR, Graham J, Oleen-Burkey M, et al. Cost effectiveness of glatiramer acetate and natalizumab in relapsing-remitting multiple sclerosis. Appl Health Econ Health Policy. 2009;7(2):91–108.CrossRefGoogle Scholar
  23. 23.
    Aguiar PN Jr, Perry LA, Penny-Dimri J, et al. The effect of PD-L1 testing on the cost-effectiveness and economic impact of immune checkpoint inhibitors for the second-line treatment of NSCLC. Ann Oncol. 2017;28(9):2256–63.CrossRefGoogle Scholar
  24. 24.
    Huang M, Lou Y, Pellissier J. Cost effectiveness of Pembrolizumab vs. standard-of-care chemotherapy as first-line treatment for metastatic NSCLC that expresses high levels of PD-L1 in the United States. Pharmacoeconomics. 2017;35(8):831–44.CrossRefGoogle Scholar
  25. 25.
    Jakubowiak AJ, Campioni M, Benedict Á, et al. Cost-effectiveness of adding carfilzomib to lenalidomide and dexamethasone in relapsed multiple myeloma from a US perspective. J Med Econ. 2016;19(11):1061–74.CrossRefGoogle Scholar
  26. 26.
    Gandra SR, Villa G, Fonarow GC, et al. Cost-effectiveness of LDL-C lowering with evolocumab in patients with high cardiovascular risk in the United States. Clin Cardiol. 2016;39(6):313–20.CrossRefGoogle Scholar
  27. 27.
    King JB, Shah RU, Bress AP, et al. Cost-effectiveness of sacubitril-valsartan combination therapy compared with enalapril for the treatment of heart failure with reduced ejection fraction. JACC Heart Fail. 2016;4(5):392–402.CrossRefGoogle Scholar
  28. 28.
    Gaziano TA, Fonarow GC, Claggett B, Chan WW, et al. Cost-effectiveness analysis of sacubitril/valsartan vs enalapril in patients with heart failure and reduced ejection fraction. JAMA Cardiol. 2016;1(6):666–72.CrossRefGoogle Scholar
  29. 29.
    Bucher HC, Guyatt GH, Griffith LE, et al. The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997;50:683–91.CrossRefGoogle Scholar
  30. 30.
    Science & Innovation Theater. (2019). ICER: payer perspectives on the use and usage of ICER reports [PowerPoint slides].
  31. 31.
    Bell CM, Urbach DR, Ray JG, et al. Bias in published cost effectiveness studies: systematic review. BMJ. 2006. Scholar
  32. 32.
    Norris P, Herxheimer A, Lexchin J, Mansfield P. Drug promotion: what we know, what we have yet to learn. World Health Organization and Health Action International; 2005.Google Scholar
  33. 33.
    Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: a systematic review. BMJ. 2003;326:1167–70.CrossRefGoogle Scholar
  34. 34.
    Neumann PJ, Sandberg EA, Bell CM, et al. Are pharmaceuticals cost-effective? A review of the evidence. Health Aff. 2000;19(2):92–109.CrossRefGoogle Scholar
  35. 35.
    Chopra SS. Industry funding of clinical trials: benefit or bias? JAMA. 2003;290(1):113–4.CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Boston Health Economics, LLCBostonUSA

Personalised recommendations