Engaging with the Institute for Clinical and Economic Review (ICER), a health technology assessment organization located in the USA, can pose a challenge for drug manufacturers. Drug manufacturers can engage with ICER by providing information and feedback at several points during ICER’s assessment process. Engagement offers drug manufacturers an opportunity to help ensure that ICER’s assessments reflect all available science and input, and to respond to the organization’s draft reports before they reach their final form. However, some may see it as legitimizing potentially controversial findings from ICER about a drug’s  value. To our knowledge, however, no formal investigation has pursued this question. We undertook an empirical analysis to understand how often manufacturers engage with ICER and whether engagement may influence ICER’s cost-effectiveness estimates.

We examined pharmaceutical assessments conducted by ICER from June 2017 to December 2021. These assessments are available on ICER’s website [1]. First, we recorded how frequently manufacturers engage with ICER, measured as the number of manufacturer engagements divided by the number of engagement opportunities. For this analysis, the unit of observation was each unique manufacturer. We designated an ICER draft evidence report an “engagement opportunity” for a manufacturer (the frequency’s denominator) if that report assessed at least one drug produced by that manufacturer. For example, an ICER report that evaluated two drugs produced by manufacturer “A” and one drug produced by manufacturer “B” would represent two engagement opportunities (one for “A” and one for “B”). We designated a drug manufacturer’s interaction with ICER an “engagement” (the frequency’s numerator) if the manufacturer submitted at least one written comment in response to an ICER draft evidence report. For example, if manufacturer “A” submitted three comments in response to the draft report and manufacturer “B” submitted no comments to that report, the report would represent one engagement. Limiting attention to the one report and two manufacturers in this example, the engagement frequency is 50% (two manufacturers had an opportunity to engage and one did so). We note that this type of engagement is but one way manufacturers can engage with ICER, with others including the provision of data, suggestions of clinical experts, patient representatives, and other stakeholders, comments on draft documents, and participation in ICER’s public meetings [2].

Second, we calculated the proportion of cost-effectiveness estimates that “improved” between the draft and revised evidence reports, stratifying these results by whether the manufacturer of the drug under evaluation engaged with ICER. For this analysis, the unit of observation was each unique incremental cost-effectiveness ratio. We designated changes as “improved” if ICER reduced its estimated cost-effectiveness ratio from the draft to the revised report because a lower cost-effectiveness ratio indicates that the intervention achieves health gains at a lower cost. We evaluated the association between “improved” and company engagement using a chi-square test of independence and designated a p value < 0.05 to be statistically significant.

Our final analysis was analogous to our second analysis, although this last analysis assessed the association between company engagement with ICER and whether the ratio reported in ICER’s revised report was “favorable” (i.e., no more than $150,000 per quality-adjusted life-year). We reviewed 35 ICER assessments representing 88 manufacturer engagement opportunities. Manufacturers engaged in 80 (91%) of those instances. From the draft to the revised reports, cost-effectiveness ratios improved in 55% of the instances in which manufacturers engaged, and in 22% of the instances in which manufacturers did not engage (Table 1; p = 0.06). Finally, product cost-effectiveness ratios in the revised evidence report were favorable in 34% of the instances when the manufacturer engaged, and in 9% of the instances when the manufacturer did not engage (Table 1; p = 0.09).

Table 1 Is manufacturer engagement with ICER associated with improvements in cost effectiveness from the draft to the revised report and favorable revised results?
Full size table

Note that while the third analysis counts all ratios that appear in a revised ICER report, the second analysis compares each draft report ratio to a corresponding revised report ratio. Because the second analysis counts ratios only if they appear in both the draft and revised reports, and because some ratios in the revised report have no corresponding draft report value, the sample for the second analysis (n = 92) is smaller than the sample for the third analysis (n = 133).

Our analysis shows that the vast majority of manufacturers engage with ICER when that organization assesses their products. There is a nonsignificant association between manufacturer engagement and (1) improved cost-effectiveness ratios from the draft to the revised evidence reports and (2) reporting of a favorable ratio in the revised report. These findings suggest that manufacturers should continue to engage with ICER, as the vast majority have.

We note our study’s limited sample size and that our analysis does not account for clustering introduced because some manufacturers had multiple opportunities to comment on ICER’s reports. Furthermore, our results may reflect “reverse causality.” For example, when ICER is unlikely to improve its ratio between the draft and revised reports, or when it is unlikely to report a favorable value for a ratio, a manufacturer may choose not to engage either because doing so consumes resources that the company anticipates will yield little return or because it perceives that engaging with ICER will lend legitimacy to the group’s findings.

Companies should understand that ICER infrequently makes substantial changes to its results from the draft to the revised report, and that public comments infrequently affect the cost-effectiveness estimates [3, 4]. Nonetheless, Cohen et al. found that a company can improve its chances of influencing ICER’s revised findings by submitting comments that are clear, offer a specific alternative to ICER’s approach, and make a case that the suggested revision’s impact will be meaningful [3]. Research by Ronquest et al. found that ICER was more likely to make changes to its base-case analysis based on comments that include specific alternative data or a published article that supported the changes [4].

Commenting on ICER’s draft scoping document, which ICER releases at the beginning of its assessment process, represents an additional opportunity for manufacturers to engage. A sensitivity analysis broadening our definition of “engagement” to include commenting on ICER’s draft scoping document, its draft evidence report, or both did not substantially influence our results.

Finally, as noted earlier, manufacturers can engage with ICER in other ways, such as by providing data, suggesting clinical experts, patient representatives, and other stakeholders, and by making statements and/or by serving on the policy roundtable at ICER’s public meetings [2]. Further research should explore the influence of these other types of engagement on cost-effectiveness findings and on other outcomes in the assessment process, such as council votes on value.