Skip to main content

Real-World Evidence: Bridging Gaps in Evidence to Guide Payer Decisions

Abstract

Randomized controlled trials (RCTs) are preferred by payers for health technology assessments and coverage decisions. However, the inclusion of a highly selective patient population and the rigorously controlled conditions in RCTs may not be reflective of real-world clinical practice. Real-world evidence (RWE) obtained from an analysis of real-world data (RWD) from observational studies can bridge gaps in evidence not addressed by RCTs and is thus valuable to public and private payers for decision-making. Through a broad literature search to obtain insights into payers’ experience, we found that payers have concerns about real-world studies with respect to data quality, poor internal validity, potential bias, and lack of meaningful endpoints. However, they valued RWE to fill evidence gaps not addressed by RCTs, such as high-quality, real-world, long-term effectiveness and safety data; head-to-head drug comparisons; cost analyses for tiering formulary placement; medication use and adherence patterns; identification of relevant responder and non-responder patient subpopulations; and patient-reported outcomes (PROs). RWE can be used to assess clinically meaningful endpoints and gauge the impact of interventions on the quality of healthcare. Here, we review how payers use or can use RWD on the comparative effectiveness and safety of treatments, PROs, medication adherence and persistence, prescribing patterns, healthcare resource utilization, and patient characteristics and/or biomarkers associated with treatment response when making health technology assessments and payer coverage decisions across therapeutic areas.

FormalPara Key Points for Decision Makers
Although payers consider randomized controlled trials (RCTs) the gold standard for decision-making, they also value real-world evidence (RWE) for evidence gaps not filled by RCTs.
RWE can provide high-quality data on long-term effectiveness and safety in real-world settings, head-to-head comparisons, cost analyses for formulary placements, medication use and adherence patterns, and patient-reported outcomes; and consequently, can aid payers in making health technology assessments and coverage decisions across therapeutic areas.
Some payer concerns limit the use of RWE: study design limitations, lack of transparency in research methods and analyses used, timeliness of results for pharmacy and therapeutic committee decisions, potential bias, and a lack of training to evaluate observational studies.

Introduction

Real-world evidence (RWE) obtained from an analysis of real-world data (RWD) is a valuable tool that can be used to aid healthcare decision-making [1]. RWD could be derived prospectively or retrospectively from multiple sources, including electronic health records, claims databases, pragmatic trials, or registries, but they may also be gathered from patient-generated sources such as smartphones, wearable devices, and survey data [2, 3]. RWD supplement clinical trial data, which are usually obtained from randomized controlled trials (RCTs). However, RCTs include highly selective patient populations in tightly controlled experimental settings [4]. While evidence obtained from RCTs is considered most reliable, the study populations may not be representative of real-world patients and the study logistics may not be representative of routine clinical practice.

RWE has several applications in the United States (US). Besides aiding the pharmaceutical industry with drug development pathways [1] and the US Food and Drug Administration (FDA) with regulatory approval decisions and safety monitoring [2], RWE is also used by public and private payers while making health technology assessments and payer coverage decisions, initially and during reassessment [5]. Initially, claims-based epidemiological data can be used to identify the patient population(s) that may be eligible for a drug and to determine preliminary cost estimates. During reassessments, RWE can help guide decisions governing reconsideration of coverage, discounts, and formulary tiering based on effectiveness and safety observed during real-world drug use [5]. Here, we discuss the value of RWE from a payer’s perspective, highlight the applications of RWE to payers, and illustrate how RWE has been or might be used to address gaps in evidence using clinical examples across therapeutic areas.

Insights from Payer-Experience Studies

Limited literature exists on how payers utilize available clinical data for their decision-making. Using a broad literature review (see electronic supplementary material [ESM]), we identified six articles summarizing payer surveys (Supplementary Table, see ESM). RCTs were the preferred source of data for coverage decisions made by payers, especially those related to initial market access and pharmacy and therapeutic committee decisions [6,7,8,9]. Payers also used systematic reviews and meta-analyses to assess available evidence and replicability of findings across sources [6]. Although payers continued to regard RCTs as the gold standard, they valued RWE to fill evidence gaps not addressed by RCTs, such as long-term effectiveness and safety [6, 7, 9]; head-to-head drug comparisons [6,7,8,9,10]; cost analyses for tiering formulary placement [6, 7, 9, 11]; medication use patterns, including medication adherence [7, 9]; identification of relevant responder and non-responder patient subpopulations [8, 9]; and patient-reported outcome (PRO) data [10]. Notably, PRO data were considered especially useful in oncology compared with other therapeutic areas, given the high symptom burden and increased need for palliative care [10].

Overall, payers considered efficacy, effectiveness, safety, FDA approval status, availability of alternative treatments, and acquisition cost of drugs to be important factors in formulary decision-making [6]. In addition, payers considered clinical and economic aspects of treatments as equally important for making coverage decisions [11]. They valued confirmatory evidence that the long-term, real-world consequences of covering treatments matched the expected benefits, particularly for high-cost curative therapies [11].

Many payers cited concerns regarding data quality, study design flaws, potential bias, and lack of meaningful endpoints as barriers to using RWE when making decisions [6, 7, 9]. Notably, a need was identified for continuing education about the evaluation and use of RWE to better understand the study methods, findings, and applicability to respective organizations [7]. Taken together, these payer insights indicate that, while they rely on RCTs for evaluation of comparative efficacy and safety of treatments, high-quality RWE on long-term safety and effectiveness, healthcare resource utilization (HCRU) and costs, treatment patterns and medication adherence, PROs, and identification of patient subgroups most suitable for treatments are being used or are of potential use to payers.

Role of Real-World Evidence in Bridging Evidence Gaps

To expand upon our findings from the literature, we sought to illustrate applications of RWE for payers (Fig. 1), including how RWE has been or could be used to fill gaps in evidence across therapeutic areas to aid payer decision-making.

Fig. 1
figure 1

Applicability of RWE to payers. RWD real-world data, RWE real-world evidence

Comparative Effectiveness

A relatively small proportion of RCTs are head-to-head, active-comparator studies [12]. RWE can help fill this void by providing insights into the comparative effectiveness of drugs in routine clinical practice [1]. Furthermore, as available treatment options increase, clarity is needed about which patients are most likely to benefit from the range of therapies. An example is the lack of definitive RWE on the effectiveness of warfarin and direct oral anticoagulants (DOACs; e.g., apixaban, rivaroxaban, and dabigatran) for non‐valvular atrial fibrillation (NVAF) in special patient populations, such as those who are obese, the elderly, or those who have specific comorbidities. Large administrative claims databases are an ideal RWD source and they enable drug comparisons that may or may not have been performed in RCTs; for example, a number of studies have evaluated real-world effectiveness of DOACs versus warfarin in large databases [13, 14]. Patient registries are also a valuable source for evaluating the comparative effectiveness of treatments in special patient populations; for example, registry and claims-based studies have evaluated anticoagulation in patients with atrial fibrillation and comorbid diabetes mellitus, peripheral heart disease, and a high risk of bleeding [15]; the comparative effectiveness of DOACs in morbidly obese patients [16]; and older patients with heart failure [17].

Pragmatic clinical trials (PrCTs) have features of RCTs (e.g., randomization) and real-world observational studies (e.g., routine clinical practice, prospectively collected and clinically relevant RWD) [18]. Therefore, they can be used to fill evidence gaps across therapeutic areas and to address some of payers’ concerns about RCTs (e.g., restrictive eligibility criteria and rigorous monitoring) and real-world studies (e.g., study design limitations, difficulty in interpreting results, poor internal validity, and timeliness of results) [6, 18, 19]. For example, patients with chronic kidney disease often have diverse comorbid conditions, limiting the generalizability of RCTs that have highly restrictive eligibility criteria [19]. PrCTs may be ideally suited to address questions related to the effectiveness of available chronic kidney disease treatments because of their real-world settings and corresponding patients, ability to generate clinically relevant results with high external validity and applicability, and low costs. Some PrCTs have also been criticized because of their more complex study designs compared with RCTs. For example, classification of the Salford Lung Study, which evaluated comparative effectiveness of fluticasone furoate + vilanterol combination in patients with chronic obstructive pulmonary disease (COPD) within the pragmatic framework, was questioned because of a different target population than that approved by regulatory authorities [20] and a possibility that the Hawthorne effect—a change in subjects’ behavior because of their awareness of being observed—may have impacted treatment outcomes [21].

Safety Monitoring

Real-world studies can provide information on safety signals and adverse drug reactions associated with routine clinical practice. Notably, retrospective observational claims data studies have found a lower all-cause mortality risk with DOACs than warfarin among patients aged ≥ 80 years [22], and among patients aged ≥ 65 years with coronary/peripheral artery disease [23]. Long-term use of inhaled corticosteroid (ICS) in patients with COPD has been associated with an increased risk of non-fatal pneumonia and other local and systemic adverse events [24], and while longer duration RCTs are needed to better understand safety of long-term ICS use beyond a couple of years, they can be very costly. Comparable safety outcomes have been reported in some RCTs [25, 26]; however, results of real-world studies [27,28,29] that can be conducted at a much reduced cost have supported head-to-head RCTs that in general have reported pneumonia less often among patients receiving non-ICS-containing treatments (long-acting β2-agonist [LABA] or long-acting muscarinic antagonist [LAMA] + LABA) than among those receiving ICS-containing treatments (LABA + ICS or LAMA + LABA + ICS) [30,31,32,33,34].

Medication Adherence and Persistence

Medication adherence (taking medications as prescribed [timing, dosage, and frequency]) and persistence (taking medication for the prescribed period of time) are particularly important in chronic diseases such as diabetes, heart disease, and respiratory illnesses, where complex treatment regimens can negatively affect medication-taking behavior [35]. Patients in general, and their medication-taking behavior specifically, are closely monitored in RCTs; thus, adherence, persistence, and compliance outcomes may not reflect actual, real-world patient behavior. In contrast, investigating medication-taking behaviors is uniquely suited to real-world studies, where routine follow-up and monitoring occurs [36].

Likewise, RWE can be invaluable when assessing medication-taking behavior as well as inhaler use and technique among patients with COPD or asthma. In routine clinical practice, patients are not regularly trained or monitored for proper inhaler technique [37, 38]. As the factors associated with adherence are multidimensional, well-designed clinical studies specifically probing reasons patients are more or less adherent would provide information about patient preferences. Addressing these patient preferences might improve adherence and clinical outcomes. Several real-world studies illustrate this point. In a claims-based study (IQVIA™ Real-World Data Adjudicated Claims) of adherence and persistence among patients with COPD using multiple inhaler triple therapy, patients were more adherent with two versus three inhalers, suggesting simplified treatment regimens may increase adherence and persistence [39]. However, corresponding clinical outcomes were not investigated, precluding conclusions about whether or not the use of fewer inhalers was associated with comparatively better clinical outcomes. However, in another real-world survey of physicians and COPD patients, increased inhaler satisfaction was associated with significantly increased treatment compliance, improved health outcomes, and fewer exacerbations [40].

Finally, identifying patient demographics and characteristics associated with poor adherence/treatment discontinuation was the focus of several studies [41,42,43]. Addressing some of the barriers to optimal adherence (e.g., forgetting to refill prescriptions), persistence, and compliance (e.g., taking one puff instead of two each day) could help improve clinical outcomes and lower costs [44, 45].

Healthcare Resource Utilization and Costs

One factor related to higher healthcare spending in the US compared with other countries is the high cost of pharmaceutical products [46]. Pharmaceutical cost is indeed a concern for payers, but payers are also focused on interventions that may reduce the high cost of potentially avoidable emergency department and hospitalization use [47]. While payers value RCT evidence in making treatment decisions, economic endpoints are often not included in RCTs [48]. When evaluating the economic costs and benefits of treatments, payers recognize that estimates may be more reliable for mature treatments than for new treatments, and that estimates are often supported by retrospective database analyses and decision models [49]. Hospital readmissions are a potentially avoidable contributor to HCRU that place a substantial burden on the healthcare system [50]. The US Centers for Medicare & Medicaid Services uses 30-day readmission rates as a measure of healthcare quality [51], and through its Hospital Readmissions Reduction Program (HRRP), Medicare payments are reduced to hospitals with excess readmissions for patients initially hospitalized with acute myocardial infarction, COPD, heart failure, pneumonia, coronary artery bypass graft surgery, or total knee/hip arthroplasty [51]. Real-world studies can help determine readmission rates and identify precipitating factors. Further, and importantly, large nationally representative data sources can easily be used. In COPD, analysis of the Nationwide Readmissions Database, containing inputs from 40 US states from 2013 to 2014, indicated a 30-day readmission rate of 19.2% following an acute exacerbation [52]. Of interest, this rate was lower than the estimate of 20.5% found in a study using data from 15 US states during 2008, before notification of the HRRP implementation [53]. Recognizing that readmission risk after discharge from a COPD hospitalization is interwoven with mortality risk in the real world, Ohar and colleagues demonstrated that a comprehensive care plan focusing on management of COPD and comorbid conditions for patients discharged from an acute COPD hospital stay was associated with a reduction in 30- and 90-day readmissions in a real-world exploratory evaluation [54].

Of note, country-specific guidance on assessing economic- and cost-related data for new drugs is provided by the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) [55, 56]. Payers are also increasingly interested in not only the costs associated with chronic and long-term treatment [8] but also the high up-front costs associated with curative drugs whose offsetting cost benefits may not be realized for decades [11]. Over the last decade, the development of drugs that are curative and/or life-changing for individuals with certain diseases has sparked a greater interest in the value of new drugs. Of note, forecasts were dire regarding the budget impact of paying for curative treatments for hepatitis C [57], though some estimates may have overestimated the actual budget impact [58]. The US does not have universal healthcare coverage; historically, formal budget impact analyses have been conducted infrequently in the US [59], but use has increased in recent years. In a study of budget impact modeling versus real-world expenses for high-cost drugs, real-world expenses were usually lower than forecasted by budget impact models [60], suggesting that the payers recognized the value of these drugs, but also managed use to specific patient populations as opposed to having open access for all patients. RWD can be particularly useful in providing data on disease incidence, treatment patterns, and disease-related costs for impact estimates [61, 62].

Results of a retrospective study of 1648 patients with NVAF conducted using medical and pharmacy claims data showed that although dabigatran had higher pharmacy costs than warfarin, costs were offset by lower medical costs in up to 12 months of follow-up because of significantly fewer physician visits than with warfarin [63]. However, only direct costs were assessed; evaluation of other parameters, such as work productivity, would have helped elucidate the complete costs associated with such treatments. In another real-world study in elderly Medicare beneficiaries with NVAF, dabigatran was associated with lower cost and higher quality-adjusted life-years compared with rivaroxaban [64]. The study was, however, limited by a short follow-up time.

Patient-Reported Outcomes

A review of ClinicalTrials.gov for the years 2007 through 2013 reported that > 25% of clinical trials used PRO measures, with the highest use among trials for neoplasms (> 70%), mental and behavioral conditions (approximately 40%), and nervous system conditions (approximately 25%) [65]. Recognizing that PRO instruments can provide information on benefit and risk of pharmaceutical products based on PRO data, the US FDA in 2009 issued guidance on incorporating PROs in product labeling claims [66]. This was followed several years later by the FDA patient-focused drug development initiative [67]. Payers are also increasingly recognizing the importance of patient perspectives and PROs in market access and reimbursement decisions [10]. Of note, while disease impact on work productivity costs (to include absenteeism and/or presenteeism) may be included in economic evaluations, and may be particularly meaningful for a condition such as irritable bowel disease that impacts a much younger population [68], to date productivity costs have been minimally included in cost-effectiveness evaluations oriented to payers [69, 70].

When using RWD to develop RWE, results of PROs and claims-based studies may not always align [71]. For example, in one study, 47.9% of patients with NVAF were adherent to oral anticoagulant therapy per pharmacy claims data (using proportion of days covered), while only 37.2% were adherent according to patient self-reporting (using the 8-item Morisky Medication Adherence Scale [MMAS-8]). The different findings may, in part, be attributed to the different methodologies: claims-based data provide long-term adherence data, while the MMAS-8 provides data about a patient’s recall of recent medication use [71]. Further research is warranted to elucidate the sensitivities of different measures in predicting various outcomes. These results also highlight the need to understand the limitations associated with different methodologies and the need to consider multiple RWD sources while making decisions.

Informing Personalized Medicine

Patient characteristics and/or biomarkers associated with treatment responses can be evaluated in long-term observational studies, and a number of clinical factors correlated with response to drugs across therapeutic areas in real-world studies [72, 73]. However, not all clinical factors identified in RCTs or real-world studies are sufficiently predictive and/or useful in routine clinical settings. Not surprisingly, identification and validation of biomarkers predictive of treatment response have gained attention, leading to establishment of registries that include biomarker data [74,75,76,77]. Registry data are already being used to identify predictive biomarkers of treatment response across disease states [78, 79]. RWE can also reveal unmet treatment needs in patient subgroups. Findings from IDEAL (Identification and Description of sEvere Asthma patients in a cross-sectionaL study) [80]—a prospective, global, observational study conducted to identify and characterize patients with severe asthma eligible for biologic therapy with mepolizumab, omalizumab, or reslizumab—highlight that limited treatment options beyond high-dose ICS and oral corticosteroids are available for most patients with severe asthma.

Discussion

While RCTs remain the gold standard for making coverage and reimbursement decisions, payers recognize the role high-quality RWE can play [7,8,9, 81]. Payers are interested in real-world, long-term, comparative effectiveness and safety data from head-to-head comparisons with current standard of care, as well as RWD about prescription patterns and medication adherence, HCRU, costs and economic endpoints, PROs, and the identification of distinct patient subpopulations for personalized medicine [6,7,8,9, 11]. We illustrated the applications of RWE in bridging evidence gaps. However, real-world research has the potential to play a larger role in addressing gaps in evidence with RWD across therapeutic areas. For instance, COPD is associated with a high hospital readmission rate post-exacerbation, but information about the interventions most effective at preventing readmissions is often conflicting [82,83,84]. Furthermore, information about the impact of comorbidities on outcomes of non-pharmacological COPD treatments, such as pulmonary rehabilitation, is limited [85]. Linking poor medication adherence to different treatments to clinical and economic outcomes in chronic diseases such as COPD will also be meaningful to payers interested in the long-term clinical and financial aspects of treatments. In the absence of head-to-head RCTs, emerging RWE can fill this evidence gap. In addition, RWE will be crucial to understand how high costs of biologics compare with traditional therapies. Exploring the potential use of biologics, not just as late-stage treatment but at earlier stages, may also be possible through investigation of claims databases.

A distinct gap exists between payers’ expectations and the current state of comparative effectiveness research [9]. Payers also differ in the extent to which they use RCTs and RWE to inform their decisions. Moreover, the relative value of each type of evidence used by payers varies and is not transparent, making a study design that fulfills different payer demands difficult [8]. Payers prefer to use RCT data validated in real-world studies, where outcomes in their plans’ membership reflect RCT findings. Practically, PrCTs could assume a more prominent role in payers’ decision-making considering their use of randomization, inclusion of appropriate comparator arms, and generation of data that are reflective of real-world populations [8].

Factors that limit the use of RWE include concerns about study design limitations; poor internal validity; lack of transparency in research methods and analyses used; delays in obtaining results in time for pharmacy and therapeutic committee decisions; potential bias; and a lack of budget, skilled support staff, and training to evaluate observational studies [7,8,9]. While assessment tools are available for the interpretation and use of RWE aimed at payers [86,87,88], no guidance is available on how to interpret the entire body of evidence (RCTs and RWE) available to payers. With the availability of data from different clinical trials and real-world settings, the complexity of evidence available to the payer has increased exponentially, necessitating the need for advanced tools to analyze the high volumes of data efficiently.

Conclusion

Although RCTs remain crucial for coverage and reimbursement decisions by payers, RWE can be employed to assess clinically meaningful endpoints, gauge the impact of interventions on the quality of healthcare, and help payers make appropriate data-driven decisions.

References

  1. Khosla S, White R, Medina J, Ouwens M, Emmas C, Koder T, et al. Real world evidence (RWE)—a disruptive innovation or the quiet evolution of medical evidence generation? F1000Research. 2018;7.

  2. FDA. Framework for FDA’s Real-World Evidence Program. [serial online] 2018 December 2018. https://www.fda.gov/media/120060/download. Accessed 18 June 2019.

  3. Berger M, Daniel G, Frank K, Hernandez A, McClellan M, Okun S. A framework for regulatory use of real-world evidence. White paper prepared by the Duke Margolis Center for Health Policy. 2017.

  4. Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials. 2015;16:495.

    PubMed  PubMed Central  Google Scholar 

  5. Hampson G, Towse A, Dreitlein WB, Henshall C, Pearson SD. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res. 2018;7(12):1133–43.

    PubMed  Google Scholar 

  6. Leung MY, Halpern MT, West ND. Pharmaceutical technology assessment: perspectives from payers. J Manag Care Pharm. 2012;18(3):256–64.

    PubMed  Google Scholar 

  7. Malone DC, Brown M, Hurwitz JT, Peters L, Graff JS. Real-world evidence: useful in the real world of US payer decision making? How? When? And what studies? Value Health. 2018;21(3):326–33.

    PubMed  Google Scholar 

  8. Moloney R, Mohr P, Hawe E, Shah K, Garau M, Towse A. Payer perspectives on future acceptability of comparative effectiveness and relative effectiveness research. Int J Technol Assess Health Care. 2015;31(1–2):90–8.

    PubMed  PubMed Central  Google Scholar 

  9. Wang A, Halbert RJ, Baerwaldt T, Nordyke RJ. US payer perspectives on evidence for formulary decision making. Am J Manag Care. 2012;18(5 Spec No. 2):Sp71–6.

  10. Brogan AP, DeMuro C, Barrett AM, D'Alessio D, Bal V, Hogue SL. Payer perspectives on patient-reported outcomes in health care decision making: oncology examples. J Manag Care Spec Pharm. 2017;23(2):125–34.

    PubMed  Google Scholar 

  11. Yeung K, Suh K, Garrison LP Jr, Carlson JJ. Defining and managing high-priced cures: healthcare payers' opinions. Value Health. 2019;22(6):648–55.

    PubMed  Google Scholar 

  12. Lathyris DN, Patsopoulos NA, Salanti G, Ioannidis JP. Industry sponsorship and selection of comparators in randomized clinical trials. Eur J Clin Invest. 2010;40(2):172–82.

    CAS  PubMed  Google Scholar 

  13. Coleman C, Bunz T, Eriksson D, Meinecke AK, Sood N. Effectiveness and safety of rivaroxaban vs warfarin in people with non-valvular atrial fibrillation and diabetes: an administrative claims database analysis. Diabet Med. 2018;35(8):1105–10.

    CAS  PubMed  Google Scholar 

  14. Baker WL, Beyer-Westendorf J, Bunz TJ, Eriksson D, Meinecke AK, Sood NA, et al. Effectiveness and safety of rivaroxaban and warfarin for prevention of major adverse cardiovascular or limb events in patients with non-valvular atrial fibrillation and type 2 diabetes. Diabetes Obes Metab. 2019;21(9):2107–14.

    CAS  PubMed  Google Scholar 

  15. Esteve-Pastor MA, Rivera-Caravaca JM, Roldan-Rabadan I, Roldan V, Muniz J, Rana-Miguez P, et al. Quality of oral anticoagulation with vitamin K antagonists in ‘real-world’ patients with atrial fibrillation: a report from the prospective multicentre FANTASIIA registry. Europace. 2018;20(9):1435–41.

    PubMed  Google Scholar 

  16. Peterson ED, Ashton V, Chen YW, Wu B, Spyropoulos AC. Comparative effectiveness, safety, and costs of rivaroxaban and warfarin among morbidly obese patients with atrial fibrillation. Am Heart J. 2019;212:113–9.

    CAS  PubMed  Google Scholar 

  17. Amin A, Reeves ABG, Li X, Dhamane A, Luo X, Di Fusco M, et al. Effectiveness and safety of oral anticoagulants in older adults with non-valvular atrial fibrillation and heart failure. PLoS ONE. 2019;14(3):e0213614.

    CAS  PubMed  PubMed Central  Google Scholar 

  18. Zuidgeest MGP, Goetz I, Groenwold RHH, Irving E, van Thiel G, Grobbee DE. Series: pragmatic trials and real world evidence: paper 1. Introduction. J Clin Epidemiol. 2017;88:7–13.

    PubMed  Google Scholar 

  19. de Boer IH, Kovesdy CP, Navaneethan SD, Peralta CA, Tuot DS, Vazquez MA, et al. Pragmatic clinical trials in CKD: opportunities and challenges. J Am Soc Nephrol. 2016;27(10):2948.

    PubMed  PubMed Central  Google Scholar 

  20. Dal-Ré R. Could phase 3 medicine trials be tagged as pragmatic? A case study: the Salford COPD trial. J Eval Clin Pract. 2018;24(1):258–61.

    PubMed  Google Scholar 

  21. Sedgwick P, Greenwood N. Understanding the Hawthorne effect. BMJ. 2015;351:h4672.

    PubMed  Google Scholar 

  22. Deitelzweig S, Keshishian A, Li X, Kang A, Dhamane AD, Luo X, et al. Comparisons between oral anticoagulants among older nonvalvular atrial fibrillation patients. J Am Geriatr Soc. 2019;67(8):1662–71.

    PubMed  PubMed Central  Google Scholar 

  23. Lopes RD, Steffel J, Di Fusco M, Keshishian A, Luo X, Li X, et al. Effectiveness and safety of anticoagulants in adults with non-valvular atrial fibrillation and concomitant coronary/peripheral artery disease. Am J Med. 2018;131(9):1075–85.e4.

    CAS  PubMed  Google Scholar 

  24. Price D, Yawn B, Brusselle G, Rossi A. Risk-to-benefit ratio of inhaled corticosteroids in patients with COPD. Prim Care Respir J. 2013;22(1):92–100.

    PubMed  Google Scholar 

  25. Papi A, Vestbo J, Fabbri L, Corradi M, Prunier H, Cohuet G, et al. Extrafine inhaled triple therapy versus dual bronchodilator therapy in chronic obstructive pulmonary disease (TRIBUTE): a double-blind, parallel group, randomised controlled trial. Lancet. 2018;391(10125):1076–84.

    CAS  PubMed  Google Scholar 

  26. Ferguson GT, Rabe KF, Martinez FJ, Fabbri LM, Wang C, Ichinose M, et al. Triple therapy with budesonide/glycopyrrolate/formoterol fumarate with co-suspension delivery technology versus dual therapies in chronic obstructive pulmonary disease (KRONOS): a double-blind, parallel-group, multicentre, phase 3 randomised controlled trial. Lancet Respir Med. 2018;6(10):747–58.

    CAS  PubMed  Google Scholar 

  27. Janson C, Johansson G, Stallberg B, Lisspers K, Olsson P, Keininger DL, et al. Identifying the associated risks of pneumonia in COPD patients: ARCTIC an observational study. Respir Res. 2018;19(1):172.

    PubMed  PubMed Central  Google Scholar 

  28. Yang H-H, Lai C-C, Wang Y-H, Yang W-C, Wang C-Y, Wang H-C, et al. Severe exacerbation and pneumonia in COPD patients treated with fixed combinations of inhaled corticosteroid and long-acting beta2 agonist. Int J Chronic Obstr Pulm Dis. 2017;12:2477.

    Google Scholar 

  29. Suissa S, Dell'Aniello S, Ernst P. Comparative effectiveness and safety of LABA-LAMA vs LABA-ICS treatment of COPD in real-world clinical practice. Chest. 2019;155(6):1158–65.

    PubMed  Google Scholar 

  30. Crim C, Calverley P, Anderson J, Celli B, Ferguson G, Jenkins C, et al. Pneumonia risk in COPD patients receiving inhaled corticosteroids alone or in combination: TORCH study results. Eur Respir J. 2009;34(3):641–7.

    CAS  PubMed  Google Scholar 

  31. Rodrigo GJ, Castro-Rodriguez JA, Plaza V. Safety and efficacy of combined long-acting β-agonists and inhaled corticosteroids vs long-acting β-agonists monotherapy for stable COPD: a systematic review. Chest. 2009;136(4):1029–38.

    CAS  PubMed  Google Scholar 

  32. Crim C, Dransfield MT, Bourbeau J, Jones PW, Hanania NA, Mahler DA, et al. Pneumonia risk with inhaled fluticasone furoate and vilanterol compared with vilanterol alone in patients with COPD. Ann Am Thorac Soc. 2015;12(1):27–34.

    PubMed  Google Scholar 

  33. Calverley PM, Anderson JA, Celli B, Ferguson GT, Jenkins C, Jones PW, et al. Salmeterol and fluticasone propionate and survival in chronic obstructive pulmonary disease. N Engl J Med. 2007;356(8):775–89.

    CAS  PubMed  Google Scholar 

  34. Lipson DA, Barnhart F, Brealey N, Brooks J, Criner GJ, Day NC, et al. Once-daily single-inhaler triple versus dual therapy in patients with COPD. N Engl J Med. 2018;378(18):1671–80.

    CAS  PubMed  Google Scholar 

  35. World Health Organization. Adherence to long-term therapies: evidence for action. World Health Organization; 2003.

  36. Coleman CI, Pandya S, Wang L, Baser O, Cai J, Ingham M, et al. Treatment patterns, glycemic control & bodyweight with canagliflozin 300 mg versus GLP1RAs in type II diabetes patients. J Comp Eff Res. 2019;8(11):889–905.

    PubMed  Google Scholar 

  37. Giraud V, Allaert F-A, Roche N. Inhaler technique and asthma: feasability and acceptability of training by pharmacists. Respir Med. 2011;105(12):1815–22.

    PubMed  Google Scholar 

  38. Lingner H, Burger B, Kardos P, Criée C, Worth H, Hummers-Pradier E. What patients really think about asthma guidelines: barriers to guideline implementation from the patients’ perspective. BMC Pulm Med. 2017;17(1):13.

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Bogart M, Stanford RH, Laliberte F, Germain G, Wu JW, Duh MS. Medication adherence and persistence in chronic obstructive pulmonary disease patients receiving triple therapy in a USA commercially insured population. Int J Chronic Obstr Pulm Dis. 2019;14:343–52.

    CAS  Google Scholar 

  40. Chrystyn H, Small M, Milligan G, Higgins V, Gil EG, Estruch J. Impact of patients' satisfaction with their inhalers on treatment compliance and health status in COPD. Respir Med. 2014;108(2):358–65.

    PubMed  Google Scholar 

  41. Deshpande M, Chewning B, Mott D, Thorpe JM, Young HN. Asthma medication use among US adults 18 and older. Res Social Adm Pharm. 2014;10(6):e113–23.

    PubMed  Google Scholar 

  42. Deshpande M, Chewning B, Mott D, Thorpe JM, Young HN. Asthma medication use among late midlife US adults. J Asthma. 2016;53(3):261–8.

    CAS  PubMed  Google Scholar 

  43. Tavakoli H, FitzGerald JM, Lynd LD, Sadatsafavi M. Predictors of inappropriate and excessive use of reliever medications in asthma: a 16-year population-based study. BMC Pulm Med. 2018;18(1):33.

    PubMed  PubMed Central  Google Scholar 

  44. Bender BG, Cvietusa PJ, Goodrich GK, Lowe R, Nuanes HA, Rand C, et al. Pragmatic trial of health care technologies to improve adherence to pediatric asthma treatment: a randomized clinical trial. JAMA Pediatr. 2015;169(4):317–23.

    PubMed  Google Scholar 

  45. Apter AJ. Understanding adherence requires pragmatic trials: lessons from pediatric asthma. JAMA Pediatr. 2015;169(4):310–1.

    PubMed  Google Scholar 

  46. Papanicolas I, Woskie LR, Jha AK. Health care spending in the United States and other high-income countries. JAMA. 2018;319(10):1024–39.

    PubMed  Google Scholar 

  47. Joynt KE, Gawande AA, Orav EJ, Jha AK. Contribution of preventable acute care spending to total spending for high-cost Medicare patients. JAMA. 2013;309(24):2572–8.

    PubMed  Google Scholar 

  48. Mitchell JM, Patterson JA. The inclusion of economic endpoints as outcomes in clinical trials reported to clinicaltrials.gov. J Manag Care Spec Pharm. 2020;26(4):386–93.

    PubMed  Google Scholar 

  49. AMCP format for formulary submissions. Guidance on submission of pre-approval and post-approval clinical and economic information and evidence. https://www.amcp.org/Resource-Center/format-formulary-submissions/AMCP-Format-for-Formulary-Submissions-4.1. Accessed 7 May 2020.

  50. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418–28.

    CAS  PubMed  Google Scholar 

  51. Centers for Medicare and Medicaid Services. Hospital Readmissions Reduction Program (HRRP). 2019. https://www.cms.gov/medicare/medicare-fee-for-service-payment/acuteinpatientpps/readmissions-reduction-program.html. Accessed 9 June 2020.

  52. Jacobs DM, Noyes K, Zhao J, Gibson W, Murphy TF, Sethi S, et al. Early hospital readmissions after an acute exacerbation of chronic obstructive pulmonary disease in the nationwide readmissions database. Ann Am Thorac Soc. 2018;15(7):837–45.

    PubMed  PubMed Central  Google Scholar 

  53. Elixhauser A, Au DH, Podulka J. Readmissions for Chronic Obstructive Pulmonary Disease, 2008: Statistical Brief #121. In: Healthcare cost and utilization project (HCUP) statistical briefs. Rockville: Agency for Healthcare Research and Quality (US); 2006. https://www.ncbi.nlm.nih.gov/books/NBK65392/. Accessed 9 Oct 2019.

  54. Ohar JA, Loh CH, Lenoir KM, Wells BJ, Peters SP. A comprehensive care plan that reduces readmissions after acute exacerbations of COPD. Respir Med. 2018;141:20–5.

    CAS  PubMed  Google Scholar 

  55. Eldessouki R, Dix SM. Health care system information sharing: a step toward better health globally. Value Health Reg Issues. 2012;1(1):118–20.

    PubMed  Google Scholar 

  56. The International Society of Pharmacoeconomics and Outcomes Research (ISPOR). Pharmacoeconomic guidelines around the world. 2019. https://tools.ispor.org/peguidelines/ Accessed 7 Aug 2019.

  57. Iyengar S, Tay-Teo K, Vogler S, Beyer P, Wiktor S, de Joncheere K, et al. Prices, costs, and affordability of new medicines for hepatitis C in 30 countries: an economic analysis. PLoS Med. 2016;13(5):e1002032.

    PubMed  PubMed Central  Google Scholar 

  58. Geenen JW, Boersma C, Klungel OH, Hövels AM. Accuracy of budget impact estimations and impact on patient access: a hepatitis C case study. Eur J Health Econ. 2019;20(6):857–67.

    PubMed  PubMed Central  Google Scholar 

  59. Bilinski A, Neumann P, Cohen J, Thorat T, McDaniel K, Salomon JA. When cost-effective interventions are unaffordable: integrating cost-effectiveness and budget impact in priority setting for global health programs. PLoS Med. 2017;14(10):e1002397.

    PubMed  PubMed Central  Google Scholar 

  60. Snider JT, Sussell J, Tebeka MG, Gonzalez A, Cohen JT, Neumann P. Challenges with forecasting budget impact: a case study of six ICER reports. Value Health. 2019;22(3):332–9.

    PubMed  Google Scholar 

  61. Grewal S, Ramsey S, Balu S, Carlson JJ. Cost-savings for biosimilars in the United States: a theoretical framework and budget impact case study application using filgrastim. Expert Rev Pharmacoecon Outcomes Res. 2018;18(4):447–54.

    PubMed  Google Scholar 

  62. Mauskopf J, Earnshaw S. A methodological review of US budget-impact models for new drugs. Pharmacoeconomics. 2016;34(11):1111–31.

    PubMed  Google Scholar 

  63. Jain R, Fu AC, Lim J, Wang C, Elder J, Sander SD, et al. Health care resource utilization and costs among newly diagnosed and oral anticoagulant-naive nonvalvular atrial fibrillation patients treated with dabigatran or warfarin in the United States. J Manag Care Spec Pharm. 2018;24(1):73–82.

    PubMed  Google Scholar 

  64. Peng S, Deger KA, Ustyugova A, Gandhi P, Qiao N, Wang C, et al. Cost-effectiveness analysis of dabigatran versus rivaroxaban for stroke prevention in patients with non-valvular atrial fibrillation using real-world evidence in elderly US Medicare beneficiaries. Curr Med Res Opin. 2018;34(1):55–63.

    PubMed  Google Scholar 

  65. Vodicka E, Kim K, Devine EB, Gnanasakthy A, Scoggins JF, Patrick DL. Inclusion of patient-reported outcome measures in registered clinical trials: evidence from ClinicalTrials.gov (2007–2013). Contemp Clin Trials. 2015;43:1–9.

    CAS  PubMed  Google Scholar 

  66. Guidance for industry. Patient-reported outcome measures: use in medical product development to support labeling claims. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims. Accessed 7 May 2020.

  67. Perfetto EM, Burke L, Oehrlein EM, Epstein RS. Patient-focused drug development: a new direction for collaboration. Med Care. 2015;53(1):9–17.

    PubMed  Google Scholar 

  68. Williet N, Sandborn WJ, Peyrin-Biroulet L. Patient-reported outcomes as primary end points in clinical trials of inflammatory bowel disease. Clin Gastroenterol Hepatol. 2014;12(8):1246–56.e6.

    PubMed  Google Scholar 

  69. Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA. 2016;316(10):1093–103.

    PubMed  Google Scholar 

  70. Lensberg BR, Drummond MF, Danchenko N, Despiégel N, François C. Challenges in measuring and valuing productivity costs, and their relevance in mood disorders. Clinicoecon Outcomes Res. 2013;5:565–73.

    PubMed  PubMed Central  Google Scholar 

  71. Stephenson JJ, Shinde MU, Kwong WJ, Fu AC, Tan H, Weintraub WS. Comparison of claims vs patient-reported adherence measures and associated outcomes among patients with nonvalvular atrial fibrillation using oral anticoagulant therapy. Patient Prefer Adherence. 2018;12:105–17.

    PubMed  PubMed Central  Google Scholar 

  72. Battat R, Kopylov U, Bessissow T, Bitton A, Cohen A, Jain A, et al. Association between ustekinumab trough concentrations and clinical, biomarker, and endoscopic outcomes in patients with Crohn’s disease. Clin Gastroenterol Hepatol. 2017;15(9):1427–34.e2.

    CAS  PubMed  Google Scholar 

  73. Sergeant JC, Hyrich KL, Anderson J, Kopec-Harding K, Hope HF, Symmons DPM, et al. Prediction of primary non-response to methotrexate therapy using demographic, clinical and psychosocial variables: results from the UK Rheumatoid Arthritis Medication Study (RAMS). Arthr Res Ther. 2018;20(1):147.

    Google Scholar 

  74. Ibfelt EH, Jensen DV, Hetland ML. The Danish nationwide clinical register for patients with rheumatoid arthritis: DANBIO. Clin Epidemiol. 2016;8:737.

    PubMed  PubMed Central  Google Scholar 

  75. Dream. 2019. https://www.dreamregistry.nl/nl/. Accessed 12 June 2019.

  76. ISAR. International Severe Asthma Registry. https://isaregistries.org/. Accessed 9 June 2020.

  77. Plant D, Barton A. Adding value to real-world data: the role of biomarkers. Rheumatology (Oxford). 2020;59(1):31–8.

    CAS  PubMed  Google Scholar 

  78. Massey J, Plant D, Hyrich K, Morgan AW, Wilson AG, Spiliopoulou A, et al. Genome-wide association study of response to tumour necrosis factor inhibitor therapy in rheumatoid arthritis. Pharmacogenom J. 2018;18(5):657.

    CAS  Google Scholar 

  79. Plant D, Maciejewski M, Smith S, Nair N, Hyrich K, Ziemek D, et al. Profiling of gene expression biomarkers as a classifier of methotrexate nonresponse in patients with rheumatoid arthritis. Arthr Rheumatol (Hoboken, NJ). 2019;71(5):678–84.

    CAS  Google Scholar 

  80. Albers FC, Müllerová H, Gunsoy NB, Shin J-Y, Nelsen LM, Bradford ES, et al. Biologic treatment eligibility for real-world patients with severe asthma: the IDEAL study. J Asthma. 2018;55(2):152–60.

    CAS  PubMed  Google Scholar 

  81. Hurwitz JT, Brown M, Graff JS, Peters L, Malone DC. Is real-world evidence used in P&T monographs and therapeutic class reviews? J Manag Care Spec Pharm. 2017;23(6):613–20.

    PubMed  Google Scholar 

  82. Bracken N. Reducing readmissions in COPD patients. Am Nurse Today. 2016;11(7):24–9.

    Google Scholar 

  83. Press VG, Au DH, Bourbeau J, Dransfield MT, Gershon AS, Krishnan JA, et al. Reducing chronic obstructive pulmonary disease hospital readmissions. An Official American Thoracic Society Workshop Report. Ann Am Thorac Soc. 2019;16(2):161–70.

    PubMed  PubMed Central  Google Scholar 

  84. Feemster LC, Au DH. Penalizing hospitals for chronic obstructive pulmonary disease readmissions. Am J Respir Crit Care Med. 2014;189(6):634–9.

    PubMed  PubMed Central  Google Scholar 

  85. Franssen FME, Rochester CL. Comorbidities in patients with COPD and pulmonary rehabilitation: do they matter? Eur Respir Rev. 2014;23(131):131.

    PubMed  Google Scholar 

  86. Berger ML, Martin BC, Husereau D, Worley K, Allen JD, Yang W, et al. A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force Report. Value Health. 2014;17(2):143–56.

    PubMed  PubMed Central  Google Scholar 

  87. Dreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J Manag Care Spec Pharm. 2016;22(10):1107–13.

    PubMed  Google Scholar 

  88. Pearson SD, Dreitlein WB, Towse A, Hampson G, Henshall C. A framework to guide the optimal development and use of real-world evidence for drug coverage and formulary decisions. J Comp Eff Res. 2018;7(12):1145–52.

    PubMed  Google Scholar 

Download references

Acknowledgements

Writing, editorial support, and formatting assistance was provided by Michelle Rebello, PhD, CMPP, Saurabh Gagangras, PhD, Vidula Bhole, MD, MHSc, and Maribeth Bogush, PhD, of Cactus Life Sciences (part of Cactus Communications), which was contracted and compensated by Boehringer Ingelheim Pharmaceuticals, Inc. (BIPI) for these services. BIPI was given the opportunity to review the manuscript for medical and scientific accuracy as well as intellectual property considerations.

Author information

Authors and Affiliations

Authors

Contributions

Both authors contributed to the conception, preparation, literature search, and analysis, commented on all versions, and critically revised the manuscript. Both authors read and approved the final manuscript. In summary, the authors meet the criteria for authorship as recommended by the International Committee of Medical Journal Editors. The authors received no direct compensation related to the development of this manuscript.

Corresponding author

Correspondence to Melissa H. Roberts.

Ethics declarations

Funding

The writing support was funded by Boehringer Ingelheim Pharmaceuticals, Inc. (BIPI).

Conflict of interest

MHR did not receive funding for this manuscript; however, outside of this manuscript, she has received grant funding from GlaxoSmithKline, AstraZeneca, Boehringer Ingelheim, Endo Pharmaceuticals, Sunovion, and Pfizer. GTF did not receive funding for this manuscript; however, outside of this manuscript, he has received grant funding from Boehringer Ingelheim, GlaxoSmithKline, AstraZeneca, Novartis, Theravance, Sunovion, Verona, Altavant, Sanofi, and Chiesi; consulting/advisory board fees from Boehringer Ingelheim, GlaxoSmithKline, AstraZeneca, Novartis, Sunovion, Innoviva, Mylan, Circassia, Theravance, Verona, and Sanofi; and speaker’s fee from Boehringer Ingelheim, GlaxoSmithKline, AstraZeneca, Theravance, and Sunovion.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Roberts, M.H., Ferguson, G.T. Real-World Evidence: Bridging Gaps in Evidence to Guide Payer Decisions. PharmacoEconomics Open 5, 3–11 (2021). https://doi.org/10.1007/s41669-020-00221-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41669-020-00221-y