INTRODUCTION

Despite the seriousness of the nation’s current economic woes, prominent health policy makers and Obama administration officials see rising health-care costs as the key long-term fiscal challenge to the US.1,2 Furthermore, many health service researchers and policy analysts assert that if medical practice were more evidence-based, substantial savings could be achieved without loss of access or quality of care.3,4 Accordingly, better information on comparative clinical effectiveness is seen to be a critical element of the solution to the national health-care cost crisis,5–7 and the recently enacted American Recovery and Reinvestment Act of 2009 committed $1.1 billion over 2 years to expand these efforts.8 Since the marketplace does not produce sufficient quantities of high-quality clinical effectiveness research,6,9 a broad range of stake holders agree on the need for public intervention to provide more and better information on comparative clinical effectiveness.3,4,6,7,10–12,13 Underneath this broad consensus, however, debate rages on what kind of CER should be promoted, who should fund it, and how it should be conducted. The resolution of this debate has significant implications for both the quality and quantity of future evidence guiding clinical practice. This article will summarize key elements of the current debate, focusing on the policy issues of greatest importance to internists seeking better evidence to inform decisions at the bedside.

WHAT IS COMPARATIVE EFFECTIVENESS RESEARCH?

Policy makers have defined comparative effectiveness research (CER) as “a rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients. Such a study may compare similar treatments, such as competing drugs, or it may analyze very different approaches, such as surgery and drug therapy. The analysis may focus only on the relative medical benefits and risks of each option, or it may also weigh both the costs and the benefits of those options.”9 The focus of CER is effectiveness under average conditions in diverse populations and clinical practice settings, in contrast to traditional clinical trials investigating intervention efficacy under ideal conditions.

Comparative effectiveness research will necessarily employ a broad range of research methodologies. These include not only randomized, controlled clinical trials, but also systematic reviews of existing research, practical clinical trials, medical registries, “coverage with evidence development” projects by payers, and observational studies using large data sets (such as electronic health records from HMO networks).7,9,14–16 These raise a variety of methodological challenges as well as controversies regarding the standard of evidence needed to promote, or restrict, use of a clinical service.14,16–20,20–24

Beyond these substantive methodological issues, there is considerable policy debate regarding other aspects of the potential scope for comparative effectiveness research. Most advocates insist that CER should have an emphasis on developing information useful from the health-care consumer’s point of view (as opposed to the perspective of the payer or the investigator).25–27 Thus, CER should be informative about the outcomes that patients most care about, such as ability to function and the quality (not just length) of life. Advocates are also concerned that CER provides more information than simply the perspective of the “average” patient; accordingly, many consumer advocacy groups urge that CER explicitly address the specific needs of various sub-populations (e.g., women, children, ethnic minorities).25–28

Perhaps no issue in the CER debate is more controversial than the role of assessment of the comparative “value” or cost of clinical services. This subject was explored in depth in a recent series of Annals articles,29–31 and anxieties of manufacturers and some patient advocacy groups have been highlighted in the recent editorializing over the comparative effectiveness research provisions in the American Recovery and Reinvestment Act of 2009.32–35 In brief, opponents of CER view the inclusion of cost or value within the context of comparative effectiveness research as equivalent to payer-oriented cost-effectiveness analysis (CEA). Such CEA is assumed a precursor to nationwide coverage determinations and rationing of access to expensive but effective therapies. Advocates for patients with rare conditions, and for disease groups with particularly expensive treatments, are of course particularly alert to these concerns, as are the developers of expensive innovations.36

This fear of centralized national coverage determinations has even complicated discussion of the nature of reports to be developed by publically supported CER. Some advocates would prefer that any publically funded CER effort should provide only summaries of studies, rather than publish recommendations (such as the United States Preventive Services Task Force does) or clinical practice guidelines.11,25,28,37

WHO SHOULD PAY FOR CER?

Clinical research is time-consuming and costly, thereby contributing to its underproduction by existing private interests.6,9 Manufacturers are not required by the Food and Drug Administration to demonstrate comparative effectiveness of new drugs or devices, and physicians routinely introduce new practices and procedures without strong evidence of relative clinical effectiveness. Since robust CER studies could result in a loss in market share, manufacturers and providers are not motivated to invest in the full range of clinical effectiveness information of greatest benefit to patients and health professionals. Furthermore, health-care purchasers like health plans and large employers cannot justify substantial corporate investments in CER. To adequately inform the practice community, the findings of CER must be disseminated in an open and transparent fashion. Therefore, individual health-care purchasers cannot gain a unique financial advantage from investments in CER.6,9 Similarly, it is difficult to envision a “Consumers Report” type of business model using patient subscriptions or per-use charges to finance CER.

Public financing of CER has its own difficulties. With over 16% of gross domestic product (GDP) devoted to health care, the research on comparative clinical effectiveness involves “high stakes,” resulting in “winners”—providers or manufacturers whose health care services are found to be superior—as well as “losers” whose services are of lesser value, or even harmful. In a future where health-care decisions are strongly guided by comparative effectiveness research, studies will have enormous financial and other consequences for important health-care stakeholders. Consider the implications of a negative study on a manufacturers’ key source of revenue (e.g., erythrocyte-stimulating agents for Amgen) or on a clinical dominant service line (e.g., angioplasty for interventional cardiologists; hip and knee surgery rehabilitation for inpatient rehabilitation facilities).

Not surprisingly, the typical annual Congressional appropriations dedicated to CER have been miniscule; support for the only explicit federal CER program, the “Effective Health Care Program” of the Agency for Healthcare Research and Quality (AHRQ), was only $30 million for 2008 compared to over $29 billion for the National Institutes of Health (NIH).9,11,16,25 Despite the one-time infusion of CER funds to the National Institutes of Health (NIH), AHRQ, and the Department of Health and Human Services (HHS) under the 2009 Recovery Act, policy makers are skeptical regarding the feasibility of the annual Congressional appropriation ever being a viable source for substantial, sustained investments in CER.38 The appropriations process, a yearly political exercise weighing the allocation of government funds to activities as diverse as space exploration, cancer cures, road construction, veterans care, and Head Start, is fraught with risks for political intrusion into the selection, production, or dissemination of CER.

The political backlash directed through the appropriations process at the Agency for Health Care Policy and Research (AHCPR) is a case study of these risks. Established in 1989, AHCPR had statutory responsibilities for such CER-type activities as outcomes research and practice guideline development. It received substantially increased funding over its predecessor, the National Center for Health Services Research, to initiate major initiatives like Patient Outcomes Research Teams. These multidisciplinary centers focused on particular medical problems, reviewed and synthesized available research, analyzed practice variations and patient outcomes, and evaluated the effects of disseminating their findings. By 1995, PORT back pain research and related back pain management guidelines had questioned the value of back surgery in many clinical circumstances as well as the growing use of new surgical appliances like the pedicle screw. The hostility of back surgeons and device manufacturers to this work led to calls to eliminate the Agency as an unwarranted intrusion into the physician-patient relationship. Signaling this displeasure, in June of 1995 the House/Senate budget committee conference report called for elimination of AHCPR. Subsequent negotiations in the latter stages of the appropriations process prevented complete de-funding of AHCPR. Nonetheless, as a result of this controversy, the Agency suffered a severe budget reduction, with attendant loss of programs in outcomes research and clinical guidelines development, and, in 1999, a name change (to AHRQ).39,40

This example is the best known of a variety of failed efforts in the US to undertake publically supported comparative assessments of health-care services. Informed by these experiences, CER policy makers have sought to secure the advantages of public financing for CER without the complications of annual Congressional appropriations. This requires the establishment of a dedicated trust fund, or some other mechanism of mandatory recurring funding, such as a tax on private health insurance, a new payroll tax, or a mandatory tap on the general federal revenue. While these approaches remove the public financing of comparative effectiveness research from the annual appropriations process, they introduce new challenges regarding how these funds should be managed.6,7,9,41

HOW SHOULD CER INVESTMENTS BE MANAGED?

Because of the high stakes nature of CER, as well as recent examples of “pressure” placed on Congress and/or Federal agencies regarding the interpretation and use of clinical evidence,39,40,42,43 policy makers have proposed a variety of governance options, weighing the risks of political intrusion into the selection, creation, and dissemination of comparative effectiveness information. The CER process will have a variety of steps, each of which will be of great interest to one or more stakeholders. These include the selection of priority topics for research (which drugs, procedures, and/or devices should be scrutinized first), the framing of study questions (e.g., study drug effects on blood pressure control or on quality-adjusted life-years), the choice of research method (e.g., commission a systematic review or a practical clinical trial), the phrasing of study conclusions (e.g., “trend toward benefit for glucose control” vs “unlikely to have any meaningful impact on patient quality of life”), and the dissemination of findings.6,7,9,16 Special interests will be tempted to exert influence on all these processes, and various executive branch officials could provide a point of contact to steer the results in the direction preferred by a key stake holder. And of course Congress can express pleasure or displeasure with study findings through various investigative or legislative actions.

In considering options for governing the conduct of comparative effectiveness research, policy makers have identified a number of important principles, exemplified in the Institute of Medicine (IOM) report Knowing What Works in Health Care.11 These include accountability, consistency, efficiency, feasibility, objectivity, responsiveness, scientific rigor, and transparency. While each of these principles is relevant to CER, they often represent competing, albeit important, priorities. For example, responsiveness might compete with transparency or efficiency (as defined by avoiding unnecessary duplication of resources), and scientific rigor almost always competes with feasibility and ability to respond quickly. A more complex challenge addressed in the IOM report is that of coordination of efforts in order to ensure the authoritativeness of comparative effectiveness research studies. In the US, competing or conflicting clinical research findings may give providers broad discretion to make “preference-sensitive” or “supply-sensitive” medical decisions.44,45 Accordingly these conflicting lines of clinical evidence may contribute substantially to wasteful variations in health-care expenditures and to inflationary increases in health-care spending. In light of these issues, policy experts have proposed several organizational options to manage this high-stakes scientific work.6,9,16,41,46

One proposed approach to governing publically funded CER is a Federally-funded Research and Development Center (FFRDC).46 FFRDCs are hybrid organizations47 that came to prominence at the close of World War II through national defense and nuclear energy laboratories (e.g., Oak Ridge, Los Alamos). They provide executive branch departments with the option of establishing a government funded entity that is operated by a non-federal organization unfettered by civil service rules or other federal management laws. The theoretical advantages of an FFRDC for CER include close government oversight combined with some freedom from regulation. Transparency and accountability may suffer, however, since FFRDCs are not subject to laws governing the establishment and management of advisory committees nor to federal employee regulations. Furthermore, it is unclear how an FFRDC could achieve the IOM’s goal of coordinated activities across agencies, having neither authority over executive branch employees nor the ability to readily transfer funds to other government agencies with relevant CER capabilities [e.g., AHRQ, NIH, Veterans Administration (VA), Centers for Disease Control and Prevention (CDC)]. Furthermore, since the FFRDC contract must be periodically renewed, executive branch officials will have ample opportunity to express displeasure and exert influence over CER processes. Congress can also use its oversight authority to challenge findings as well as exert annual appropriations pressure on the federal agency parent of the FFRDC. Thus, the integrity and clinical utility of the findings of an FFRDC for CER could be highly dependent on the agency, the FFRDC contractor, and the advisory board established to manage it. In the 110th Congress, proposals in both the House48 and the Senate49 have referred to an FFRDC model as a potential strategy to address the need for enhancing CER.

Another proposed approach to managing CER is a Congressionally chartered non-profit organization,47,50 such as the National Academy of Sciences with its Institute of Medicine. A Congressionally chartered “CER institute” could conduct timely and responsive projects on behalf of a governing board and could ensure consistency in developing and applying standards to CER. Since such an institute would exist outside the executive branch, however, it would have no authority to coordinate with existing federal CER efforts or infrastructure, nor (at least under current law) to contract with federal agencies to conduct CER projects on its behalf. Accordingly, the new institute likely would need to duplicate at least some, and perhaps much, of the functions and infrastructure of existing CER-related federal research agencies to achieve its mission, hardly the most efficient strategy. While achieving a high degree of political insulation, this approach also achieves the greatest degree of autonomy. Congressionally chartered corporations are not subject to federal advisory commission rules or other federal regulations (unless these are explicitly addressed in enabling legislation), so the institute could exert considerable latitude in how it chose to define and implement key IOM principles like accountability, objectivity, scientific rigor, and transparency. Accordingly, the quality and clinical implications of its CER reports would be highly dependent on the governing board for the institute, the staff it employed, and the research contractors it selected. Once such a CER institute were established, it would not be easy to correct deficiencies; any change in the charter or in the funding would require a law to be passed by both the House and the Senate and signed by the President. This barrier to interference gives the institute strong protection from the political process; given the US history of past political interference in CER efforts and the increasingly high stakes involved in the production of evidence, this approach has its strong proponents. The “S. 3408: Comparative Effectiveness Research Act of 2008” introduced by Senators Baucus and Conrad has such a structure, establishing a Health Care Comparative Effectiveness Research Institute with a 21-member Board of Governors.51

The most conventional approach to a public CER effort is that proposed in the IOM report Knowing What Works in Health Care: “Congress should direct the secretary of the US Department of Health and Human Services to designate a single entity (the Program) with authority, overarching responsibility, sustained resources, and adequate capacity to ensure production of credible, unbiased information about what is known and not known about clinical effectiveness.” And “The secretary of Health and Human Services should appoint a Clinical Effectiveness Advisory Board to oversee the Program.”11 Legislatively, this approach is similar to the House-passed version of comparative effectiveness research, section 904 of the Children’s Health and Medicare Protection Act.9 This approach may achieve a number of the IOM’s proposed CER principles. Accountability is achieved through a federal agency (e.g., AHRQ) managing the CER portfolio with the oversight of an advisory board. HHS could achieve efficiencies by using extant infrastructure at AHRQ52,53 (e.g., Evidence-based Practice Centers, Centers for Education and Research on Therapeutics, etc.) and at the NIH (e.g., development and implementation of large clinical trials) to answer priority research questions. Transparency would be enhanced through the Federal Advisory Committee Act (FACA), while the management of the research process by federal agencies contracting with university researchers ensures public accountability. Nonetheless, because of the Constitution's “separation of powers,” a non-political advisory board outside the President’s control can merely advise, not direct the actions of a federal research agency; executive branch leadership retains ultimate control over the agency’s responsiveness to recommendations, timeliness of projects, adherence to standards of evidence, and coordination and consistency of CER reports. Congress can manage this risk, however, through close oversight and legislative intervention if advisory board recommendations were ignored.

CONCLUSION

For public CER efforts to be successful, there must be substantial and sustained federal investment. In an era of skyrocketing federal deficits, severe economic recession, and demands for increased public investments in traditional biomedical research, achieving such financing will be a challenge. Even if the CER financing and governance problems are satisfactorily resolved, other controversial policy issues remain, including the range of stake holder perspectives and expertise to include in the governing board; the processes to assure proper public input into CER projects and publications; the development of findings and implications (i.e., recommendations, guidelines, etc.); and the role for “value” (e.g., cost, quality of life, and cost effectiveness) in evaluating and reporting comparative clinical effectiveness. Of course, for any new public investment in CER to realize a return to the taxpayers, the new information must actually be put to use by payers, providers, and patients to increase the use of highly effective treatments and minimize the use of ineffective or inefficient ones.4,9

In considering solutions to other health policy dilemmas, leaders in the American College of Physicians (ACP) and elsewhere have recently looked across our borders to find useful models abroad.54 There are several vibrant and robust examples of policy and practice-relevant CER in other countries,11,55 but while these models are encouraging, they likely provide insufficient guidance on US implementation. The reasons involve not just differences in law and polity, but more substantively, the much larger role health care plays across many sectors of the US economy. Continued health care expenditure growth and technology dissemination are built into the business plans of many prominent corporations and institutions. Therefore, the stakes for CER may be much higher for more, and more prominent, advocates in the US than in many other industrialized countries, and the challenges to manage the process thereby more complex.

Internal medicine, with its long history of commitment to scientific medical practice and its leadership in evidence–based medicine, has an enormous stake in the outcome of this debate. We cannot countenance, nor can the nation afford, another federal effort in CER to be paralyzed by a twenty-first century version of AHCPR’s back pain malady, nor can we allow an unaccountable and opaque “CER Institute” to use private contractors to serve special interests; such an outcome could also be harmful, risking misinformation to patients and clinicians while wasting money on redundant projects and bureaucracy. To solve our health-care problems, the US would be well served by reestablishing the appropriate function of federal health research agencies in public concerns. Given the broad range of interests in CER, it is appropriate, even necessary, for an independent, multi-stakeholder process (with a prominent role for clinicians and consumers) to identify the clinical questions that the American people most need answering. A dedicated trust fund immune from political influence should support this essential work. And then the substantial infrastructure already developed through past investments in AHRQ, NIH, VA, CDC, etc., must be put to effective use. These resources will be critical both in answering the clinical effectiveness questions confronted by patients and their physicians, as well as solving the implementation science and knowledge translation problems familiar to administrators and policymakers.