Abstract
“What healthcare treatment works best, for whom, and under what circumstances?” is the central question of comparative effectiveness research (CER). This paper first defines CER, and then briefly discusses its policy origins, the Patient-Centered Outcomes Research Institute (PCORI), and the relationship of CER to other evidence-based medicine approaches. PCORI methodology standards for the conduct of CER are described. Specific evidentiary challenges including the need to include observational data, and assess heterogeneity of treatment effects, are identified. The last section of this paper focuses on how evidence from a systematic review, a common study design in CER, is interpreted for decision-making. Methods for assessing the strength of evidence across a group of studies are described. These assessments are used for decisions at the individual patient level as well as the public health perspective, for example to determine clinical practice guidelines. The challenges inherent in these methods are presented, along with the results from a reliability study. The implications for CER decisions and decision-makers are important to consider.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berkman, N.D., Lohr, K.N., Ansari, M., McDonagh, M., Balk, E., Whitlock, E., Reston, J., Bass, E., Butler, M., Gartlehner, G., Hartling, L., Kane, R., McPheeters, M., Morgan, L., Morton, S.C., Viswanathan, M., Sista P., Chang, S.: Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. Methods Guide for Comparative Effectiveness Reviews (Prepared by the RTI-UNC Evidence-based Practice Center under Contract No. 290-2007-10056-I). AHRQ Publication No. 13(14)-EHC130-EF. Agency for Healthcare Research and Quality, Rockville (2013)
Berkman, N.D., Lohr, K.N., Morgan, L.C., Kuo, T.M., Morton, S.C.: Interrater reliability of grading strength of evidence varies with the complexity of the evidence in systematic reviews. J. Clin. Epidemiol. 66(10), 1105–1117 (2013)
Federal Coordinating Council for Comparative Effectiveness Research: Report to the President and the Congress (2009)
Goodman, S.N.: Quasi-random reflections on randomized controlled trials and comparative effectiveness research. Clin. Trials 9, 22–26 (2012)
Greenfield, S., Sox, H.S.: Comparative effectiveness research: a report from the Institute of Medicine. Ann. Intern. Med. 151(3), 203–205 (2009)
Hill, A.B.: Reflections on the controlled trial. (Heberden Oration, 1965). Ann. Rheum. Dis. 25, 107–113 (1966)
Institute of Medicine (IOM): Initial National Priorities for Comparative Effectiveness Research. National Academies Press, Washington, DC (2009)
Institute of Medicine (IOM): In: Eden, J., Levitt, L., Berg, A., Morton, S.C. (eds.) Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press, Washington, DC (2011)
Luce, B.R., Drummond, M., Jönsson, B., Neumann, P.J., Schwartz, J.S., Siebert, U., Sullivan, S.D.: EBM, HTA, and CER: clearing the confusion. Milbank Q. 88(2), 256–276 (2010)
Messerli, F.H.: Chocolate consumption, cognitive function, and Nobel Laureates. NEJM 367, 1562–1564 (2012)
Mustafa, R.A., Santesso, N., Brozek, J., Akl, E.A., Walter, S.D., Norman, G., Kulasegaram, M., Christensen, R., Guyatt, G.H., Falck-Ytter, Y., Chang, S., Murad, M.H., Vist, G.E., Lasserson, T., Gartlehner, G., Shukla, V., Sun, X., Whittington, C., Post, P.N., Lang, E., Thaler, K., Kunnamo, I., Alenius, H., Meerpohl, J.J., Alba, A.C., Nevis, I.F., Gentles, S., Ethier, M.C., Carrasco-Labra, A., Khatib, R., Nesrallah, G., Kroft, J., Selk, A., Brignardello-Petersen, R., Schünemann, H.J.: The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. J. Clin. Epidemiol. 66(7), 736–742 (2013)
Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee: The PCORI Methodology Report. www.pcori.org/research-we-support/research-methodology-standards (2013). Accessed 22 Jan 2014
Sackett, D.L., Rosenberg, W.M.C., Gray, J.A.M., Haynes, R.B., Richardson, W.S.: Evidence based medicine: what it is and what it isn’t. BMJ 312(7023), 71–72 (1996)
Slutsky, J.R., Clancy, C.M.: AHRQ’s Effective Health Care Program: why comparative effectiveness matters. Am. J. Med. Qual. 24, 67–70 (2009)
Thorpe, K.E., Zwarenstein, M., Oxman, A.D., Trweek, S., Furberg, C.D., Altman, D.G., Tunis, S., Bergel, E., Harvey, I., Magid, D.J., Chalkidou, K.: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J. Clin. Epidemiol. 62, 464–475 (2009)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Morton, S.C. Evidentiary challenges in comparative effectiveness research. Health Serv Outcomes Res Method 14, 159–165 (2014). https://doi.org/10.1007/s10742-014-0119-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10742-014-0119-8