Implementation Research: Beyond the Traditional Randomized Controlled Trial

  • Amanda H. Salanitro
  • Carlos A. Estrada
  • Jeroan J. Allison


Implementation research is a new scientific discipline emerging from the recognition that the public does not derive sufficient or rapid benefit from advances in the health sciences. One often-quoted estimate claims that it takes an average of 17 years for even well-established clinical knowledge to be fully adopted into routine practice. In this chapter, we will discuss particular barriers to evidence implementation, present tools for implementation research, and provide a framework for designing implementation research studies, emphasizing the randomized trial. The reader is advised that this chapter only provides a basic introduction to several concepts for which new approaches are rapidly emerging. Therefore, our goal is to stimulate interest and promote additional in-depth learning for those who wish to develop new implementation research projects or better understand this exciting field.


Continue Medical Education Implementation Research Computerize Provider Order Entry Lead Physician Cluster RCTs 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Berwick DM. Disseminating innovations in health care. JAMA 2003; 289:1969–75.PubMedCrossRefGoogle Scholar
  2. 2.
    Lenfant C. Shattuck lecture–clinical research to clinical practice–lost in translation? N Engl J Med 2003; 349:868–74.PubMedCrossRefGoogle Scholar
  3. 3.
    Kiefe CI, Sales A. A state-of-the-art conference on implementing evidence in health care. Reasons and recommendations. J Gen Intern Med 2006; 21 Suppl 2:S67–70.PubMedGoogle Scholar
  4. 4.
    McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003; 348:2635–45.PubMedCrossRefGoogle Scholar
  5. 5.
    Rubenstein LV, Pugh J. Strategies for promoting organizational and practice change by advancing implementation research. J Gen Intern Med 2006; 21 Suppl 2:S58–64.PubMedCrossRefGoogle Scholar
  6. 6.
    Kiefe CI, Safford M, Allison JJ. Forces influencing the care of complex patients: a framework. In: Academy Health Annual Meeting, 2007. Orlando, Fl; 2007.Google Scholar
  7. 7.
    Unequal treatment: confronting racial and ethnic disparities in health care. Washington, DC: National Academies Press; 2003.Google Scholar
  8. 8.
    Allison JJ. Health disparity: causes, consequences, and change. Med Care Res Rev 2007; 64:5S–6S.PubMedCrossRefGoogle Scholar
  9. 9.
    Chin MH, Walters AE, Cook SC, Huang ES. Interventions to reduce racial and ethnic disparities in health care. Med Care Res Rev 2007; 64:7S–28S.PubMedCrossRefGoogle Scholar
  10. 10.
    Kilbourne AM, Switzer G, Hyman K, Crowley-Matoka M, Fine MJ. Advancing health disparities research within the health care system: a conceptual framework. Am J Public Health 2006; 96:2113–21.PubMedCrossRefGoogle Scholar
  11. 11.
    Smedley BD, Stith AY, Nelson AR. Unequal treatment: confronting racial and ethnic disparities in health care. Washington, DC: Institute of Medicine; 2003.Google Scholar
  12. 12.
    Flaskerud JH, Nyamathi AM. Attaining gender and ethnic diversity in health intervention research: cultural responsiveness versus resource provision. ANS Adv Nurs Sci 2000; 22:1–15.PubMedGoogle Scholar
  13. 13.
    Hayward RA, Asch SM, Hogan MM, Hofer TP, Kerr EA. Sins of omission: getting too little medical care may be the greatest threat to patient safety. J Gen Intern Med 2005; 20:686–91.PubMedCrossRefGoogle Scholar
  14. 14.
    Kohn LT, Corrigan JM, Donaldson MS. To err is human: building a safer health system. Washington, DC: Institute of Medicine; 1999.Google Scholar
  15. 15.
    Public Health Action Plan to Combat Antimicrobial Resistance Centers for Disease Control and Prevention, 1999. (Accessed November 2007, at
  16. 16.
    Snow V, Mottur-Pilson C, Gonzales R. Principles of appropriate antibiotic use for treatment of acute bronchitis in adults. Ann Intern Med 2001; 134:518–20.PubMedGoogle Scholar
  17. 17.
    Wenzel RP, Fowler AA, 3rd. Clinical practice. Acute bronchitis. N Engl J Med 2006; 355:2125–30.PubMedCrossRefGoogle Scholar
  18. 18.
    Roumie CL, Halasa NB, Grijalva CG, et al. Trends in antibiotic prescribing for adults in the United States–1995 to 2002. J Gen Intern Med 2005; 20:697–702.PubMedCrossRefGoogle Scholar
  19. 19.
    Metlay JP, Camargo CA, Jr., MacKenzie T, et al. Cluster-randomized trial to improve antibiotic use for adults with acute respiratory infections treated in emergency departments. Ann Emerg Med 2007; 50:221–30.PubMedCrossRefGoogle Scholar
  20. 20.
    Crossing the quality chasm: a new health system for the 21st century. Washington, DC: Institute of Medicine; 2001.Google Scholar
  21. 21.
    Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100, 000 lives campaign: setting a goal and a deadline for improving health care quality. JAMA 2006; 295:324–7.PubMedCrossRefGoogle Scholar
  22. 22.
    Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med 2007; 357:608–13.PubMedCrossRefGoogle Scholar
  23. 23.
    Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ 2003; 327:1459–61.PubMedCrossRefGoogle Scholar
  24. 24.
    Brennan TA, Gawande A, Thomas E, Studdert D. Accidental deaths, saved lives, and improved quality. N Engl J Med 2005; 353:1405–9.PubMedCrossRefGoogle Scholar
  25. 25.
    Berwick D. The stories beneath. Medical Care 2007; 45:1123–5.PubMedCrossRefGoogle Scholar
  26. 26.
    Bhattacharyya O, Reeves S, Garfinkel S, Zwarenstein M. Designing theoretically-informed implementation interventions: fine in theory, but evidence of effectiveness in practice is needed. Implement Sci 2006;1 Feb 23:5.CrossRefGoogle Scholar
  27. 27.
    Effective health care: getting evidence into practice. National Health Service Center for Reviews and Dissemination, Royal Society of Medicine Press, 1999; 5(1). (Accessed November 2007, at
  28. 28.
    Rogers EM. Diffusion of innovations (5th ed.). New York: Free Press; 2003.Google Scholar
  29. 29.
    Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health 2007; 28:413–33.PubMedCrossRefGoogle Scholar
  30. 30.
    Sung NS, Crowley WF, Jr., Genel M, et al. Central challenges facing the national clinical research enterprise. JAMA 2003; 289:1278–87.PubMedCrossRefGoogle Scholar
  31. 31.
    Nelson EC, Batalden PB, Huber TP, et al. Microsystems in health care: part 1. Learning from high-performing front-line clinical units. Joint Comm J Qual Im 2002; 28:472–93.Google Scholar
  32. 32.
    Zerhouni EA. Medicine. The NIH roadmap. Science 2003; 302:63–72.PubMedCrossRefGoogle Scholar
  33. 33.
    Zerhouni EA. US biomedical research: basic, translational, and clinical sciences. JAMA 2005; 294:1352–8.PubMedCrossRefGoogle Scholar
  34. 34.
    Chao SR. The state of quality improvement and implementation research: expert views–workshop summary. Washington, DC: National Academies Press; 2007.Google Scholar
  35. 35.
    Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff (Millwood) 2005; 24:138–50.CrossRefGoogle Scholar
  36. 36.
    Shojania KG, McDonald KM, Wachter RM, Owens DK. Closing The Quality Gap: A Critical Analysis of Quality Improvement Strategies, Volume 1—Series Overview and Methodology. Technical Review 9. (Contract No. 290–02-0017 to the Stanford University—UCSF Evidence-based Practices Center). AHRQ Publication No. 04–0051-1. Rockville, MD: Agency for Healthcare Research and Quality. August 2004.Google Scholar
  37. 37.
    Williams GC, Deci EL. Activating patients for smoking cessation through physician autonomy support. Med Care 2001; 39:813–23.PubMedCrossRefGoogle Scholar
  38. 38.
    Williams GC, McGregor H, Zeldman A, Freedman ZR, Deci EL, Elder D. Promoting glycemic control through diabetes self-management: evaluating a patient activation intervention. Patient Educ Couns 2005; 56:28–34.PubMedCrossRefGoogle Scholar
  39. 39.
    Turner RC, Waivers LE, O’Brien K. The effect of patient-carried reminder cards on the performance of health maintenance measures. Arch Intern Med 1990; 150:645–7.PubMedCrossRefGoogle Scholar
  40. 40.
    Bodenheimer T, Lorig K, Holman H, Grumbach K. Patient self-management of chronic disease in primary care. JAMA 2002; 288:2469–75.PubMedCrossRefGoogle Scholar
  41. 41.
    Eng E, Parker E, Harlan C. Lay health advisor intervention strategies: a continuum from natural helping to paraprofessional helping. Health Educ Behav 1997; 24:413–7.PubMedCrossRefGoogle Scholar
  42. 42.
    Swider SM. Outcome effectiveness of community health workers: an integrative literature review. Public Health Nurs 2002; 19:11–20.PubMedCrossRefGoogle Scholar
  43. 43.
    Institute of Medicine. Clinical practice guidelines: directions for a new program. Washington, DC: National Academy Press; 1990.Google Scholar
  44. 44.
    Grimshaw J, Eccles M, Thomas R, et al. Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966–1998. J Gen Intern Med 2006; 21 Suppl 2:S14–20.PubMedGoogle Scholar
  45. 45.
    Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282:1458–65.PubMedCrossRefGoogle Scholar
  46. 46.
    Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 2005; 294:716–24.PubMedCrossRefGoogle Scholar
  47. 47.
    Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999; 282:867–74.PubMedCrossRefGoogle Scholar
  48. 48.
    Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995; 274:700–5.PubMedCrossRefGoogle Scholar
  49. 49.
    Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA 2002; 288:1057–60.PubMedCrossRefGoogle Scholar
  50. 50.
    Centor R, Casebeer L, Klapow J. Using a combined CME course to improve physicians’ skills in eliciting patient adherence. Acad Med 1998; 73:609–10.PubMedCrossRefGoogle Scholar
  51. 51.
    Fordis M, King JE, Ballantyne CM, et al. Comparison of the instructional efficacy of Internet-based CME with live interactive CME workshops: a randomized controlled trial. JAMA 2005; 294:1043–51.PubMedCrossRefGoogle Scholar
  52. 52.
    Soumerai SB, Avorn J. Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA 1990; 263:549–56.PubMedCrossRefGoogle Scholar
  53. 53.
    Valente TW, Pumpuang P. Identifying opinion leaders to promote behavior change. Health Educ Behav 2007; 34(6):881–96.PubMedCrossRefGoogle Scholar
  54. 54.
    Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA 2001; 285:2871–9.PubMedCrossRefGoogle Scholar
  55. 55.
    Davidoff F, Batalden P. Toward stronger evidence on quality improvement. Draft publication guidelines: the beginning of a consensus project. Qual Saf Health Care 2005; 14:319–25.PubMedCrossRefGoogle Scholar
  56. 56.
    Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs Health Care System on the quality of care. N Engl J Med 2003; 348:2218–27.PubMedCrossRefGoogle Scholar
  57. 57.
    Payne TH. Computer decision support systems. Chest 2000; 118:47S–52S.PubMedCrossRefGoogle Scholar
  58. 58.
    Walton RT, Harvey E, Dovey S, Freemantle N. Computerised advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev 2001:CD002894.Google Scholar
  59. 59.
    Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 2005; 116:1506–12.PubMedCrossRefGoogle Scholar
  60. 60.
    Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF. High rates of adverse drug events in a highly computerized hospital. Arch Intern Med 2005; 165:1111–6.PubMedCrossRefGoogle Scholar
  61. 61.
    Scalise D. Technology. CPOE: are you really ready? Hosp Health Netw 2006; 80:14, 6.PubMedGoogle Scholar
  62. 62.
    Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007; 14:415–23.PubMedCrossRefGoogle Scholar
  63. 63.
    Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005; 293:1239–44.PubMedCrossRefGoogle Scholar
  64. 64.
    Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery bypass graft report cards. Circulation 2005; 111:1257–63.PubMedCrossRefGoogle Scholar
  65. 65.
    Rosenthal MB, Frank RG, Li Z, Epstein AM. Early experience with pay-for-performance: from concept to practice. JAMA 2005; 294:1788–93.PubMedCrossRefGoogle Scholar
  66. 66.
    Glickman SW, Ou FS, DeLong ER, et al. Pay for performance, quality of care, and outcomes in acute myocardial infarction. JAMA 2007; 297:2373–80.PubMedCrossRefGoogle Scholar
  67. 67.
    Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 356:486–96.PubMedCrossRefGoogle Scholar
  68. 68.
    Murray DM. Design and analysis of group-randomized trials. New York: Oxford University Press; 1998.Google Scholar
  69. 69.
    Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276:637–9.PubMedCrossRefGoogle Scholar
  70. 70.
    Moher D, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001; 285:1987–91.PubMedCrossRefGoogle Scholar
  71. 71.
    Campbell MK, Elbourne DR, Altman DG. CONSORT statement: extension to cluster randomised trials. BMJ 2004; 328:702–8.PubMedCrossRefGoogle Scholar
  72. 72.
    Elbourne DR, Campbell MK. Extending the CONSORT statement to cluster randomized trials: for discussion. Stat Med 2001; 20:489–96.PubMedCrossRefGoogle Scholar
  73. 73.
    Casarett D, Karlawish JH, Sugarman J. Determining when quality improvement initiatives should be considered research: proposed criteria and potential implications. JAMA 2000; 283:2275–80.PubMedCrossRefGoogle Scholar
  74. 74.
    Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA 2000; 283:2701–11.PubMedCrossRefGoogle Scholar
  75. 75.
    Lynn J, Baily MA, Bottrell M, et al. The ethics of using quality improvement methods in health care. Ann Intern Med 2007; 146:666–73.PubMedGoogle Scholar
  76. 76.
    Van den Broeck J, Cunningham SA, Eeckels R, Herbst K. Data cleaning: detecting, diagnosing, and editing data abnormalities. PLoS Med 2005; 2:e267.PubMedCrossRefGoogle Scholar
  77. 77.
    Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication: International Committee of Medical Journal Editors. Available at (last accessed November 2007).
  78. 78.
    Scriven M. Beyond formative and summative evaluation. In: McLaughlin MW, Phillips DC, eds. Evaluation and education: 90th yearbook of the National Society for the Study of Education. Chicago, IL: University of Chicago Press; 1991:18–64.Google Scholar
  79. 79.
    Weston CB, McAlpine L, Bordonaro T. A model for understanding formative evaluation in instructional design. Education Tech Research Dev 1995; 43:29–49.CrossRefGoogle Scholar
  80. 80.
    Delbecq AL, Van de Ven AH, Gustafson DH. Group techniques for program planning: a guide to nominal group and Delphi processes. Glenview, IL: Scott Foresman; 1975.Google Scholar
  81. 81.
    Krueger RA, Casey MA. Focus groups: a practical guide for applied research (3rd ed.). Thousand Oaks, CA: Sage; 2000.Google Scholar
  82. 82.
    Nielsen J, Mack R. Usability inspection methods. New York: Wiley; 1994.Google Scholar
  83. 83.
    Strauss A, Corbin J. Basics of qualitative research: grounded theory, procedures, and techniques. Newbury Park, CA: Sage; 1990.Google Scholar
  84. 84.
    Casebeer LL, Strasser SM, Spettell CM, et al. Designing tailored Web-based instruction to improve practicing physicians’ preventive practices. J Med Internet Res 2003; 5:e20.PubMedCrossRefGoogle Scholar
  85. 85.
    Bootzin RR. The role of expectancy in behavior change. In: White L, Turskey B, Schwartz G, eds. Placebo: theory, research, and mechanisms. New York: Guilford Press; 1985:196–210.Google Scholar
  86. 86.
    Gross D. On the merits of attention-control groups. Res Nurs Health 2005; 28:93–4.PubMedCrossRefGoogle Scholar
  87. 87.
    Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ 2001; 322:355–7.PubMedCrossRefGoogle Scholar
  88. 88.
    Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ 2003; 327:785–9.PubMedCrossRefGoogle Scholar
  89. 89.
    Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health 2004; 94:423–32.PubMedCrossRefGoogle Scholar
  90. 90.
    Lachin JM. Statistical considerations in the intent-to-treat principle. Control Clin Trials 2000; 21:167–89.PubMedCrossRefGoogle Scholar
  91. 91.
    Schulz KF, Grimes DA. Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet 2002; 359:781–5.PubMedCrossRefGoogle Scholar
  92. 92.
    Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol 1986; 51:1173–82.PubMedCrossRefGoogle Scholar
  93. 93.
    Preacher KJ, Hayes AF. SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behav Res Methods Instrum Comput 2004; 36:717–31.PubMedGoogle Scholar
  94. 94.
    Donner A, Klar N. Pitfalls of and controversies in cluster randomization trials. Am J Public Health 2004; 94:416–22.PubMedCrossRefGoogle Scholar
  95. 95.
    Beach ML. Primer on group randomized trials. Eff Clin Pract 2001; 4:42–3.PubMedGoogle Scholar
  96. 96.
    Campbell MK, Fayers PM, Grimshaw JM. Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research. Clin Trials 2005; 2:99–107.PubMedCrossRefGoogle Scholar
  97. 97.
    Sculpher M. Evaluating the cost-effectiveness of interventions designed to increase the utilization of evidence-based guidelines. Fam Pract 2000;17 Suppl 1:S26–31.PubMedCrossRefGoogle Scholar
  98. 98.
    Institute of Medicine. Advancing quality improvement research: challenges and opportunities–workshop summary. Washington, DC: National Academies Press; 2007. (Accessed November 2007, at

Copyright information

© Springer Science + Business Media B.V 2008

Authors and Affiliations

  • Amanda H. Salanitro
    • 1
  • Carlos A. Estrada
    • 1
    • 2
    • 3
  • Jeroan J. Allison
    • 3
    • 4
  1. 1.Veterans’ Administration National Quality Scholars ProgramBirmingham VA Medical CenterBirmingham
  2. 2.University of Alabama at Birmingham
  3. 3.Deep South Center on EffectivenessBirmingham VA Medical CenterBirmingham
  4. 4.University of Alabama at Birmingham

Personalised recommendations