The European Journal of Health Economics

, Volume 8, Issue 2, pp 111–121 | Cite as

Systematic review of economic evaluations and cost analyses of guideline implementation strategies

  • Luke Vale
  • Ruth Thomas
  • Graeme MacLennan
  • Jeremy Grimshaw
Original paper


To appraise the quality of economic studies undertaken as part of evaluations of guideline implementation strategies, we conducted a systematic review of such studies published between 1966 and 1998. Studies were assessed against BMJ economic evaluations guidelines for each stage (guideline development, implementation and treatment). Of 235 studies identified, 63 reported some information on cost. Only 3 studies provided evidence that their guideline was effective and efficient, 38 reported treatment costs only, 12 implementation and treatment costs, 11 implementation costs alone, and 2 guideline development, implementation and treatment costs. No study gave reasonably complete information on costs. Thus, very few satisfactory economic evaluations of guideline implementation strategies have been performed. Current evaluations have numerous methodological defects and rarely consider all relevant costs and benefits. Future evaluations should focus on evaluating the implementation of evidence-based guidelines.


Cost-effectiveness analysis Physician (or health care professional) behaviour Practice guidelines Quality improvement Systematic review 



The work reported in this paper was part of a larger study funded by the UK NHS Research and Development, Health Technology Assessment Programme and would have not have possible without the work of Cynthia Fraser, Craig Ramsay, Paula Whitty, Martin Eccles, Lloyd Matowe, Liz Shirran, Michel Wensing, Rob Dikstra, Cam Donaldson and Allen Hutchison. We would also like to thank Liz Grant for the preparation of the final report, part of which forms the basis for this paper. The Health Services Research Unit and the Health Economics Research Unit are core funded by the Scottish Executive Health Department. The views expressed are those of the authors and not necessarily those of the funding bodies.


  1. 1.
    Institute of Medicine: Guidelines for Clinical Practice: From Development to use. National Academic Press, Washington (1992)Google Scholar
  2. 2.
    Grimshaw, J.M., Russell, I.T.: Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 342, 1317–1322 (1993)CrossRefGoogle Scholar
  3. 3.
    NHS Centre for Reviews and Dissemination and Nuffield Institute for Health. Implementing clinical guidelines: Can guidelines be used to improve clinical practice? Eff. Health Care 1, 1–12 (1994)Google Scholar
  4. 4.
    Oxman, A.D., Thomson, M.A., Davis, D.A., Haynes, R.B.: No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ 153, 1423–1431 (1995)Google Scholar
  5. 5.
    Davis, D.A., Taylor-Vaisey, A.: Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ 157, 408–416 (1997)Google Scholar
  6. 6.
    Worrall, G., Chaulk, P., Freake, D.: The effects of clinical practice guidelines on patient outcomes in primary care: a systematic review. CMAJ 156, 1705–1712 (1997)Google Scholar
  7. 7.
    Wensing, M., Van der Weijden, T., Grol, R.: Implementing guidelines and innovations in general practice: which interventions are effective? Br. J. Gen. Pract. 48, 991–999 (1998)Google Scholar
  8. 8.
    Thomas, L., Cullum, N., McColl, E., Rousseau, N., Soutter, J., et al.: Guidelines in professions allied to medicine. Cochrane Database Syst. Rev. (1) (1999), Art. No.: CD000349. DOI: 10.1002/14651858.CD000349 (1999)Google Scholar
  9. 9.
    Drummond, M., OBrien, B., Stoddart, G., Torrance, G.: Methods for Economic Evaluation of Health Care Programme, 2nd edn. Oxford University Press, New York (1997)Google Scholar
  10. 10.
    McIntosh, E.: Economic evaluation of guideline implementation studies. In: Thorsen, T., Makela, M. (eds.) Changing Professional Practice: Theory and Practice of Clinical Guidelines Implementation. Danish Institute for Health Services Research, Copenhagen (1999)Google Scholar
  11. 11.
    Mason, J., Wood, J., Freemantle, N.: Designing evaluations of interventions to change professional practice. J. Health Serv. Res. Policy 4, 106–111 (1999)Google Scholar
  12. 12.
    Mason, J., Freemantle, N., Nazareth, I., Eccles, M., Haines, A., Drummond, M.: When is it cost-effective to change the behaviour of health professionals. JAMA 286, 2988–2992 (2001)CrossRefGoogle Scholar
  13. 13.
    Grimshaw, J., Thomas, R., MacLennan, G., Fraser, C., Ramsay, C., et al.: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol. Assess. 8(6) (2004)Google Scholar
  14. 14.
    Bero, L.A., Grilli, R., Grimshaw, J.M., Mowatt, G., Oxman, A.D., et al. (eds.): The Cochrane Effective Practice and Organisation of Care Group (EPOC). In: The Cochrane Library, issue 1. Oxford: Update Software. Updated quarterly (2002)Google Scholar
  15. 15.
    Drummond, M., Jefferson, T.: For the BMJ Working Party on guidelines for authors and peer-reviewers of economic submissions to the British Medical Journal Guidelines for authors and peer-reviewers of economic submissions to the British Medical Journal. BMJ 313, 275–283 (1996)Google Scholar
  16. 16.
    Carney, P.A., Dietrich, A.J., Keller, A., Landgraf, J., O’Connor, G.T.: Tools, teamwork, and tenacity: an office system for cancer prevention. J. Fam. Pract. 35, 388–394 (1992)Google Scholar
  17. 17.
    Leviton, L.C., Goldenberg, R.L., Baker, C.S., Schwartz, R.M., Freda, M.C., et al.: Methods to encourage the use of antenatal corticosteroid therapy for fetal maturation: a randomized controlled trial. JAMA 281, 46–52 (1999)CrossRefGoogle Scholar
  18. 18.
    Anonymous: Integrated care for diabetes: clinical, psychosocial and economic evaluation. Diabetes Integrated Care Evaluation Team. BMJ 308, 1208–1212 (1994)Google Scholar
  19. 19.
    Morrissey, J.P., Harris, R.P., Kincade-Norburn, J., McLaughlin, C., Garrett, J.M., et al.: Medicare reimbursement for preventive care. Changes in performance of services, quality of life, and health care costs. Med. Care 33, 315–331 (1995)CrossRefGoogle Scholar
  20. 20.
    Pearce, M.J., Begg, E.J.: Encouraging consensus cost effective drug therapy: five years experience with a hospital drug utilisation review programme. N. Z. Med. J. 110, 92–95 (1997)Google Scholar
  21. 21.
    van Walraven, C., Goel, V., Chan, B.: Effect of population-based interventions on laboratory utilization: a time-series analysis. JAMA 280, 2028–2033 (1998)CrossRefGoogle Scholar
  22. 22.
    Avorn, J., Soumerai, S.B., Taylor, W., Wessels, M.R., Janousek, J., et al.: Reduction of incorrect antibiotic dosing through a structured educational order form. Arch. Intern. Med. 148, 1720–1724 (1988)CrossRefGoogle Scholar
  23. 23.
    Brady, W.J., Hissa, D.C., McConnell, M., Wones, R.G.: Should physicians perform their own quality assurance audits? J. Gen. Intern. Med. 3, 560–565 (1998)CrossRefGoogle Scholar
  24. 24.
    Kouides, R.W., Bennett, N.M., Lewis, B., Cappuccio, J.D., Barker, W.H., et al.: Performance-based physician reimbursement and influenza immunization rates in the elderly. The Primary-Care Physicians of Monroe County. Am. J. Prev. Med. 14, 89–95 (1998)CrossRefGoogle Scholar
  25. 25.
    Dranitsaris, G., Warr, D., Puodziunas, A.: A randomized trial of the effects of pharmacist intervention on the cost of antiemetic therapy with ondansetron. Support. Care Cancer 3, 183–189 (1995)CrossRefGoogle Scholar
  26. 26.
    MacCosbe, P.E., Gartenberg, G.: Modifying empiric antibiotic prescribing: experience with one strategy in a medical residency program. Hosp. Formul. 20, 986–999 (1985)Google Scholar
  27. 27.
    Thomas, R., Grimshaw, J., McClinton, S., McIntosh, E., Mollison, J., et al.: An evaluation of a guideline-based open-access urological investigation service (URGE). Final project report to Chief Scientist Office of Scottish Office Home and Health Department. HSRU, Aberdeen (1998)Google Scholar
  28. 28.
    Barnett, G.O., Winickoff, M.D., Dorsey, M.D., Lurie, R.S.: Quality assurance through automated monitoring and concurrent feedback using a computer-based medical information system. Med. Care 16, 962–970 (1978)CrossRefGoogle Scholar
  29. 29.
    Watson, M.C.: The development, implementation and evaluation of prescribing guidelines in general practice. Ph.D. thesis (University of Bristol) (1998)Google Scholar
  30. 30.
    Katon, W., Von Korff, M., Lin, E., Simon, G., Walker, E., et al.: Collaborative management to achieve depression treatment guidelines. J. Clin. Psychiatry 58, 20–23 (1997)Google Scholar
  31. 31.
    Weingarten, S.R., Riedinger, M.S., Conner, L., Lee, T.H., Hoffman, I., et al.: Practice guidelines and reminders to reduce duration of hospital stay for patients with chest pain. An interventional trial. Ann. Intern. Med. 120, 257–263 (1994)Google Scholar
  32. 32.
    Weingarten, S.R., Riedinger, M.S., Hobson, P., Noah, M.S., Johnson, B., et al.: Evaluation of a pneumonia practice guideline in an interventional trial. Am. J. Respir. Crit. Care Med. 153, 1110–1115 (1996)Google Scholar
  33. 33.
    Gurwitz, J.H., Noonan, J.P., Soumerai, S.B.: Reducing the use of H2-receptor antagonists in the long-term-care setting. J. Am. Geriatr. Soc. 40, 359–364 (1992)Google Scholar
  34. 34.
    Winickoff, R., Wilner, S., Neisuler, R., Barnett, G.: Limitations of provider interventions in hypertension quality assurance. Am. J. Public Health 75, 43–46 (1985)CrossRefGoogle Scholar
  35. 35.
    Rosser, W.W., McDowell, I., Newell, C.: Use of reminders for preventive procedures in family medicine. CMAJ 145, 807–814 (1991)Google Scholar
  36. 36.
    Jefferson, T., Demicheli, V., Vale, L.: Methodological reviews of economic evaluations in health care. JAMA 287, 2809–2812 (2002)CrossRefGoogle Scholar
  37. 37.
    Fergusson, D., van Walraven, C., Coyle, D., Laupacis, A.: Economic evaluations of technologies to minimize perioperative transfusion: a systematic review of published studies. International Study of Peri-operative Transfusion (ISPOT) investigators. Transfus. Med. Rev. 2, 106–117 (1999)CrossRefGoogle Scholar
  38. 38.
    Dempsey, C.L.: Nursing home-acquired pneumonia: outcomes from a clinical process improvement program. Pharmacotherapy 15 (1 Pt 2), 33S–38S (1995)Google Scholar
  39. 39.
    Jones, D.L., Kroenke, K., Landry, F.J., Tomich, D.J., Ferrel, R.J.: Cost savings using a stepped-care prescribing protocol for nonsteroidal anti-inflammatory drugs. JAMA 275, 926–930 (1996)CrossRefGoogle Scholar
  40. 40.
    Morrison, J.C., Sumrall, D.D., Chevalier, S.P., Robinson, S.V., Morrison, F.S., Wiser, W.L.: The effect of provider education on blood utilization practices. Am. J. Obstet. Gynecol. 169, 1240–1245 (1993)Google Scholar
  41. 41.
    Shojania, K.G., Yokoe, D., Platt, R., Fiskio, J., Ma’luf, N., Bates, D.W.: Reducing vancomycin use utilizing a computer guideline: results of a randomized controlled trial. J. Am. Med. Inform. Assoc. 5, 554–562 (1998)Google Scholar
  42. 42.
    Suwangool, P., Moola-Or, P., Waiwatana, A., Sitthi-Amorn, C., Israsena, S., Hanvanich, M.: Effect of a selective restriction policy on antibiotic expenditure and use: an institutional model. J. Med. Assoc. Thai. 74, 272–275 (1991)Google Scholar
  43. 43.
    Williams, A.: The cost benefit approach. Br. Med. Bull. 30(3), 252–256 (1974)Google Scholar
  44. 44.
    Drummond, M., OBrien, B., Stoddart, G., Torrance, G.: Methods for Economic Evaluation of Health Care Programme, 1st edn. Oxford University Press, New York (1987)Google Scholar
  45. 45.
    Anonymous: The Guideline Development Process: 1 Information for the Public and the NHS. The National Institute for Clinical Excellence. Oaktree Press, London (2001)Google Scholar
  46. 46.
    Truman, B.I., Smith-Akin, C.K., Hinman, A.R., Gebbie, K.M., Brownson, R., et al. and Task Force on Community Preventive Services: Developing the guide to community preventive services-overview and rationale. Am. J. Prev. Med. 18, 18–26 (2000)Google Scholar
  47. 47.
    Bloor, K., Freemantle, N., Khadjesari, Z., Maynard, A.: Impact of NICE guidance on laparoscopic surgery for inguinal hernias: analysis of interrupted time series. BMJ 326, 578 (2003)CrossRefGoogle Scholar
  48. 48.
    Sculpher, M.: Evaluating the cost-effectiveness of interventions designed to increase the utilisation of evidence-based guidelines. Fam. Pract. 17, S26–S31 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  • Luke Vale
    • 1
  • Ruth Thomas
    • 2
  • Graeme MacLennan
    • 2
  • Jeremy Grimshaw
    • 3
  1. 1.Health Services Research Unit and Health Economics Research UnitUniversity of AberdeenAberdeenScotland
  2. 2.Health Services Research UnitUniversity of AberdeenAberdeenScotland
  3. 3.Ottawa Health Research InstituteOttawaCanada

Personalised recommendations