Skip to main content

Changing Clinical Practice Through Patient Specific Reminders Available at the Time of the Clinical Encounter: Systematic Review and Meta-Analysis

Journal of General Internal Medicine volume 27, pages 974–984 (2012)Cite this article

Abstract

OBJECTIVE

To synthesise current evidence for the influence on clinical behaviour of patient-specific electronically generated reminders available at the time of the clinical encounter.

DATA SOURCES

PubMed, Cochrane library of systematic reviews; Science Citation Index Expanded; Social Sciences Citation Index; ASSIA; EMBASE; CINAHL; DARE; HMIC were searched for relevant articles.

STUDY ELIGIBILITY CRITERIA, PARTICIPANTS AND INTERVENTIONS

We included controlled trials of reminder interventions if the intervention was: directed at clinician behaviour; available during the clinical encounter; computer generated (including computer generated paper-based reminders); and generated by patient-specific (rather than condition specific or drug specific) data.

STUDY APPRAISAL AND SYNTHESIS METHODS

Systematic review and meta-analysis of controlled trials published since 1970. A random effects model was used to derive a pooled odds ratio for adherence to recommended care or achievement of target outcome. Subgroups were examined based on area of care and study design. Odds ratios were derived for each sub-group. We examined the designs, settings and other features of reminders looking for factors associated with a consistent effect.

RESULTS

Altogether, 42 papers met the inclusion criteria. The studies were of variable quality and some were affected by unit of analysis errors due to a failure to account for clustering. An overall odds ratio of 1.79 [95% confidence interval 1.56, 2.05] in favour of reminders was derived. Heterogeneity was high and factors predicting effect size were difficult to identify.

LIMITATIONS

Methodological diversity added to statistical heterogeneity as an obstacle to meta-analysis. The quality of included studies was variable and in some reports procedural details were lacking.

CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS

The analysis suggests a moderate effect of electronically generated, individually tailored reminders on clinician behaviour during the clinical encounter. Future research should concentrate on identifying the features of reminder interventions most likely to result in the target behaviour.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Figure 1.
Figure 2.
Figure 3.
Figure 4.

References

  1. Balas EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD, Brown GD. The clinical value of computerized information services - A review of 98 randomized clinical trials. Arch Fam Med. 1996;5(5):271–8.

    PubMed  Article  CAS  Google Scholar 

  2. Bennett JW, Glasziou PP, Sim I. Review: Computerised reminders and feedback can improve provider medication management. Evid Base Med. 2003;8(6):190.

    Article  Google Scholar 

  3. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA. 2005;293(10):1223–38.

    PubMed  Article  CAS  Google Scholar 

  4. Kawamoto K, Houlihan C, Balas E, Lobach D. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.

    PubMed  Article  Google Scholar 

  5. Mitchell E, Sullivan F. A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97. BMJ. 2001;322(7281):279–82E.

    PubMed  Article  CAS  Google Scholar 

  6. Montgomery AA, Fahey T. A systematic review of the use of computers in the management of hypertension. Journal of Epidemiology and Community Health. 1998;52(8):520–5.

    PubMed  Article  CAS  Google Scholar 

  7. Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3(6):399–409.

    PubMed  Article  CAS  Google Scholar 

  8. Shiffman RN, Liaw Y, Brandt CA, Corb GJ. Computer-based guideline implementation systems: a systematic review of functionality and effectiveness. J Am Med Inform Assoc. 1999;6(2):104–14.

    PubMed  Article  CAS  Google Scholar 

  9. Dexheimer JW, Talbot TR, Sanders DL, Rosenbloom ST, Aronsky D. Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. J Am Med Inform Assoc. 2008;15(3):311–20.

    PubMed  Article  Google Scholar 

  10. Shojania KG, Jennings A, Mayhew A, et al. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;3:CD001096.

    PubMed  Google Scholar 

  11. Shojania KG, Jennings A, Mayhew A, et al. Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ. 2010;182:E216–25.

    PubMed  Article  Google Scholar 

  12. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org (last accessed 24.1.12).

  13. Adams G, Gulliford MC, Ukoumunne OC, et al. Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol. 2004;57(8):785–794.

    PubMed  Article  Google Scholar 

  14. Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, et al. A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc. 2008;15(4):424–9.

    PubMed  Article  Google Scholar 

  15. Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999;106(2):144–50.

    PubMed  Article  CAS  Google Scholar 

  16. Burack RC, Gimotty PA, George J, Simon MS, Dews P, Moncrease A. The effect of patient and physician reminders on use of screening mammography in a health maintenance organization. Results of a randomized controlled trial. Cancer. 1996;78(8):1708–21.

    PubMed  Article  CAS  Google Scholar 

  17. Burack RC, Gimotty PA, George J, McBride S, Moncrease A, Simon MS, et al. How reminders given to patients and physicians affected pap smear use in a health maintenance organization: results of a randomized controlled trial. Cancer. 1998;82(12):2391–400.

    PubMed  Article  CAS  Google Scholar 

  18. Chambers CV, Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center. J Am Board Fam Pract. 1991;4(1):19–26.

    PubMed  CAS  Google Scholar 

  19. Dexter PR, Wolinsky FD, Gramelspacher GP, Zhou XH, Eckert GJ, Waisburd M, et al. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med. 1998;128(2):102–10.

    PubMed  CAS  Google Scholar 

  20. Dexter PR, Perkins S, Overhage JM, Maharry K, Kohler RB, McDonald CJ. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med. 2001;345(13):965–70.

    PubMed  Article  CAS  Google Scholar 

  21. Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002;325(7370):941.

    PubMed  Article  Google Scholar 

  22. Filippi A, Sabatini A, Badioli L, Samani F, Mazzaglia G, Catapano A, et al. Effects of an automated electronic reminder in changing the antiplatelet drug-prescribing behavior among Italian general practitioners in diabetic patients: an intervention trial. Diabetes Care. 2003;26(5):1497–500.

    PubMed  Article  Google Scholar 

  23. Frank O, Litt J, Beilby J. Opportunistic electronic reminders: improving performance of preventive care in general practice. Aust Fam Physician. 2004;33(1/2):87–90.

    PubMed  Google Scholar 

  24. Hicks LS, Sequist TD, Ayanian JZ, Shaykevich S, Fairchild DG, Orav EJ, et al. Impact of computerized decision support on blood pressure management and control: A randomized controlled trial. J Gen Intern Med. 2007;23(4):429–41.

    Article  Google Scholar 

  25. Judge J, Field TS, DeFlorio M, Laprino J, Auger J, Rochon P, et al. Prescribers' responses to alerts during medication ordering in the long term care setting. J Am Med Inform Assoc. 2006;13(4):385–90.

    PubMed  Article  Google Scholar 

  26. Kenealy T, Arroll B, Petrie KJ. Patients and computers as reminders to screen for diabetes in family practice. Randomized-controlled trial. J Gen Intern Med. 2005;20(10):916–21.

    PubMed  Article  Google Scholar 

  27. Kralj B, Iverson D, Hotz K, Ashbury FD. The impact of computerized clinical reminders on physician prescribing behavior: Evidence from community oncology practice. Am J Med Qual. 2003;18(5):197–203.

    PubMed  Article  Google Scholar 

  28. Krall MA, Traunweiser K, Towery W. Effectiveness of an electronic medical record clinical quality alert prepared by off-line data analysis. Medinfo. 2004;11(Pt 1):135–9.

    Google Scholar 

  29. Kucher N, Koo S, Quiroz R, Cooper JM, Paterno MD, Soukonnikov B, et al. Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med. 2005;352(10):969–77.

    PubMed  Article  CAS  Google Scholar 

  30. Litzelman DK, Dittus RS, Miller ME, Tierney WM. Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med. 1993;8(6):311–7.

    PubMed  Article  CAS  Google Scholar 

  31. Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK. Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009;16(1):66–71.

    PubMed  Article  Google Scholar 

  32. McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE. Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Med Inform Internet Med. 2001;26(3):191–201.

    PubMed  Article  CAS  Google Scholar 

  33. McDonald CJ. Use of a computer to detect and respond to clinical events: Its effect on clinical behavior. Ann Intern Med. 1976;84:162–7.

    CAS  Google Scholar 

  34. McDonald CJ, Wilson GA, McCabe GP Jr. Physician response to computer reminders. JAMA. 1980;244(14):1579–81.

    PubMed  Article  CAS  Google Scholar 

  35. McDowell I, Newell C, Rosser W. A randomized trial of computerized reminders for blood pressure screening in primary care. Med Care. 1989;27(3):297–305.

    PubMed  Article  CAS  Google Scholar 

  36. McDowell I, Newell C, Rosser W. Computerized reminders to encourage cervical screening in family practice. J Fam Pract. 1989;28(4):420–4.

    PubMed  CAS  Google Scholar 

  37. Murray MD, Harris LE, Overhage JM, Zhou XH, Eckert GJ, Smith FE, et al. Failure of Computerized Treatment Suggestions to Improve Health Outcomes of Outpatients with Uncomplicated Hypertension: Results of a Randomized Controlled Trial. Pharmacotherapy. 2004;24(3):324–37.

    PubMed  Article  Google Scholar 

  38. Overhage JM, Tierney WM, McDonald CJ. Computer reminders to implement preventive care guidelines for hospitalized patients. Arch Intern Med. 1996;156(14):1551–6.

    PubMed  Article  CAS  Google Scholar 

  39. Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc. 1997;4(5):364–75.

    PubMed  Article  CAS  Google Scholar 

  40. Rosser WW, McDowell I, Newell C. Use of Reminders for Preventive Procedures in Family Medicine. CMAJ. 1991;145(7):807.

    PubMed  CAS  Google Scholar 

  41. Rosser WW, Hutchison BG, McDowell I, Newell C. Use of Reminders to Increase Compliance with Tetanus Booster Vaccination. CMAJ. 1992;146(6):911–7.

    PubMed  CAS  Google Scholar 

  42. Rossi RA, Every NR. A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med. 1997;12(11):672–8.

    PubMed  Article  CAS  Google Scholar 

  43. Rothschild J, McGurk S, Honour M, Lu L, McClendon A, Srivastava P, et al. Assessment of education and computerized decision support interventions for improving transfusion practice. Transfusion. 2007;47(2):228–39.

    PubMed  Article  Google Scholar 

  44. Safran C, Rind DM, Davis RB, Ives D, Sands DZ, Currier J, et al. Guidelines for management of HIV infection with computer-based patient's record. Lancet. 1995;346(8971):341–6.

    PubMed  Article  CAS  Google Scholar 

  45. Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, et al. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. Journal of the American Medical Informatics Association. 2005;12(4):431–7.

    PubMed  Article  Google Scholar 

  46. Shea S, Sideli RV, DuMouchel W, Pulver G, Arons RR, Clayton PD. Computer-generated informational messages directed to physicians: effect on length of hospital stay. J Am Med Inform Assoc. 1995;2(1):58–64.

    PubMed  Article  CAS  Google Scholar 

  47. Tamblyn R, Huang A, Perreault R, Jacques A, Roy D, Hanley J, et al. The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ. 2003;169(6):549–56.

    PubMed  Google Scholar 

  48. Tamblyn R, Huang A, Taylor L, Kawasumi Y, Bartlett G, Grad R, et al. A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care. J Am Med Inform Assoc. 2008;15(4):430–8.

    PubMed  Article  Google Scholar 

  49. Tape TG, Campbell JR. Computerized medical records and preventive health care: success depends on many factors. Am J Med. 1993;94(6):619–25.

    PubMed  Article  CAS  Google Scholar 

  50. Tierney WM, McDonald CJ, Martin DK, Rogers MP. Computerized display of past test results. Effect on outpatient testing. Ann Intern Med. 1987;107(4):569–74.

    PubMed  CAS  Google Scholar 

  51. Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003;18(12):967–76.

    PubMed  Article  Google Scholar 

  52. Tierney W, Overhage J, Murray M, Harris L, Zhou X, Eckert G, et al. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005;40(2):477–97.

    PubMed  Article  Google Scholar 

  53. van Wyk MA, van der Lei J, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. A randomized trial. Ann Intern Med. 2001;134(4):274–81.

    Google Scholar 

  54. White KS, Lindsay A, Pryor TA, Brown WF, Walsh K. Application of a computerized medical decision-making process to the problem of digoxin intoxication. J Am Coll Cardiol. 1984;4(3):571–6.

    PubMed  Article  CAS  Google Scholar 

  55. Holt TA, Thorogood M, Griffiths F, Munday S, Friede T, Stables D. Automated electronic reminders to facilitate cardiovascular disease prevention: randomised controlled trial. Brit J Gen Pract. 2010;60(573):e137–43.

    Article  Google Scholar 

  56. Gandhi TJ, Sequist TD, Poon EG, et al. Primary care clinicians’ attitudes towards electronic clinical reminders and clinical practice guidelines. Proceedings of the AMIA Symposium 2003:848.

  57. Krall MA, Sittig DF. Subjective assessment of usefulness and appropriate presentation mode of alerts and reminders in the outpatient setting. Proceedings of the AMIA Symposium 2001:334–338.

  58. Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. JAMIA. 2005;12:438–447.

    PubMed  Google Scholar 

  59. Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM. Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005;38:189–199.

    PubMed  Article  Google Scholar 

  60. Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL. A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Informat Decis Making. 2006;6:6.

    Article  Google Scholar 

  61. Krall MA, Sittig DF. Clinicians’ assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proceedings of the AMIA Symposium 2002:400–404.

  62. Agrawal A, Mayo-Smith MF. Adherence to computerized clinical reminders in a large healthcare delivery network. Medinfo. 2004;11(1):111–114.

    Google Scholar 

  63. Mayo-Smith MF, Agrawal A. Factors associated with improved completion of computerized clinical reminders across a large healthcare system. Int J Med Informat. 2007;76(10):710–716.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Dr Simon Gates for analytical advice and Samantha Johnson for assistance with database searching and retrieval of papers. We also thank the authors of studies for which clarification was required for providing useful information.

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Funding source

Funded through internal sources. No external funding.

Author information

Authors and Affiliations

  1. Department of Primary Care Health Sciences, University of Oxford, 2nd floor, 23-38 Hythe Bridge Street, Oxford, OX1 2ET, UK

    Tim A. Holt PhD

  2. Health Sciences Research Institute, University of Warwick, Coventry, UK

    Tim A. Holt PhD, Margaret Thorogood PhD & Frances Griffiths PhD

Authors
  1. Tim A. Holt PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Margaret Thorogood PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frances Griffiths PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tim A. Holt PhD.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Holt, T.A., Thorogood, M. & Griffiths, F. Changing Clinical Practice Through Patient Specific Reminders Available at the Time of the Clinical Encounter: Systematic Review and Meta-Analysis. J GEN INTERN MED 27, 974–984 (2012). https://doi.org/10.1007/s11606-012-2025-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-012-2025-5

KEY WORDS

  • reminder systems
  • electronic health records
  • computer systems
  • decision support systems, clinical