Journal of General Internal Medicine

, Volume 33, Issue 7, pp 1043–1051 | Cite as

The Impact of Automated Notification on Follow-up of Actionable Tests Pending at Discharge: a Cluster-Randomized Controlled Trial

  • Anuj K. Dalal
  • Adam Schaffer
  • Esteban F. Gershanik
  • Ranganath Papanna
  • Katyuska Eibensteiner
  • Nyryan V. Nolido
  • Cathy S. Yoon
  • Deborah Williams
  • Stuart R. Lipsitz
  • Christopher L. Roy
  • Jeffrey L. Schnipper
Original Research



Follow-up of tests pending at discharge (TPADs) is poor. We previously demonstrated a twofold increase in awareness of any TPAD by attendings and primary care physicians (PCPs) using an automated email intervention


To determine whether automated notification improves documented follow-up for actionable TPADs


Cluster-randomized controlled trial


Attendings and PCPs caring for adult patients discharged from general medicine and cardiology services with at least one actionable TPAD between June 2011 and May 2012


An automated system that notifies discharging attendings and network PCPs of finalized TPADs by email

Main Measures

The primary outcome was the proportion of actionable TPADs with documented action determined by independent physician review of the electronic health record (EHR). Secondary outcomes included documented acknowledgment, 30-day readmissions, and adjusted median days to documented follow-up.

Key Results

Of the 3378 TPADs sampled, 253 (7.5%) were determined to be actionable by physician review. Of these, 150 (123 patients discharged by 53 attendings) and 103 (90 patients discharged by 44 attendings) were assigned to intervention and usual care groups, respectively, and underwent chart review. The proportion of actionable TPADs with documented action was 60.7 vs. 56.3% (p = 0.82) in the intervention vs. usual care groups, similar for documented acknowledgment. The proportion of patients with actionable TPADs readmitted within 30 days was 22.8 vs. 31.1% in the intervention vs. usual care groups (p = 0.24). The adjusted median days [95% CI] to documented action was 9 [6.2, 11.8] vs. 14 [10.2, 17.8] (p = 0.04) in the intervention vs. usual care groups, similar for documented acknowledgment. In sub-group analysis, the intervention had greater impact on documented action for patients with network PCPs compared with usual care (70 vs. 50%, p = 0.03).


Automated notification of actionable TPADs shortened time to action but did not significantly improve documented follow-up, except for network-affiliated patients. The high proportion of actionable TPADs without any documented follow-up (~ 40%) represents an ongoing safety concern.

Clinical Trials Identifier



tests pending at discharge patient safety health information technology 



This study was supported by a grant from CRICO/Risk Management Foundation of the Harvard Medical Institutions. The funding agency played no role in the conduct of the study, collection, management, analysis, and interpretation of the data, or the preparation, review, or approval of the manuscript. Dr. Dalal had full access to all the data in the study and takes responsibility for the integrity and the accuracy of the data analysis.


This study was supported by a grant from CRICO-Risk Management Foundation.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Supplementary material

11606_2018_4393_MOESM1_ESM.docx (18 kb)
ESM 1 (DOCX 17 kb)


  1. 1.
    Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138(3):161–167.CrossRefGoogle Scholar
  2. 2.
    Forster AJ, Clark HD, Menard A, et al. Adverse events among medical patients after discharge from hospital. CMAJ. 2004;170(3):345–349.PubMedPubMedCentralGoogle Scholar
  3. 3.
    Roy CL, Poon EG, Karson AS, et al. Patient safety concerns arising from test results that return after hospital discharge. Ann Intern Med. 2005;143(2):121–128.CrossRefGoogle Scholar
  4. 4.
    Callen J, Georgiou A, Li J, Westbrook JI. The safety implications of missed test results for hospitalised patients: a systematic review. BMJ Quality & Safety. 2011;20(2):194–199.CrossRefGoogle Scholar
  5. 5.
    El-Kareh R, Roy C, Brodsky G, Perencevich M, Poon EG. Incidence and predictors of microbiology results returning post-discharge and requiring follow-up. Journal of Hospital Medicine. 2011;6(5):291–296.CrossRefGoogle Scholar
  6. 6.
    Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test results for ambulatory patients: a systematic review. J Gen Intern Med. 2012;27(10):1334–1348.CrossRefGoogle Scholar
  7. 7.
    Dalal AK, Pesterev BM, Eibensteiner K, Newmark LP, Samal L, Rothschild JM. Linking acknowledgement to action: closing the loop on non-urgent, clinically significant test results in the electronic health record. J Am Med Inform Assoc. 2015;22(4):905–908.CrossRefGoogle Scholar
  8. 8.
    El-Kareh R, Roy C, Williams DH, Poon EG. Impact of automated alerts on follow-up of post-discharge microbiology results: a cluster randomized controlled trial. J Gen Intern Med. 2012;27(10):1243–50.CrossRefGoogle Scholar
  9. 9.
    Liao JM, Roy CL, Eibensteiner K, Nolido N, Schnipper JL, Dalal AK. Lost in transition: discrepancies in how physicians perceive the actionability of the results of tests pending at discharge. J Hosp Med. 2014;9(6):407–409.CrossRefGoogle Scholar
  10. 10.
    Gandhi TK, Kachalia A, Thomas EJ, et al. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Annals of Internal Medicine. 2006;145(7):488–496.CrossRefGoogle Scholar
  11. 11.
    Wahls T, Haugen T, Cram P. The continuing problem of missed test results in an integrated health system with an advanced electronic medical record. Jt Comm J Qual Patient Saf. 2007;33(8):485–492.CrossRefGoogle Scholar
  12. 12.
    Wahls TL, Cram PM. The frequency of missed test results and associated treatment delays in a highly computerized health system. BMC Fam Pract. 2007;8:32.CrossRefGoogle Scholar
  13. 13.
    Singh H, Thomas EJ, Mani S, et al. Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential? Arch Intern Med. 2009;169(17):1578–1586.CrossRefGoogle Scholar
  14. 14.
    Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record-based settings. JAMA Intern Med. 2013;173(8):702–704.CrossRefGoogle Scholar
  15. 15.
    Cutrona SL, Fouayzi H, Burns L, et al. Primary care providers’ opening of time-sensitive alerts sent to commercial electronic health record InBaskets. J Gen Intern Med. 2017;32(11):1210–1219.CrossRefGoogle Scholar
  16. 16.
    Dalal AK, Schnipper JL, Poon EG, et al. Design and implementation of an automated email notification system for results of tests pending at discharge. J Am Med Inform Assoc. 2012;19(4):523–528.CrossRefGoogle Scholar
  17. 17.
    Dalal AK, Roy CL, Poon EG, et al. Impact of an automated email notification system for results of tests pending at discharge: a cluster-randomized controlled trial. J Am Med Inform Assoc. 2014;21(3):473–480.CrossRefGoogle Scholar
  18. 18.
    Singh H, Vij MS. Eight recommendations for policies for communicating abnormal test results. Jt Comm J Qual Patient Saf. 2010;36(5):226–232.CrossRefGoogle Scholar
  19. 19.
    Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med. 2010;362(12):1066–1069.CrossRefGoogle Scholar
  20. 20.
    Callen J, Georgiou A, Li J, Westbrook JI. The impact for patient outcomes of failure to follow up on test results. how can we do better? EJIFCC. 2015;26(1):38–46.PubMedPubMedCentralGoogle Scholar
  21. 21.
    Were MC, Li X, Kesterson J, et al. Adequacy of hospital discharge summaries in documenting tests with pending results and outpatient follow-up providers. J Gen Intern Med. 2009;24(9):1002–1006.CrossRefGoogle Scholar
  22. 22.
    Harris M, Zalis M. QPID: building clinical applications on top of EMR semantic search. radiological society of north america 2011 scientific assembly and annual meeting, November 26–December 2, 2011. Accessed January 30, 2018.
  23. 23.
    Chow SC, Shao J, Wang H. Sample size calculations in clinical research. New York: Marcel Dekker; 2003.Google Scholar
  24. 24.
    Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27.CrossRefGoogle Scholar
  25. 25.
    McGreevy KM, Lipsitz SR, Linder JA, Rimm E, Hoel DG. Using median regression to obtain adjusted estimates of central tendency for skewed laboratory and epidemiologic data. Clin Chem. 2009;55(1):165–169.CrossRefGoogle Scholar
  26. 26.
    Georgiou A, McCaughey EJ, Tariq A, et al. What is the impact of an electronic test result acknowledgement system on emergency department physicians’ work processes? A mixed-method pre-post observational study. Int J Med Inform. 2017;99:29–36.CrossRefGoogle Scholar
  27. 27.
    Roy CL, Rothschild JM, Dighe AS, et al. An initiative to improve the management of clinically significant test results in a large health care network. Jt Comm J Qual Patient Saf. 2013;39(11):517–527.CrossRefGoogle Scholar
  28. 28.
    Lacson R, Prevedello LM, Andriole KP, et al. Four-year impact of an alert notification system on closed-loop communication of critical test results. AJR Am J Roentgenol. 2014;203(5):933–938.CrossRefGoogle Scholar
  29. 29.
    Lacson R, O'Connor SD, Sahni VA, et al. Impact of an electronic alert notification system embedded in radiologists’ workflow on closed-loop communication of critical results: a time series analysis. BMJ Qual Saf. 2016;25(7):518–24.CrossRefGoogle Scholar
  30. 30.
    Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: Implications for patient safety and continuity of care. JAM. 2007;297(8):831–841.CrossRefGoogle Scholar
  31. 31.
    O'Leary KJ, Liebovitz DM, Feinglass J, et al. Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary. J Hosp Med. 2009;4(4):219–225.CrossRefGoogle Scholar
  32. 32.
    Walz SE, Smith M, Cox E, Sattin J, Kind AJ. Pending laboratory tests and the hospital discharge summary in patients discharged to sub-acute care. J Gen Intern Med. 2011;26(4):393–398.CrossRefGoogle Scholar
  33. 33.
    Dinescu A, Fernandez H, Ross JS, Karani R. Audit and feedback: an intervention to improve discharge summary completion. J Hosp Med. 2011;6(1):28–32.CrossRefGoogle Scholar
  34. 34.
    Singh H, Thomas EJ, Sittig DF, et al. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain? Am J Med. 2010;123(3):238–244.CrossRefGoogle Scholar
  35. 35.
    Babbott S, Manwell LB, Brown R, et al. Electronic medical records and physician stress in primary care: results from the MEMO study. Journal of the American Medical Informatics Association : JAMIA. 2013;21:e100–e106.CrossRefGoogle Scholar
  36. 36.
    Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 2016;91(7):836–848.CrossRefGoogle Scholar
  37. 37.
    Davis Giardina T, Singh H. Should patients get direct access to their laboratory test results? an answer with many questions. JAMA. 2011;306(22):2502–2503.CrossRefGoogle Scholar
  38. 38.
    Giardina TD, Modi V, Parrish DE, Singh H. The patient portal and abnormal test results: an exploratory study of patient experiences. Patient experience journal. 2015;2(1):148–154.PubMedPubMedCentralGoogle Scholar
  39. 39.
    Dalal AK, Schnipper JL. Care team identification in the electronic health record: a critical first step for patient-centered communication. J Hosp Med. 2016;11(5):381–385.CrossRefGoogle Scholar
  40. 40.
    Dalal AK, Schnipper J, Massaro A, et al. A web-based and mobile patient-centered “microblog” messaging platform to improve care team communication in acute care. J Am Med Inform Assoc. 2017;24(e1):e178–e184.PubMedGoogle Scholar
  41. 41.
    Hyman JL, Luks HJ, Sechrest R. Online professional networks for physicians: risk management. Clin Orthop Relat Res. 2012;470(5):1386–1392.CrossRefGoogle Scholar
  42. 42.
    Ayele RA, Lawrence E, McCreight M, et al. Study protocol: improving the transition of care from a non-network hospital back to the patient's medical home. BMC Health Serv Res. 2017;17(1):123-017-2048-z.CrossRefGoogle Scholar
  43. 43.
    Sittig DF, Singh H. Improving test result follow-up through electronic health records requires more than just an alert. J Gen Intern Med. 2012;27(10):1235–1237.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2018

Authors and Affiliations

  • Anuj K. Dalal
    • 1
    • 2
    • 3
  • Adam Schaffer
    • 1
    • 2
    • 3
    • 4
  • Esteban F. Gershanik
    • 1
    • 2
    • 3
  • Ranganath Papanna
    • 1
    • 2
    • 3
  • Katyuska Eibensteiner
    • 1
  • Nyryan V. Nolido
    • 1
  • Cathy S. Yoon
    • 1
  • Deborah Williams
    • 1
    • 5
  • Stuart R. Lipsitz
    • 1
    • 3
  • Christopher L. Roy
    • 1
    • 2
    • 3
  • Jeffrey L. Schnipper
    • 1
    • 2
    • 3
  1. 1.Division of General Internal Medicine and Primary CareBrigham and Women’s HospitalBostonUSA
  2. 2.Hospital Medicine UnitBrigham and Women’s HospitalBostonUSA
  3. 3.Harvard Medical SchoolBostonUSA
  4. 4.CRICO/Risk Management Foundation of the Harvard Medical InstitutionsBostonUSA
  5. 5.Partners HealthCare, Inc.BostonUSA

Personalised recommendations