Skip to main content

The Impact of Automated Notification on Follow-up of Actionable Tests Pending at Discharge: a Cluster-Randomized Controlled Trial

Abstract

Background

Follow-up of tests pending at discharge (TPADs) is poor. We previously demonstrated a twofold increase in awareness of any TPAD by attendings and primary care physicians (PCPs) using an automated email intervention

Objective

To determine whether automated notification improves documented follow-up for actionable TPADs

Design

Cluster-randomized controlled trial

Subjects

Attendings and PCPs caring for adult patients discharged from general medicine and cardiology services with at least one actionable TPAD between June 2011 and May 2012

Intervention

An automated system that notifies discharging attendings and network PCPs of finalized TPADs by email

Main Measures

The primary outcome was the proportion of actionable TPADs with documented action determined by independent physician review of the electronic health record (EHR). Secondary outcomes included documented acknowledgment, 30-day readmissions, and adjusted median days to documented follow-up.

Key Results

Of the 3378 TPADs sampled, 253 (7.5%) were determined to be actionable by physician review. Of these, 150 (123 patients discharged by 53 attendings) and 103 (90 patients discharged by 44 attendings) were assigned to intervention and usual care groups, respectively, and underwent chart review. The proportion of actionable TPADs with documented action was 60.7 vs. 56.3% (p = 0.82) in the intervention vs. usual care groups, similar for documented acknowledgment. The proportion of patients with actionable TPADs readmitted within 30 days was 22.8 vs. 31.1% in the intervention vs. usual care groups (p = 0.24). The adjusted median days [95% CI] to documented action was 9 [6.2, 11.8] vs. 14 [10.2, 17.8] (p = 0.04) in the intervention vs. usual care groups, similar for documented acknowledgment. In sub-group analysis, the intervention had greater impact on documented action for patients with network PCPs compared with usual care (70 vs. 50%, p = 0.03).

Conclusions

Automated notification of actionable TPADs shortened time to action but did not significantly improve documented follow-up, except for network-affiliated patients. The high proportion of actionable TPADs without any documented follow-up (~ 40%) represents an ongoing safety concern.

Clinical Trials Identifier

NCT01153451

BACKGROUND

More than half of preventable adverse events after discharge are related to poor communication.1,2 In prior work, we determined that 41% of patients left the hospital before all test results were finalized. Of the results of tests pending at discharge (TPADs), 9.4% were considered actionable by independent review.3 Although physicians had access to an integrated, electronic health record (EHR), they were unaware of 62% of these actionable TPADs.3

But awareness is only part of the problem: what matters clinically is whether appropriate follow-up occurs. Across institutions, the proportion of TPADs with documented follow-up is variable (20 to 62%), representing a substantial problem.4 Depending on the result, follow-up may include establishing a diagnosis, altering therapy, arranging an ambulatory appointment, initiating referral to a consultant, ordering additional tests, or communicating with patients.5,6,7 Although documented follow-up has been evaluated for certain actionable TPADs (e.g., microbiology),5,8 there are few data for other test types, services, clinicians, and EHRs. This research is difficult because clinicians may have different perspectives about whether specific TPADs are actionable depending on context, specialty, and role on the care team.9 Failure to follow-up actionable TPADs could lead to missed diagnoses, delayed treatments, unnecessary healthcare utilization, malpractice litigation, and preventable harm.4,10,11,12,13

Current vendor EHRs can be configured to notify clinicians of important results, such as TPADs, within an “InBasket”; however, time sensitive alerts are often missed and cognitively overloaded clinicians may not follow-up actionable data in a timely manner.14,15 Although clinicians may choose to receive notifications external to the EHR via institutional email or a “push” notification to a mobile app, few take advantage of this functionality. Furthermore, these notifications are typically generic, do not specify the actual result or criticality, do not classify TPADs as a special type of result, and do not provide a mechanism to communicate clinical context and transfer responsibility to the ambulatory clinician who will follow-up with the patient.

We previously designed, developed, and implemented an automated system that notified the responsible discharging attending and copied the primary care physician (PCP) of TPADs via secure, network email. Our system was unique not only because it sent email notifications external to our EHR (i.e., useful even if providers were not using the EHR at that moment), but also because it flagged TPADs as high risk, delineated responsibility for follow-up, and provided a simple mechanism (“reply all”) for attendings to communicate clinical context to ambulatory clinicians who could then assume responsibility.16 We demonstrated a twofold increase in self-reported awareness of any TPAD by attendings (76 vs. 38%) and PCPs (57 vs. 33%) assigned to the intervention group.17 In theory, improved awareness by the responsible discharge attending via this “active” notification should lead to appropriate follow-up, even for patients with non-network PCPs, but as noted above, this is by no means assured. The aim of this study was to evaluate the impact of our system on documented follow-up of actionable TPADs, defined as evidence of documented action or acknowledgment in our EHR by any network-affiliated clinician (attending, subspecialist, PCP). Understanding the benefit of automated notification systems for actionable TPADs on actual follow-up is crucial to resolving this safety concern.18,19,20

METHODS

Study Design, Setting, and Participants

The Partners Human Research Committee approved the study and granted a waiver of consent. We conducted a cluster randomized controlled trial at Brigham and Women’s Hospital, a 793-bed tertiary care hospital in Boston, MA, affiliated with Partners HealthCare, Inc., an integrated healthcare delivery network. Patients discharged from general medicine and cardiology services with at least one TPAD between June 2011 and May 2012 were assigned to intervention or usual care groups based upon the randomization status of their discharging attending and PCP.

Physicians caring for eligible patients included hospitalists, traditional internists, cardiologists, and other subspecialists attending on general medicine services, as well as PCPs. All Partners physicians could access the network EHR and enterprise email securely from a clinical workstation, personal computer, or encrypted mobile device. Non-network PCPs did not have access to Partners clinical information systems, but discharge summaries were faxed or mailed within 48 hours of discharge.

Intervention

The system automatically sent notifications to network clinicians’ institutional email addresses, and was configured to minimize alert fatigue. Each notification was flagged as an “Important Post-Discharge Test Result” in the subject heading, and the discharging attending was assigned responsibility in the body of the email. Network PCPs were carbon copied, thereby allowing the attending to communicate clinical context and transfer responsibility after discharge.16 Non-network PCPs were not notified by email; however, the attending could choose to communicate with non-network PCPs (via phone, letter, etc.) and document these actions in the EHR.

Randomization

The randomization procedures were previously described.17 Briefly, attendings and PCPs (network and non-network) were independently randomized, and patients were assigned to the intervention or usual care group if the attending and PCP were in concordant study arms. Patients of physicians randomized to discordant study arms were excluded to avoid contamination. Non-network patients were included in the intention-to-treat analysis, though non-network PCPs did not receive automated emails.

Identification of Actionable TPAD Results

We adapted an established algorithm to identify actionable TPADs.3,21 All normal, near-normal, and benign results were excluded. Two board-certified, internal medicine physicians (A-reviewers: AS, AD), blinded to randomization status, independently reviewed the discharge summary to determine whether the TPAD was listed, and to categorize the TPAD as definitely actionable, probably actionable, probably not actionable, or definitely not actionable according to a list of actions designated a priori (Box 1). A-reviewers adjudicated all discrepancies until consensus was reached. Thus, patients with TPADs who did not have at least one actionable TPAD identified by independent physician review were excluded.

Box 1 Types of actions taken by responsible providers after acknowledging an actionable TPAD

Determination of Documented Acknowledgement and Actions

Two different physicians (B-reviewers: EG, RP), blinded to randomization, independently reviewed the EHR to identify follow-up documentation for each actionable TPAD identified by A-reviewers. We defined follow-up as explicit documentation of acknowledgment or action (independent events) in the EHR’s note repository similarly to El-Kareh.5 B-reviewers used a natural language processing search tool, Queriable Patient Inference Dossier (QPID)—widely used at our institution to identify clinical information at the point-of-care—to retrieve relevant documentation.22 Specifically, B-reviewers queried the EHR for all documentation pertinent to the actionable TPAD (clinic and hospital notes, phone calls, medication lists, etc.) for up to 6 months after discharge. For patients with a high volume of documentation, reviewers used broad but specific search terms (e.g., A1c) to improve sensitivity when conducting queries while ensuring retrieval of the most pertinent subset of documentation related to the actionable TPAD.

After retrieving relevant documentation, B-reviewers determined whether the result was acknowledged; the name and specialty of the individual who acknowledged the result; the format of acknowledgment (phone call, hospital, or clinic note); and whether one or more actions (Box 1) were documented. B-reviewers resolved all discrepancies between themselves. Chart review was limited to our network’s EHR; out-of-network records of enrolled patients were not reviewed.

Outcomes

The primary outcome was the proportion of actionable TPADs with documented action. Secondary outcomes included the proportion of actionable TPADs with documented acknowledgment, patients with actionable TPADs readmitted within 30-days of hospitalization, and median days to documented follow-up. We quantified types of actions (Box 1) documented.

Sample Size

Based on El-Kareh, we expected the proportion of actionable TPADs with documented follow-up to increase from 13 to 28% when responsible physicians were notified.8 We assumed that patients had approximately one actionable TPAD (Roy et al. observed 1.08 actionable TPADs per patient).3 To achieve 80% power with an alpha of 0.05,23 we estimated that we would need to review records of 144 discharged patients with actionable TPADs to detect the effect size noted above (288 in both groups combined), accounting for clustering by attending (intra-class correlation coefficient of 0.03, cluster size of ten patients with an actionable TPAD per physician based on our prior study17) and a 50% reduction in sample size due to exclusion of patients cared for by physicians randomized to discordant study arms. Sample size calculations were conducted using NCSS PASS, version 12 (Kaysville, UT). We identified this cohort by randomly sampling TPADs from all available patient-discharges for up to 1 year after our prior study.17

Covariates

We collected patient demographics including age, gender, ethnicity, socioeconomic, and insurance status, Elixhauser comorbidity score,24 network affiliation, and length of stay. We collected physician demographics including age, gender, post-graduate year, specialty (general internist, hospitalist, cardiologist, other specialist), and number of years employed from hospital administrative databases, publicly available information (American Board of Internal Medicine), and practice managers.

Statistical Analysis

Patient and physician characteristics were described using means with standard deviations, medians with inter-quartile ranges, and proportions as appropriate. We analyzed the primary outcome as the proportion of actionable TPADs with documented action. We analyzed documented acknowledgment similarly. We analyzed hospital readmissions as the proportion of patients discharged with an actionable TPAD who were readmitted to a network-affiliated hospital within 30 days of index hospitalization. The effect of dichotomous variables on all outcomes was first analyzed using Fisher’s exact test. To measure days to documented follow-up, we calculated adjusted medians with 95% CIs using median regression.25 We then used multivariable logistic regression to analyze action, acknowledgment, and readmission adjusted for a priori selected covariates and general estimating equations to cluster by attending. In sub-group analyses, we used multivariable logistic regression with an interaction term (study arm*characteristic-type) to determine effect modification. Two-sided p values < 0.05 were considered significant. Analyses were conducted using SAS v9.4 (Cary, NC).

RESULTS

Of the 12,999 TPADs (Fig. 1) available during the study period, 3378 were sampled-based on our randomization schema. Of these, 2552 TPADs were excluded based on our screening protocol (normal, near-normal, or benign results). Of the remaining 826 TPADs, 253 (7.5% of all TPADs sampled) were determined to be actionable and underwent subsequent chart review. Of the 253 actionable TPADs, 150 (123 patients discharged by 53 attendings) and 103 (90 patients discharged by 44 attendings) were previously assigned to intervention and usual care groups, respectively, based on provider randomization (Table 1).

Fig. 1
figure 1

Consort diagram.

Table 1 Characteristics of Actionable TPADs by Service, Test Type, Physicians, and Patients

In general, characteristics of actionable TPADs, patients with those TPADs, and physicians caring for them (Table 1) were equally balanced in the intervention and usual care groups. Most TPADs were generated on patients discharged from general medicine services. The most frequent test type was chemistry/hematology. Most discharging attendings were male and had 5 years or less of experience. Most patients were white and had incomes of $47,000 or less, public/government insurance, one actionable TPAD, and a non-network PCP. Additionally, the proportion of actionable TPADs listed as pending in the discharge summary was 50.7 and 49.5% in the intervention and control groups, respectively.

The proportion of actionable TPADs with documented action was not statistically different for the intervention arm compared to usual care (60.7 vs. 56.3%; adjusted and clustered OR 1.07 [0.58–1.99]; p = 0.82); this finding was similar for documented acknowledgment (Table 2). There were fewer readmissions in the intervention compared to usual care group, but this was not statistically significant (22.8 vs. 31.1%; adjusted OR 0.69 [0.37–1.28]; p = 0.24). The adjusted median days to documented action was significantly shorter for the intervention compared to usual care (9 [6.2, 11.8] vs. 14 [10.2, 17.8]; adjusted OR 0.64 [0.43, 0.97]; p = 0.04); this finding was similar for documented acknowledgment.

Table 2 Primary and Secondary Outcomes

An analysis of the effect of covariates (Appendix A online) on the primary outcome demonstrated that the proportion of actionable TPADs with documented action was significantly higher for pathology compared to other test types (77.2 vs. 48.5%; adjusted OR 4.11 [2.24, 7.56], p < 0.001). There was a non-significant trend towards increased proportion of actionable TPADs with documented action for patients discharged by hospitalists compared to non-hospitalists (62.3 vs. 54.4%; adjusted OR 1.45 [0.71, 2.98], p = 0.31). The effects were similar for documented acknowledgment.

Of the 152 actionable TPADs with documented acknowledgment across both study arms, 148 (97.4%) had evidence of documented action on the same day. Seventy-three (48.0%), 67 (44.1%), and 12 (7.9%) were pathology, chemistry/hematology, and microbiology test types, respectively; and acknowledgment was documented by an ambulatory clinician in 124 (81.6%). Of the 101 actionable TPADs for which no documented acknowledgment was identified, documented action was found in just one (1%); this actionable TPAD was a low vitamin D level, and the action was a new vitamin D supplement on the medication list. A total of 536 types of actions (Table 3) were documented for the 149 actionable TPADs across both study arms (mean (SD): 3.58 ± 1.30 actions per actionable TPAD); there were no significant differences between intervention and usual care groups for types of actions.

Table 3 Types of Documented Actions Recorded in EHR for Actionable TPADs

In sub-group analyses (Table 4), we observed significantly greater impact of the intervention on documented follow-up of actionable TPADs among patients with network PCPs (70 vs. 50%, p = 0.03). There was no evidence of effect modification of the intervention by clinical service, provider type or experience, test type, or number of actionable TPADs per patient.

Table 4 Sub-group Analyses: Documented Action for Actionable TPADs by Characteristic

The proportion of actionable TPADs without documented follow-up was 39.9% for acknowledgment, and 41.1% for action. The majority were chemistry/hematology tests and from non-network patients. Types of actionable TPADs without documented follow-up included abnormal vitamin levels (n = 25), pathology specimens (n = 15, including 6 malignancies); abnormal rheumatology titers (n = 9, including an extremely elevated anti-DS DNA titer); abnormal endocrine results (n = 8); abnormal viral tests (n = 8, including an elevated HCV viral load); positive microbiology cultures (4); and miscellaneous (32). See Appendix B online.

DISCUSSION

We report findings from a cluster-randomized controlled trial assessing the impact of automated notification of TPADs on documented follow-up in the EHR. For nearly all actionable TPADs, acknowledgment and action were documented at the same time, typically by an ambulatory clinician. Although we did not observe significant improvement in the proportion of actionable TPADs with documented follow-up, we did observe significant improvement in time to documented follow-up favoring the intervention. In covariate analyses, the pathology test type was most commonly associated with documented follow-up. Most commonly documented actions included establishing, changing, or confirming a diagnosis and communicating results with the patient. In sub-group analyses by PCP affiliation, we observed significant improvement in documented follow-up favoring the intervention for patients of network PCPs. Notably, a high proportion of actionable TPADs (~ 40%) did not have follow-up documented in the EHR, including a few high-risk results (e.g., malignancies).

Our inability to detect improvement in the primary outcome has several possible explanations. First, most patients in our study (~ 66%) had non-network PCPs, which likely diluted the effect of our intervention. The lack of improvement for non-network patients likely reflects two phenomena: lower impact of the intervention on care for those patients because non-network PCPs were not contacted directly (and inpatient attendings did not usually contact them indirectly), and an artifactual reduction in follow-up rates (among patients in both study arms) due to lack of documentation available to review (see limitations, below). Second, we did not expect follow-up documentation to be as high as observed in usual care (~ 60%). In prior studies, physician awareness of actionable TPADs (72 h after TPADs were reported) was 38%,3,17 and the proportion of TPADs with documented follow-up within 3-days of result finalization was 20–30%.8,26 We conducted chart review for up to 6 months after TPADs were finalized, which may have led to our higher estimates of follow-up. Third, documentation of pending results in the discharge summary may have influenced subsequent follow-up under usual care. However, the proportion of actionable TPADs listed in the discharge summary was modest (about 50%, despite concurrent efforts to improve discharge documentation), and ambulatory clinicians (not inpatient attendings) typically documented follow-up, so this is unlikely to fully explain our findings. Fourth, two other electronic notification systems were operational during the study period: one for abnormal pathology results and another for critical radiology results. The pathology system automatically notified attendings of abnormal results by email; however, unlike our intervention, this system did not provide a description of the result in the body of the email and did not concurrently notify PCPs.16,17 The radiology system—automated notification of critical results (ANCR)—requested electronic acknowledgment by the ordering clinician for clinically significant radiology results (e.g., pulmonary nodule) via a pager or email notification and escalated this request to the attending if acknowledgment was not recorded within a pre-specified timeframe depending on result severity.27,28,29 Both co-interventions may have improved follow-up for actionable pathology and radiology TPADs across both study arms. In fact, we identified only 4 (1.6%) actionable radiology TPADs in our sample; most clinically significant radiology results were acknowledged before discharge.28,29

Most studies have reported variable rates of TPAD documentation (10–80%) in the discharge summary.21,30,31,32,33 Few studies have reported documentation of follow-up actions in the EHR during the period after TPADs are finalized. While our study showed that increased awareness facilitated by automated notification did not significantly improve overall follow-up documentation, it did shorten time to documented follow-up independent of co-interventions, and was associated with a trend towards fewer readmissions. The significant improvement in time to follow-up has important ramifications for certain types of TPADs and should be underscored. Specifically, automated notification may be most relevant for non-immediately life-threatening TPADs (e.g., positive H. pylori Ag) that do not have systems in place to notify clinicians, “fall through the cracks” and have the potential to manifest in harm and re-hospitalization if appropriate treatments are not initiated in a timely manner.13,34

Our study had several limitations. First, review of outside EHR records was out of scope, and as mentioned above, this likely contributed to the apparent lack of effect of the intervention, especially for patients with non-network PCPs. Second, the study was limited to non-surgical services at a single academic institution; however, an automated notification strategy should be generalizable despite service- and institution-specific practice differences. Third, due to the volume of TPADs screened, we opted to exclude normal or near-normal results. We acknowledge that certain normal results (negative blood culture; benign pathology results) may prompt meaningful action (stopping empiric antibiotics; calling patients to provide reassurance), but measuring this was beyond the scope of our study. Fourth, actions restricted to orders but not documented in notes would have been missed. The search tool we used retrieved documentation alone; we did not measure recall and precision of our retrieval strategy. Fifth, we did not conduct a formal evaluation of the impact of our intervention on cognitive burden, but information overload and burn-out attributable to EHRs can clearly contribute to missed test results.14,35,36 Finally, we did not assess patients’ involvement in managing TPADs—in addition to technology and governance, patient engagement is a promising strategy to ensure follow-up.18,20,27,37,38

Conclusion and Implications

Automated notification as we describe is an effective strategy for facilitating closed loop communication of actionable TPADs, potentially decreasing time to action. Most major EHR vendors can extend existing functionality to externally notify clinicians of actionable TPADs via institutional email, secure messaging, or mobile app “push” notifications; flag importance in subject headings or notification banners; facilitate knowledge transfer via messaging threads between key inpatient and ambulatory clinicians; and request acknowledgment electronically.7,16,17,39,40 Institutions considering this type of “active” strategy would benefit from notifying non-network as well as network PCPs—technology to send out-of-network messages securely is increasingly available with growth of online physician directories and professional networks.41 Leveraging EHRs, communication tools, and online resources in this way can address safety concerns related to discontinuity within and across healthcare systems.30,42

Still, automated notifications are insufficient to fully resolve this problem.43 To ensure timely action is taken in the ambulatory setting, tools to transfer responsibility to follow-up providers, facilitate order entry for referrals and future studies, and identify results for which appropriate actions have not been executed are needed, especially for non-immediately life-threatening results.34 Furthermore, enterprise policies regarding appropriate acknowledgment and follow-up intervals are necessary to ensure accountability.27 Finally, future research is needed to evaluate the effectiveness of EHR-integrated notification technologies (including newer “smart” notification platforms), impact on cognitive burden, and utility in empowering patients and care partners to follow-up test results.

References

  1. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138(3):161–167.

    Article  Google Scholar 

  2. Forster AJ, Clark HD, Menard A, et al. Adverse events among medical patients after discharge from hospital. CMAJ. 2004;170(3):345–349.

    PubMed  PubMed Central  Google Scholar 

  3. Roy CL, Poon EG, Karson AS, et al. Patient safety concerns arising from test results that return after hospital discharge. Ann Intern Med. 2005;143(2):121–128.

    Article  Google Scholar 

  4. Callen J, Georgiou A, Li J, Westbrook JI. The safety implications of missed test results for hospitalised patients: a systematic review. BMJ Quality & Safety. 2011;20(2):194–199.

    Article  Google Scholar 

  5. El-Kareh R, Roy C, Brodsky G, Perencevich M, Poon EG. Incidence and predictors of microbiology results returning post-discharge and requiring follow-up. Journal of Hospital Medicine. 2011;6(5):291–296.

    Article  Google Scholar 

  6. Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test results for ambulatory patients: a systematic review. J Gen Intern Med. 2012;27(10):1334–1348.

    Article  Google Scholar 

  7. Dalal AK, Pesterev BM, Eibensteiner K, Newmark LP, Samal L, Rothschild JM. Linking acknowledgement to action: closing the loop on non-urgent, clinically significant test results in the electronic health record. J Am Med Inform Assoc. 2015;22(4):905–908.

    Article  Google Scholar 

  8. El-Kareh R, Roy C, Williams DH, Poon EG. Impact of automated alerts on follow-up of post-discharge microbiology results: a cluster randomized controlled trial. J Gen Intern Med. 2012;27(10):1243–50.

    Article  Google Scholar 

  9. Liao JM, Roy CL, Eibensteiner K, Nolido N, Schnipper JL, Dalal AK. Lost in transition: discrepancies in how physicians perceive the actionability of the results of tests pending at discharge. J Hosp Med. 2014;9(6):407–409.

    Article  Google Scholar 

  10. Gandhi TK, Kachalia A, Thomas EJ, et al. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Annals of Internal Medicine. 2006;145(7):488–496.

    Article  Google Scholar 

  11. Wahls T, Haugen T, Cram P. The continuing problem of missed test results in an integrated health system with an advanced electronic medical record. Jt Comm J Qual Patient Saf. 2007;33(8):485–492.

    Article  Google Scholar 

  12. Wahls TL, Cram PM. The frequency of missed test results and associated treatment delays in a highly computerized health system. BMC Fam Pract. 2007;8:32.

    Article  Google Scholar 

  13. Singh H, Thomas EJ, Mani S, et al. Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential? Arch Intern Med. 2009;169(17):1578–1586.

    Article  Google Scholar 

  14. Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record-based settings. JAMA Intern Med. 2013;173(8):702–704.

    Article  Google Scholar 

  15. Cutrona SL, Fouayzi H, Burns L, et al. Primary care providers’ opening of time-sensitive alerts sent to commercial electronic health record InBaskets. J Gen Intern Med. 2017;32(11):1210–1219.

    Article  Google Scholar 

  16. Dalal AK, Schnipper JL, Poon EG, et al. Design and implementation of an automated email notification system for results of tests pending at discharge. J Am Med Inform Assoc. 2012;19(4):523–528.

    Article  Google Scholar 

  17. Dalal AK, Roy CL, Poon EG, et al. Impact of an automated email notification system for results of tests pending at discharge: a cluster-randomized controlled trial. J Am Med Inform Assoc. 2014;21(3):473–480.

    Article  Google Scholar 

  18. Singh H, Vij MS. Eight recommendations for policies for communicating abnormal test results. Jt Comm J Qual Patient Saf. 2010;36(5):226–232.

    Article  Google Scholar 

  19. Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med. 2010;362(12):1066–1069.

    CAS  Article  Google Scholar 

  20. Callen J, Georgiou A, Li J, Westbrook JI. The impact for patient outcomes of failure to follow up on test results. how can we do better? EJIFCC. 2015;26(1):38–46.

    PubMed  PubMed Central  Google Scholar 

  21. Were MC, Li X, Kesterson J, et al. Adequacy of hospital discharge summaries in documenting tests with pending results and outpatient follow-up providers. J Gen Intern Med. 2009;24(9):1002–1006.

    Article  Google Scholar 

  22. Harris M, Zalis M. QPID: building clinical applications on top of EMR semantic search. radiological society of north america 2011 scientific assembly and annual meeting, November 26–December 2, 2011. http://archive.rsna.org/2011/11012779.html. Accessed January 30, 2018.

  23. Chow SC, Shao J, Wang H. Sample size calculations in clinical research. New York: Marcel Dekker; 2003.

    Google Scholar 

  24. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27.

    CAS  Article  Google Scholar 

  25. McGreevy KM, Lipsitz SR, Linder JA, Rimm E, Hoel DG. Using median regression to obtain adjusted estimates of central tendency for skewed laboratory and epidemiologic data. Clin Chem. 2009;55(1):165–169.

    CAS  Article  Google Scholar 

  26. Georgiou A, McCaughey EJ, Tariq A, et al. What is the impact of an electronic test result acknowledgement system on emergency department physicians’ work processes? A mixed-method pre-post observational study. Int J Med Inform. 2017;99:29–36.

    Article  Google Scholar 

  27. Roy CL, Rothschild JM, Dighe AS, et al. An initiative to improve the management of clinically significant test results in a large health care network. Jt Comm J Qual Patient Saf. 2013;39(11):517–527.

    Article  Google Scholar 

  28. Lacson R, Prevedello LM, Andriole KP, et al. Four-year impact of an alert notification system on closed-loop communication of critical test results. AJR Am J Roentgenol. 2014;203(5):933–938.

    Article  Google Scholar 

  29. Lacson R, O'Connor SD, Sahni VA, et al. Impact of an electronic alert notification system embedded in radiologists’ workflow on closed-loop communication of critical results: a time series analysis. BMJ Qual Saf. 2016;25(7):518–24.

    Article  Google Scholar 

  30. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: Implications for patient safety and continuity of care. JAM. 2007;297(8):831–841.

    CAS  Article  Google Scholar 

  31. O'Leary KJ, Liebovitz DM, Feinglass J, et al. Creating a better discharge summary: improvement in quality and timeliness using an electronic discharge summary. J Hosp Med. 2009;4(4):219–225.

    Article  Google Scholar 

  32. Walz SE, Smith M, Cox E, Sattin J, Kind AJ. Pending laboratory tests and the hospital discharge summary in patients discharged to sub-acute care. J Gen Intern Med. 2011;26(4):393–398.

    Article  Google Scholar 

  33. Dinescu A, Fernandez H, Ross JS, Karani R. Audit and feedback: an intervention to improve discharge summary completion. J Hosp Med. 2011;6(1):28–32.

    Article  Google Scholar 

  34. Singh H, Thomas EJ, Sittig DF, et al. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain? Am J Med. 2010;123(3):238–244.

    Article  Google Scholar 

  35. Babbott S, Manwell LB, Brown R, et al. Electronic medical records and physician stress in primary care: results from the MEMO study. Journal of the American Medical Informatics Association : JAMIA. 2013;21:e100–e106.

    Article  Google Scholar 

  36. Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 2016;91(7):836–848.

    Article  Google Scholar 

  37. Davis Giardina T, Singh H. Should patients get direct access to their laboratory test results? an answer with many questions. JAMA. 2011;306(22):2502–2503.

    Article  Google Scholar 

  38. Giardina TD, Modi V, Parrish DE, Singh H. The patient portal and abnormal test results: an exploratory study of patient experiences. Patient experience journal. 2015;2(1):148–154.

    PubMed  PubMed Central  Google Scholar 

  39. Dalal AK, Schnipper JL. Care team identification in the electronic health record: a critical first step for patient-centered communication. J Hosp Med. 2016;11(5):381–385.

    Article  Google Scholar 

  40. Dalal AK, Schnipper J, Massaro A, et al. A web-based and mobile patient-centered “microblog” messaging platform to improve care team communication in acute care. J Am Med Inform Assoc. 2017;24(e1):e178–e184.

    PubMed  Google Scholar 

  41. Hyman JL, Luks HJ, Sechrest R. Online professional networks for physicians: risk management. Clin Orthop Relat Res. 2012;470(5):1386–1392.

    Article  Google Scholar 

  42. Ayele RA, Lawrence E, McCreight M, et al. Study protocol: improving the transition of care from a non-network hospital back to the patient's medical home. BMC Health Serv Res. 2017;17(1):123-017-2048-z.

    Article  Google Scholar 

  43. Sittig DF, Singh H. Improving test result follow-up through electronic health records requires more than just an alert. J Gen Intern Med. 2012;27(10):1235–1237.

    Article  Google Scholar 

Download references

Acknowledgments

This study was supported by a grant from CRICO/Risk Management Foundation of the Harvard Medical Institutions. The funding agency played no role in the conduct of the study, collection, management, analysis, and interpretation of the data, or the preparation, review, or approval of the manuscript. Dr. Dalal had full access to all the data in the study and takes responsibility for the integrity and the accuracy of the data analysis.

Funding

This study was supported by a grant from CRICO-Risk Management Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anuj K. Dalal MD.

Ethics declarations

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Electronic supplementary material

ESM 1

(DOCX 17 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dalal, A.K., Schaffer, A., Gershanik, E.F. et al. The Impact of Automated Notification on Follow-up of Actionable Tests Pending at Discharge: a Cluster-Randomized Controlled Trial. J GEN INTERN MED 33, 1043–1051 (2018). https://doi.org/10.1007/s11606-018-4393-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11606-018-4393-y

KEY WORDS

  • tests pending at discharge
  • patient safety
  • health information technology