This study took place at Mayo Clinic Hospital, Saint Mary’s Campus, a 1,265-bed academic medical center, and involved the Mayo Rochester IM residency program, which includes 144 categorical residents and 24 preliminary residents. We specifically aimed to evaluate the handoff quality for IM residents rotating on general medicine services from October 2013 through January 2014. This site has four general medicine resident services, comprising one postgraduate year 3 (PGY-3) and three postgraduate year 1 (PGY-1) residents (2 categorical, 1 preliminary) per service, rotating monthly. Overnight coverage is provided by two PGY-1 residents, each covering two services, and a night-float supervising PGY-3 resident covering all four services. The day-to-night handoff takes place at 6:00 p.m., at which time PGY-1 residents hand off to one another and PGY-3 residents hand off to one another. The PGY-1 and PGY-3 handoffs occur separately.
To assess the information regarding patient risk provided during the handoff, we utilized the PAR, a validated tool used to quantify physician judgment of the stability of medical patients.12 The PAR has been shown to be reliable and to provide an accurate, objective measurement of a patient’s risk of clinical deterioration over the subsequent 24-hour period. The PAR is a symmetric seven-point scale (1=’Extremely unlikely’, 4=’Neither likely nor unlikely’, 7=’Extremely likely’), where a higher PAR score reflects higher perceived risk. Immediately after the day-to-night handoff, both daytime and overnight PGY-1 and PGY-3 residents were asked to independently assign a PAR score for each patient. For PGY-1 and PGY-3 residents receiving handoff, this score was to be based solely on the information provided from the daytime residents, without supplementation from the electronic medical records or other sources. Residents were instructed not to discuss PAR scores before, during, or after the handoff.
Scores were placed into a private box, and results were entered into a secure database by one of two study authors (JTR or DJK). Retrospective chart review was performed for each patient in order to obtain demographic information (age, sex, hospital day, length of stay, previous ICU stay, previous code (cardiac and/or respiratory arrest), previous rapid response team [RRT] call) and to identify patients who had experienced a clinical deterioration event, defined as RRT calls, ICU transfers, or code calls within 24 hours of handoff. Criteria for calling an RRT included an acute and persistent change in physiologic parameters (heart rate, oxygenation, etc.), signs and symptoms suggestive of myocardial ischemia or stroke, or concern on the part of any care team member regarding the patient’s clinical status. While calling an RRT was not included in the original PAR, at our institution the RRT was developed in an effort to recognize signs of deterioration, and activation of the RRT is a common and preferred method to transfer a patient to the ICU. Therefore, RRT calls were included as a clinical deterioration outcome in this study.
The daily 6:00 p.m. handoff between daytime and nighttime interns was estimated to provide up to 120 unique resident pairings of PAR scores over the duration of the study, exceeding the 44 resident sign-out pairs that would be required to achieve 90 % power to detect a 0.5 SD difference in scores at the 0.05 alpha level. Based on the published findings of the original PAR study, the team structure of our four medicine services, a four-month study period, and recent ICU transfer/Code 45/RRT rates, we estimated that we would have 90 % power to detect an odds ratio of 1.4 for the association between PAR score and occurrence of a clinical deterioration event at the 0.05 alpha level.
This study was reviewed and deemed exempt by the Mayo Clinic Institutional Review Board.
To evaluate the agreement between the handoff giver and receiver, inter-rater reliability was assessed using intraclass correlation coefficients (ICCs) and their corresponding 95 % confidence intervals (CIs). The mean and standard deviation (SD) of PAR scores by the givers and receivers were reported, and paired t tests were used to assess magnitude and direction of disagreement.
The distribution of patient sex, prevalence of prior events, and giver PAR scores were reported as counts and percentages of patients, distinct hospitalizations, and patient-days, respectively. Patient age and length of stay (LOS) were summarized using median and interquartile range (IQR).
Event prevalence was calculated within each PAR score and reported as count (%) of patient-days. There were no observed clinical deterioration events on patient-days assigned a PAR of 7 by PGY-3 givers, so ratings of 6 or 7 were grouped as “6+” a posteriori to allow statistical estimation in subsequent associative analyses.
To assess potential associations between giver PAR score and observed clinical deterioration events, logistic regression models were fit using generalized estimating equations to estimate odds ratios (ORs) of an event within 24 hours of handoff for PAR scores of 6+, 5, 4, 3, and 2 vs. 1. Available patient demographics were adjusted for simultaneously, and repeated assessments of patients were accounted for via an exchangeable correlation structure. Area under receiver operating characteristic curves (AUROCs) were reported to assess performance of PAR score thresholds as predictors of subsequent clinical deterioration.
All analyses described above were performed separately by PGY (1 and 3) and reported as such below. The threshold for significance was set at 0.05 throughout. Calculations were made using SAS statistical software (version 9.3; SAS Institute Inc., Cary, North Carolina).