BACKGROUND

In view of concerns for patient safety and quality of care, the Institute of Medicine (IOM) and Accreditation Council for Graduate Medical Education (ACGME) have increased requirements for trainee supervision, including that an attending-level supervisor be immediately available during all shifts.1,2 In response, many academic programs have implemented an overnight academic hospitalist (OAH) program in order to improve clinical care and to meet these supervisory requirements.3,4 However, there has been significant variation in implementation practices, with 25–61 % of hospitalist programs including an OAH model and less than 40 % of OAH positions having defined supervisory roles with house staff.5,6

Despite an implicit belief that on-site supervision improves patient safety and outcomes, there is little evidence describing the influence of an OAH program on patient-level outcomes.3,7,8 In light of the additional ACGME restrictions on house staff shift length and duty hours, more medicine and specialty programs are expected to adopt OAH programs to maintain overnight patient care responsibilities.5 A formal assessment of a defined OAH model will assist implementation efforts in other programs by investigating the impact on patient outcomes and revenue generation. A more thorough understanding of the impact of the OAH on clinical care outcomes, resident supervision, and revenue generation will determine whether the role contributes to the mission of medicine departments and whether the position is financially sustainable in the academic setting.

In 2012, our hospital implemented an OAH program, with explicit patient care and resident education responsibilities. The principal responsibility of the OAH was providing on-site supervision of medicine house staff. In this study, we sought to (1) compare patient-level outcomes before and after implementation of the OAH program, and (2) describe OAH revenue generation and contributions to patient care during the first year of the program.

METHODS

Study Design

To evaluate the impact of the new OAH program implemented in September 2012 on patient-level outcomes, we compared outcomes before and after implementation by conducting a retrospective medical record review of all patients admitted to the internal medicine service at our hospital between April 1, 2011, and May 31, 2014. Billing records were searched to obtain OAH physician charges billed and collected during assigned shifts for 1 year following initiation of the program. To determine clinical tasks performed by OAHs, we conducted a prospective assessment using survey-based daily work logs for 1 year following implementation (October 2012–September 2013). This study was approved by our institutional review board as minimal risk/exempt research.

Study Setting

The study was conducted at the Penn State Hershey Medical Center, a 501-bed university-based acute care hospital in central Pennsylvania. The 64 internal medicine beds are divided between a general medicine unit and intermediate care unit, with nurse-to-patient ratios of 1:4 and 1:3, respectively. The medicine residency program includes 69 internal medicine residents and 14 combined internal medicine-pediatrics residents. From 7:00 a.m. to 7:00 p.m., three teaching teams provided care for patients admitted to the resident-covered medicine service. These teams comprised one academic hospitalist, one second/third-year “senior” resident, two first-year residents, and two or more medical students. Prior to September 2012, patients on the teaching service between 7:00 a.m. and 7:00 p.m. were cared for by a single first-year “night float” resident. New patients were admitted by one of two senior residents, who were also available to assist the night float resident. Attending faculty were not present for on-site direct supervision, but primarily provided “oversight” and/or “indirect supervision” (supervising physicians were not physically present but were immediately available by phone or page).9

Intervention

In September 2012, our hospital implemented a year-round OAH program to comply with IOM and ACGME recommendations. The OAH shift was staffed by one hospitalist physician, and coincided with the hours of the night float residents and nursing shifts (7:00 a.m. to 7:00 p.m.). For the first 9 months, the role was staffed by a rotating pool of four hospitalists, with a reduction in July 2013 to a two-physician rotation (26 weeks/year). Within the first month of implementation, the department of medicine leadership developed explicit responsibilities for the OAH role (Table 1). Primary responsibilities included on-site direct house staff supervision for all clinical duties, specifically patient admissions. Secondary roles included emergency support for the intensive care unit (ICU) team when the ICU fellow was unavailable, and patient admissions to the medical service during high-volume periods. The OAH was responsible for handoffs to daytime hospitalists to maintain continuity of care.

Table 1 Defined Responsibilities for the Role of Overnight Academic Hospitalist (2012–2014)
Full size table

Inclusion and Exclusion Criteria for Patients

All patients aged 18 and older admitted to the general medicine service between April 1, 2011, and August 26, 2012 (pre-implementation cohort), and between August 27, 2012, and May 31, 2014 (post-implementation cohort) between the hours of 7:00 p.m. and 6:59 a.m. were eligible for inclusion. The admission order, which is required for each new patient admission and is placed by the admitting medicine team, was used as the time-point variable for inclusion in the nighttime cohort. We excluded patients who were admitted during nights when an OAH was not present due to illness (19 nights).

Study Outcomes

We evaluated each patient admission for five primary outcomes specified a priori: (1) in-hospital mortality; (2) 30-day hospital readmission, defined as any readmission to any service in our hospital within 30 days of discharge; (3) hospital length of stay (LOS); (4) upgrades in care to the ICU, defined as any transfer to a higher level of care from the acute or intermediate care unit to the ICU during the night of admission or; (5) upgrades in care to the ICU during the hospitalization. Patient-level covariates hypothesized to influence the primary outcomes included age, gender, race, International Classification of Diseases codes (required to determine the major diagnostic category, or MDC), and severity-of-illness scores. Severity of illness was defined by the All Patient Refined Diagnosis Related Groups (APR-DRG) weight, a metric that has been used in prior studies as a measure of severity of illness and mortality risk, and predictors of readmission risk.10,11 To capture revenue generation by the OAH, physician billing data from the first 365 OAH-staffed shifts following implementation were collected for (1) all patient encounters for physician services (e.g., initial hospital care and consultations, discharges, and critical illness) and (2) procedures. Data were obtained from our hospital’s clinical data warehouse, which includes data from the electronic medical record (EMR), bed tele-tracking system, and billing records.

To capture data not available from the EMR or billing records, each OAH was asked to complete an electronic “work log” immediately following his/her shift (September 2012–October 2013). The log template was developed for the purposes of this study (Appendix 1), and quantified the following tasks: (1) number of patient admissions before and after midnight, (2) consultations before and after midnight, (3) cardiac arrest/rapid response events, (4) procedures, and (5) time spent in house staff supervision. Used in our prior studies, this near-time data collection method was chosen in order to obtain more accurate data than could be retrieved by remote-time surveys.12,13

Statistical Analysis

Comparisons between night admissions before and after OAH implementation for demographic variables and covariates were made using Chi-square tests and two-sample t tests. Comparisons between night admissions before and after OAH implementation for dichotomous outcomes (upgrades in care, in-hospital mortality, 30-day readmissions) were assessed using logistic regression and for continuous outcomes (LOS) using linear regression; LOS was log-transformed to meet the assumptions of linear regression. Covariates controlled for in the logistic or linear regression model included age, gender, race, MDCs, APR-DRG severity-of-illness and mortality scores, and time of admission. Model assumptions were checked with residual plots. To adjust for underlying temporal changes independent of the intervention, an interrupted time series model was employed. This analysis used a segmented linear regression on the rate over time of dichotomous outcome variables and on the average over time of continuous outcome variables. Finally, to address the potential threat of inaccuracy in the “admit order time” during change-of-shift times, we performed a sensitivity analysis wherein we excluded patients admitted during the first hour (7:00–7:59 p.m.) and last hour (6:00–6:59 a.m.) of the night shift in both cohorts. All analyses were performed using SAS (Version 9.4; SAS Institute Inc., Cary, NC, USA) and Stata (IC-8; StataCorp LP, College Park, TX, USA) statistical software.

RESULTS

Patient Characteristics

A total of 6484 patients admitted to the medicine service during night shifts (7:00 p.m. to 6:59 a.m.) between April 1, 2011, and May 31, 2014 were identified (Table 2). Of these, 2722 patients (42 %) were admitted before implementation and 3762 (58 %) after implementation of the OAH program. Patient characteristics were similar between the two groups with respect to age (63.5 vs. 63.4 years, p=0.93), gender (55.4 % vs. 53.9 % female, p=0.23), and race (90.2 % vs. 89.2 % white, p=0.39). The APR-DRG mortality risk scores showed no differences (p=0.18), while APR-DRG severity-of-illness scores were lower in the post-implementation cohort (p=0.004).

Table 2 Baseline Characteristics of Patients Admitted to the Internal Medicine Service (n=6484) Before and After Implementation of an Overnight Academic Hospitalist Program
Full size table

Patient-Level Outcomes

Compared to patients admitted before implementation, patients admitted after implementation had similar in-hospital mortality rates (1.1 % vs. 0.9 %, p=0.38), 30-day readmission rates (14.8 % vs. 15.6 %, p=0.39), lengths of stay (3.09 vs. 3.08 days, p=0.86), and upgrades in care to intensive care unit the night of admission (0.4 % vs. 0.7 %, p=0.11) or during hospitalization (3.5 % vs. 4.2 %, p=0.20) (see Table 3. Our results were unchanged when excluding patients admitted during the first and last hour of the night shift. The interrupted time series analyses did not show any differences in patient-level outcomes.

Table 3 Rates of Mortality, 30-Day Readmission, Length of Stay, and Upgrade to Intensive Care Unit (ICU) Before and After Overnight Academic Hospitalist Program Implementation
Full size table

Revenue and Billing

During the first year of the program, OAHs billed for 1209 patient encounters (3.3 per shift), including 803 initial hospital care encounters, 123 observation care encounters, and 76 initial consultations. Overnight academic hospitalists billed 63 total procedures, including 49 central venous lines. A total of $588,445 was charged, $163,751 was paid to the hospital, and 3,324 relative value units (RVUs) were attributed to the program during the first year after implementation (see Table 4).

Table 4 Billed Charges and Revenue Generated by Overnight Academic Hospitalists During First Year of Program Implementation (2012–2013, n=365 shiftsa)
Full size table

Overnight Academic Hospitalist Work Tasks

Of the 365 eligible shifts, logs were completed for 317 (87 %). Overnight academic hospitalists reported staffing and billing a mean 3.44 new patient admissions per shift, 2.6 before midnight and 0.9 after midnight; 57 (1.5 % of 3762 total) admissions were performed independently (Table 5). In total, 156 procedures were supervised or performed independently (mean 0.49 per shift). The most common procedures were central venous catheter insertion (n=102, 65 % of total), arterial line insertion (17 % of total), and abdominal paracentesis (10 % of total). The mean time per shift spent in supervisory tasks with house staff was 237 min. On average, OAHs reported 12 min per shift spent discussing patient care issues with other interprofessional providers (e.g., nursing staff).

Table 5 Overnight Academic Hospitalist Self-Report Work Logs During First Year of the Program (n=317a)
Full size table

DISCUSSION

In this study, we found no differences in patient-level outcomes with respect to in-hospital mortality and 30-day readmission rates, patient length of stay, or upgrades in care to the ICU between periods before and after implementation of an OAH program. Based on billing data, the revenue generated by OAHs was modest and was insufficient to offset the salary expense. These results raise questions regarding the cost–benefit of the OAH program, specifically with regard to patient outcomes and house staff supervision and training.

Overnight academic hospitalists are relatively new to academic medical centers, with significant variation in definition, job description, and implementation.3 Over the past ten years, an increasing number of studies have investigated the impact of hospitalists on outcomes, noting effects primarily in resource utilization and cost reduction, and less so in quality outcomes, patient satisfaction, and provider satisfaction.14–18 Prior attempts to demonstrate an impact of overnight attending-level physicians on patient outcomes have been limited to the ICU, and revealed minimal impact.19,20 To the best of our knowledge, our results are the first to describe patient-level outcomes following implementation of an OAH program for general medicine units. These results fail to provide a rationale for modeling an OAH program based solely on objective improvements in patient-level outcomes. Likewise, the clinical revenue generated from physician billing by the OAHs, although highly dependent on local variables such as patient volume, was insufficient to independently support the position’s salary. While the total clinical and supervisory time accounted for in the OAH surveys was approximately 4 h per 12-h shift, it is unclear whether expansion of OAH roles into other RVU-generating areas is feasible. Furthermore, the diversification of OAH responsibilities carries the risk of limiting their availability to supervise house staff. Future studies in other institutions are needed to validate these findings.

There are several limitations to our study. First, we chose a before-and-after study design, since a concurrent control within our medicine service or a randomized study design would have been impractical, and thus our results were subject to several unmeasured confounders unrelated to the intervention.21 Specifically, quality improvement initiatives occurring during the study period, such as a sepsis alert protocol, may have influenced our results. We indirectly accounted for this factor using the APR-DRG, and we used an interrupted time series to correct for secular trends.10,22 Second, several variables were taken from electronic timestamps (e.g., “admit order time”), which may have been vulnerable to unidentifiable inaccuracies such as delayed order placement. However, our sensitivity analysis showed no differences in studied outcomes between groups. Next, our data lacked sufficient power to detect small differences in infrequent outcomes such as in-hospital mortality or upgrades in care, which might have been detected with a larger sample size. Our ability to accurately account for time spent by OAHs was potentially limited by our instrument design and the inherent biases of self-reported data, specifically social desirability bias. Lastly, our results are from a single academic hospital and OAH model, and may not be generalizable to other medical centers, including both smaller and larger academic programs and those with variable referral patterns.

Contrary to our hypothesis, we found no change in patient-level outcomes following implementation of the OAH. We believe there are several possible explanations for these findings. First, the assessed outcomes may be too infrequent or insensitive to detect the actual impact of supervision. Length of stay and readmission rates are influenced by numerous factors independent of the initial admission decision-making and the infrequent management changes made by overnight teams during patients’ subsequent hospital days. Second, near-time outcomes such as upgrades in care are difficult to interpret, as increased upgrades may demonstrate appropriate care rather than mismanagement of a deteriorating patient. The trend toward increased ICU-upgrade rates may reflect improved situational awareness amongst admitting teams and nursing staff and the contributions of the OAH in decision-making. Lastly, s traditional model of indirect supervision may have been sufficient to provide good patient outcomes.

Implementation of our OAH program required significant financial investment, including two full-time equivalent (FTE) positions at competitive salaries based on published benchmarks.23 Primary revenue generated by OAHs stemmed from billing admissions prior to midnight and from supervising procedures, which overall were infrequent. Compared to daytime academic hospitalists within the general internal medicine division, the physician-work RVUs generated by OAHs were in the lowest tenth percentile, demonstrating the limited revenue-generating nature of the role despite the overall robust volume experienced by the internal medicine service. Although OAHs were not required to bill admissions after midnight, this is not considered a source of lost revenue, as professional billing for these patients was captured by daytime physicians. And while total revenue generation is sensitive to hospital setting, these findings at a large academic hospital with a busy internal medicine service cast doubt regarding the return on economic investment for implementing a relatively expensive “academic-heavy” OAH program in smaller settings, especially with minimal immediate impact on patient-level outcomes.

The IOM and the Clinical Learning Environment Review (CLER) program, as part of the ACGME Next Accreditation System (NAS) recommendations, are promoting continued focus on patient safety, quality improvement, transitions of care, supervision, and duty hours oversight.24–27 While the OAH role contributes to meeting house staff supervision requirements, the nighttime presence of an attending physician is a paradigm shift for graduate medical education.5,28 Night hours have traditionally provided opportunities for house staff physicians to garner autonomy and independence, leading to significant advances in skill development.29 Additionally, indirect supervision by attending physicians requires astute assessment of house staff physicians’ skills, the decision to entrust them with specific activities, and their ability to accurately communicate medical information.30,31 With the transition from traditional time-based training toward competency-based graduate medical education, academic medical centers and residency programs are challenged to provide house staff with the skills to perform the activities necessary for independent, autonomous practice.32 The increasing use of OAHs requires close monitoring of the degree and quality of direct on-site supervision in order to preserve house staff development and growth.3,5,7,28 A deliberate effort to preserve resident autonomy will be needed to secure this essential aspect of house staff education.

The responsibilities of our OAHs represent a prototype for other programs and subsequent research in this area. If, as our results suggest, the creation of an OAH program cannot be justified by improved patient outcomes, the position must then be justified by other means, either through deliberate design optimizing sustained revenue generation or through intentional curricular design and professional development in order to improve the education and maturation of resident physicians. Other benefits not quantified by this study could also be considered, including patient and physician satisfaction, and select patient-reported outcome measures. As Hanson et al. hypothesized, the presence of an OAH may lead to a lower rates of adverse events, medical errors, diagnostic testing, and high-cost bed triage.28 Studies have also demonstrated that inexperienced physicians have higher cost profiles, supporting the hypothesis that an experienced OAH could help reduce expenditures.33 Potential benefits also include enhanced resident education, opportunities for resident physicians to collaborate with attending physicians in medical decision-making, and optimized care delivery process measures (e.g., appropriate medication selection). Investigation of these more granular outcomes affected by on-site supervision may provide a more accurate understanding of the value of the OAH. Measuring these outcomes will likely require time and labor-intensive methods not readily captured in clinical data.

In summary, the implementation of an OAH program at a large, well-staffed academic hospital was not associated with significant changes in patient-level outcomes, and required financial subsidization by hospital leadership. Institutions that are considering adoption of an OAH program will need to balance relative cost with other, less tangible benefits in resident education, patient satisfaction, physician quality of life, and patient outcomes.