BACKGROUND

In 2009, the Accreditation Council for Graduate Medical Education (ACGME) updated internal medicine residency program guidelines to recommend that internal medicine residencies increase ambulatory training time and develop training schedules to reduce conflicts between inpatient and outpatient responsibilities.1 These recommendations were the culmination of two decades of effort by many professional societies—including the Society of General Internal Medicine, the American College of Physicians, the Association of Program Directors in Internal Medicine (APDIM), the Alliance for Academic Internal Medicine, and the American Board of Internal Medicine (ABIM)—calling for a major redesign of ambulatory resident training to better prepare residents for independent outpatient practice.2,3,4,5,6,7,8,9,10,11,12,13

Although not formally required by the ACGME, many internal medicine residency programs interpreted the updated 2009 guidelines as an implicit recommendation for the implementation of “block” ambulatory scheduling, also known as “X + Y” scheduling,12 where residents have X weeks of uninterrupted inpatient rotations, followed by Y weeks of uninterrupted outpatient rotations. This was a marked transition away from traditional ambulatory scheduling, in which residents are expected to conduct primary care continuity clinics for one half-day per week irrespective of other concurrent clinical responsibilities. Traditional scheduling was thought to create a conflict between continuity clinic and inpatient responsibilities, resulting in a disjointed ambulatory training experience and negative resident attitudes towards ambulatory training.12 Additionally, many programs found it challenging for residents to attain the minimum required clinic sessions per ACGME in the traditional scheduling model, and wanted to create scheduling models that improved the continuity of care residents had with their patient panel.14 Therefore, many programs were eager to pilot alternate training models. The ACGME-sponsored Education Innovation Project (EIP), which was a 10-year pilot project initiated in 2006 that allowed 21 high-performing internal medicine residency programs to operate under 40% fewer requirements in order to foster innovation in accreditation, facilitated early adoption of block scheduling by participating programs. By 2015, nearly half (43%) of all internal medicine residency programs—including those without EIP support—had adopted block scheduling.14

Despite the popularity and prevalence of block scheduling, it is unclear whether this strategy has had the intended effect on residency-related outcomes. Thus, the objective of our study was to conduct a high-quality synthesis of the current literature to assess the impact of block ambulatory scheduling on residency-related outcomes, including resident satisfaction with ambulatory training, resident-perceived conflict between inpatient and outpatient responsibilities, continuity of care, ambulatory training time, patient satisfaction, and patient health outcomes compared to traditional ambulatory scheduling.

METHODS

Data Sources and Searches

We systematically searched Ovid MEDLINE, Ovid MEDLINE InProcess, EBSCO CINAHL, EBSCO ERIC, and the Cochrane Library (Cochrane Database of Systematic Reviews, Cochrane Database of Reviews of Effect and Cochrane Central Register of Controlled Trials), from inception through March 2017 for studies of residency programs comparing traditional to block ambulatory scheduling. Subject headings and text words for the concepts of graduate medical education, internship, or residency were combined with those for care setting (ambulatory care, ambulatory care facilities, hospital outpatient clinics, primary health care) or scheduling (continuity of patient care, personnel staffing and scheduling, appointments and schedules, rotations, clinics or schedules, including block/random/4 + 1/6 + 2 schedules). Reference lists of included articles were hand-searched to identify additional eligible studies. The search strategies are provided in detail in Online Appendix 1.

Study Selection

Two authors (A.D. and H.L.) independently conducted title/abstract and full-text review. Disagreements were resolved through discussion and consensus. Inclusion criteria were (1) full text in English; (2) study population included residents in primary care specialties, which we defined as family medicine, pediatrics, internal medicine, and obstetrics/gynecology; (3) the study evaluated a difference between traditional ambulatory scheduling (defined as primary care clinic scheduled weekly, including during inpatient rotations) and block ambulatory scheduling (defined as primary care clinic scheduled only during dedicated longitudinal primary care or other specialty outpatient rotations, with rotations lasting anywhere from 1 week to 1 year in duration), (4) the presence of any type of comparison group as a control (i.e., pre-post, historical, or concurrent comparison groups were all acceptable); and (5) at least one of the following outcomes of interest was reported: resident satisfaction with ambulatory training, resident-perceived conflict between inpatient and outpatient duties, continuity of care, change in ambulatory training time (i.e., number of scheduled resident continuity clinic sessions), patient satisfaction, and/or patient health outcomes.

Data Extraction and Quality Assessment

Using a standardized data extraction form, two reviewers (A.D. and H.L.) independently extracted data on the setting, number of participants, definitions of traditional and block ambulatory scheduling, and outcomes of interest as described above. Disagreements were resolved through discussion and consensus. If consensus could not be achieved, a third author (O.K.N.) reviewed discrepancies. Corresponding authors were contacted by e-mail for missing data, with up to three attempts at contact made per study.

For resident satisfaction, we extracted data for the survey item or average composite score for multiple survey items that most closely matched the question “What is your overall satisfaction with ambulatory training?” Similarly, for resident-perceived conflict, we extracted data for the survey item that most closely matched the question “Does the current schedule minimize conflict between inpatient and outpatient responsibilities?” Both outcomes were assessed across all studies using each study’s own 5-point Likert scales. To facilitate comparison across studies, we calculated the effect size—the absolute difference in scores between comparison groups divided by the standard deviation associated with that measure—when possible.15 Generally, values of ≥ 0.2, ≥ 0.5, and ≥ 0.8 are considered small, medium, and large effect sizes respectively.15, 16 If standard deviations were not reported, we calculated them from reported 95% confidence intervals when possible.

We extracted data on continuity of care both for “physician continuity,” defined as the number of total visits during which a physician saw a patient in his or her own panel, and “patient continuity,” defined as the number of total visits during which a patient was seen by his or her own physician. We calculated the percent difference in continuity as the difference in visit proportions with block scheduling compared to traditional scheduling for both measures of continuity.

For ambulatory training time, we extracted data on the number of clinic sessions that occurred during the study period. Since studies varied in the length of follow-up time, we calculated the percent change in clinic sessions with block versus traditional scheduling to facilitate comparison across studies.

For patient satisfaction, we extracted the reported mean overall patient satisfaction survey scores. Because none of the studies reporting this outcome used the same or comparable surveys, we calculated the percent change in patient satisfaction with block versus traditional scheduling to facilitate comparison across studies.

For patient health outcomes, we extracted data on any preventive or chronic disease measures that were reported in included studies (e.g., proportion of patients with diabetes with LDL < 100). Because reported patient health outcomes varied widely across studies, we qualitatively described these results in the manuscript text.

Of note, there were four studies that assessed the effect of block scheduling on different outcomes within subgroups of a twelve-site cohort of programs participating in the ACGME Education Innovations Program (EIP).17,18,19,20 In these studies, the subgroups were traditional scheduling, combination block and traditional scheduling, and block scheduling. Only the data for the block scheduling group was extracted.

We assessed the quality of included studies using both the Medical Education Research Study Quality Instrument (MERSQI) and the Newcastle Ottawa Scale for Education (NOS-E) for methodological quality assessment of educational intervention studies.21, 22 Because of minimal variation in NOS-E scores, we included only MERSQI scores in the main manuscript text; details of the NOS-E scoring are available in Online Appendix 2.

Data Synthesis and Analysis

For data synthesis and analysis, we grouped studies by those of programs with “short” ambulatory blocks of 1-week duration. “Other” ambulatory blocks were defined as ambulatory blocks lasting ≥ 3 weeks. These definitions were based on face validity, given the lack of a uniform standard duration for ambulatory blocks among residency training programs. In addition, grouping together programs for analysis that have ambulatory blocks lasting ≥ 3 weeks to 1 year in duration has previously been done in the literature.17,18,19, 23

Several issues precluded the performance of a meta-analysis, including heterogeneity in study settings, block scheduling interventions, and outcome measurements; statistical heterogeneity; and missing data needed for meta-analysis of outcomes data. Thus, aside from a subgroup meta-analysis of resident satisfaction scores among programs implementing “short” ambulatory blocks for which we report a pooled standard mean difference (equivalent to a pooled estimate of effect size), we otherwise focused on qualitatively synthesizing the results for specific outcomes of interest and methodological quality of studies.

For tests of statistical significance, we included reported p values when available from the original text of included studies. If p values were not reported, we calculated p values using two-sample tests for comparison of means or proportions as appropriate to the reported outcome measure when adequate data were available either in the study text or through author correspondence. We conducted all statistical analyses using Stata 12.1 (StataCorp, College Station, TX).

RESULTS

Of 8139 articles identified from our search strategy, 39 articles met criteria for full-text review, and 11 were included in our final analysis (Fig. 1).17,18,19,20, 23,24,25,26,27,28,29 Although we searched for studies across various primary care specialties, all identified studies took place within internal medicine residency programs in the USA between 2008 and 2016 (Table 1).

Figure 1
figure 1

Study selection flowchart.

Full size image
Table 1 Characteristics of Included Studies (n = 11 studies)
Full size table

Of the 11 included studies, 7 were single-site studies.23,24,25,26,27,28,29 The remaining 4 studies were multi-site studies that included the twelve-site cohort of programs participating in the EIP.17,18,19,20, 23 Two of the single-site studies took place in programs that were also part of this larger twelve-site cohort.28, 29 One study received ACGME EIP support for block scheduling implementation and evaluation but was not part of the Francis et al. evaluations.23 Overall, about half of the studies were pre-post evaluations (n = 5).23, 24, 27,28,29 One single-site study25 and the four studies from the twelve-site cohort had concurrent comparison groups17,18,19,20; one study used a historical comparison group.26

Block ambulatory scheduling strategies were heterogeneous, and study sample sizes ranged from 38 to 463 residents with between 5 and 12 months of study follow-up time (Table 1).

Resident Satisfaction

Seven studies assessed changes in resident satisfaction with ambulatory training associated with block versus traditional ambulatory scheduling (Table 2).17, 23,24,25, 27,28,29 Five studies found improvements in resident satisfaction with ambulatory training following the implementation of block clinic scheduling.20, 24, 25, 27, 28 Three of these five studies were of programs with short ambulatory blocks,24, 25, 27 which showed the largest and statistically significant changes (p ≤ 0.05) in resident satisfaction (effect size range + 0.7 to + 0.9, subgroup meta-analysis with pooled standard mean difference of + 0.83, 95% CI + 0.59 to + 1.08, p < 0.001; I2 = 0.0%). One study28 of the other ambulatory block designs found an increase in resident satisfaction (0.9 point mean increase on a 5-point Likert scale, p = 0.02); however, there were no statistically significant changes in resident satisfaction among the rest of the “other” ambulatory block design studies.

Table 2 Effect of Block Scheduling on Residency-Related Outcomes
Full size table

Conflict Between Inpatient and Outpatient Responsibilities

Five studies assessed change in resident-perceived conflict between inpatient and outpatient responsibilities with block scheduling (Table 2).17, 23,24,25, 28 All five studies demonstrated statistically significant improvements in this outcome; i.e., all showed statistically significant reductions in resident-perceived conflict between inpatient and outpatient responsibilities with a range of effect sizes from 0.3 to 2.6 for studies with available data (n = 4).17, 24, 25, 28 Two of these five studies were evaluations of programs with short ambulatory blocks, which showed the largest improvements in perceived inpatient-outpatient conflict (effect sizes 1.4 and 2.6, p ≤ 0.001 and p ≤ 0.01 respectively).24, 25 The three remaining studies did not have short ambulatory blocks,17, 23, 28 but all showed statistically significant improvements in perceived conflict as outlined in Table 2.

Care Continuity

Four studies assessed changes in physician continuity of care with block scheduling, i.e., the proportion of visits in the study period during which a resident physician saw his or her own assigned primary care patients (Table 2).19, 23, 26, 29 Of these, three19, 26, 29 demonstrated worse physician continuity (− 10.2 to − 15.0% of visits with own assigned patients), while only one study28 found an improvement in continuity (+ 14.0% of visits) with block scheduling. Only one study26 took place in a program with a short ambulatory block.

Four studies evaluated changes in patient continuity of care with block scheduling, i.e., the proportion of visits in the study period during which a patient saw his or her assigned resident physician.19, 26, 28, 29 There was an inconsistent effect of block scheduling on patient continuity, with two studies19, 28 reporting improved continuity (+ 14.0 to + 35.5% visits with own physician, Table 2), and two studies26, 29 reporting worse continuity (− 8.7 to − 10.2% visits with own physician). Only one study took place in a program with a short ambulatory block.26

Ambulatory Training Time

Five studies assessed changes in ambulatory training time, i.e., changes in the number of scheduled resident continuity clinic sessions with block scheduling (Table 2).19, 23, 24, 26, 27 Three studies were conducted in programs with short ambulatory blocks,24, 26, 27 and had the largest relative increases in ambulatory time (range, + 32.0 to + 66.7% more clinics compared to traditional scheduling). The remaining two studies19, 23 were conducted in programs without short ambulatory blocks. Of these, one23 found a modest increase in ambulatory time (+ 11%) and one19 found a decrease in ambulatory time (− 13%).

Patient Satisfaction

Three studies assessed changes in patient satisfaction with block scheduling, using a variety of measurement tools (Table 2).18, 28, 29 No studies found any changes in patient satisfaction with transitioning to block from traditional ambulatory scheduling.

Patient Health Outcomes

Three studies evaluated the effect of block scheduling on various preventive and chronic disease management quality metrics as surrogates for patient health outcomes.20, 28, 29 Each study assessed between 5 and 20 metrics. Only one patient health outcome metric was reported across all three studies: the proportion of patients with diabetes with a LDL serum level < 100 mg/dL. There was no consistent effect of block scheduling on LDL serum levels, or on any other patient health outcome measure.

Quality Assessment of Study Methods

Most studies were of fair to moderate quality by the MERSQI criteria22 due to lack of a concurrent control group (n = 5)23, 24, 27,28,29; being conducted at a single site (n = 7)23,24,25,26,27,28,29; the use of resident-reported rather than “objective” outcomes (n = 4)17, 23, 24, 27; the use of primarily descriptive analyses (n = 1)23; and lack of assessment of patient health outcomes (n = 8)17,18,19, 23,24,25,26,27 (Table 3). NOS-E scores ranged from 2 to 3 out of a possible maximum of 6, with 91% of studies (10 of 11 studies) with the same score of 2, suggesting overall fair methodologic quality (Appendix Table 2).

Table 3 Quality Assessment of Study Methodsa
Full size table

DISCUSSION

In this systematic review, we identified 11 studies comparing the effect of block to traditional ambulatory scheduling on various residency-related outcomes. Included studies were generally of fair to moderate methodologic quality. Overall, block scheduling resulted in moderate to large improvements in resident satisfaction with ambulatory training and in resident-perceived conflict between inpatient and outpatient responsibilities, as well as large relative increases in the number of clinic sessions scheduled for ambulatory training time. However, these gains may have come at the expense of worsened care continuity, with the majority of studies that examined physician continuity of care reporting decreases in continuity.19, 26, 29

One of the major initial motivations for implementing block scheduling was to improve continuity for residents in their primary care practices14; therefore, it is important to recognize that “blocking” resident clinic time was not shown to improve residents’ accessibility to their assigned patients. There are many systems-level factors that residents have previously identified that affect continuity of care, including clinic triage scripts that deemphasize continuity, residents’ inefficient use of non-physician care resources, and an imbalance of clinic scheduling that does not favor continuity over acute visits.30 Therefore, there are certainly other factors which could have affected continuity of care in these studies. However, with residents spending extended weeks at a time entirely removed from the ambulatory setting, it is not surprising that continuity may have been effected. It has become necessary in the block system for residents on their ambulatory block to see the patients of colleagues who are not on their ambulatory block, which could decrease the overall proportion of visits that residents spend seeing patients from their own panel.

Of note, only one26 of the studies that examined physician continuity of care had a short ambulatory block; therefore, it is not clear if short ambulatory blocks in general result in decreased physician continuity of care. There were other differences noted in the short ambulatory block studies. Programs with short ambulatory blocks had a larger magnitude of improvement in ambulatory training time and reduction in resident-perceived conflict between inpatient and outpatient responsibilities compared to those without short ambulatory blocks. Resident satisfaction was significantly improved with implementation of short ambulatory block designs. It was not clear that resident satisfaction changed with implementation of “other” designs. While one “other” ambulatory study28 did show an increase in resident satisfaction, that study implemented the so-called long-block, which was 12 months of clinic implemented in the 17th–28th months of residency. The long-block’s unique design may have caused its results to be somewhat disparate from the rest of the “other” ambulatory block designs.

The short ambulatory block designs showing different results from the “other” ambulatory designs was unexpected. One explanation for these differences could be the presence of the EIP. All of the “other” ambulatory block design studies17,18,19,20, 23, 28, 29 included programs that were enrolled in the ACGME-sponsored EIP. These programs were high-performing by definition of their inclusion in the EIP. Therefore, these programs may have had residents more satisfied with their ambulatory training in general, with less inpatient and outpatient conflict built into their scheduling, and with higher numbers of ambulatory sessions already in place. Therefore, drastic improvements in these outcomes may have been less likely to be seen. In light of this confounding factor, it is not possible to state that the short ambulatory block design is superior to other ambulatory block designs.

An additional consideration for programs looking into block scheduling is the potentially prohibitive cost of administrative resources needed to implement block scheduling. The 2015 APDIM survey found that one in five internal medicine residency program directors did not have the needed administrative resources to implement a change to block scheduling.14 In our systematic review, all of the programs without short ambulatory blocks (including three single-site studies23, 28, 29 and all programs in the multi-site Francis et al studies17,18,19,20) received EIP support for the implementation of block scheduling. There may be fewer costs and challenges associated with implementing short ambulatory blocks, but this information was not explicitly reported in any of the studies in this review. Therefore, when considering the transition to block scheduling, program leadership would do well to consider the administrative resources necessary, as well as possible methods for ensuring continuity of care within each program’s specific clinic system, prior to the implementation of block scheduling.

Our findings should be interpreted in light of several limitations. First, though we conducted a systematic search, we may have overlooked articles in the gray literature. Second, the quality of studies was limited by the fact that most were observational single-site studies. Although multi-site randomized controlled trials are challenging to conduct for medical education interventions, this limitation could be addressed in future studies through initiatives by professional societies such as the ACGME, AAIM, or ABIM to coordinate multi-site pragmatic and/or cluster-randomized RCTs, if this area of investigation remains a priority. Third, few studies assessed the effect of block scheduling on patient satisfaction, chronic disease management, or preventive care measures, limiting any robust conclusions on the effect of block scheduling on patient-related outcomes. Fourth, there was marked heterogeneity among included studies in study setting, types of block scheduling interventions, and outcome measurement tools. We addressed heterogeneity in outcome measurements by standardizing our reporting of outcomes to use effect sizes and percent differences when possible to facilitate comparisons. For future studies, this limitation could be addressed with coordinated multi-site evaluations of curricular innovations that plan for use of the same standardized and/or validated tools for evaluation across sites. Finally, this body of literature is limited because it largely ignores the impact block scheduling may have had on inpatient residency–related outcomes.

CONCLUSIONS

In conclusion, there is fair to moderate quality evidence to support that block scheduling results in marked improvements in resident satisfaction, perceived conflict between inpatient and outpatient responsibilities, and ambulatory training time but possibly worsens physician care continuity. Residency programs considering the transition to block ambulatory scheduling should weigh the potential benefits of block scheduling with potential tradeoffs in continuity and costs of administrative resources needed for the transition.