INTRODUCTION

Internal medicine (IM) residency programs train approximately 20% of all physicians and one-third of the primary care workforce nationally (1) and are therefore essential to the provision of a high functioning healthcare system. However, as the healthcare system has evolved to adapt a disease prevention model, where the majority of medical care occurs in the ambulatory setting, training programs have maintained a disproportionate footprint in the inpatient setting (2). This has led to a perceived training gap among IM residents in their preparedness for ambulatory practice and ultimately dissuades them from pursuing primary care as a specialty (3), a field where there is a significant shortage of physicians.

IM residency programs have tried to address the urgent public health need for more primary care physicians in the USA by creating primary care tracks embedded within their broader educational programs. These programs have been shown to increase the likelihood of entering a career in primary care (1, 4, 5) and address educational gaps for residents who will spend most of their careers in outpatient practice (6, 7). More recently, the Accreditation Council for Graduate Medical Education (ACGME) has changed their training requirements for IM to include 10 months of ambulatory training (8). However, little is known about how to provide sufficient training in the outpatient setting, especially given the constraints of meeting the needs of teaching hospitals, which remain the centerpiece of GME funding streams. Several themes have emerged from the literature on ambulatory training models for categorical programs: (1) establishment of clear delineation between inpatient and outpatient duties; (2) provision of longitudinal care; and (3) maximizing the number of consecutive weeks of ambulatory time, all improve trainee and patient satisfaction, as well as patient continuity (9,10,11,12,13,14,15). However, there is no standard approach to adapting these themes among the over 100 primary care programs in the country (16), with no information on the effect of different training structures on educational or patient outcomes. Importantly, primary care residents have identified the structure of their training, specifically the lack of time dedicated to primary care and the lack of cohesion among their training blocks, as a major reason for not ultimately pursuing primary care as a career (3).

The aim of this study was to examine a novel approach to primary care training that applies the principles of clinical immersion. Our intervention includes two separate 6-month primary care blocks that alternate with 6-month blocks of standard categorical training during the second and third post-graduate years (Figure 1). We hypothesize that this model will improve the educational experience for trainees and the clinical experience for their patients while still meeting training requirements in the inpatient setting.

Figure 1
figure 1

Comparison of the number of primary care training weeks between the categorical program (C) and primary care immersion training (PC). Shaded cells represent full weeks where ambulatory care is practiced Monday-Friday; asterisks represent single clinic days during outpatient training rotations. Primary care residents experience a 6-month immersive block in primary care during each of the second and third post-graduate years.

Full size image

METHODS

Setting and Participants

Our study is set within a large urban university-affiliated academic medical center serving a largely urban population with high medical and psychosocial complexity. Participants included the 162 interns and residents from our internal medicine residency program. Each year, six residents from each of the junior and senior classes participate in the primary care track, having applied through a separate internship match or joined later in their internship year. Data from this study were derived from the twelve primary care and 86 categorical residents from the class of 2018 and an 11-year historical cohort of primary care track graduates, some of whom completed a non-immersive primary care track (2008–2014) and the rest of whom completed the intervention (2015–2019).

Intervention

We developed an innovative primary care track in 2014 that includes a novel 6-month block of primary care immersion during both the second and third post-graduate years. These immersive blocks alternate with 6-month blocks of standard categorical training (Figure 1). During their time on these immersion blocks, residents work with a faculty member in primary care education leadership to develop an individualized learning plan based on an elective preference survey distributed prior to meeting (see Supplemental Material). Residents spend five afternoons per week in two different continuity clinics and three mornings per week in specialty continuity clinics of their choosing. Specialty clinics are tailored in duration and setting to the individual resident’s goals and may be adjusted throughout the block based on learning needs. Residents also receive didactic and experiential learning in primary care innovation, quality improvement, leadership, and social justice (Figure 2).

Figure 2
figure 2

Sample week during primary care immersion training. A standard week schedule for our primary care training program. Residents spend 3 afternoons per week in their primary continuity clinic, which is assigned at the start of internship. They select a second continuity clinic in a different practice model (private practice, community health center, academic health center, innovative primary care start-ups, the VA, or a primary care subspecialty clinic such as women’s health or obesity medicine). They spend 3 mornings per week in subspecialty clinics based on an individualized learning plan (based on content and number of weeks) and some required experiences (addiction medicine, dermatology, women’s health, musculoskeletal medicine, and geriatrics). They have one morning set aside for follow-up of administrative tasks (notes, phone calls), and protected time for scholarly training in medical education, leadership, social justice, and primary care innovation. Our residents take the first call from Friday at 5 PM until Saturday at 5 PM with a supervising faculty member; this experience includes a Saturday urgent care clinic.

Full size image

Outcomes and Data Collection Elements

Educational Outcomes

  • Perceived comfort with primary care skills: Each year our program distributes an annual program evaluation as part of the ACGME’s common program requirement of continuous improvement. This survey is distributed to all residents at the end of the academic year. To enhance our assessment of ambulatory training, we added the following measures to the survey with a 1–5 scale of increasing agreement in comfort with:

    • Population health management

    • Chronic disease management of type 2 diabetes mellitus and hypertension

    • Ambulatory care skills, including telephone management, running on time in clinic, after-visit task management, and panel management

    • Patient experience, including ability to practice longitudinal care and form relationships

    We compared levels of comfort with ambulatory care between the 12 residents in the primary care track and the 86 residents in the categorical program.

  • Perceived career preparation and overall satisfaction: We developed an alumni survey and distributed it to all graduates of our primary care programs from 2008 to 2019. Alumni from 2008 to 2014 had completed our historical primary care track, which did not include immersive training blocks, and alumni from 2015 to 2019 completed the intervention. We used a 1–5 scale of an agreement to measure the perceived degree of preparation for a career in primary care and overall satisfaction with the training experience.

  • Objective measures of ambulatory educational experience: We compared the following objective ambulatory educational measures between the 12 residents in the primary care track to the 86 residents in the categorical program:

    • Total number of ambulatory clinics during the course of residency: We performed a manual count in the resident scheduling software.

    • Panel size upon graduation, using a standard field in our electronic medical record

    • Panel complexity upon graduation, using a chart audit tool that classifies patients as high, medium, and low risk that residents complete before graduating. Risk level is determined by the graduating resident and is based on the presence of an unstable condition actively being worked up, number and stability of chronic medical conditions, psychosocial complexity that interferes with medical care, a need for an opioid treatment agreement, or any other physician concern that increases risk. We present the average percentage of high-risk patients in each panel.

    • Continuity with the primary faculty preceptor, based on the number of patient encounters billed with the primary preceptor divided by the total number of encounters for that resident in our hospital’s professional fee billing database

    • Percentage of time residents were able to work with an ambulatory subspecialist >6 times

    • Number of home visits completed per year

    • Total number of inpatient training blocks, including on the inpatient general medicine wards, intensive care units, and inpatient specialty wards (e.g., inpatient oncology, cardiology, and solid organ transplant) based on a manual count

Clinical Outcomes

To measure residents’ performance, we used standard clinical performance measures as defined by Healthcare Effectiveness Data and Information Set (HEDIS), including the following: colorectal, breast, and cervical cancer screening rates; diabetes control (percentage of patients with diabetes mellitus with glycated hemoglobin (HbA1C) less than 9%), and hypertension control (percent of patients with hypertension with blood pressure less than 140/90). We queried our population health database to ascertain these rates for each practitioner and compared rates between the 12 residents in the primary care track to the 86 residents in the categorical program.

Analysis

Educational Outcomes

  • Perception of educational experience: We compared mean scores for perceptions of the educational experience for the twelve residents in the intervention to their peers in the standard training program using t-tests of significance.

  • Objective measures of education experience: We compared the number of training experiences and panel characteristics for the six senior residents in the intervention to their senior resident peers in the standard training program using t-tests of significance.

  • Career preparation and overall training satisfaction: We compared median scores on 1–5 scales between alumni who had completed the intervention (2015–2019) to a historical cohort of primary care track alumni who had not (2008–2014)

Clinical Outcomes

We modeled rates of colorectal, breast, and cervical cancer screening and control of type 2 diabetes and hypertension using a log-binomial regression model with generalized estimating equation methods to account for clustering at the level of the physician, with training program as the independent variable and the binary outcome (e.g., screening) as the dependent variable.

All statistical analyses were performed using SAS version 9.3 (Cary, NC).

Ethical Issues

The institutional review board at Beth Israel Deaconess Medical Center approved the protocol as exempt from further review as an educational research project. Participation by individual physicians was voluntary, and all data were kept confidential.

RESULTS

Compared to their peers in the standard categorical training program, residents who participated in the intervention spent more time training in the ambulatory setting (242 vs. 154 general medicine clinics/3 years, p<0.0001, and 14 vs. 6 weeks of ambulatory subspecialty clinics/3 years) and, to a lesser degree, had fewer inpatient training blocks (23 vs. 28 inpatient general medicine weeks/3 years, p<0.001, 17 vs. 21 inpatient specialty weeks/3 years, p<0.001 and 16 vs. 19 intensive care unit weeks/3 years, p<0.001). Both cohorts were able to exceed ACGME training standards in all settings (Table 1).

Table 1 Balance of inpatient and ambulatory training time compared to ACGME standards
Full size table

Compared to their peers in the categorical program, residents who participated in the intervention reported significantly higher levels of comfort with a variety of primary care topics, including population health management (p=0.02), interpreting ambulatory blood pressure monitoring, and incorporating automated blood pressure in patient care (p<0.01). They also reported higher levels of comfort with a set of ambulatory care skills such as telephone management (p=0.002) and panel management (p=0.004), and they participated in significantly more home visits (p=0.0007). They reported being better able to see their patients in follow-up (4.3/5 vs. 2.6/5, p<0.0001). Finally, they had significantly more time working with ambulatory subspecialists, with 91% vs. 41%, p<0.001, reporting being able to work consistently with a specialist, better integration into their primary care clinic site (4.6/5 vs. 3.7/5. p=0.0009), and more satisfaction with the amount of ambulatory training they received (4.3/5 vs. 3.6/5, p=0.008) (Table 2).

Table 2 Educational experience among residents in a standard internal medicine training program vs. an immersive primary care track
Full size table

Compared to residents in the categorical training program, residents participating in the intervention reported higher levels of perceived preparation for a career in primary care (4.4/5 vs. 3.3/5, p<0.0001). Furthermore, we queried an 11-year historical cohort of primary care, 54 of whom completed the traditional primary care track (without a 6-month immersive block) and 24 of whom completed the intervention model. Eighty-five percent of alumni from the historical track and 79% of alumni from the intervention track responded to our survey. Residents, those who had experienced the immersive training model, reported significantly higher satisfaction with their training (5/5 vs. 4/5, p=0.04) and better career preparation compared to those who had not (3/5 vs. 4/5, p<0.001) (Table 3).

Table 3 Alumni perceptions of primary care training from an 11-year historical cohort (2008–2019)
Full size table

Finally, intervention residents performed better on preventive health measures across all domains, with only diabetes control being significant (85 vs. 76%, p<0.001) (Figure 3).

Figure 3
figure 3

Rates of colorectal, breast, and cervical cancer screening; and rates of control of type 2 diabetes mellitus and primary hypertension among patients with type 2 diabetes mellitus in residents who participated in the immersive primary care training program from compared to a standard internal medicine residency program from 2016 to 2019. p values were derived using a log-binomial regression model with a generalized estimating equation to account for clustering at the level of the physician, with time as the independent variable and performance rates as the dependent variable. Clinical performance among residents in a standard internal medicine training program vs. an immersive primary care track on routine population health measures

Full size image

DISCUSSION

In this training intervention nested within a large internal medicine residency program, we found that a novel structure that introduces two 6-month immersive primary care blocks into the second and third post-graduate years significantly increased the amount of ambulatory exposure, both subjective and objective educational outcomes, and likely improved the care of resident patients. This approach to training was achieved with minimal impact on inpatient time and still exceeded ACGME training requirements. Our study is the first of its kind to evaluate the impact of a primary care training program on both educational and clinical outcomes, and, therefore, provides a potential model for primary care training in internal medicine programs around the country.

A robust experience for the subset of residents planning a career in primary care is an essential component of their training. IM residency training is currently structured around the care of acutely ill hospitalized patients, a model developed in the past when most of medical care occurred in the inpatient setting. Furthermore, this model has been perpetuated by the way graduate medical education is funded, where taxpayer dollars are paid directly to teaching hospitals to distribute funds (2) based on the number of resident-staffed hospital beds. This funding structure poses a potential threat to innovation in teaching methods for an evolving healthcare workforce, as it neglects other skills that both general and specialty physicians will need in a modern healthcare system, where most patients are managed using a preventive disease model in the ambulatory setting. The essential principle embedded in our intervention is not different from that which was applied to the original hospital-based model: that trainees require immersion in the environment in which they will practice medicine. Specifically, residents pursuing primary care careers require immersion in ambulatory care to develop the essential skills required of this particular workforce. By providing two 6-month continuous blocks of primary care, we allow trainees to adapt their critical thinking to diseases that evolve over the course of months to years rather than hours to days, without the frequent disruptions that are inevitable when residents move between different clinical settings in a standard alternating call block model (Figure 1). We allow them to form continuous relationships so that they can adapt their clinical knowledge to their patient’s lives outside of the hospital. This model addresses the concerns raised in a national cohort study of primary care trainees, who named the structure of their training programs as the most negative aspect of their training (3).

This model nearly doubles exposure to ambulatory settings in both general and specialty IM, with an educationally insignificant impact on inpatient training in these domains (Table 1). The model was able to significantly increase exposure to the ambulatory setting, where the majority of residents will practice upon completion of training, at the expense of 5 fewer weeks of inpatient general medicine wards and 3 fewer weeks of intensive care unit time over the span of 156 weeks of residency training, while still exceeding ACGME training requirements for both (Table 1). On balance, participants in the primary care track received a fuller perspective of the breadth and depth of IM, arguably a perspective that all residents need, even those not planning careers in primary care.

We conducted a robust analysis of our hypotheses on both subjective and objective measures of educational experience and clinical care. Trainees who participated in the intervention rated their subjective educational and patient care experience on a variety of ambulatory outcomes significantly higher than those in a standard categorical training program. Notably, our intervention is modeled after the prominent healthcare delivery system for ambulatory IM, designed around the principles of primary and secondary prevention, chronic disease management, and limited inpatient care: a population health model. This is the model of care where the majority of graduates will be practicing, even among those planning to specialize, indicating a potential training gap imposed by current residency training models. Participants in the intervention noted significantly higher levels of comfort with population health management than their peers in the standard categorical program. Furthermore, participants also reported increased comfort in specific components of ambulatory care, including chronic disease management and core set of ambulatory care skills. Finally, participants also perceived a significantly improved educational experience, with better integration into their clinic site, career preparation, and overall satisfaction with the amount of ambulatory training. These findings were confirmed among an 11-year cohort of primary care alumni, who reported both increased satisfaction with training and better career preparation.

Perceptions of improved educational and clinical experience were supported by many objective measures. Participants in the intervention had significantly more primary care clinical sessions and larger panel sizes with a non-significant but notable increase in medical complexity. They also were able to attend significantly more ambulatory specialty clinics and home visits. There was an increase in educational continuity, with more opportunities to work with their primary outpatient preceptors, but this was not statistically significant. It is likely that improving educational continuity is beyond the scope of this intervention, as it would have required changes to faculty schedules in addition to resident schedules. Clinical outcomes were also better for all cancer screening and chronic disease measures; however, only diabetes control reached statistical significance. This particular analysis was likely underpowered given the small sample size of residents participating in the intervention and may not completely represent the effect of the program on clinical outcomes.

Our analysis provides a place to start in the work ahead of determining the ideal way to structure primary care training programs, serving as a point of comparison to those with shorter primary care blocks, as is currently the standard. Finding the balance of adequate immersion in ambulatory care and feasibility for residency programs, who answer to the staffing needs of teaching hospitals in addition to the training needs of residents, will be important work moving forward as our healthcare system continues to evolve and orient itself toward prevention of disease in ambulatory settings. Our intervention may also have a wider impact on categorical IM training. The ACGME now recommends 10 months of ambulatory training to stay current with the shifting healthcare system (5). Our model may provide a blueprint for residency programs as they adapt their programming to meet these expectations. However, these standards will be difficult to scale up beyond the primary care programs to the entire IM resident population without significant changes to the funding structure of graduate medical education, where the teaching hospital, and therefore its staffing needs, remains the centerpiece through which GME funds are distributed. Perhaps, if ambulatory training were higher quality for all IM residents, we would reverse secular trends and have more internists seeking primary care careers, and other ambulatory specialties that face shortages of physicians.

Our study is the first of its kind to formally evaluate a primary care training program on both educational and clinical outcomes. We provide our readers with both subjective and objective measures of both categories, as well the perspectives from practicing physicians in our alumni cohort. Our study is limited by our single center study design and our small sample size, which likely resulted in an underpowered analysis, especially in our ability to capture the impact on clinical outcomes. Our treatment assignments were not determined by randomization, limiting our ability to attribute some of the subjective educational outcomes to the different programs. Finally, due to a lack of a validated survey in the literature, we used local expert opinion to develop our constructs which can impact validity, furthermore, some of our outcomes were self-reported and are subject to individual variation in interpretation.

In summary, we found that a novel immersive primary care training program significantly improved both subjective and objective measures of educational experience among residents in a large IM residency program with a minimal impact on inpatient training and no impact on meeting training requirements. We also found an important signal toward improvements in patient care as well. Providing outstanding educational programming to physicians in training while maintaining our contract to provide excellent clinical care to the patients we serve are missions that should be structured to achieve synergy. Increasing the quality and quantity of ambulatory training is one way to achieve such synergy, not only by providing better care during residency training, but also by turning out better prepared physicians that are facile at practicing in the model of healthcare delivery in the present and future. While the healthcare system at large is evolving toward keeping care in the ambulatory setting, the funding structures for medical education must change in parallel to meet the health needs of patients.