BACKGROUND

Ambulatory practice and education is a key aspect of training for the nearly 8000 internal medicine (IM) residents across the country.1, 2 Though most of their residency time is spent on inpatient rotations, IM residents are likely, with the exception of those practicing hospital medicine, to spend the majority of their clinical practice in the outpatient setting, where close to 1 billion patient encounters occur annually.3 To date, efforts to measure the quality of care in the outpatient setting have primarily focused on patients seen by post-training staff physicians, although there has been increased interest in assessing the care of patients seen by resident physicians.4,5,6

Prior studies have shown mixed results when comparing patient outcomes of resident and staff primary care physicians (PCPs), with some studies actually finding better outcomes for resident patients.7,8,9,10,11 However, as the demands of the health system grow and resident work hour regulations increase, few studies have examined contemporary primary care outcomes of patients cared for by resident PCPs.12, 13 Additionally, even less is known about the patient experience when receiving primary care from residents compared to staff PCPs.14,15,16 As resident physicians often care for a larger proportion of outpatients from underserved backgrounds compared with staff physicians, understanding whether differences in quality of care exist among resident and staff PCPs may present an opportunity to improve equity of care within academic health centers.17, 18 Identifying areas where quality outcomes of resident and staff PCPs differ may also help guide training directors and primary care health system leaders in how best to optimize the outpatient setting for education and high-quality patient care.19,20,21

In this analysis, we sought to assess outcomes of patients seen in the same clinics by resident and staff PCPs at a large academic medical center in the modern medical education era, and to examine whether differences persist after controlling for patient demographic, clinical, and socioeconomic characteristics, along with practice-based features. The objectives of our analysis were to assess whether differences exist between resident and staff physician PCPs for the following outcomes: (1) chronic disease management, specifically related to diabetes mellitus (DM) and coronary artery disease (CAD); (2) preventive cancer screening including breast, cervical, and colorectal; (3) resource utilization; and (4) patient-reported experience of care.

METHODS

Study Setting and Population

The cohort for this analysis was derived from the Massachusetts General Hospital (MGH) Practice-Based Research Network (PBRN) and included patients receiving longitudinal primary care at MGH. The PBRN includes 16 primary care clinics where resident and staff physicians provide longitudinal outpatient care, including four community health centers. At MGH, resident physicians are supervised by staff physicians within the same clinical spaces and have access to the same resources as staff physicians including care coordination and allied health professionals.22 Patients seen within this network are connected to specific staff physicians using a previously validated algorithm.23 Those patients not attributed to a specific staff physician are then connected with a primary care practice.24 Among patients seen in a given practice but not connected with a specific staff physician, we worked with: (1) the residency program to embed a resident PCP field within the electronic medical record, (2) practice administrative staff to assign patients to the specific resident PCP who followed them, and (3) information technology staff to reassign residents who completed training with new residents taking over the patient panel.

Inclusion criteria for the cohort included patients seen within one of 16 primary care clinics affiliated with MGH between January 1, 2014, and December 31, 2016, who were linked to a specific resident or staff PCP. Excluded patients were those who were not linked to a specific resident or staff PCP, those who died, or those who left the network. The Institutional Review Board at Partners Healthcare approved this study.

Patient Demographic and Clinical Factors

Data was obtained from electronic data sources including registration information for patient demographic characteristics and electronic medical records for problem list and billing code diagnoses, appointment information, laboratory and procedure data, and emergency department (ED) and hospital admission data.25 Patient-level demographic information included age, gender, self-reported race/ethnicity, spoken language preference, insurance status, educational attainment, and poverty level based on zip code–derived median household income (algorithms for which have been previously described).26 Clinical information included the clinic site, patient years in the primary care network, mental illness (including depression), substance use disorder (using billing codes and problem list diagnoses), Charlson Comorbidity Index (an index for medical comorbidity), and Higashi score (a sum of common comorbid conditions).27, 28

Outcomes

For all eligible patients in the cohort, we assessed outcomes using data from the calendar year 2016. This included items derived from Healthcare Effectiveness Data and Information Set (HEDIS) performance measures based upon their role in evaluating quality of care.29 Data were obtained from the Research Patient Data Registry (RPDR), an electronic data repository within the Partners Healthcare System.25 Quality of care included outcomes of chronic disease management, specifically for patients with DM and CAD. The algorithms used to identify patients diagnosed with DM and CAD have been previously described.30,31,32 We assessed hemoglobin A1C (HbA1c) and low-density lipoprotein (LDL) testing rates for DM patients, and LDL testing rates for CAD patients within the past year. We also assessed whether HbA1c and LDL were at clinical goal for each subpopulation. HbA1c at goal (< 9%) was defined as the most recent HbA1c value within the past year. For lipid control, LDL at goal (< 100 mg/dl) was defined as the most recent LDL value within the past year, or on a high-intensity statin for patients with CAD or on a high/moderate-intensity statin for patients with DM.33

Further, we examined quality of care measures for preventive cancer screening including breast, cervical, and colorectal cancer, among those who were eligible for testing. Women 50 to 74 years of age were considered up-to-date for routine breast cancer screening if they had a record of having a mammogram in the past 2 years. Women 21 to 64 years of age were considered up-to-date for routine cervical cancer screening if they had a record of having a Papanicolaou test in the past 3 years, or, for women 30 years and older, having an HPV test in the past 5 years. Patients 50 to 75 years of age were considered up-to-date for routine colorectal cancer screening if they had a record of having a colonoscopy in the past 10 years or sigmoidoscopy, barium enema, or computed tomography colonography in the past 5 years.34 These data were also obtained using the RPDR. The chronic disease management and preventive cancer screening outcomes selected are quality measures used by the institution’s population health management program.35

Additionally, resource utilization measures were evaluated, including ED visits, hospital admissions, and percentage of high-cost imaging (outpatient CT, MRI, and nuclear medicine tests) per 100 patients. ED visits and hospital admissions were particularly chosen given the well-described association between having a PCP and these outcomes.36, 37 Lastly, we examined patient-reported healthcare experiences using Consumer Assessment of Healthcare Providers and System (CAHPS) scores to assess timeliness of appointments, care, and information, and provider communication with patients. Patients in the cohort were randomly selected for surveys following outpatient visits as part of routine quality assessment efforts.

Statistical Analysis

We compared characteristics of patients seen by resident versus staff PCPs using chi-square or t tests as appropriate. To reduce the effect of confounding due to baseline differences between resident and staff patients, we present results from regression models adjusting for sociodemographic and clinical factors including age, gender, race/ethnicity, spopken language, insurance status, education level, area-based median household income, clinic type (health center vs. non-health center), Charlson Comorbidity Index, Higashi score, and patient time in network. We used logistic regression models to evaluate quality of care and patient-reported health experience outcomes, and Poisson regression models for resource utilization outcomes. To take into account the clustering of patients within providers, we used the generalized estimating equations (GEE) approach in all analyses.

As a sensitivity analysis to account for broad sociodemographic and clinical differences and the likely differential distribution of these factors between resident and staff PCP patients, we further compared outcomes from 1:1 propensity score–matched subgroups of resident and staff PCP patients.38 The propensity score model included the same sociodemographic and clinical factors listed above. We used a greedy matching algorithm where each resident patient (case) was first matched with a staff PCP patient (control) within five digits of the propensity score. For those that did not match, cases were then matched to controls on four digits of the propensity score. This continued down to a one-digit match. Analyses were performed using SAS version 9.4 (Cary, NC).

RESULTS

The cohort for this study included 10,870 patients who were followed by resident PCPs and 132,404 patients who were followed by staff PCPs (Table 1). Resident patients were younger and included a smaller proportion of women. Resident patients included a higher proportion of non-white patients and patients whose primary spoken language was not English. A higher proportion of resident patients were not insured or were covered by Medicaid, were living in poverty, and had not completed a high school education. Resident patients also had a shorter time in network than staff patients. Resident patients were more likely to have DM and CAD as well as depression and substance use disorder. Resident patients also had more comorbid conditions as assessed by the Charlson and Higashi scores. These sociodemographic and clinical differences were not observed in the propensity score–matched sample cohorts of 10,784 resident and staff PCP patients (online Appendix Table 1).

Table 1 Demographic Characteristics of Patients
Full size table

Chronic Disease and Preventive Cancer Screening Quality Measures

Within the cohort, fewer resident patients with CAD had an LDL screening test performed in the past year, and when they did, fewer had an LDL at goal (adjusted OR [aOR] 0.77 [95% CI 0.65, 0.92]) (Table 2). Similarly, a smaller proportion of resident patients with DM had an HbA1c test performed within the past year and when tested fewer had an HbA1c at goal (aOR 0.73 [95% CI 0.62, 0.85]). A smaller proportion of resident patients with diabetes had an LDL screening test performed and of those tested there was no significant difference in those who had their lipids at goal (aOR 0.93 [95% CI 0.79, 1.08]). When examining preventive cancer screening, eligible resident patients were less likely to have completed breast (aOR 0.56 [95% CI 0.49, 0.63]), cervical (aOR 0.66 [95% CI 0.60, 0.74]), or colorectal (aOR 0.72 [95% CI 0.65, 0.79] cancer screening tests (Table 3). Similar results were observed in the propensity score–matched sample cohorts (online Appendix Tables 2a and 2b).

Table 2 Quality of Care Outcomes Between Staff and Resident PCP Patients: Chronic Disease Management
Full size table
Table 3 Quality of Care Outcomes Between Staff and Resident PCP Patients: Preventive Cancer Screening
Full size table
Table 4 Resource Utilization Between Resident and Staff PCP Patients
Full size table

Resource Utilization

When comparing resource utilization between groups, resident physician patients had a higher mean rate of ED visits and hospital admissions, per 100 patients (adjusted rate ratio, ARR 1.53 [95% CI 1.42, 1.66] and 1.81 [95% CI 1.65, 1.99]) respectively (Table 4). In contrast, resident patients had a lower rate of outpatient high-cost imaging, per 100 patients, ARR 0.46 (95% CI 0.40, 0.52) compared to staff patients. Similar findings were observed in the propensity score–matched sample cohorts (online Appendix Table 3).

Patient-Reported Health Experience

When assessing patient-reported health experience scores, we found that resident patients reported less satisfaction with obtaining an appointment for urgent and routine care; on the other hand, we found no difference between resident and staff PCP patients in timeliness of receiving answers to phone questions (Table 5). The majority of patients felt their physicians explained things clearly and showed respect, although staff physician scored marginally higher on these features of communication (Table 5). Given the differential response rate to the survey questions, a propensity score–matched cohort was not examined for patient-reported health experience.

Table 5 Patient-Reported Health Experience Between Resident and Staff PCPs
Full size table

DISCUSSION

Primary care is a key point of entry to the health care system and, as the setting where the majority of patients receive their care, it remains crucial to improving the overall health of a society.39, 40 Assessing quality of care is integral for highly functioning primary care practice networks.41 For primary care practices that include resident physicians as part of IM training programs, this has led to increased interest in evaluating the care of patients seen by resident physicians. This is particularly important given primary care training opportunities are fraught with challenges, including trainee scheduling and supervision, which may impact the quality of care for patients seen by resident physicians.42

In this study, we examined the association between provider type (resident vs. staff PCP) and quality of primary care outcomes, resource utilization, and patient-reported experience of care. First, we found that resident patients were more likely to have demographic and socioeconomic characteristics associated with underserved populations such as living in poverty, English not being one’s primary language, and use of Medicaid insurance. Second, after controlling for available clinical and sociodemographic factors, resident patients were less likely to complete chronic disease quality measures, specifically HbA1c and LDL testing, and have these measures at goal. Third, resident patients were less likely to complete preventive cancer screening, specifically breast, cervical, and colorectal. Fourth, resident patients had higher rates of ED visits and hospital admissions than staff patients. Lastly, patient-reported satisfaction was similar between resident and staff PCP patients, though patients generally rated access and communication more positively when they had a staff PCP.

Similar to prior studies, we have shown that there are major differences in the characteristics of patients seen by resident and staff physicians, though the direction of these differences has varied.43 For example, using the National Hospital Ambulatory Medical Care Survey (NHAMCS) data from 1997 to 2004, Zallman and colleagues found that resident patients were more likely to be from underrepresented minority backgrounds and to have Medicaid insurance. Yet resident physicians in this study performed better in various processes of care measures including prescribing angiotensin-converting enzyme inhibitors for patients with congestive heart failure and diuretics for those with hypertension.8 While the NHAMCS study examined process of care measures, particularly those reinforced by inpatient training, it did not measure clinical outcomes such as LDL and HbA1c measures, a feature unique to the current analysis. In a separate study of primary care clinics at New York Hospital–Cornell Medical Center, Charlson et al. found that, during a 1-year period, resident patients had higher ambulatory costs than attending physicians, particularly related to radiology costs and consultations.7 This study focused on comparing costs associated with high-resource utilization between resident and staff PCP patients and did not examine important chronic disease and preventive screening quality measures evaluated in the current analysis. These prior studies report outcomes from many years ago and may not reflect the current resident education or primary care practice environment. Additionally, neither study examined patient experience of care, a factor distinctive to the present study. In a more recent analysis that evaluated inpatient to outpatient transitions of resident patients compared to staff patients, Doctoroff et al. found that resident patients had lower rates of timely post-discharge follow-up than staff physicians.44 However, these data were limited to patients who were recently hospitalized and did not examine preventive care outcomes.

Differences in quality of care outcomes between resident and staff physicians may arise for several reasons. First, variability in clinical skills and the inherently limited experience of residents, particularly in the ambulatory setting, may lead to lower quality of care, including achieving chronic disease management targets and preventive cancer screening tests, among resident patients.45,46,47,48,49 Second, the disruption in continuity caused by patient transition of care every 2 to 3 years to another resident provider may impact the management of chronic diseases and preventive care, resulting in delays in testing and adverse patient outcomes.50,51,52,53 Third, as reflected in this cohort, resident PCPs tend to have patient panels with a higher proportion of medically and socially complex individuals compared to staff PCPs which may result in disparate patient outcomes.54, 55 Fourth, residency continuity clinic scheduling models have faced challenges in balancing the needs of inpatient and outpatient care, sometimes resulting in significant gaps in residents’ presence in clinic, which can impact continuity with patients as well as completion of patient care tasks.56 In addition, resident PCPs may not utilize resources available within the practice, such as allied health professionals, electronic medical record reporting, and population health coordinators, as effectively as their staff counterparts.57 While each resident PCP evaluated in this analysis had access to these resources, we did not examine whether differential utilization of team-based care exists between resident and staff physicians. Lastly, resident physician patients were followed for fewer years within the primary care practice compared to staff patients; though we controlled for this factor, it may still be a marker of limited patient engagement in primary care and a potential factor affecting the outcomes observed in this study and in resident ambulatory care overall.58

The findings of this study have important implications for the quality of primary care, particularly for underserved populations. As we observed, resident physicians care for a predominantly underserved patient population. Thus, the quality of care that they offer plays an important role in the health-related disparities that are present in lower income, racial/ethnic minority patients, including chronic disease management and preventive care outcomes.59 There is a need for closer attention to how patients from various sociodemographic backgrounds are introduced into the primary care system, and how underserved patients are managed by a constantly changing cadre of resident physicians. Striking the appropriate balance is vital as the medical system strives to train physicians while providing high-quality and equitable care to all patients.

LIMITATIONS

This was one of the larger studies examining resident and staff PCP quality of care, and included care provided at 16 different primary care practices; however, being a single-institution study may limit the generalizability of our findings. In addition, since it is an observational study, we cannot exclude the possibility that residual and/or unmeasured confounding exists, even after performing propensity score matching, especially given the large differences observed. While resident PCPs practice in the same clinical settings as staff PCPs, there may be differences in the utilization of available resources that would limit the health care system equivalence between the two provider types. Additionally, we were unable to isolate the effect of resident supervision in the present analysis. Since clinical supervision has been found to be associated with improved patient- and education-related outcomes, it is possible the differences observed in this study would be even greater without resident supervision.60 Most of the data were obtained from a clinical research database which has been previously validated, yet some unavailable patient-level characteristics, such as employment status, health literacy, and social support, may have limited our ability to adjust for all the factors that affect an individual’s clinical and socioeconomic status. We did not examine provider-specific factors such as age, gender, race, or time in practice, some of which have been shown to impact patient-perceived quality and use of health care.61 Lastly, the available data did not allow us to capture differences between resident and staff PCPs in ordering rates for screening tests. While it is possible that resident patients were less likely to complete a screening test even though it was ordered, it is unlikely this would explain the large differences observed in study outcomes.

CONCLUSION

In a large primary care network affiliated with an academic medical center, we found that patients of resident physicians were less likely to achieve chronic disease quality of care and preventive cancer screening measures, had higher resource utilization including ED visits and hospital admission, and lower patient-reported satisfaction with care on certain measures. Our results highlight a potential conflict between the goals of postgraduate medical training and the quality of care provided to patients of trainees, particularly among underserved populations. Novel approaches are needed to address residency ambulatory medical education, a central issue to improving overall quality and equity of primary care.