BACKGROUND

Physician behaviors play a key role in the provision of high-value care.1,2 Value in this context has been defined as quality (care that is safe, timely, effective, efficient, equitable, and patient-centered)3 divided by cost (the “Value Equation”).4 High-value care seeks to balance quality and cost with the goal of reducing harm and improving patient outcomes.5 The importance of high-value care is underscored by the Institute for Health Care Improvement Triple Aim, which encourages healthcare systems to simultaneously improve population health, enhance patient experience, and reduce per capita cost.6 Medical education thus has a responsibility to not only equip students with a nuanced understanding of value, but also support the development of high-value practice patterns aligned with this aim.7

Prior studies have demonstrated relationships between physicians’ experiences during residency and their subsequent knowledge,8 attitudes,9 behaviors,10,11 and patient outcomes12 related to high-value care. These relationships are evident as early as internship13 and persist for up to 15 years after training.10,11 This suggests residency training contributes to an imprinting process whereby learners develop practice patterns resembling those they observe in the learning environment.11,12 Studies like these highlight the importance of the graduate medical education training environment. However, comparatively little is known about the potential impact of regional practice patterns on medical student behaviors and experiences—despite the fact that their professional identity formation is already well-underway.14 Clinical clerkships, for example, represent a particularly formative time in the professional development of physicians15,16,17 and a time when early practice patterns begin to develop. It is thus possible that the practice patterns medical students experience in clinical settings imprint their behaviors even before entering residency training.

Practice patterns can be evaluated using measures of regional healthcare intensity (HCI) which reflects the number and types of services patients receive within a particular geographic region. Regional HCI varies widely across the USA, and this cannot be completely explained by differences in illness severity, patient preferences, socioeconomic factors, or pricing.18 Instead, this variation appears to reflect differences in practice style.18,19 Higher HCI, while costly, could increase the overall value of healthcare by improving patient outcomes. However, regions with higher spending do not consistently demonstrate better health outcomes, access to care, or patient satisfaction—illustrating that more intense care is not necessarily better care.18,20,21

OBJECTIVES

We previously demonstrated that medical students observe conflicting physician role-modeling behaviors with respect to high-value care and that students training in regions with higher HCI report observing fewer cost-conscious role-modeling behaviors than students training in regions with less intense use of healthcare resources.22 In this study, we aimed to build upon our prior work by (a) surveying medical students about their own behaviors related to value and (b) examining the relationship between student behaviors and HCI in the region of their medical school.

METHODS

Study Design and Participants

We conducted a cross-sectional survey of all third- and fourth-year students at nine US medical schools: Brody School of Medicine at East Carolina University; University of California, Davis School of Medicine; University of California, San Francisco School of Medicine; Indiana University School of Medicine; Mayo Clinic School of Medicine; Pennsylvania State University College of Medicine; University of Michigan Medical School; Vanderbilt University School of Medicine; and Warren Alpert Medical School of Brown University. These nine participating schools were recipients of an American Medical Association (AMA) Accelerating Change in Medical Education grant23 and have variable geographic locations, class sizes, missions, and private/public status. Each school’s institutional review board (IRB) approved or exempted this study.

Measures

Survey items measured high-value (n = 10) and low-value (n = 9) medical student behaviors. These items were developed by the study team (which includes experienced educators, content experts, and survey researchers), achieving consensus through discussion via a series of conference calls, in-person meetings, and a review of the literature to identify relevant medical student behaviors (such as the “Choosing Wisely for Medical Students” guidelines).24 Students were asked to indicate how often in the current academic year they had engaged in each behavior: 0 = never, 1 = rarely (1–2 times), 2 = sometimes (3–5 times), 3 = often (6 or more times).

As previously described,22 we measured regional HCI using hospital referral region (HRR)–level per capita data from the Dartmouth Atlas of Health Care’s End-of-Life Chronic Illness Care database.25 These data reflect HCI during the last 2 years of life for Medicare beneficiaries aged 67 years or older with chronic illnesses who died (decedents) in 2014. Consistent with prior studies,1,8,22 we used end-of-life data in order to compare the intensity of care provided to a cohort of comparably ill patients with a life expectancy of exactly 2 years. We elected to use regional-level data rather than hospital-level data because most students train at more than one hospital, usually in close proximity. The primary HRR for each medical school was considered to be the HRR encompassing the majority of hospitals where students from that school rotate.

We measured HCI in the primary HRR associated with each medical school using the hospital care intensity index (a composite measure of hospital days and inpatient physician visits), ratio of physician visits per decedent relative to the average number of physician visits per decedent in the USA, and ratio of medical specialty to primary care physician visits per decedent. These data were all adjusted for sex, age, race, and chronic condition (cancer, congestive heart failure, coronary artery disease, peripheral vascular disease, diabetes with end-organ damage, dementia, liver disease, pulmonary disease, and chronic renal failure).26 As previously described,22 we selected these measures of HCI based on the premise that they would be more visible to students than direct measures of spending.

Data Collection

We e-mailed a letter to students inviting them to participate in the study between January and March 2017. The letter emphasized that participation was voluntary and responses would be anonymous. Each letter included a link to the electronic survey. As an incentive for participation, students at eight of the nine participating medical schools were invited to enter a lottery to win one of 70 $250 cash cards; one school’s IRB did not allow an incentive. Up to three reminders were sent to non-responders, and informed consent was implied upon survey completion.

Data Analysis

We reported response rates using the American Association for Public Opinion Research response rate 2 (RR2) definition27 and compared the sex, age, and year in school of respondents with those of the total sampled population. Descriptive summary statistics were reported as frequencies with percentages, and we examined the internal consistency reliability of the high- and low-value behavior items using Cronbach’s alpha.

To summarize student behaviors, we calculated median item ratings for each student across the high- and low-value behavior items. Students were then dichotomized into two groups (those with a median rating of < 1 and ≥ 1) for each set of items, reflecting the natural cut-point of our 4-point behavioral frequency rating scale (0 = never, ≥ 1 = one or more times). These groups were used as dependent variables in unadjusted and adjusted (controlling for age, sex, and year in school) logistic regression models examining associations with regional HCI.

All tests were two-sided, and P values < 0.05 were considered statistically significant. We conducted sensitivity analyses excluding responses from students who were not offered an incentive for participation. Analyses were performed using SAS version 9.3 (SAS Inc., Cary, NC).

RESULTS

Of 2623 third- and fourth-year medical students invited to participate, 1304 (50%) responded. The distributions of respondents with respect to sex, age, and year of training were similar to those of the overall sample (Table 1) and US medical students in general.28

Table 1 Characteristics of Study Participants: Respondents and Overall Sample (Numbers in Each Column May Not Sum to the Total N for That Column due to Missing Data. Aggregate Data for the Overall Sample Were Obtained from Institutional Records)

High-Value Behaviors

Most students reported engaging in at least one high-value behavior on clinical rotations in their current academic year (Fig. 1), such as asking their supervisor to clarify how a test or procedure would change patient management (1211/1256, 96%), pointing out potential risks of unnecessary testing (1135/1230, 92%), and initiating a conversation with a patient about whether a test is necessary (1117/1238, 90%; Table 2). The majority of students also reported asking their supervisor questions about costs of care (1085/1234, 88%), trying to determine costs (1099/1259, 87%), asking for clarification on a test they believed may have been ordered inappropriately (1062/1238, 86%), and screening for financial hardship (961/1230, 78%). However, over one-quarter of students reported never talking to patients about costs when discussing treatment options (333/1230, 27%) and never seeking cost-effectiveness data to inform proposed care plans (347/1256, 28%), while over half of students (686/1257, 55%) reported never including details about costs in their case presentations.

Figure 1
figure 1

Percentage of third- and fourth-year medical students who reported in engaging in high- and low-value behaviors on clinical rotations during their current academic year, 2017 survey.

Table 2 High- and Low-Value Behaviors Among Third- and Fourth-Year US Medical Students, 2017 Survey

Low-Value Behaviors

Most students also reported engaging in at least one low-value behavior on clinical rotations in their current academic year (Fig. 1), such as suggesting a test for the sole purpose of anticipating what their supervisor would want (1143/1220, 94%) or showing off their ability to generate a broad differential diagnosis (1072/1218, 88%). Many medical students also reported suggesting a test solely to satisfy curiosity (958/1217, 79%), protect the medical team from potential liability (938/1215, 77%), appease a patient (892/1216, 73%), and save time (752/1213, 62%). Forty-four percent (533/1217) suggested a test or procedure solely to build clinical experience. Nearly two-thirds (803/1235, 65%) knowingly suggested a test that would not change patient management, and 54% (663/1232) knowingly suggested a more invasive test when a less invasive test was available (this item included an example from the Choosing Wisely for Medical Education guidelines24 illustrating the less invasive test was also appropriate in order to emphasize the focus on value).

Internal Consistency Reliability

Cronbach’s alphas for the high- and low-value behavior items were 0.82 and 0.79 respectively.

Relationship Between Regional Healthcare Intensity and Medical Student Behaviors

Measures of HCI did not significantly differ between HRRs that were (n = 9) and were not (n = 297) associated with a participating medical school (Table 3). Students training in regions with higher HCI had lower odds of reporting high-value behaviors, but these associations were not statistically significant (Table 4). Conversely, students training in higher intensity regions had higher odds of reporting low-value behaviors. These associations were statistically significant for two of our three measures of HCI in unadjusted analyses. The association with physician visits per decedent relative to the US average remained significant after adjusting for age, sex, and year in school. This finding did not change when responses from students who were not offered an incentive for participation were excluded from the analysis (data not shown).

Table 3 Regional Healthcare Intensity of Hospital Referral Regions That Were (n = 9) and Were Not (n = 297) Associated with a Participating School (the Primary Hospital Referral Region Associated with Participating Medical School Was Considered to be the Hospital Referral Region That Encompassed the Majority of Hospitals Where Students from That School Rotate)
Table 4 Relationship Between Regional Healthcare Intensity and High- and Low-Value Medical Student Behaviors

DISCUSSION

This large, multisite survey study describes high- and low-value behaviors among third- and fourth-year medical students and demonstrates that students training in regions with higher HCI report engaging in more low-value behaviors. This highlights the importance of the undergraduate medical education learning environment and reinforces the need to foster healthcare teams and organizational systems that promote the delivery of the best care at the lowest cost.

The high-value behaviors most frequently reported by students primarily involved interactions with the healthcare team (e.g., questioning how a test will change patient management, asking a supervisor about costs of care) or interactions with patients (e.g., initiating a conversation about whether a test is necessary, screening for financial hardship). Student responses suggest they are aware of and willing to point out potential risks of unnecessary testing, make efforts to determine the costs of healthcare services, and consider the impact of costs on patients. These results are encouraging, especially given students’ more junior role on the healthcare team and relative inexperience navigating clinical settings.

However, our findings also highlight opportunities for improvement. The high-value behaviors reported least frequently by students involve incorporating cost information into patient care plans (e.g., seeking cost-effectiveness data, including cost in case presentations). These types of behaviors may thus deserve special attention and focus in undergraduate medical education curricula, especially given the numerous barriers students endorse with respect to high value.22 Such barriers may stem from the students themselves (e.g., inadequate knowledge and skills, competing priorities), from faculty on clinical rotations (e.g., suboptimal role-modeling behaviors, no expectation or reinforcement of high-value behaviors),22,29 from the local learning environment (e.g., limited or no consideration of high-value behaviors on clerkship evaluations), or from the broader healthcare system (lack of cost transparency, limited cost-effectiveness research, time pressures).30,31

Third- and fourth-year students also report engaging in numerous low-value behaviors—including those specifically identified by Choosing Wisely Canada as behaviors trainees should avoid.24 Notably, students appear to have insight into the wasteful nature of these behaviors (e.g., proposing a test for the sole purpose of anticipating what a supervisor would want; knowingly recommending a test that would not change patient management), indicating lack of awareness is not the only driver of such behaviors. Rather, students appear to be influenced by many of the same drivers of wasteful behaviors reported by residents and practicing physicians such as saving time, satisfying curiosity, appeasing patients, and protecting the healthcare team from liability (so-called defensive medicine).32,33,34,35,36,37,38,39,40

Other drivers of wasteful behaviors may be unique to trainees such as suggesting a test solely to anticipate what a supervisor would want, show off one’s ability to generate a broad differential diagnosis, or build clinical experience. Students are keenly aware of the impact of clerkship evaluations on the likelihood of matching into desired residency programs and may thus engage in these types of low-value behaviors to create an impression of competence or otherwise ingratiate themselves to their supervisors.40,41,42 Students are also sensitive to hierarchy within healthcare teams,41,43 and may engage in wasteful behaviors to avoid upsetting or being disrespectful to more senior team members.44 To address this issue, medical schools could consider faculty development strategies that encourage attending physicians to explicitly reinforce high-value behaviors and discourage low-value behaviors.32,45 Items assessing such behaviors could also be added to clerkship evaluation forms and other assessments to elevate the importance of these issues in the eyes of both faculty and students.46

The importance of the clinical learning environment is further underscored by our finding that medical students training in regions with more intense use of healthcare resources reported significantly more low-value behaviors than students training in lower intensity regions. This suggests the imprinting process is already underway during medical school. Students training in higher intensity regions also tended to report fewer high-value behaviors, though these associations were not statistically significant. The reason for this is unclear. One possibility is that formal high-value, cost-conscious care curricula are more effective at promoting high-value student behaviors than deterring low-value behaviors, leaving the latter more susceptible to imprinting by regional practice patterns. Taken together, the results of this study highlight medical schools’ obligation to train physicians capable of effectively stewarding healthcare resources on behalf of both patients and society.7,47

To address this issue, educators and educational leaders should consider ways to promote “purposeful imprinting.”11 Results of a recent systematic review suggest a three-pronged strategy of knowledge transmission, reflective practice (e.g., audits with feedback, interactive discussions), and provision of a supportive environment may be particularly effective.48 Schools could also give serious consideration to training students in a variety of environments, including those with less intense use of healthcare services. Such an undertaking may be logistically and politically challenging, but could expose students to practice settings enriched with individuals who model high-value care.

The generalizability of our results is supported by the inclusion of private and public medical schools that are distributed geographically across the USA. The characteristics of respondents were also similar to those of the total sampled population and US medical students in general, reducing concerns about bias resulting from systematic differences between respondents and non-respondents. The anonymous nature of the survey also decreases concerns about the impact of social desirability on student responses.

Nevertheless, our study has limitations. First, the nine participating schools were recruited through the AMA Accelerating Change in Medical Education initiative,23 so the findings reported here may not reflect the behaviors of all US medical students. Second, our classification of medical student behaviors as high or low value, while based on the literature, may not apply in all situations or fully capture the complex trade-offs between financial and non-financial resources made in clinical practice.49 For example, repeating a test to save time, while classified as a low-value behavior for the purposes of this study, could increase the value of care in high-acuity situations when time is of the essence or if the effort required to procure recent test results would divert team members from more pressing tasks. Conversely, behaviors classified as high value for the purposes of this study could reduce the value of care if they undermine the doctor-patient relationship (e.g., talking to a patient about costs when discussion treatment options) or shift the focus to costs apart from quality (e.g., including cost details in case presentations). Such trade-offs are common but difficult to measure through formal cost analyses. Third, our survey may have omitted key behaviors that were not identified in our review of the literature. Fourth, student responses were based on recall and hence may not accurately or completely reflect their actual past behaviors. Fifth, while we sought to provide content validity for survey items (e.g., by grounding them in the Choosing Wisely for Medical Education guidelines24) and demonstrated their internal consistency reliability, additional construct validity evidence is needed to support the use of resulting scores as a measure student behaviors. Sixth, our HCI measures only reflect care provided to Medicare beneficiaries. However, prior studies have demonstrated that healthcare resource utilization among Medicare beneficiaries reflects HCI among Medicaid beneficiaries50 and commercially insured patients51 in the same region.

CONCLUSIONS

This study describes high- and low-value medical student behaviors on clinical rotations and demonstrates that students training in regions with higher HCI have higher odds of reporting low-value behaviors. Further studies are needed to determine if the association between regional HCI and low-value behaviors persists into residency and to clarify the relative impact of undergraduate versus graduate medical education experiences on physicians’ subsequent practice patterns. Strategies whereby medical education can mold the clinical learning environment and mitigate its potentially negative impacts on students also warrant further investigation.