Medical students need to develop knowledge, skills, and attitudes related to cultural competence and cultural humility [1]. Students are often required to take classroom-based courses in cultural competence and cultural humility; however, teaching these concepts is less frequent or systematic in the critical clinical years. Previous research has demonstrated that cultural competence training can improve knowledge, attitudes, and skills of health professions trainees [2, 3]. However, these studies mostly included small sample sizes and self-selected participants.

Fewer studies have measured acquisition of skills as it applies to clinical encounters or after the intervention to assess if the desired changes persist [1, 2]. A recent literature review highlighted the lack of best practices with regard to teaching cultural competence in healthcare and noted that most studies focused on self-reported knowledge or attitudes [4].

In the context of persistent health disparities and recent national attention on structural violence, health professionals are grappling with how to respond individually and professionally. How to train the next generation is a crucial part of this discussion [5]. Previous literature suggests that interpersonal skills, individualized treatment, and effective communication are important components of culturally sensitive healthcare and that culturally sensitive physicians may help to improve health equity outcomes [6]. This study sought to examine if an educational intervention utilizing cultural competence and cultural humility could improve clinical skills in actual patient encounters.

Methods

The authors adapted an educational intervention that has been studied previously among psychiatry residents and shown to improve relevant knowledge, skills, and attitudes [3]. In addition to utilizing the DSM-5 Cultural Formulation Interview [7], the class features case examples and utilizes reflective practices to highlight cultural humility components, including self-evaluation, the culture of medicine, and power differentials in clinical encounters [6]. Rapid cycle improvement [8] was employed to collect feedback from the first couple participant cohorts and minor changes to content made. This was employed through an anonymous survey soliciting feedback, which was explicitly noted to be voluntary and for quality improvement purposes.

This study was submitted to the IRB at SUNY Upstate Medical University and determined to be exempt since it was determined not to adversely impact either the students’ opportunity to learn required educational content or to impact the assessment of educators who provide instruction.

The educational intervention is a mandatory addition to the psychiatry clerkship for third-year medical students, and the clinical evaluation assessments are a routine part of their clerkship assessment. The intervention posed no more risk than normal activity and informed consent was not obtained.

Participants included two cohorts of third-year medical students at SUNY Upstate Medical University. The pre-intervention group consists of students who completed their clinical year in 2018–2019 (N = 80) and who did not participate in the intervention. The intervention group includes students who completed their clinical year in 2019–2020 (N = 91). Eligibility criteria included having three time points of evaluations in order to estimate change in performance scores, in other words, at least one clerkship prior to psychiatry and at least one clerkship subsequent to psychiatry. Given eligibility criteria, 71% of the pre-intervention cohort of students and 67% of the intervention cohort were included in the analyses. Data were collected for medical students from the beginning of the 2019 academic year until mid-March, 2020, when clerkships switched to remote learning due to COVID-19 pandemic.

The 1-h interactive didactic utilized cultural competence curriculum previously studied with psychiatry residents [3] and adapted it to include locally relevant cases and reflective practice exercises. It included the DSM-5 Cultural Formulation Interview, health disparities experienced by racial and ethnic minorities, and the rationale for cultural competence training. Cultural humility was emphasized through discussion and practice of asking questions nonjudgmentally and utilizing validation techniques. In addition, as in the previously studied curriculums, the intervention included reflection exercises with debriefs to focus on the culture of medicine and power differentials in clinical encounters. The interactive exercise invited participants to reflect and discuss in small groups the following questions: “When patients work with me, what identity of mine are they most aware of? When patients work with me, what identity of mine are they least aware of? Which identity is hardest to discuss with a patient who identifies differently?” This intervention is facilitated by a child psychiatrist or a child psychiatry fellow trained in giving the didactic by first observing the process, being observed with feedback and then leading on their own.

Whereas the previous study utilizing this intervention determined effectiveness by examining self-reports of change in knowledge, skills, and attitudes, this study examined whether behavior in clinical encounters could be improved. The primary outcome was thus changes in student performances as measured by supervising physicians in the clerkship before and after the psychiatry clerkship, where the intervention took place.

Supervising physicians assess students during each clerkship rotation using a clinical evaluation tool that includes domains linked to the Accreditation Council for Graduate Medical Education (ACGME) competencies. The same clinical assessment form is used by supervising physicians across all clerkship rotations. There are four domains including [9] (1) relates to patients in a respectful, caring, empathetic manner; (2) engages in professional behaviors including reliability, dependability, and punctuality; (3) demonstrates accountability, contribution, and commitment to patient care; and (4) develops insightful, focused, pertinent questions based on clinical scenarios that have stayed consistent between the two clerkship years. Three of the domains were hypothesized to have been influenced by the intervention, while one of the domains was not addressed by the intervention (engaging in professional behaviors including reliability, dependability and punctuality), and would therefore serve as a control. In ascending order, potential ratings on each domain are developing, meets, exceeds, and exemplary. Faculty raters are trained annually in assessing medical students in patient encounters, and multiple clinical assessments occur in the context of each clerkship.

The primary outcome was student performance on the relevant clinical assessment items, compared between the two cohorts. In order to meaningfully compare the performance of two cohorts, the authors adjusted the four-point categorical scale on the assessment forms (developing, meets, exceeds, exemplary) so that it was binary and reflected grading as seen on Dean’s letters (“Honors distinction” vs. “No honors distinction”). This decision was made post hoc. The authors conducted two sets of Chi-square tests, using SPSS [10]. The first set of analyses compared student performance between the two cohorts in the rotation prior to the psychiatry clerkship, where the educational intervention took place. Secondly, we ran the same set of analyses analyzing student performance in both cohorts after completion of the psychiatry clerkship, where the educational intervention took place. Given the student schedules, clerkship schedules were different for each student. A post-hoc analysis was done to look at differences in clerkships before and after psychiatry in the pre-intervention vs intervention year before and after the intervention, and no significant differences were noted. The didactic was delivered within the first half of the psychiatry clerkship, and the clinical evaluation assessments can occur throughout the month. For this reason, comparison between scores within the psychiatry clerkship itself was not thought to be as meaningful. The data were de-identified but not anonymized because they were linked over time.

Results

All students in the 2019–2020 participated in the educational intervention. Given the inclusion criteria for analysis, the 2018–2019 cohort consists of 79 students with an average of 38 clinical evaluations per student. The 2019–2020 cohort consists of 90 students with an average of 34 clinical evaluations per student. Student feedback from the first few cohorts led to minor changes to content such as utilizing more locally relevant cases and focusing more on clinical applications than background information.

Results of the chi-square analysis indicate no statistical differences between pre-intervention and intervention cohorts in student performance prior to the psychiatry clerkship (Table 1). However, after taking the psychiatry clerkship where the intervention took place, student assessments in the intervention group are statistically significantly higher in all three domains that were hypothesized to have been influenced by the intervention than student assessments in the pre-intervention cohort (Table 1).

Table 1 Percentage of clinical evaluations in the two top tiers (exceeds or exemplary)
Full size table

Specifically, in the domains of “relating to patients in a respectful, caring, empathetic manner”, “demonstrates accountability, contribution and commitment to patient care”, and “develops insightful, focused, pertinent questions based on clinical scenarios”, students who completed the intervention demonstrate greater competencies, as measured by percent of students earning top scores on clinical evaluations in the clerkship taken subsequent to psychiatry, after the intervention (2.7% difference of students earning top two scores, P value 0.05; 3.8% difference, P value 0.01 and 5.1% difference, P-value 0.0, respectively). No statistically significant difference in these scores was found in scores between the two cohorts in the clerkship prior to psychiatry, where the intervention occurs. While statistically significant, the measures of association for these items are tenuous (Cramer’s V test < 0.2 for all items). The strongest clinical assessment item, “develops insightful, focused, pertinent questions based on clinical scenarios”, had a Cramer’s V of 0.06 which may indicate a small effect. A future study with a more robust intervention (including intervention for faculty to reinforce concepts learned in this training) is expected to yield more substantial results. No statistically significant difference in these assessments was found in scores between the two cohorts in the clerkship prior to psychiatry, where the intervention occurs.

Discussion

Cultural competence and cultural humility are both crucial to providing quality healthcare and to working towards reducing healthcare inequities. This study supports the notion that these concepts can be modelled and taught during critical clinical years of medical education. Clinical improvement in empathy and communication is promising, given that healthcare provider empathy towards diverse patient populations is critical to being able to provide patient-oriented interviewing and care and has been linked to improved patient satisfaction and health outcomes [11]. Moreover, the focus of outcome on change in clinical skills seen in the month after the educational intervention suggests that even brief educational interventions can be impactful.

More recent attention on merging cultural competence and cultural humility training as well as a focus on structural competency highlights recognition that medical training must include individual and structural factors that contribute to patient interactions [5, 6]. Strategies to help medical students in the clinical years to critically assess power and privilege and improve their ability to evaluate and treat patients from different cultures, races, and ethnicities are critical, particularly at a time when structural inequities and racism are being highlighted nationally.

Certainly, the domains assessed as primary outcomes in this study are important but broad and may be impacted by a number of other factors including working with a particularly skilled mentor or other curricular interventions, such as an increased focus on general wellness during the 2019–2020 cohort. An additional limitation of this study includes changed evaluation criteria for grading during this clerkship over the course of the 2 years being studied from conjunctive to compensatory grading, which may have changed behavior in patient encounters. However, a few other clerkships also made changes to their grading rubric during the intervention year, which included setting minimum requirements for a grade of A to include earning a set score on the ACGME competencies. Although this may have affected the improvement seen between pre-psychiatry and post-psychiatry clerkships, it is unlikely to explain the change, given that students rotate through clerkships in various orders and are as likely to have taken an updated clerkship after their psychiatry clerkship as they were to take one prior to the psychiatry clerkship.

Notably, this study did not conduct a traditional post-intervention assessment examining knowledge or attitudes; however, the authors were more interested in targeting behavior change in actual patient scenarios. A strength of this study is the methodology, which utilized data points in two time frames to approximate changes in scores over time and helps to control for possible differences between the two cohorts at baseline.

This study’s focus on investigating a more distal outcome (clinical assessments one month after the class) is ambitious given the brevity of the intervention; the results are promising and must be interpreted with caution, given the lag time between the class and ongoing assessments and other potential variables. While the core faculty remains essentially unchanged over the 2 years studied, resident physicians change each year, and this may have made an impact on clinical assessment outcomes. Additionally, there may be other variables from larger socio-political events that might affect student performance. Thus, these findings may not be replicated in future years. Future research is needed to investigate whether more robust interventions such as training core faculty to reinforce concepts in clinical rotations would provide more clinically significant and impactful effects.

While previous studies have demonstrated that such brief interventions can increase trainee knowledge, attitudes, and skills, an ultimate goal of cultural competence and cultural humility training is to improve patient outcomes and decrease health inequities. Towards this end, future studies should aim to compliment didactic portions with continued modelling and teaching in clinical encounters on the wards, develop best practices for all clinician educators in cultural competence and humility, and focus on how to teach these skills and support medical trainees in their formative clinical years.