Introduction

“The ultimate aim of a curriculum on disparities is that learners develop a professional commitment to eliminating inequities in health care quality and understand and accept their role in eliminating racial and ethnic healthcare disparities [1].”

In the USA, IndigenousFootnote 1 people receive poorer healthcare compared to non-Indigenous people and have worse health outcomes [2]. These disparities occur due to reduced access to primary care, use of lower-quality healthcare facilities, language barriers, and economic and educational disparities [3, 4]. In addition, ethnicity and race also play a role in the quality of care and treatment patients receive within the medical system [5]. When controlling for access-related factors, stereotyping, discrimination, and implicit bias are associated with health disparities for racial and ethnic minorities [6]. For instance, a survey of implicit bias among 154 providers found that one third agreed with negative stereotypes about Indigenous people; 84% both preferred non-Hispanic White patients to Indigenous patients and believed Indigenous children were more challenging and their caregivers less compliant compared to non-Hispanic White patients [7]. These biased beliefs have consequences for patients’ outcomes [8] and negatively affects patient safety [9]. From surgery outcomes [10] to maternal mortality rates [11] and for medical students [12], residents [13], surgeons [14], and nurses [15], race plays a role in adverse health outcomes.

About half of Indigenous people say that they experience racism very often [16]. In the medical setting in particular, Indigenous people are 2.6 times more likely to report racially based maltreatment [17] and are 10 times more likely to experience discrimination compared to non-Hispanic White patients [18]. A recent study that gathered experiences of Indigenous patients reported a variety of adverse experiences [8]. These patients often felt that they were experiencing bias, treated poorly, threatened, and dismissed. They felt that their medical problems were trivialized; that they were treated unkindly, especially in discussions regarding pain; and that these experiences led to worsened care. Racism is related to health disparities in the area of mental health, cancer, obesity, diabetes, heart disease, and even adverse birth outcomes [19,20,21].

The Institute of Medicine (IOM), the National Institutes of Health (NIH), and the Association of American Medical Colleges (AAMC) have all discussed the importance of providers and healthcare organizations providing culturally competent and equitable care [22,23,24]. Lack of attention to cultural factors in the clinic is related to a number of negative outcomes, including non-adherence, unnecessary visits and tests, and longer hospital stays [6]. Further, almost half of all providers (44%) say that addressing culture in clinic settings is “somewhat to extremely stressful,” [25] indicating a poor level of confidence and competence in the area that requires further training. However, there is evidence that training healthcare professionals about cultural issues can be effective in increasing knowledge on the topic [1] and improving health outcomes for racial and ethnic minority patients [26,27,28]. In a notable example, after providers at Massachusetts General Hospital received online cross-cultural training, the proportion of African-American patients who felt they received a lower quality of care than most White, English-speaking patients decreased from 25% in 2004 to 7% in 2012 [29].

While effective methods to improve cultural competency of healthcare providers exist, many challenges remain in delivery and effectiveness. Less than half of all fourth-year medical students demonstrate competency during common cross-cultural situations; the majority feel unskilled in providing basic cross-cultural care [30]. One partially successful medical-student training program aiming to prepare them for cross-cultural healthcare found improvements in assessment skills, collaboration with an interpreter, and level of preparedness in working with ethnic and cultural minority patients. However, the training did not change how students work with patients who use complementary and alternative medicine, have non-Western medicine health beliefs, distrust the medical system, are transgender patients, are religious minorities (including an inability to identify how religious beliefs affect clinical care and treatment), or are new immigrants [26]. Clearly some content and skills around cultural learning are more effectively acquired than others, demonstrating considerable room for improvement about how to teach this material effectively. Training carried out in classrooms, discussion groups, or with online learning methods might not be sufficient. Also, it is difficult to create and maintain a program around cultural effectiveness in a medical school despite the critical need for such programs [31]. In a review of five programs that implemented cultural competency curricula during residency, only 3 remain today [32]. However, even very little effort can increase learning in the area of cultural effectiveness and social justice. For example, two students at the John A. Burns School of Medicine at University of Hawaii at Manoa in the USA created and evaluated a program that taught medical students about social justice using online discussion groups and social media. They discovered that 92% of students increased knowledge of social justice [33]. This may be the first program that has successfully improved students’ social justice beliefs and has disseminated this information.

The Society for General Internal Medicine Health Disparities Task Force recommends three critical components for training in racial and cultural health disparities. It is important to address: (1) attitudes such as unconscious bias, stereotyping, and mistrust; (2) knowledge and history of health disparities using a social determinants of health lens; and (3) skills to effectively communicate, assess, and treat across cultures [1]. These components are important because, unfortunately, unconscious bias concerning patients’ race and social class plays a role in health professionals’ (including residents, surgeons, and nurses) decision-making [13]. Even providers who are aware of the social determinants of health rate patients’ health outcomes as being 90% attributable to patient factors, and over 40% did not believe there was strong evidence of the role of race in health outcomes [34]. Unconscious bias coupled with inaccurate beliefs of the cause of health outcomes for patients is magnified for Indigenous patients who experience prevalent stereotyping and bias within the healthcare system. Therefore, training that starts with basic information about the history of local communities, their political status, and cultural beliefs can help to reduce the perpetuation of false stereotypes that may contribute to unequal care.

The purpose of this project was to evaluate whether medical students who had received an Indigenous health lecture series at a single USA medical school had improved learning at the completion of the program and 6 months after the training. The hypothesis was that training would have a positive effect on students’ learning in the areas of knowledge, social justice, cultural empathy, cultural intelligence, and cultural humility at the completion of the program and at the follow-up period.

The Intervention

First-year medical students in 2014 (year 1) and in 2015 (year 2) received 7 h of coordinated lectures over 1 to 2 weeks about Indigenous peoples’ history, culture, and health [35]. The majority of students (year 1 = 96.66%; year 2 = 88.33%) attended these lectures, as they are part of their core curriculum, yet attendance is not taken, and therefore, the courses are not mandatory. This content was delivered mainly by Indigenous faculty from the School of Medicine, American Indian Studies and Education. There were slight differences in the curriculum from year 1 to year 2 given faculty availability and updates that were made based on feedback from the faculty and students (see Appendix Table 4).

Method

To evaluate the program, a one-group (non-randomized), pre-post-test quasi-experimental design was implemented. Students completed online questionnaires three times: 1 week before the lectures (pre-test), 2 weeks after the end of the lectures (post-test), and 6 months after the lectures (follow-up). However, in the year 1 post-test evaluation, four measures (standardized Social Justice Scale, Scale of Ethnocultural Empathy, Cultural Intelligence Scale, and Multigroup Ethnic Identity Measure) of the five total measures proved unavailable due to technical errors in the survey software, which prevented retrieving the data for analysis. The pre-test also requested demographic information, including age, gender, race/ethnicity, and current residence via zip code.

Instruments

Indigenous Knowledge and Beliefs

To measure the content learning of participants, the author created the Indigenous knowledge and beliefs (IKB) scale. This scale was adapted from a lesson plan assessment developed by anti-racism educators at the University of Calgary [36], and the content was tailored to focus on the Indigenous people local to the region (see Appendix Table 5). The scale consists of ten short-answer items such as “Currently, how many federally recognized American Indian nations are there in Minnesota?” or “What was the main goal of the boarding school era?” Respondents could attempt to answer the question or select “I don’t know.” This measure has two summary scores: the Indigenous knowledge and beliefs summary score and the Indigenous knowledge and beliefs attempts summary score. For the Indigenous knowledge and beliefs summary score, possible individual item scores ranged from 0 to 1, with 0 indicating an incorrect answer and 1 indicating a correct answer. The 10 items were averaged to create an IKB summary score with higher scores indicating a higher knowledge of both local and broad Indigenous knowledge and more culturally appropriate beliefs.

In order to account for self-efficacy on the lecture topics, a calculated attempt (i.e., those individuals who responded anything other than “I don’t know”) score called the Indigenous knowledge and beliefs attempts summary score was also created using the average number of “attempts” whether they answered the question correctly or not. Individual item scores ranged from 0 to 1, with 0 indicating no attempt and 1 indicating an attempt.

Social Justice

The standardized Social Justice Scale (SJS) was used to capture medical student’s opinions about social justice (e.g., I feel confident in my ability to talk to others about social injustices and the impact of social conditions on health and well-being; I believe that it is important to respect and appreciate people’s diverse social identities) [36, 37]. This scale was designed to measure social justice–related values, attitudes, perceived behavioral control, subjective norms, and intentions based on a 4-factor conception of Ajzen’s theory of planned behavior. Confirmatory factor analysis and analyses for reliability and validity were used to test the properties of the original scale. Internal consistency was good for all four subscales: attitudes (α = .95), subjective norms (α = .82), perceived behavioral control (α = .84), and intentions (α = .88). The original measure was shortened from 29 items down to 17, which included the original 4 subscales (see Appendix Table 6).

Ethnocultural Empathy

Students were asked to answer 16 questions (e.g., I share the anger of those who face injustice because of their racial and ethnic backgrounds) from the Scale of Ethnocultural Empathy (ECE) that includes 4 scales: empathic feeling and expression, empathic perspective taking, acceptance of cultural differences, and empathic awareness [38]. This scale was shortened from 31 to 16 questions for relevancy and brevity (see Appendix Table 7). High internal consistency and test–retest reliability estimates of the original scale have been demonstrated [38].

Cultural Intelligence–Specified Indigenous Culture

Students were asked 10 items (out of 20 questions from the original measure; see Appendix Table 8) from a standardized scale examining cultural intelligence–specified for Indigenous culture (Cultural Intelligence Scale (CIS)) [39]. Items were on a 7-point Likert scale ranging from “1 = disagree strongly” to “7 = agree strongly” (e.g., I check the accuracy of my cultural knowledge as I interact with people from Indigenous cultures; I know the legal systems of Indigenous cultures). The original measure has demonstrated good reliability (α = .70–.86) and significant convergent and discriminant validity [39].

Multicultural Ethnic Identity

Students were asked to answer the 16-question Multigroup Ethnic Identity Measure (MEIM) [40]. Twelve items use a 4-point Likert scale (“1 = strongly agree” to “4 = strongly disagree”; e.g., I have a strong sense of belonging to my own ethnic group), and 4 items are open-ended and ask about cultural identity of self and of parents (e.g., my father’s ethnicity is?). Studies have demonstrated good reliability for both high school (α = .81) and college samples (α = .90) [40].

Analysis

Descriptive statistics were examined for all student characteristics including age, race/ethnicity, and gender. Additionally, paired t tests were conducted to examine changes in student responses between pre, post, and follow-up surveys for all study measures (i.e., Indigenous knowledge and beliefs, cultural intelligence–specified Indigenous knowledge, multicultural ethnicity identity, social justice, and ethnocultural empathy) with the exception noted above of missing data for four measures for the 2014 cohort post-test. Surveys for the three time points were linked using demographic information (birth date, zip code, gender), and a participant identification number was created. All analyses were performed in SAS, version 9.4 (SAS Institute, Cary, NC).

Results

Year 1

In year 1, there were a total of 60 students and 58 students (97%) filled out pre-lecture surveys; however, the number of participants that was used in the analysis is reduced given that the participants needed to have completed a survey in at least one other time period as is customary with paired t tests (see Table 2). Approximately half (55%) were male, 91% were White, and the mean age was 25.2 years (SD = 2.8); see Table 1. As shown in Table 2, students’ IKB summary scores increased significantly between pre- and post-lecture surveys (mean difference = 0.35, p < .0001) and between pre and follow-up (mean difference = 0.22, p < .0001). Student’s IKB summary scores, however, decreased significantly between post and follow-up (mean difference = − 0.09, p = .02). Student’s IKB summary attempt scores increased significantly between pre- and post-lecture surveys (mean difference = 0.42, p < .0001) and between pre and follow-up (mean difference = 0.36, p < .0001), but no differences were noted between post and follow-up (p = .69). Although student’s empathy, intelligence, and identity scores increased between pre and follow-up lecture, none were found to be statistically significant. Similarly, social justice scores decreased slightly between pre and follow-up, but the difference was non-significant.

Table 1 Sample demographics by year
Full size table
Table 2 Student scores from year 1 surveys (pre, post, and follow-up)
Full size table

Year 2

In year 2, there were a total of 60 students, of which 53 students (88%) filled out pre-lecture surveys and fewer represented in the analysis (see Table 3). Slightly less than half (45%) of the students were male, 81% were White, and the mean age was 23.8 years (SD = 1.8); see Table 1. Similar to year 1, student’s IKB summary scores at year 2 (Table 3) increased significantly between pre- and post-lecture surveys (mean difference = 0.15, p < .0001) and between pre and follow-up (mean difference = 0.09, p < .001). The decrease in IKB summary scores between post and follow-up was not significant (p = .11). Students’ IKB summary attempt scores at year 2 also increased significantly between pre- and post-lecture surveys (mean difference = 0.34, p < .0001) and between pre and follow-up (mean difference = 0.21, p < .0001). A significant decrease between student’s post and follow-up IKB summary attempt scores was noted (p = .02).

Table 3 Student scores from year 2 surveys (pre, post, and follow-up)
Full size table

Student’s empathy (ECE) scores at year 2 (Table 3) increased significantly between pre- and post-lecture surveys (mean difference = 0.24, p < .0001) but not between pre and follow-up (mean difference = 0.01, p = .88), whereas post-ECE scores were significantly higher than follow-up ECE scores (mean difference = 0.17, p = .03). Student intelligence (CIS) post and follow-up scores were significantly higher than pre-lecture scores (pre-post mean difference = 1.02, pre-follow-up mean difference = 0.46, p < .001), while post-CIS scores were significantly higher than follow-up CIS scores (mean difference = 0.50, p < .0001). No significant differences were noted in student’ social justice and identity scores between pre, post, or follow-up lecture surveys.

Discussion

This project tested the effects of adding seven didactic hours to a first-year medical school curriculum mostly delivered by Indigenous faculty. The curriculum was nested in a university with a mission of American Indian Health and a Center for American Indian Minority Health, which houses the American Indian medical student recruitment core.

Our results suggest that Indigenous health lectures have lasting effects on students’ knowledge up to at least 6 months. Knowledge is the first component in most models of cultural effectiveness, dexterity, and humility. In fact, three recommendations to address structural racism in medicine include the following: “First, learn about, understand, and accept the United States’ racist roots. Second, understand how racism has shaped our narrative about disparities. Third, define and name racism [41].” Similarly, in an international consensus statement around medical education, colonization, racism, and privilege were named as the strongest barriers to Indigenous health [42]. Authors expressed the importance of decolonizing western medical education through changes in the medical infrastructure led by Indigenous people.

The study intervention, however, had less strong of an effect on students’ growth in other domains. For instance, cultural intelligence was approaching significance in year 1 from pre-test to follow-up (mean difference = 0.23, p = .06) and was significant in year 2 from pre-test to follow-up (mean difference = 0.46, p = .006), indicating that there was a trend of improvement in this measure. While student’s ethnocultural empathy (ECE) scores at year 1 did not reach a level of significance, in year 2 (Table 2), ECE increased significantly between pre- and post-lecture surveys (mean difference = 0.24, p < .0001) but not between pre and follow-up (mean difference = 0.01, p = .88); see Table 3. Although there were improved effects, possibly due to updated content in the second year, the intervention did not have long-lasting effects on student’s empathy. This is interesting because students continued to maintain their knowledge about Indigenous people, which, one would think, would affect beliefs and, therefore, empathy, yet it did not in the long term. It is possible that although one learns and continues to retain the information, the effect of actively learning and discussing the topic affects empathy, but not in isolation and not over time. It is also possible that the dose was insufficient for lasting effects, which requires more testing.

Other programs that have been successful at improving the knowledge, skills, and beliefs of students around Indigenous health have added additional components besides didactic learning including experiential learning opportunities. For instance, the University of Arkansas Medical Sciences Northwest campus created an interdisciplinary program to address health disparities with a focus on Marshallese patient populations [43]. This program was comprised of two didactic lectures co-created by a Native Marshallese physician, an experiential component within the clinic, and a service learning project within the community. Students who participated in this program reported significantly improved cultural competence and readiness for interpersonal learning, in addition to improved knowledge, skills, and behaviors towards working with Marshallese patients. At the John A. Burns School of Medicine, Native Hawaiian specific cultural competence training began in 2003 and now electives include a Dean’s certificate in Hawaiian Native Health as well as the Kalaupapa service learning project [44,45,46]. Evaluations of this program indicated that over 95% of medical students in the Native Hawaiian health elective either agreed or strongly agreed that they have a better understanding of tools that can be used for Native Hawaiian people to heal from cultural trauma. Students also reported that they learned about Indigenous perspectives and beliefs and it reinforced their commitment to rural and underserved areas [45]. It appears that experiential learning opportunities, whether in clinic or in community, provide a boost to the effects of teaching didactic content resulting in improved readiness and skills to work with Indigenous patients.

In the case of social justice beliefs and cultural humility in this study, neither was significantly improved at post-test or follow-up. These domains appear more difficult to change than knowledge and skills. However few, there are programs that have had success in teaching social justice, which may provide strategic examples for other developing medical curriculums. In particular, at the University of Otago in New Zealand, a study was conducted to assess social accountability within Indigenous medical education. Social accountability is a similar concept to social justice. Stakeholders including students, teachers, and community members were interviewed about how this Indigenous curriculum created social accountability to address Indigenous health disparities. They reported that they believed that the following components of the curriculum created social accountability for health professionals: (1) advocacy for Māori health in the community, university, and clinics; (2) putting social engagement into practice; and (3) creating a transformative practice around advocating for Māori health [47]. The University of Otago puts social accountability into practice through its commitment to address Māori health disparities mandated through the Treaty of Waitangi, using a strategic plan to recruit Māori students and teaching students more Indigenous health content than any other university in the world [48]. Through qualitative interviews, researchers discovered that this Indigenous health curriculum taught students how to advocate for patients and become change agents to reduce Indigenous health disparities. In other words, this curriculum has resulted in improved social justice beliefs and actions of students. In this case, a government decree, recruitment efforts, and a larger quantity of hours taught about Indigenous health resulted in improvements in social justice beliefs.

Although not Indigenous focused, a preclinical social justice program at Mount Sinai requires “a didactic course, faculty and student mentorship, research projects in social justice, longitudinal policy and advocacy service projects, and a career seminar series [49].” Students were interviewed in regard to their learning from this program, and the most common theme was that students were able to apply their learning about health disparities and social determinants of health to clinical encounters. This is an important outcome and highlights the need to teach social justice in a variety of ways (e.g., didactic, research project, and service project) with mentorship for both students and faculty.

In searching for best practices to improve social justice, an area that did not see improvement in our study, it is important to look to other programs that have had long-standing missions around social justice to guide future projects given the lack of evaluation research in this area. In particular, at the University of Hawaii at Manoa, core competencies in medical social justice cover knowledge, skills, and beliefs regarding social determinants of health, health justice advocacy, and community needs [50]. The purpose of this social justice program is to implement a student-driven curriculum to expose learners to various components of social justice in health and medicine by using their own inputs for content and design [51]. At the Geisel School of Medicine at Dartmouth, a social justice expert group convened that led to curriculum revision and improved student competencies in social justice [52]. The addition of student-led curriculum and social justice competencies creates a bar for all students to meet and also requires more faculty to put social justice content into their lectures [53] which may prove to increase the social justice competency of students.

Limitations

The purpose of the research project was to test the learning of medical students before and after they received a series of seven lectures. No other program in the USA has implemented Indigenous health content within the core curriculum for all students and tested it. Therefore, the first goal of this investigation was to determine if the training was feasible and to test the hypothesis that training has a positive effect on students’ learning in the areas of knowledge, social justice, cultural empathy, cultural intelligence, and cultural humility. A one-group (non-randomized), pre-post-test was implemented using a quasi-experimental design and a convenience sample of first-year medical students. The author recognizes that with this design, there are substantial threats to both internal and external validity given the selection of participants and the lack of control groups and randomization. Measures were condensed in this study to reduce respondent fatigue. Due to modifications to some of the measures, it is possible that the reliability and consistency of these measures were not maintained. Further, while our team tried to make improvements to the lecture during year 2, small differences to the intervention exist and therefore cohorts 1 and 2 did not receive the exactly the same intervention. A power analysis was not completed before the data was analyzed. Given this limitation, some of the analyses may have lacked power possibly explaining some of the null results.

This intervention included a lecture series created by local stakeholders and was tailored to represent tribes of the region and may not be representative to other communities and tribes. For instance, stakeholders were very outspoken about the needs of the local Indigenous community and what they wanted their healthcare professionals to know about them, which may not be the case in all other communities. It is important to note with over 570 tribes in the USA alone, one must not generalize and assume similarities exist between them.

Implications

There are many barriers for effective healthcare for Indigenous people. Without a medical provider who knows about your culture and health beliefs available at your local clinic, or an administrator that advocates for your community, healthcare is unlikely to effectively service Indigenous patients in that community. According to the AAMC, in 2015, only 0.2% of US medical school (not including osteopathic medicine programs) applicants were Indigenous, only 21 of 19,553 US medical student graduates in 2017 were Indigenous, only 0.6% are active physicians, and only 0.5% are full-time faculty [54]. Further, Indigenous representation in medical education has decreased, not increased, in the USA over the last several years and there are very few Indigenous faculty in medical schools (0.1%) or in the sciences [55]. Fewer underrepresented potential faculty are being hired as assistant professors when compared to those from well-represented groups, and simulation models suggest that this inequity will not change for the next 60 years at least [56]. Therefore, more needs to be done to include Indigenous students and faculty in the medical field. Content important to underrepresented groups should be reflected in the curriculum; however, this cannot be done without Indigenous advocates and allies.

Future Directions

The solution to addressing Indigenous health disparities is multifaceted and requires intervention at multiple points. Intentionality is important. Programs that set out to teach values and beliefs around patient-centeredness and social justice advocacy have succeeded in reaching these goals [57]. For example, to improve social justice beliefs and behaviors, successful programs create core competencies and train faculty. Further, medical school initiatives and missions focused around underserved populations increase the number of providers practicing in those areas [58]. It is important to be intentional and strategic about the goal of addressing health disparities and also address the root of the problem. For deep structural changes to happen with regard to Indigeneity, decolonizing approaches could address rebalancing of systems of hierarchy in educational systems, critically analyzing the role of colonization in medicine and healthcare, and addressing root causes of disparate outcomes like racism and inequity. Although the Indigenous curriculum in this study resulted in some positive learning outcomes, future projects will work to be more holistic in nature including engaging more Indigenous medical education experts, more patient voices, and more administrative collaboration, as well as work to create learning objectives, core competencies, and a variety of and an increased amount of learning opportunities such as mentoring, service learning, and student-led initiatives.

Conclusion

Culturally appropriate care in the USA is critical to the improved health of Indigenous people and requires a trained health workforce. This study found that medical student knowledge could be persistently increased by a lecture series, but other key values were less strongly affected. Future work should aim to discover what types of interventions (e.g., didactic vs. experiential), what amount, and at what intervals of training will produce significant improvements in healthcare providers’ knowledge, beliefs, and skills. At this intersection of health justice and education justice, Indigenous health content is critical; it is especially important for programs located on or near communities with higher rates of Indigenous patients who can work to reduce health disparities directly.