Skip to main content
SpringerLink
Log in

Reimagining Merit and Representation: Promoting Equity and Reducing Bias in GME Through Holistic Review

  • Empirical Report
  • Published:
Academic Psychiatry Aims and scope Submit manuscript

Abstract

Objective

This study aims to evaluate the capacity of a holistic review process in comparison with non-holistic approaches to facilitate mission-driven recruitment in residency interview screening and selection, with particular attention to the promotion of race equity for applicants underrepresented in medicine (URM).

Methods

Five hundred forty-seven applicants to a psychiatry residency program from US allopathic medical schools were evaluated for interview selection via three distinct screening rubrics—one holistic approach (Holistic Review; HR) and two non-holistic processes: Traditional (TR) and Traditional Modified (TM). Each applicant was assigned a composite score corresponding to each rubric, and the top 100 applicants in each rubric were identified as selected for interview. Odds ratios (OR) of selection for interview according to URM status and secondary outcomes, including clinical performance and lived experience, were measured by analysis of group composition via univariate logistic regression.

Results

Relative to Traditional, Holistic Review significantly increased the odds of URM applicant selection for interview (TR-OR: 0.35 vs HR-OR: 0.84, p < 0.01). Assigning value to lived experience and de-emphasizing USMLE STEP1 scores contributed to the significant changes in odds ratio of interview selection for URM applicants.

Conclusions

Traditional interview selection methods systematically exclude URM applicants from consideration without due attention to applicant strengths or potential contribution to clinical care. Conversely, holistic screening represents a structural intervention capable of critically examining measures of merit, reducing bias, and increasing URM representation in residency recruitment, screening, and selection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Talamantes E, Henderson M, Fancher T, Mullan F. Closing the gap - making medical school admissions more equitable. N Engl J Med. 2019;380:803–5.

    Article  Google Scholar 

  2. Deville C, Hwang W-T, Burgos R, Chapman C, Both S, Thomas CR. Diversity in graduate medical education in the United States by race, ethnicity, and sex, 2012. JAMA Intern Med. 2015;175:1706–9.

    Article  Google Scholar 

  3. An American Crisis. The growing absence of black men in medicine and science. Washington, DC: The National Academies Press; 2018.

    Google Scholar 

  4. Komaromy M, Grumbach K, Drake M, Vranizan K, Lurie N, Keane D, et al. The role of Black and Hispanic physicians in providing health care for underserved populations. N Engl J Med. 1996;334:1305–10.

    Article  CAS  Google Scholar 

  5. Walker KO, Moreno G, Grumbach K. The association among specialty, race, ethnicity, and practice location among California physicians in diverse specialties. J Natl Med Assoc. Elsevier Masson SAS. 2012;104:46–52. https://doi.org/10.1016/S0027-9684(15)30126-7.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Hsu P, Balderas-Medina Anaya Y, Hayes Bautista D. 5 Centuries to reach parity: an analysis of how long it will take to address California’s Latino physician shortage. UCLA Lat Policy Polit Initiat. 2018; Available from: https://latino.ucla.edu/wp-content/uploads/2018/10/5-Centuries-to-Reach-Parity-UCLA-LPPI.pdf. Accessed 8 December 2019.

  7. Nivet MA. Diversity 3.0: a necessary systems upgrade. Acad Med. 2011;86:1487–9.

    Article  Google Scholar 

  8. Edmond MB, Deschenes JL, Eckler M, Wenzel RP. Racial bias in using USMLE Step 1 scores to grant internal medicine residency interviews. Acad Med. 2001;76:1253–6.

    Article  CAS  Google Scholar 

  9. Boatright D, Ross D, O’Connor P, Moore E, Nunez-Smith M. Racial disparities in medical student membership in the alpha omega alpha honor society. JAMA Intern Med. 2017;177:659–65.

    Article  Google Scholar 

  10. Ross DA, Boatright D, Nunez-Smith M, Jordan A, Chekroud A, Moore EZ. Differences in words used to describe racial and gender groups in Medical Student Performance Evaluations. PLoS One. 2017;12:e0181659.

    Article  Google Scholar 

  11. Wijesekera T, Kim M, Moore EZ, Sorenson O, Ross DA. All other things being equal: exploring racial and gender disparities in medical school honor society induction. Acad Med. 2019;94:562–9.

    Article  Google Scholar 

  12. Aibana O, Swails JL, Flores RJ, Love LT. Bridging the gap: holistic review to increase diversity in graduate medical education. Acad Med. 2019;94:1137–41.

    Article  Google Scholar 

  13. Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student perspectives on the “step 1 climate” in preclinical medical education. Acad Med. 2019;94:302–4.

    Article  Google Scholar 

  14. Freeman AD, Freeman AD. Legitimizing racial discrimination through antidiscrimination law: a critical review of supreme court doctrine. Minn Law Rev. 1978;62.

  15. Boyd RW. The case for desegregation. Lancet Elsevier Ltd. 2019;393:2484–5.

    Article  Google Scholar 

  16. Coleman AL, Lipper KE, Taylor TE, Palmer SR. Roadmap to diversity and educational excellence: key legal and educational policy foundations for medical schools. A Report for the Association of American Medical Colleges. Washington, DC; 2014. Available from: https://store.aamc.org/downloadable/download/sample/sample_id/192

  17. Dort JM, Trickey AW, Kallies KJ, Joshi ART, Sidwell RA, Jarman BT. Applicant characteristics associated with selection for ranking at independent surgery residency programs. J Surg Educ Elsevier. 2015;72:e123–9.

    Article  Google Scholar 

  18. UCSF. URM Definition [Internet]. 2019. p. 2019. Available from: https://diversity.ucsf.edu/URM-definition. Accessed 8 December 2019.

  19. Cuddy MM, Swanson DB, Clauser BE. A multilevel analysis of examinee gender and USMLE step 1 performance. Acad Med. 2008;83:58–62.

    Article  Google Scholar 

  20. Veloski JJ, Callahan CA, Xu G, Hojat M, Nash DB. Prediction of students’ performances on licensing examinations using age, race, sex, undergraduate GPAs, and MCAT scores. Acad Med. 2000;75:S28–30.

    Article  CAS  Google Scholar 

  21. Girotti JA, Chanatry JA, Clinchot DM, McClure SC, Sein AS, Walker IW, et al. Investigating group differences in examinees’ preparation for and performance on the new MCAT exam. Acad. Med. 2019.

  22. Moynahan KF. The current use of United States medical licensing examination step 1 scores: holistic admissions and student well-being are in the balance. Acad Med. 2018;93:963–5.

    Article  Google Scholar 

  23. Bewick V, Cheek L, Ball J. Statistics review 8: qualitative data - tests of association. Crit Care. 2004;8:46–53.

    Article  Google Scholar 

  24. Bonilla-Silva E. Rethinking racism: toward a structural interpretation. Am Sociol Rev. 2006;62:465.

    Article  Google Scholar 

  25. Agapoff JR, Tonai C, Eckert DM, Gavero G, Goebert DA. Challenges and perspectives to the rise in general psychiatry residency applications. Acad Psychiatry. 2018;42:674–6.

    Article  Google Scholar 

  26. AAMC. 2019 FACTS: Electronic Residency Application Service (ERAS) [Internet]. 2020 [cited 2020 Feb 23]. Available from: https://www.aamc.org/data-reports/students-residents/interactive-data/2019-facts-electronic-residency-application-service-eras-data. Accessed 23 February 2020.

  27. Johnson TJ, Ellison AM, Dalembert G, Fowler J, Dhingra M, Shaw K, et al. Implicit bias in pediatric academic medicine. J Natl Med Assoc. Elsevier Inc. 2017;109:156–63. https://doi.org/10.1016/j.jnma.2017.03.003.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Iglehart JK. Diversity dynamics - challenges to a representative U.S. medical workforce. N Engl J Med. 2014;371:1471–4.

    Article  Google Scholar 

  29. Ford CL, Airhihenbuwa CO. The public health critical race methodology: praxis for antiracism research. Soc Sci Med. 2010;71:1390–8.

    Article  Google Scholar 

  30. Krieger N, Alegría M, Almeida-Filho N, Barbosa da Silva J, Barreto ML, Beckfield J, et al. Who, and what, causes health inequities? Reflections on emerging debates from an exploratory Latin American/North American workshop. J Epidemiol Community Health. 2010;64:747–9.

    Article  Google Scholar 

  31. Sklar DP. Racial violence, academic medicine, and academic medicine. Acad Med. 2015;90:1577–80.

    Article  Google Scholar 

  32. Mgbako O. The Unicorn. JAMA. 2019;321:149–50. Available from: http://www.nber.org/papers/w24787.

    Article  Google Scholar 

  33. Teherani A, Hauer KE, Fernandez A, King TE, Lucey C. How small differences in assessed clinical performance amplify to large differences in grades and awards: a cascade with serious consequences for students underrepresented in medicine. Acad Med. 2018;93:1286–92.

    Article  Google Scholar 

  34. Smith DG. Building institutional capacity for diversity and inclusion in academic medicine. Acad Med. 2012;87:1511–5.

    Article  Google Scholar 

  35. ACGME. Common program requirements (residency) [Internet]. Acgme. 2019. Available from: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRResidency2019.pdf. Accessed 23 February 2020.

  36. Bandiera G, Abrahams C, Ruetalo M, Hanson MD, Nickell L, Spadafora S. Identifying and promoting best practices in residency application and selection in a complex academic health network. Acad Med. 2015;90:1594–601.

    Article  Google Scholar 

  37. Fernandez A. Further incorporating diversity, equity, and inclusion into medical education research. Acad Med. 2019;94:S5–6.

  38. Cyrus KD. Medical education and the minority tax. JAMA. 2017;317:1833–4.

    Article  Google Scholar 

  39. Osseo-asare A, Balasuriya L, Huot SJ, Keene D, Berg D, Nunez-smith M, et al. Minority resident physicians’ views on the role of race/ethnicity in their training experiences in the workplace. JAMA Netw Open. 2018;1:1–11.

    Article  Google Scholar 

  40. Palepu A, Carr PL, Friedman RH, Amos H, Ash AS, Moskowitz MA. Minority faculty and academic rank in medicine. J Am Med Assoc. 1998;280:767–71.

    Article  CAS  Google Scholar 

  41. Diaz T, Navarro JR, Chen EH. An institutional approach to fostering inclusion and addressing racial bias: implications for diversity in academic medicine. Teach Learn Med Routledge; 2019;0:1–7.

Download references

Acknowledgments

Thanks go to Evelyn Carter, PhD, for her contribution to conception of this Holistic Review rubric. Additional thanks to Dr. Theodore Hall and Lingqi Tan, PhD, for their insightful feedback. We additionally thank members of the “2018-19 Recruitment Task Force” including Drs. Erin Samplin, T.C. Scotton, Maria Gabriela Aguilera Nunez, and Evelyn Nelson. Finally, we thank each of the many residents who volunteered their time and energy to participate in Holistic Review.

Funding

N.B. is supported the by the APA SAMHSA Minority Fellowship, Grant 1H79SM080388-01. E.B. is supported by funding from the Los Angeles County Department of Probation and from the National Institute on Drug Abuse of the National Institutes of Health under the AACAP NIDA K12 program, Grant # K12DA000357, the California Community Foundation, and the UCLA Pritzker Center for Strengthening Children and Families. Dr. Goodsmith was supported by the VA Office of Academic Affiliations through the UCLA National Clinician Scholars Program. The contents do not represent the views of the U.S. Department of Veterans Affairs, the United States Government, or affiliated institutions.

Author information

Authors and Affiliations

  1. University of California, Los Angeles, Los Angeles, CA, USA

    Nicolás E. Barceló, Sonya Shadravan, Christine R. Wells, Brittany Tarrant, Trevor Shaddox, Yvonne Yang, Eraka Bath & Katrina DeBonis

  2. Greater Los Angeles Veterans Affairs Healthcare System, Los Angeles, CA, USA

    Nichole Goodsmith

Authors
  1. Nicolás E. Barceló
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sonya Shadravan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christine R. Wells
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nichole Goodsmith
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Brittany Tarrant
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Trevor Shaddox
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yvonne Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Eraka Bath
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Katrina DeBonis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolás E. Barceló.

Ethics declarations

This study qualified for exemption by the Institutional Review Boards at UCLA Medical Center.

Disclosures

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barceló, N.E., Shadravan, S., Wells, C.R. et al. Reimagining Merit and Representation: Promoting Equity and Reducing Bias in GME Through Holistic Review. Acad Psychiatry 45, 34–42 (2021). https://doi.org/10.1007/s40596-020-01327-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40596-020-01327-5

Keywords

Search

Navigation