Abstract
The relative impact of admissions factors and curricular measures on the first medical licensing exam (United States Medical Licensing Exam [USMLE] Step 1) scores is examined. The inclusion of first-year and second-year curricular measures nearly doubled the variance explained in Step 1 scores from the amount explained by the combination of preadmission demographic characteristics and admissions factors. In addition, the relationship between the Medical College Admissions Test (MCAT) and Step 1 scores becomes counterintuitive in models that include curricular measures, where students with the lowest combined admissions metrics (MCAT, grade-point average) score higher, on average, than those with some of the highest admissions metrics. Overreliance on traditional metrics in admissions decisions can exclude students from medical school who have the ability to succeed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Addams, A. N., Bletzinger, R. B., Sondheimer, H. M., White, S. E., & Johnson, L. M. (2010). Roadmap to diversity: Integrating holistic review practices into medical school admission processes. Washington, DC: Association of American Medical Colleges.
Andriole, D. A., & Jeffe, D. B. (2010). Prematriculation variables associated with suboptimal outcomes for the 1994–1999 cohort of US medical school matriculants. JAMA, 304(11), 1212–1219.
Association of American Medical Colleges (AAMC). (2014). Table 17: MCAT scores and GPA for applicants and matriculants to U.S. medical schools, 2003–2013. Retrieved from https://www.aamc.org/download/321494/data/factstable17.pdf.
Association of American Medical Colleges (AAMC). (2015). Roadmap to residency: Understanding the process of getting into residency. Retrieved from http://members.aamc.org/eweb/upload/2015%20AAMC%20Roadmap%20to%20Residency.pdf.
Association of American Medical Colleges (AAMC). (n.d.a.). GME funding: How to fix the doctor shortage [Website]. Retrieved from https://www.aamc.org/advocacy/campaigns_and_coalitions/fixdocshortage/
Association of American Medical Colleges (AAMC). (n.d.b.). Taking the MCAT [Website]. Retrieved from https://students-residents.aamc.org/applying-medical-school/taking-mcat-exam/.
Astin, A. W. (1993). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. Phoenix, AZ: Oryx Press.
Astin, A. W., & Antonio, A. L. (2012). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. Lanham, MD: Rowman & Littlefield Publishers.
Burns, E. R., & Garrett, J. (2015). Student failures on first-year medical basic science courses and the USMLE Step 1: A retrospective study over a 20-year period. Anatomical Sciences Education, 8(2), 120.
Davis, D., Dorsey, J. K., Franks, R. D., Sackett, P. R., Searcy, C. A., & Zhao, X. (2013). Do racial and ethnic group differences in performance on the MCAT exam reflect test bias? Academic Medicine, 88(5), 593–602.
Donnon, T., Paolucci, E. O., & Violato, C. (2007). The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Academic Medicine, 82(1), 100–106.
Emmett, R. I. (1993). A descriptive analysis of medical school application forms. Academic Medicine, 68(7), 564–569.
Enarson, C., & Cariaga-Lo, L. (2001). Influence of curriculum type on student performance in the United States Medical Licensing Examination Step 1 and Step 2 exams: Problem-based learning vs. lecture-based curriculum. Medical Education, 35(11), 1050–1055.
Eva, K. W., Reiter, H. I., Rosenfeld, J., & Norman, G. R. (2004). The ability of the multiple mini-interview to predict preclerkship performance in medical school. Academic Medicine, 79(10), S40–S42.
Fredieu, J. R., & Snyder, C. W. (2015). Positive impact of a master of science in applied anatomy program on USMLE Step 1 performance. Anatomical Sciences Education, 8(1), 31–36.
Gandy, R. A., Herial, N. A., Khuder, S. A., & Metting, P. J. (2008). Use of curricular and extracurricular assessments to predict performance on the United States Medical Licensing Examination (USMLE) Step 1: A multi-year study. Learning Assistance Review, 13(2), 27–35.
Gohara, S., Shapiro, J. I., Jacob, A. N., Khuder, S. A., Gandy, R. A., Metting, P. J., et al. (2011). Joining the conversation: Predictors of success on the United States Medical Licensing Examinations (USMLE). Learning Assistance Review, 16(1), 11–20.
Green, M., Jones, P., & Thomas, J. X., Jr. (2009). Selection criteria for residency: Results of a national program directors survey. Academic Medicine, 84(3), 362–367.
Hoffman, K., Hosokawa, M., Blake, R., Jr., Headrick, L., & Johnson, G. (2006). Problem-based learning outcomes: Ten years of experience at the University of Missouri—Columbia School of Medicine. Academic Medicine, 81(7), 617–625.
Julian, E. R. (2005). Validity of the medical college admissions test for predicting medical school performance. Academic Medicine, 80(10), 910–917.
Kleshinski, J., Khuder, S. A., Shapiro, J. I., & Gold, J. P. (2009). Impact of preadmission variables on USMLE Step 1 and Step 2 performance. Advances in Health Sciences Education, 14(1), 69–78.
Koenig, J. A., Sireci, S. G., & Wiley, A. (1998). Evaluating the predictive validity of MCAT scores across diverse applicant groups. Academic Medicine, 73(10), 1095–1106.
Kreiter, C. D., & Kreiter, Y. (2007). A validity generalization perspective on the ability of undergraduate GPA and the medical college admission test to predict important outcomes. Teaching and Learning in Medicine, 19(2), 95–100.
McGaghie, W. C., Cohen, E. R., & Wayne, D. B. (2011). Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions? Academic Medicine, 86(1), 48–52.
Milem, J. F., O’Brien, C. L., & Bryan, W. P. (2013). The matriculated student: Assessing the impact of holistic review. In Roadmap to excellence: Key concepts for evaluating the impact of medical school holistic admission (pp. 19–29). Washington, DC: Association of American Medical Colleges. https://members.aamc.org/eweb/upload/Holistic%20Review%202013.pdf.
National Residency Match Program (NRMP). (2014). Charting outcomes in the match: Characteristics of applicants who matched to their preferred specialty in the 2014 main residency match, (5th ed.). Washington, DC: National Residency Match Program. Retrieved from http://www.nrmp.org/wp-content/uploads/2014/09/Charting-Outcomes-2014-Final.pdf.
Peterson, C. A., & Tucker, R. P. (2005). Medical gross anatomy as a predictor of performance on the USMLE Step 1. The Anatomical Record Part b: The New Anatomist, 283(1), 5–8.
Siu, E., & Reiter, H. I. (2009). What’s worked and what hasn’t as a guide toward predictive admissions tool development. Advances in Health Sciences Education, 14(5), 759–775.
StataCorp. (2015). Stata Statistical Software: Release 14. College Station, TX: StataCorp LP.
United States Medical Licensing Examination (USMLE). (2014). USMLE score interpretation guide. Retrieved from http://www.usmle.org/pdfs/transcripts/USMLE_Step_Examination_Score_Interpretation_Guidelines.pdf.
United States Medical Licensing Examination (USMLE). (n.d.a.). About the USMLE [Website]. Retrieved from http://www.usmle.org/about/.
United States Medical Licensing Examination (USMLE). (n.d.b.). Scores and transcripts [Website]. Retrieved from http://www.usmle.org/transcripts/.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sesate, D.B., Milem, J.F., McIntosh, K.L. et al. Coupling Admissions and Curricular Data to Predict Medical Student Outcomes. Res High Educ 58, 295–312 (2017). https://doi.org/10.1007/s11162-016-9426-y
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11162-016-9426-y