Abstract
Many schools seek to predict performance on national exams required for medical school graduation using prematriculation and medical school performance data. The need for targeted intervention strategies for at-risk students has led much of this interest. Assumptions that preadmission data and high stakes in-house medical exams correlate strongly with national standardized exam performance needs to be examined. Looking at prematriculation data for predicting USMLE Step 1 performance, we found that MCAT exam totals and math-science GPA had the best prediction from a set of prematriculation values (adjusted R 2 = 11.7 %) for step 1. The addition of scores from the first medical school exam improved our predictive capabilities with a linear model to 27.9 %. As we added data to the model, we increased our predictive values as expected. However, it was not until we added data from year 2 exams that we started to get step 1 prediction values that exceeded 50 %. Stepwise addition of more exams in year two resulted in much higher predictive values but also led to the exclusion of many early variables. Therefore, our best step 1 predictive value of around 76.7 % consisted of three variables from a total of 37. These data suggest that the preadmission information is a relatively poor predictor of licensure exam performance and that including class exam scores allows for much more accurate determination of students who ultimately proved to be at risk for performance on their licensure exams. The continuous use of this data, as it becomes available, for assisting at-risk students is discussed (251).
Similar content being viewed by others
References
Richardson PH, Winder B, Briggs K, Tydeman C. Grade predictions for school-leaving examinations: do they predict anything? Med Educ. 1998;32(3):294–7.
Collins JP, White GR, Kennedy JA. Entry to medical school: an audit of traditional selection requirements. Med Educ. 1995;29(1):22–8.
Husbands A, Mathieson A, Dowell J, Cleland J, MacKenzie R. Predictive validity of the UK clinical aptitude test in the final years of medical school: a prospective cohort study. BMC Med Educ. 2014;14:88. doi:10.1186/1472-6920-14-88.
Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007;82(1):100–6. doi:10.1097/01.ACM.0000249878.25186.b7.
Dunleavy DM, Kroopnick MH, Dowd KW, Searcy CA, Zhao X. The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001-2004 matriculants. Acad Med. 2013;88(5):666–71. doi:10.1097/ACM.0b013e3182864299.
Kreiter CD, Kreiter Y. A validity generalization perspective on the ability of undergraduate GPA and the medical college admission test to predict important outcomes. Teach Learn Med. 2007;19(2):95–100. doi:10.1080/10401330701332094.
Siu E, Reiter HI. Overview: what’s worked and what hasn’t as a guide towards predictive admissions tool development. Adv Health Sci Educ Theory Pract. 2009;14(5):759–75. doi:10.1007/s10459-009-9160-8.
Kleshinski J, Khuder SA, Shapiro JI, Gold JP. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv Health Sci Educ Theory Pract. 2009;14(1):69–78. doi:10.1007/s10459-007-9087-x.
Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14(1):34–42. doi:10.1207/s15328015tlm1401_9.
Al Shawwa L, Abulaban AA, Abulaban AA, Merdad A, Baghlaf S, Algethami A, et al. Factors potentially influencing academic performance among medical students. Adv Med Educ Pract. 2015;6:65–75. doi:10.2147/AMEP.S69304.
Adam J, Bore M, Childs R, Dunn J, McKendree J, Munro D et al. Predictors of professional behaviour and academic outcomes in a UK medical school: a longitudinal cohort study. Med Teach. 2015:1-13. doi:10.3109/0142159x.2015.1009023.
Puddey IB, Mercer A. Predicting academic outcomes in an Australian graduate entry medical programme. BMC Med Educ. 2014;14:31. doi:10.1186/1472-6920-14-31.
Dickman RL, Sarnacki RE, Schimpfhauser FT, Katz LA. Medical students from natural science and nonscience undergraduate backgrounds. Similar academic performance and residency selection. JAMA. 1980;243(24):2506–9.
Stratton TD, Elam CL. A holistic review of the medical school admission process: examining correlates of academic underperformance. Med Educ Online. 2014;19:22919. doi:10.3402/meo.v19.22919.
DeZee KJ, Magee CD, Rickards G, Artino Jr AR, Gilliland WR, Dong T, et al. What aspects of letters of recommendation predict performance in medical school? Findings from one institution. Acad Med. 2014;89(10):1408–15. doi:10.1097/acm.0000000000000425.
McManus IC, Woolf K, Dacre J, Paice E, Dewberry C. The academic backbone: longitudinal continuities in educational achievement from secondary school and medical school to MRCP(UK) and the specialist register in UK medical students and doctors. BMC Med. 2013;11:242. doi:10.1186/1741-7015-11-242.
Neame RL, Powis DA, Bristow T. Should medical students be selected only from recent school-leavers who have studied science? Med Educ. 1992;26(6):433–40.
Miller B, Dzwonek B, McGuffin A, Shapiro JI. From LCME probation to compliance: the Marshall University Joan C Edwards School of Medicine experience. Adv Med Educ Pract. 2014;5:377–82. doi:10.2147/amep.s70891.
Harden RM. What is a spiral curriculum? Med Teach. 1999;21(2):141–3. doi:10.1080/01421599979752.
Acknowledgments
We would like to thank Ms. Carrie Rockel for editorial assistance with this manuscript and Dr. Tracey LeGrow, Associate Dean for Academic Standards, for her insights on how she and her office currently advises “at risk” students.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gullo, C.A., McCarthy, M.J., Shapiro, J.I. et al. Predicting Medical Student Success on Licensure Exams. Med.Sci.Educ. 25, 447–453 (2015). https://doi.org/10.1007/s40670-015-0179-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-015-0179-6