Abstract
Many medical schools can successfully predict overall performance on the initial licensing examination by the end of the second year. Unfortunately, the time left to remediate students before students take exams is short. The goal of this project was to determine the feasibility of predicting licensing examination scores at the end of the first year of training. Data from three cohorts of medical students, each made up of about 230 individuals from the graduating classes of 2013 to 2015 were used. A series of prediction equations for licensing examination scores, based on class performance, was developed from the first cohort of students. The best predictive equation was applied to the second cohort of students to generate a predicted examination score which was then compared to the actual examination score. This approach was cross-validated with another cohort of students. The R-values for the selected predictive equations ranged from 0.691 to 0.814, accounting from about 48 to 66 % of the variance in actual scores. The mean difference between predicted and actual examination scores was 19 points on a three-digit scale ranging from 200 to 800 for the second cohort and 25 points for the third cohort. The data show that initial licensing examination scores can be successfully predicted 1 year before students take the exam. Success at this task would give medical schools the ability to utilize their available resources efficiently to provide additional assistance to those students in need.
Similar content being viewed by others
References
Hoffman KI. The USMLE, the NBME subject examinations, and assessment of individual academic achievement. Acad Med. 1993;68:740–7.
Williams RG. Use of NBME and USMLE examinations to evaluate medical education programs. Acad Med. 1993;68:748–52.
Berner ES, Brooks CM, Erdmann JB. Use of the USMLE to select residents. Acad Med. 1993;68:753–9.
Thundiyil JG, Modica RF, Silvestri S, Papa L. Do United States Medical Licensing Examination (USMLE) scores predict in-training test performance for emergency medicine residents? J Emerg Med. 2010;38:65–9.
Wiley A, Koenig JA. The validity of the Medical College Admission Test for predicting performance in the first two years of medical school. Acad Med. 1996;71 Suppl 10:S83–5.
Coumarbatch J, Robinson L, Thomas R, Bridge PD. Strategies for identifying students at risk for USMLE step 1 failure. Fam Med. 2010;42:105–10.
Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007;82:100–6.
Julian ER. Validity of the Medical College Admission Test for predicting medical school performance. Acad Med. 2005;80:910–7.
Kleshinski J, Khuder SA, Shapiro JI, Gold JP. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv Health Sci Educ: Theory Pract. 2009;14:69–78.
Koenig JA, Sireci SG, Wiley A. Evaluating the predictive validity of MCAT scores across diverse applicant groups. Acad Med. 1998;73:1095–106.
Zhao X, Oppler S, Dunleavy D, Kroopnick M. Validity of four approaches of using repeaters' MCAT scores in medical school admissions to predict USMLE Step 1 total scores. Acad Med. 2010;85 Suppl 10:S64–7.
Dunleavy DM, Kroopnick MH, Dowd KW, Searcy CA, Zhao X. The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001–2004 matriculants. Acad Med. 2013;88:666–71.
Evans P, Wen FK. Does the medical college admission test predict global academic performance in osteopathic medical school? J Am Osteopath Assoc. 2007;107:157–62.
Gonnella JS, Erdmann JB, Hojat M. An empirical study of the predictive validity of number grades in medical school using 3 decades of longitudinal data: implications for a grading system. Med Educ. 2004;38:425–34.
Hamdy H, Prasad K, Anderson MB, Scherpbier A, Williams R, Zwierstra R, et al. BEME systematic review: predictive values of measurements obtained in medical schools and future performance in medical practice. Medical Teacher. 2006;28:103–16.
Dawson B, Trapp RG. Basic & clinical biostatistics. 4th ed. New York: Lange Medical Books/McGraw-Hill; 2004.
Glaros AG, Adkison LR. Use of early indicators to predict success on COMLEX. Paper presented at: Annual meeting of the American Association of Colleges of Osteopathic Medicine; 2013 Apr 24–27; Baltimore, MD.
Acknowledgments
We thank Dr. Richard Winslow for his careful reading of an earlier draft of this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Glaros, A.G., Hanson, A. & Adkison, L.R. Early Prediction of Medical Student Performance on Initial Licensing Examinations. Med.Sci.Educ. 24, 291–295 (2014). https://doi.org/10.1007/s40670-014-0053-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40670-014-0053-y