Skip to main content

Advertisement

SpringerLink
Log in

Development of Statistical Models to Predict Medical Student Performance on the USMLE Step 1 as a Catalyst for Deployment of Student Services

  • Original Research
  • Published:
Medical Science Educator Aims and scope Submit manuscript

Abstract

The purpose of this study was to explore whether useful regression models could be developed to predict student performance on the US Medical Licensing Examination (USMLE) Step 1, using pre-matriculation data and student performance outcomes available as early in the curriculum as possible. Additionally, we sought to determine if we could establish a model by which we could identify students who may need supplemental instruction early in the curriculum. Regression modeling revealed that a fairly strong predictive relationship exists when combining certain internally developed summative assessments and the National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination (CBSE) with pre-matriculation variables. However, in our best regression model, > 40% of the overall variance in Step 1 scores could not be explained by the model. As a consequence, we propose using this type of modeling to facilitate timely deployment of student services to support individual students who are struggling. Future studies will focus on understanding what additional human factors fill this gap including student motivation, affect, and the factors that help students become “expert learners” (e.g., interactions with senior students, identification of test preparation materials).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007;82(1):100–6. doi:10.1097/01.ACM.0000249878.25186.b7.

    Article  Google Scholar 

  2. Wiley A, Koenig JA. The validity of the Medical College Admission Test for predicting performance in the first two years of medical school. Acad Med. 1996;71(10 Suppl):S83–5.

    Article  Google Scholar 

  3. Julian ER. Validity of the Medical College Admission Test for predicting medical school performance. Acad Med. 2005;80(10):910–7.

    Article  Google Scholar 

  4. Kleshinski J, Khuder SA, Shapiro JI, Gold JP. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv Health Sci Educ Theory Pract. 2009;14(1):69–78. doi:10.1007/s10459-007-9087-x.

    Article  Google Scholar 

  5. Gonnella JS, Erdmann JB, Hojat M. An empirical study of the predictive validity of number grades in medical school using 3 decades of longitudinal data: implications for a grading system. Med Educ. 2004;38(4):425–34. doi:10.1111/j.1365-2923.2004.01774.x.

    Article  Google Scholar 

  6. Gullo CA, McCarthy MJ, Shapiro JI, Miller BL. Predicting medical student success on licensure exams. Medical Science Educator. 2015;25(4):447–53. doi:10.1007/s40670-015-0179-6.

    Article  Google Scholar 

  7. Glaros AG, Hanson A, Adkison LR. Early prediction of medical student performance on initial licensing examinations. Medical Science Educator. 2014;24(3):291–5. doi:10.1007/s40670-014-0053-y.

    Article  Google Scholar 

  8. Johnson TR, Khalil MK, Peppler RD, Davey DD, Kibble JD. Use of the NBME Comprehensive Basic Science Examination as a progress test in the preclerkship curriculum of a new medical school. Adv Physiol Educ. 2014;38(4):315–20. doi:10.1152/advan.00047.2014.

    Article  Google Scholar 

  9. Lee MW, Soborowicz M. Evaluation of team-teaching as a means of integrating microbiology and pharmacology. Med Sci Educ. 2015;25(4):497–502. doi:10.1007/s40670-015-0167-x.

    Article  Google Scholar 

  10. Dunleavy DM, Kroopnick MH, Dowd KW, Searcy CA, Zhao X. The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001-2004 matriculants. Acad Med. 2013;88(5):666–71. doi:10.1097/ACM.0b013e3182864299.

    Article  Google Scholar 

  11. Sandars J, Cleary TJ. Self-regulation theory: applications to medical education: AMEE Guide No. 58. Med Teach. 2011;33(11):875–86. doi:10.3109/0142159X.2011.595434.

    Article  Google Scholar 

  12. Brydges R, Dubrowski A, Regehr G. A new concept of unsupervised learning: directed self-guided learning in the health professions. Acad Med. 2010;85(10 Suppl):S49–55. doi:10.1097/ACM.0b013e3181ed4c96.

    Article  Google Scholar 

  13. Adam J, Bore M, McKendree J, Munro D, Powis D. Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study. BMC Med Educ. 2012;12:69. doi:10.1186/1472-6920-12-69.

    Article  Google Scholar 

  14. Haight SJ, Chibnall JT, Schindler DL, Slavin SJ. Associations of medical student personality and health/wellness characteristics with their medical school performance across the curriculum. Acad Med. 2012;87(4):476–85. doi:10.1097/ACM.0b013e318248e9d0.

    Article  Google Scholar 

  15. Lee KB, Vaishnavi SN, Lau SK, Andriole DA, Jeffe DB. “Making the grade:” noncognitive predictors of medical students’ clinical clerkship grades. J Natl Med Assoc. 2007;99(10):1138–50.

    Google Scholar 

  16. Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14(1):34–42. doi:10.1207/S15328015TLM1401_9.

    Article  Google Scholar 

  17. Bing-You R, Hayes V, Varaklis K, Trowbridge R, Kemp H, McKelvy D. Feedback for learners in medical education: what is known? A Scoping Review Acad Med. 2017. doi:10.1097/ACM.0000000000001578.

  18. Krupat E, Pelletier SR, Dienstag JL. Academic performance on first-year medical school exams: how well does it predict later performance on knowledge-based and clinical assessments? Teach Learn Med. 2017;29(2):181–187. doi:10.1080/10401334.2016.1259109.

  19. Hairrel AR, Smith S, McIntosh D, Chico DE. Impact of pre-matriculation instruction on student acculturation and first-year academic performance in medical school. Med Sci Educ. 2016;26(4):519–23. doi:10.1007/s40670-016-0284-1.

    Article  Google Scholar 

  20. Cutrer WB, Miller B, Pusic MV, Mejicano G, Mangrulkar RS, Gruppen LD, et al. Fostering the development of master adaptive learners: a conceptual model to guide skill acquisition in medical education. Acad Med. 2017;92(1):70–5. doi:10.1097/ACM.0000000000001323.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Medical Education, Dell Medical School, The University of Texas at Austin, Health Learning Building, 1501 Red River Street, MC: Z0100, Austin, TX, 78712, USA

    Michael W. Lee

  2. Division of Health Sciences Informatics, School of Medicine, Johns Hopkins University, Baltimore, MD, 21218, USA

    Teresa R. Johnson

  3. Department of Medical Education, College of Medicine, University of Central Florida, Orlando, FL, 32816, USA

    Jonathan Kibble

Authors
  1. Michael W. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Teresa R. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan Kibble
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael W. Lee.

Ethics declarations

Conflict of Interest

The authors declare that there is no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, M.W., Johnson, T.R. & Kibble, J. Development of Statistical Models to Predict Medical Student Performance on the USMLE Step 1 as a Catalyst for Deployment of Student Services. Med.Sci.Educ. 27, 663–671 (2017). https://doi.org/10.1007/s40670-017-0452-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40670-017-0452-y

Keywords

Search

Navigation