Abstract
Purpose
Our US medical school uses National Board of Medical Examiners (NBME) tests as progress tests during the pre-clerkship curriculum to assess students. In this study, we examined students’ growth patterns using progress tests in the first year of medical school to identify students at risk for failing United States Medical Licensing Examination (USMLE) Step 1.
Method
Growth Mixture Modeling (GMM) was used to examine the growth trajectories based on NBME progress test scores in the first year of medical school. Achieving a passing score on the USMLE Step 1 at the end of the second year of medical school was used as the distal outcome, controlling for Medical College Admissions Test (MCAT) scores and underrepresented in medicine (URiM) status.
Results
A total of 518 students from a US medical school were included in the analysis. Five different growth patterns were identified based on students’ NBME test results. Seventy-eight students identified in Group 1 had the lowest starting NBME test score (mean = 33.6, 95% CI 32.0–35.2) and lowest growth rate (mean = 2.30, 95% CI 2.06–2.53). All 26 students who failed Step 1 at the end of the second year were in Group 1 (failing rate = 33%). Meanwhile Group 4 (n = 65 students) had moderate starting NBME test scores (mean = 37.9, 95% CI 36.3–39.0) but the highest growth rate with mean slope at 6.07 (95% CI 5.40–6.73). This group of students achieved significant higher USMLE Step1 scores comparing with the 3 other groups of students (P < 0.05).
Conclusions
Our study found students had heterogeneous growth patterns in progress test results in their first year of medical school. Growth patterns were highly predictive of USMLE step 1 results. This study can provide performance benchmarks for our future students to assess their progress and for medical educators to identify students who need support and guidance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Allison, P. D. (2000). Multiple imputation for missing data: A cautionary tale. Sociol Methods Res. https://doi.org/10.1177/0049124100028003003
Basco, W. T., Way, D. P., Gilbert, G. E., & Hudson, A. (2002). Undergraduate institutional MCAT scores as predictors of USMLE step 1 performance. Acad Med. https://doi.org/10.1097/00001888-200210001-00005
Ben Gal, I. (2007). Bayesian networks. In F. Ruggeri, R. S. Kennett, & F. W. Faltin (Eds.), Encyclopedia of statistics in quality and reliability. Chichester: Wiley.
Berkhout, J. J., Teunissen, P. W., Helmich, E., van Exel, J., van der Vleuten, C. P., & Jaarsma, D. A. (2017). Patterns in clinical students’ self-regulated learning behavior: A Q-methodology study. Advances in Health Sciences Education, 22(1), 105–121.
Coumarbatch, J., Robinson, L., Thomas, R., & Bridge, P. D. (2010). Strategies for identifying students at risk for USMLE step 1 failure. Fam Med, 42(2), 105–110.
Croen, L. G., Reichgott, M., & Spencer, R. K. (1991). A performance-based method for early identification of medical students at risk of developing academic problems. Acad Med. https://doi.org/10.1097/00001888-199108000-00017
De Champlain, A. F., Cuddy, M. M., Scoles, P. V., et al. (2010). Progress testing in clinical science education: Results of a pilot project between the national board of medical examiners and a US medical school. Med Teach. https://doi.org/10.3109/01421590903514655
Glew, R. H., Ripkey, D. R., & Swanson, D. B. (1997). Relationship between students’ performances on the NBME comprehensive basic science examination and the USMLE Step 1: A longitudinal investigation at one school. Acad Med. https://doi.org/10.1097/00001888-199712000-00022
Holmstrom, A. (2018). United States medical licensing examination. In M. G. Patti & P. Marco Fisichella (Eds.), The American health care system. Cham: Springer.
Kusurkar, R. A., Mak-van der Vossen, M., Kors, J., Grijpma, J. W., van der Burgt, S. M., Koster, A. S., & de la Croix, A. (2021). ‘One size does not fit all’: The value of person-centred analysis in health professions education research. Perspectives on Medical Education, 10(4), 245–251.
Morrison, C. A., Ross, L. P., Fogle, T., Butler, A., Miller, J., & Dillon, G. F. (2010). Relationship between performance on the NBME comprehensive basic sciences self-assessment and USMLE step 1 for US and Canadian medical school students. Acad Med, 85, S98–S101. https://doi.org/10.1097/ACM.0b013e3181ed3f5c
Muthén, B. (2003). Statistical and substantive checking in growth mixture modeling: Comment on Bauer and Curran (2003). Psychol Methods. https://doi.org/10.1037/1082-989X.8.3.369
Muthén B. (2004) Latent Variable Analysis. In: Handbook of Quantitative Methodology for Social Sciences.
Muthén, B., & Shedden, K. (1999). Finite mixture modeling with mixture outcomes using the EM algorithm. Biometrics. https://doi.org/10.1111/j.0006-341X.1999.00463.x
Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Struct Equ Model. https://doi.org/10.1080/10705510701575396
Vogt W. (2015) Tukey’s Honestly Significant Difference (HSD) Test. In: Dictionary of Statistics & Methodology. https://doi.org/10.4135/9781412983907.n2011
Wang, M., & Bodner, T. E. (2007). Growth mixture modeling: Identifying and predicting unobserved subpopulations with longitudinal data. Organ Res Methods. https://doi.org/10.1177/1094428106289397
Wang, L., Laird-Fick, H. S., Parker, C. J., & Solomon, D. (2021). Using Markov chain model to evaluate medical students’ trajectory on progress tests and predict USMLE step 1 scores–-A retrospective cohort study in one medical school. BMC Med Educ, 21(1), 1–9.
Williams, M., Kim, E. J., Pappas, K., Uwemedimo, O., Marrast, L., Pekmezaris, R., & Martinez, J. (2020). The impact of United States Medical Licensing Exam (USMLE) step 1 cutoff scores on recruitment of underrepresented minorities in medicine: A retrospective cross-sectional study. Health Sci Reports, 3(2), e2161.
Wrigley, W., Van Der Vleuten, C. P., Freeman, A., & Muijtjens, A. (2012). A systemic framework for the progress test: Strengths, constraints and issues: AMEE Guide No. 71. Med Teach., 34, 683–697. https://doi.org/10.3109/0142159X.2012.704437
Zhao, X., Oppler, S., Dunleavy, D., & Kroopnick, M. (2010). Validity of four approaches of using repeaters’ MCAT scores in medical school admissions to predict USMLE step 1 total scores. Acad Med. https://doi.org/10.1097/ACM.0b013e3181ed38fc
Funding
None.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All co-authors (L. Wang, H. Laird-Fick, C. Parker, Z. Liao and D. Solomon) declare that they have no competing interest.
Ethical approval
Per request from the Michigan State University's Human Research Protection Program (MSU-HRPP), a designated honest broker is used to deidentify curricular and student evaluation data collected as a normal part of the medical school’s educational programs. By the MSU-HRPP’s determination, these data are not considered human subject data. The ethical approval is obtained by Institutional Review Board Office at Michigan State University. Documentation concerning the honest broker program can be found at https://omerad.msu.edu/research/honest-broker-for-educational-scholarship.
Disclaimers
The authors alone are responsible for the content and writing of the article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, L., Laird-Fick, H., Parker, C. et al. Growth in basic science knowledge in first-year medical school and USMLE Step 1 results: a longitudinal investigation at one school. Adv in Health Sci Educ 27, 605–619 (2022). https://doi.org/10.1007/s10459-022-10104-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-022-10104-y