Skip to main content

Advertisement

SpringerLink
Log in

The Design and Development of Prediction Models for Maximizing Students’ Academic Achievement

  • Original Research
  • Published:
Medical Science Educator Aims and scope Submit manuscript

Abstract

Objectives

To enable early identification and intervention for students at risk for academic failure and for failure on the USMLE Step 1 Examination, three predictive models (pre-matriculation, end of M1, and end of M2) were developed that include both behavioral attributes (Learning and Study Strategies Inventory: LASSI subscale scores) and performance measures (internal examinations and National Board of Medical Examiners [NBME] Comprehensive Basic Science Examination [CBSE] progress tests).

Methods

Pearson product moment correlation coefficients and multivariate regression modeling were used to determine optimal combinations of independent variables. The pre-matriculation prediction model includes MCAT scores and undergraduate overall GPA. The end of M1 model includes the progress test scores on the NBME CBSE administered at the end of first year medical school (M1), LASSI subscale scores, and weighted performance on M1 internal examinations. Finally, the end of M2 model includes CBSE progress test scores at the end of M1 and second year medical school (M2), LASSI subscale scores, and weighted performance on M1 and M2 internal examinations.

Results

Our pre-matriculation, end of M1, and end of M2 models explain 24, 62, and 81% of the variation in USMLE Step 1 performance, respectively. The inclusion of LASSI subscale scores improves the end of M1 model from 60 to 62% and end of M2 model from 79 to 81% in explaining variation in USMLE Step 1 scores.

Conclusion

Continuous monitoring of student performance based on these multiple measures supports a holistic perspective on progress, areas of ongoing weakness, and potentially, identification of yet-undetected weakness for targeted interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. 2007;82(1):100–6. https://doi.org/10.1097/01.ACM.0000249878.25186.b7.

    Article  Google Scholar 

  2. Dunleavy DM, Kroopnick MH, Dowd KW, Searcy CA, Zhao X. The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001–2004 matriculants. Acad Med. 2013;88(5):666–71. https://doi.org/10.1097/ACM.0b013e3182864299.

    Article  Google Scholar 

  3. Evans P, Wen FK. Does the medical college admission test predict global academic performance in osteopathic medical school? J Am Osteopath Assoc. 2007;107(4):157–62.

    Google Scholar 

  4. Roy B, Ripstein I, Perry K, Cohen B. Predictive value of grade point average (GPA), Medical College Admission Test (MCAT), internal examinations (Block) and National Board of Medical Examiners (NBME) scores on Medical Council of Canada qualifying examination part I (MCCQE-1) scores. Can Med Educ J. 2016;7(1):e47–56.

    Google Scholar 

  5. Saguil A, Dong T, Gingerich RJ, Swygert K, LaRochelle JS, Artino AR Jr, et al. Does the MCAT predict medical school and PGY-1 performance? Mil Med. 2015;180(4S):4–11. https://doi.org/10.7205/MILMED-D-14-00550.

  6. Stegers-Jager KM, Themmen AP, Cohen-Schotanus J, Steyerberg EW. Predicting performance: relative importance of students’ background and past performance. Med Educ. 2015;49(9):933–45. https://doi.org/10.1111/medu.12779.

    Article  Google Scholar 

  7. National Resident Matching Program, Data Release and Research Committee: Results of the 2016 NRMP Program Director Survey. National Resident Matching Program, Washington, DC. 2016. http://www.nrmp.org/wp-content/uploads/2016/09/NRMP-2016-Program-Director-Survey.pdf. Accessed 12 Aug 2017.

  8. Gandy RA, Herial NA, Khuder SA, Metting PJ. Use of curricular and extracurricular assessments to predict performance on the United States Medical Licensing Examination (USMLE) Step 1: a multi-year study. The Learning Assistance Review (TLAR). 2008;13(2):27–35.

    Google Scholar 

  9. McHarg J, Bradley P, Chamberlain S, Ricketts C, Searle J, McLachlan JC. Assessment of progress tests. Med Educ. 2005;39(2):221–7. https://doi.org/10.1111/j.1365-2929.2004.02060.x.

    Article  Google Scholar 

  10. Wrigley W, Van Der Vleuten CP, Freeman A, Muijtjens A. A systemic framework for the progress test: strengths, constraints and issues: AMEE Guide No. 71. Med Teach. 2012;34(9):683–97. https://doi.org/10.3109/0142159X.2012.704437.

    Article  Google Scholar 

  11. Weinstein CE, Palmer DR. LASSI User’s Manual: for those administering the Learning and Study Strategies Inventory. Clearwater: H & H; 2002.

    Google Scholar 

  12. Guglielmino LM (1977) Development of the self-directed learning readiness scale. Doctoral Dissertation. Athens: University of Georgia

  13. West C, Kurz T, Smith S, Graham L. Are study strategies related to medical licensing exam performance? Int J Med Educ. 2014;5:199–204. https://doi.org/10.5116/ijme.5439.6491.

    Article  Google Scholar 

  14. Schutz CM, Gallagher ML, Tepe RE. Differences in learning and study strategies inventory scores between chiropractic students with lower and higher grade point averages. J Chiropr Educ. 2011;25(1):5–10. https://doi.org/10.7899/1042-5055-25.1.5.

    Article  Google Scholar 

  15. Schutz CM, Dalton L, Tepe RE. Learning and Study Strategies Inventory subtests and factors as predictors of National Board of Chiropractic Examiners Part 1 examination performance. J Chiropr Educ. 2013;27(1):5–10. https://doi.org/10.7899/JCE-D-12-012.

    Article  Google Scholar 

  16. Sleight DA, Mavis BE. Study skills and academic performance among second-year medical students in problem-based learning. Med Educ Online. 2006;11(23):1–6.

    Google Scholar 

  17. West C, Sadoski M. Do study strategies predict academic performance in medical school? Med Educ. 2011;45(7):696–703. https://doi.org/10.1111/j.1365-2923.2011.03929.x.

    Article  Google Scholar 

  18. Zahn CM, Saguil A, Artino AR Jr, Dong T, Ming G, Servey JT, et al. Correlation of National Board of Medical Examiners scores with United States Medical Licensing Examination Step 1 and Step 2 scores. Acad Med. 2012;87(10):1348–54. https://doi.org/10.1097/ACM.0b013e31826a13bd.

  19. Julian ER. Validity of the Medical College Admission Test for predicting medical school performance. Acad Med. 2005;80(10):910–7. https://doi.org/10.1097/00001888-200510000-00010.

    Article  Google Scholar 

  20. Gauer JL, Wolff JM, Jackson JB. Do MCAT scores predict USMLE scores? An analysis on 5 years of medical student data. Med Educ Online. 2016;21(1):31795. https://doi.org/10.3402/meo.v21.31795.

    Article  Google Scholar 

  21. Callahan CA, Hojat M, Veloski J, Erdmann JB, Gonnella JS. The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: a longitudinal study of 36 classes of Jefferson Medical College. Acad Med. 2010;85(6):980–7. https://doi.org/10.1097/ACM.0b013e3181cece3d.

    Article  Google Scholar 

  22. Peterson CA, Tucker RP. Medical gross anatomy as a predictor of performance on the USMLE Step 1. Anat Rec. 2005;283(1):5–8.

    Article  Google Scholar 

  23. Fredieu JR, Snyder CW. Positive impact of a master of science in applied anatomy program on USMLE Step 1 performance. Anat Sci Educ. 2015;8(1):31–6. https://doi.org/10.1002/ase.1455.

    Article  Google Scholar 

  24. Gohara S, Shapiro JI, Jacob AN, Khuder SA, Gandy RA, Metting PJ, et al. Joining the conversation: predictors of success on the United States Medical Licensing Examinations (USMLE). Learn Assist Rev (TLAR). 2011;16(1):11–20.

  25. Zhang C, Rauchwarger A, Toth C, O’Connell M. Student USMLE step 1 preparation and performance. Adv Health Sci Educ. 2005;9(4):291–7. https://doi.org/10.1007/s10459-005-3925-5.

    Article  Google Scholar 

  26. Sawhill A, Butler A, Ripkey D, Swanson DB, Subhiyah R, Thelman J, et al. Using the NBME self-assessments to project performance on USMLE Step 1 and Step 2: impact of test administration conditions. Acad Med. 2004;79(10):S55–7. https://doi.org/10.1097/00001888-200410001-00017.

  27. Morrison CA, Ross LP, Fogle T, Butler A, Miller J, Dillon GF. Relationship between performance on the NBME Comprehensive Basic Sciences Self-Assessment and USMLE Step 1 for US and Canadian medical school students. Acad Med. 2010;85(10):S98–101. https://doi.org/10.1097/ACM.0b013e3181ed3f5c.

    Article  Google Scholar 

  28. Cano F. An in-depth analysis of the Learning and Study Strategies Inventory (LASSI). Educ Psychol Meas. 2006;66(6):1023–38. https://doi.org/10.1177/0013164406288167.

    Article  Google Scholar 

  29. Green M, Angoff N, Encandela J. Test anxiety and United States medical licensing examination scores. Clin Teach. 2015;13(2):142–6. https://doi.org/10.1111/tct.12386.

    Article  Google Scholar 

  30. Tucker P, Jeon-Slaughter H, Sener U, Arvidson M, Khalafian A. Do medical student stress, health, or quality of life foretell step 1 scores? A comparison of students in traditional and revised preclinical curricula. Teach Learn Med. 2015;27(1):63–70. https://doi.org/10.1080/10401334.2014.979178.

    Article  Google Scholar 

  31. Khalil MK, Hawkins HG, Crespo LM, Buggy J. The relationship between Learning and Study Strategies Inventory (LASSI) and academic performance in medical schools. Med Sci Educ. 2017;27(2):315–20. https://doi.org/10.1007/s40670-017-0400-x.

    Article  Google Scholar 

  32. Rosenzweig S, Reibel DK, Greeson JM, Brainard GC, Hojat M. Mindfulness-based stress reduction lowers psychological distress in medical students. Teach Learn Med. 2003;15(2):88–92. https://doi.org/10.1207/S15328015TLM1502_03.

    Article  Google Scholar 

  33. Vøllestad J, Nielsen MB, Nielsen GH. Mindfulness-and acceptance-based interventions for anxiety disorders: a systematic review and meta-analysis. Br J Clin Psychol. 2012;51(3):239–60. https://doi.org/10.1111/j.2044-8260.2011.02024.x.

    Article  Google Scholar 

  34. Li AW, Goldsmith C. The effects of yoga on anxiety and stress. Altern Med Rev. 2012;17(1):21–35.

    Google Scholar 

  35. Robins LS, Fantone JC, Oh MS, Alexander GL, Shlafer M, Davis WK. The effect of pass/fail grading and weekly quizzes on first-year students’ performances and satisfaction. Acad Med. 1995;70(4):327–9. https://doi.org/10.1097/00001888-199504000-00019.

    Article  Google Scholar 

  36. Dyrbye LN, Thomas MR, Sanafelt TD. Medical student distress: causes, consequences, and proposed solutions. Mayo Clin Proc. 2005;80(12):1613–22. https://doi.org/10.4065/80.12.1613.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of South Carolina School of Medicine Greenville, 607 Grove Road, Greenville, SC, 29605, USA

    Mohammed K. Khalil, H. Gregory Hawkins, Lynn M. Crespo & James Buggy

  2. Greenville Health System, Greenville, SC, USA

    Lynn M. Crespo

Authors
  1. Mohammed K. Khalil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. Gregory Hawkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lynn M. Crespo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. James Buggy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed K. Khalil.

Ethics declarations

Conflict of Interest

The authors declare that there is no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khalil, M.K., Hawkins, H.G., Crespo, L.M. et al. The Design and Development of Prediction Models for Maximizing Students’ Academic Achievement. Med.Sci.Educ. 28, 111–117 (2018). https://doi.org/10.1007/s40670-017-0515-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40670-017-0515-0

Keywords

Search

Navigation