Abstract
Purpose To examine the predictive ability of preadmission variables on United States Medical Licensing Examinations (USMLE) step 1 and step 2 performance, incorporating the use of a neural network model. Method Preadmission data were collected on matriculants from 1998 to 2004. Linear regression analysis was first used to identify predictors of performance on step 1 and step 2. A generalized regression neural network (GRNN) as well as a feed forward neural network (FFNN) was then developed in an effort to more accurately predict step 1 and step 2 scores from these preadmission data. Results Statistically significant predictors for step 1 and step 2 included science grade point average (SGPA), the biologic science (BS) section of the Medical College Admissions Test (MCAT), college selectivity, race, and age of the applicant. Neural networks were found to predict a significant portion of the variance, and the FFNN demonstrated some superiority over that obtained with linear regression models as well as the GRNN. Conclusions The results have implications that could impact the selection of applicants to medical school and the neural networks that we developed could be used in a prospective manner.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adolmaleki, P., Buadu, L. D., Murayama, S., Murakami, J., Hashiguchi, N., Yabuuchi, H., & Masuda, K. (1997). Neural network analysis of breast cancer from MRI findings. Radiation Medicine, 15, 283–293.
Blue, A. V., Gilbert, G. E., Elam, C. L., & Basco, W. T. (2000). Does institutional selectivity aid in the prediction of medical school performance? Academic Medicine, 75, 31–33.
Case, S. M., Becker, D. F., & Swanson, D. B. (1993). Performances of men and women on NBME Part I and Part II: The more things change... Academic Medicine, 68, 25–27.
Coppini, G., Poli, R., Rucci, M., & Valli, G. (1992). A neural network architecture for understanding discrete three-dimensional scenes in medical imaging. Computers and Biomedical Research, 25, 569–585.
Dawson, B., Iwamoto, C. K., Ross, L. P., & Nungester, R. J. (1994). Performance on the National Board of Medical Examiners Part I examination by men and women of different race and ethnicity. Journal of the American Medical Association, 272, 674–679.
De Champlain, A., Sample, L., Dillon, G. F., & Boulet, J. R. (2006). Modeling longitudinal performances on the United States Medical Licensing examination and impact of sociodemographic covariates: An application of survival data analysis. Academic Medicine, 81, 108–111.
Donnon, T., Paolucci, E. O., & Violato, C. (2007). The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta analysis of the published research. Academic Medicine, 82, 100–106.
Edelin, K. C., & Ugbolue, A. (2001). Evaluation of an early medical school selection program for underrepresented minority students. Academic Medicine, 76, 1056–1059.
Furness, P., Kazi, J., Levesky, J., Taub, N., & Nicholson, M. (1999). A neural network approach to the biopsy diagnosis of early acute renal transplant rejection. Histopathology, 35, 461–467.
Johnson, D. G., Lloyd, S. M., Jones, R. F., & Anderson, J. (1986). Predicting academic performance at a predominantly black medical school. Journal of Medical Education, 61, 629–639.
Jones, R. F., & Thomae-Forgues, M. (1984). Validity of the MCAT in predicting performance in the first two years of medical school. Journal of Medical Education, 59, 455–464.
Julian E. R. (2005). Validity of the medical college admission test for predicting medical school performance. Academic Medicine, 80, 910–917.
Kasuya, R. T., Naguwa, G. S., Guerrero, A. P., Hisinuma, E. S., Lindberg, M. A., & Judd, N. K. (2003). USMLE Performances in a predominantly Asian and Pacific Islander population of medical students in a problem-based learning curriculum. Academic Medicine, 78, 483–490.
Mosier, P. D., & Jurs, P. C. (2002). QSAR/QSPR studies using probabilistic neural networks and generalized regression neural networks. Journal of Chemical Information and Computer Science, 42, 1460–1470.
Oram F. A. (Ed.) (2005). Peterson’s four-year colleges 2006. Lawrenceville, NJ: Thomson/Peterson’s.
Pilon, S., & Tandberg, D. (1997). Neural network and linear regression models in residency selection. American Journal of Emergency Medicine, 15, 361–364.
Ramsbottom-Lucier, M., Johnson, M. S., & Elam, C. (1995). Age and gender differences in students’ preadmission qualifications and medical school performances. Academic Medicine, 70, 236–239.
Roth, K. S., Riley, W. T., Brandt, R. B., & Seibel, H. R. (1996). Prediction of students’ USMLE step 2 performances based on premedical credentials related to verbal skills. Academic Medicine, 71, 176–180.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533–536.
Silver, B., & Hodgson, C. S. (1997). Evaluating GPAs and MCAT scores as predictors of NBME I and clerkship performances based on students’ data from one undergraduate institution. Academic Medicine, 72, 394–396.
Smith, S. (1998). Effect of undergraduate college major on performance in medical school. Academic Medicine, 73, 1006–1008.
Tarca, A. L., Cooke, J. E., & Mackay J. (2005). A robust neural networks approach for spatial and intensity-dependent normalization of cDNA microarray data. Bioinformatics, 21, 2674–2683.
Veloski, J. J., Callahan, C. A., Xu, G., Mohammadreza, H., & Nash, D. B. (2000). Prediction of students’ performances on licensing examinations using age, race, sex, undergraduate GPAs, and MCAT scores. Academic Medicine, 75, 28–30.
Xu, G., Veloski, J. J., Hojat, M., Gonnella, J. S., & Bacharach, B. (1993). Longitudinal comparison ofthe academic performances of Asian–American and white medical students. Academic Medicine, 68, 82–86.
Zeleznik C., Hojat, M., & Veloski, J. (1983). Baccalaureate preparation for medical school: Does type of degree make a difference? Journal of Medical Education, 58, 26–33.
Acknowledgements
The authors would like to thank Adam Jacob for his assistance in preparation of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kleshinski, J., Khuder, S.A., Shapiro, J.I. et al. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv in Health Sci Educ 14, 69–78 (2009). https://doi.org/10.1007/s10459-007-9087-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-007-9087-x