Advertisement

Journal of General Internal Medicine

, Volume 23, Issue 7, pp 1016–1019 | Cite as

Associations Between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) Scores

  • Furman S. McDonald
  • Scott L. Zeger
  • Joseph C. Kolars
Original Article

Abstract

Background

Little is known about the associations of previous standardized examination scores with scores on subsequent standardized examinations used to assess medical knowledge in internal medicine residencies.

Objective

To examine associations of previous standardized test scores on subsequent standardized test scores.

Design

Retrospective cohort study.

Participants

One hundred ninety-five internal medicine residents.

Methods

Bivariate associations of United States Medical Licensing Examination (USMLE) Steps and Internal Medicine In-Training Examination (IM-ITE) scores were determined. Random effects analysis adjusting for repeated administrations of the IM-ITE and other variables known or hypothesized to affect IM-ITE score allowed for discrimination of associations of individual USMLE Step scores on IM-ITE scores.

Results

In bivariate associations, USMLE scores explained 17% to 27% of the variance in IME-ITE scores, and previous IM-ITE scores explained 66% of the variance in subsequent IM-ITE scores. Regression coefficients (95% CI) for adjusted associations of each USMLE Step with IM-ITE scores were USMLE-1 0.19 (0.12, 0.27), USMLE-2 0.23 (0.17, 0.30), and USMLE-3 0.19 (0.09, 0.29).

Conclusions

No single USMLE Step is more strongly associated with IM-ITE scores than the others. Because previous IM-ITE scores are strongly associated with subsequent IM-ITE scores, appropriate modeling, such as random effects methods, should be used to account for previous IM-ITE administrations in studies for which IM-ITE score is an outcome.

KEY WORDS

United States Medical Licensing Examination (USMLE) Internal Medicine In-Training Examination (IM-ITE) medical knowledge assessment random effects modeling 

Notes

Acknowledgments

There was no external funding source for this study. This work was presented at the 13th Ottawa International Conference on Clinical Competence, Melbourne Australia, 3–8 March 2008.

Conflict of interest statement

None disclosed.

References

  1. 1.
    ACGME Outcome Project, ACGME General Competencies Vers. 1.3 (9.28.99) Available at: http://www.acgme.org/outcome/comp/compFull.asp#2. Accessed April 7, 2008.
  2. 2.
    Garibaldi RA, Subhiyah R, Moore ME, Waxman H. The In-Training Examination in Internal Medicine: an analysis of resident performance over time. Ann Intern Med. 2002;137(6):505–10.PubMedGoogle Scholar
  3. 3.
    Internal Medicine In-Training Examination®. Available at: http://www.acponline.org/education_recertification/education/in_training/ Accessed April 7, 2008.
  4. 4.
    Grosman RS, Fincher RM, Layne RD, Seelig CB, Berkowitz LR, Levine MA. Validity of the in-training examination for predicting American Board of Internal Medicine certifying examination scores. J Gen Intern Med. 1992;7(1):63–7.CrossRefGoogle Scholar
  5. 5.
    Cantwell JD. The Mendoza Line and in-training examination scores. Ann Intern Med. 1993;119(6):541.PubMedGoogle Scholar
  6. 6.
    Rollins LK, Martindale JR, Edmond M, Manser T, Scheld WM. Predicting pass rates on the American Board of Internal Medicine certifying examination. J Gen Intern Med. 1998;13(6):414–6.CrossRefPubMedGoogle Scholar
  7. 7.
    Waxman H, Braunstein G, Dantzker D, et al.. Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examination. J Gen Intern Med. 1994;9(12):692–4.CrossRefPubMedGoogle Scholar
  8. 8.
    Babbott SF, Beasley BW, Hinchey KT, Blotzer JW, Holmboe ES. The predictive validity of the internal medicine in-training examination. Am J Med. Aug 2007;120(8):735–40.CrossRefPubMedGoogle Scholar
  9. 9.
    United States Medical Licensing Examination. Available at: http://www.usmle.org/. Accessed April 7, 2008.
  10. 10.
    Pohl CA, Robeson MR, Veloski J. USMLE Step 2 performance and test administration date in the fourth year of medical school. Acad Med. Oct 2004;79(10 Suppl):S49–51.CrossRefPubMedGoogle Scholar
  11. 11.
    Melnick DE, Dillon GF, Swanson DB. Medical licensing examinations in the United States. J Dent Educ. May 2002;66(5):595–9. discussion 610–591.PubMedGoogle Scholar
  12. 12.
    O'Donnell MJ, Obenshain SS, Erdmann JB. Background essential to the proper use of results of step 1 and step 2 of the USMLE. Acad Med. Oct 1993;68(10):734–9.CrossRefPubMedGoogle Scholar
  13. 13.
    Andriole DA, Jeffe DB, Hageman HL, Whelan AJ. What predicts USMLE Step 3 performance? Acad Med. 2005;80(10 Suppl):S21–4, Oct.CrossRefPubMedGoogle Scholar
  14. 14.
    Cacamese SM, Eubank KJ, Hebert RS, Wright SM. Conference attendance and performance on the in-training examination in internal medicine. Med Teach. 2004;26(7):640–4.CrossRefPubMedGoogle Scholar
  15. 15.
    Dupras DM, Li JT. Use of an objective structured clinical examination to determine clinical competence. Acad Med. 1995;70(11):1029–34.CrossRefPubMedGoogle Scholar
  16. 16.
    Ellis DD, Kiser WR, Blount W. Is interruption in residency training associated with a change in in-training examination scores? Fam Med. 1997;29(3):184–6.PubMedGoogle Scholar
  17. 17.
    Hawkins RE, Sumption KF, Gaglione MM, Holmboe ES. The in-training examination in internal medicine: resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance. Am J Med. 1999;106(2):206–10.CrossRefPubMedGoogle Scholar
  18. 18.
    McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22(7):962–8, Jul.CrossRefPubMedGoogle Scholar
  19. 19.
    Ogunyemi D, Taylor-Harris de S. Factors that correlate with the U.S. Medical Licensure Examination Step-2 scores in a diverse medical student population. J Natl Med Assoc. 2005;97(9):1258–62, Sep.PubMedGoogle Scholar
  20. 20.
    Shokar GS. The effects of an educational intervention for “at-risk” residents to improve their scores on the In-training Exam. Fam Med. 2003;35(6):414–7.PubMedGoogle Scholar
  21. 21.
    Liang KY, Zeger SL. Regression analysis for correlated data. Annu Rev Public Health. 1993;14:43–68.CrossRefPubMedGoogle Scholar
  22. 22.
    Diggle P, Heagerty P, Liang K, Zeger S. Analysis of Longitudinal Data. 2nd edition. Oxford, England: Oxford University Press; 2002.Google Scholar
  23. 23.
    Pofahl WE, Swanson MS, Cox SS, et al. Performance standards improve American Board of Surgery In-Training Examination scores. Curr Surg. 2002;59(22):20–2, Mar-Apr.CrossRefPubMedGoogle Scholar
  24. 24.
    Godellas CV, Huang R. Factors affecting performance on the American Board of Surgery in-training examination. Am J Surg. 2001;181(4):294–6.CrossRefPubMedGoogle Scholar
  25. 25.
    Bull DA, Stringham JC, Karwande SV, Neumayer LA. Effect of a resident self-study and presentation program on performance on the thoracic surgery in-training examination. Am J Surg. 2001;181(2):142–4.CrossRefPubMedGoogle Scholar
  26. 26.
    Godellas CV, Hauge LS, Huang R. Factors affecting improvement on the American Board of Surgery In-Training Exam (ABSITE). J Surg Res. 2000;91(1):1–4.CrossRefPubMedGoogle Scholar
  27. 27.
    Madlon-Kay DJ. When family practice residents are parents: effect on in-training examination scores. Fam Pract Res J. 1989;9(1):53–6.PubMedGoogle Scholar
  28. 28.
    Pollak R, Baker RJ. The acquisition of factual knowledge and the role of the didactic conference in a surgical residency program. Am Surg. 1988;54(9):531–4.PubMedGoogle Scholar
  29. 29.
    Black KP, Abzug JM, Chinchilli VM. Orthopaedic in-training examination scores: a correlation with USMLE results. J Bone Joint Surg Am. 2006;88(3):671–6, Mar.CrossRefPubMedGoogle Scholar
  30. 30.
    Carmichael KD, Westmoreland JB, Thomas JA, Patterson RM. Relation of residency selection factors to subsequent orthopaedic in-training examination performance. South Med J. 2005;98(5):528–32, May.CrossRefPubMedGoogle Scholar
  31. 31.
    Sawhill AJ, Dillon GF, Ripkey DR, Hawkins RE, Swanson DB. The impact of postgraduate training and timing on USMLE Step 3 performance. Acad Med. Oct 2003;78(10 Suppl):S10–12.CrossRefPubMedGoogle Scholar
  32. 32.
    Julian ER. Validity of the Medical College Admission Test for predicting medical school performance. Acad Med. Oct 2005;80(10):910–7.CrossRefPubMedGoogle Scholar
  33. 33.
    Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research. Acad Med. Jan 2007;82(1):100–6.CrossRefPubMedGoogle Scholar

Copyright information

© Society of General Internal Medicine 2008

Authors and Affiliations

  • Furman S. McDonald
    • 1
  • Scott L. Zeger
    • 2
  • Joseph C. Kolars
    • 3
  1. 1.Residency Program Director and Associate Chair, Department of MedicineMayo Clinic College of MedicineRochesterUSA
  2. 2.Professor of Biostatistics and Epidemiology, Chair, Department of BiostatisticsJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  3. 3.Emeritus Program Director and Associate Chair, Department of Medicine, Professor of MedicineMayo Clinic College of MedicineRochesterUSA

Personalised recommendations