Journal of General Internal Medicine

, Volume 26, Issue 11, pp 1253–1257 | Cite as

A Multiple Choice Testing Program Coupled with a Year-long Elective Experience is Associated with Improved Performance on the Internal Medicine In-Training Examination

  • Bradley R. MathisEmail author
  • Eric J. Warm
  • Daniel P. Schauer
  • Eric Holmboe
  • Gregory W. Rouan
Original Research



The Internal Medicine In-Training Exam (IM-ITE) assesses the content knowledge of internal medicine trainees. Many programs use the IM-ITE to counsel residents, to create individual remediation plans, and to make fundamental programmatic and curricular modifications.


To assess the association between a multiple-choice testing program administered during 12 consecutive months of ambulatory and inpatient elective experience and IM-ITE percentile scores in third post-graduate year (PGY-3) categorical residents.


Retrospective cohort study.


One hundred and four categorical internal medicine residents. Forty-five residents in the 2008 and 2009 classes participated in the study group, and the 59 residents in the three classes that preceded the use of the testing program, 2005–2007, served as controls.


A comprehensive, elective rotation specific, multiple-choice testing program and a separate board review program, both administered during a continuous long-block elective experience during the twelve months between the second post-graduate year (PGY-2) and PGY-3 in-training examinations.


We analyzed the change in median individual percent correct and percentile scores between the PGY-1 and PGY-2 IM-ITE and between the PGY-2 and PGY-3 IM-ITE in both control and study cohorts. For our main outcome measure, we compared the change in median individual percentile rank between the control and study cohorts between the PGY-2 and the PGY-3 IM-ITE testing opportunities.


After experiencing the educational intervention, the study group demonstrated a significant increase in median individual IM-ITE percentile score between PGY-2 and PGY-3 examinations of 8.5 percentile points (p < 0.01). This is significantly better than the increase of 1.0 percentile point seen in the control group between its PGY-2 and PGY-3 examination (p < 0.01).


A comprehensive multiple-choice testing program aimed at PGY-2 residents during a 12-month continuous long-block elective experience is associated with improved PGY-3 IM-ITE performance.


Internal Medicine In-Training Exam multiple-choice testing medical knowledge 



The authors would like to thank the American Board of Internal Medicine for the use of the Maintenance of Certification program Self Evaluation Program modules in the construction of our board review testing program. Funding for this study was provided by a University of Cincinnati College of Medicine medical education research grant.

Conflict of Interest

Dr. Eric Holmboe is employed by the ABIM.


  1. 1.
    The American College of Physicians. Internal Medicine In-Training Examination. Available at: Accessed March 3, 2011.
  2. 2.
    Grossman RS, Fincher RE, Layne RD, Seelig CB, Berkowitz LR, Levine MA. Validity of the In-Training exam for predicting American Board of Internal Medicine certifying examination scores. J Gen Intern Med. 1992;7:63–67.PubMedCrossRefGoogle Scholar
  3. 3.
    Waxman H, Braunstein G, Dantzker D, et al. Performance on the internal medicine second-year In-training examination predicts the outcome of the ABIM certifying exam. J Gen Intern Med. 1994;9:692–694.PubMedCrossRefGoogle Scholar
  4. 4.
    Rollins LK, Martindale JR, Edmond M, Manser T, Scheld WM. Predicting pass rates on the American Board of Internal Medicine certifying examination. J Gen Intern Med. 1998;13:414–416.PubMedCrossRefGoogle Scholar
  5. 5.
    Babbott SF, Beasley BW, Hinchey KT, Blotzer JW, Holmboe ES. The predictive validity of the Internal Medicine In-training Examination. Am J Med. 2007;120(8):735–740.PubMedCrossRefGoogle Scholar
  6. 6.
    Holmboe E, Wang Y, Meehan T, et al. Association between maintenance of certification examination scores and quality of care for Medicare beneficiaries. Arch Intern Med. 2008;168(13):1396–1403.PubMedCrossRefGoogle Scholar
  7. 7.
    Norcini J, Lipner R, Kimball H. Certifying examination performance and patient outcomes following acute myocardial infarction. Med Educ. 2002;36(9):853–859.PubMedCrossRefGoogle Scholar
  8. 8.
    Sharp L, Bashook P, Lipsky M, Horowitz S, Miller S. Specialty board certification and clinical outcomes: the missing link. Acad Med. 2002;77:534–542.PubMedCrossRefGoogle Scholar
  9. 9.
    Garibaldi RA, Subhiyah R, Moore ME, Waxman H. The In-Training Examination in Internal Medicine: an analysis of resident performance over time. Ann Intern Med. 2002;137:505–510.PubMedGoogle Scholar
  10. 10.
    Webb TP, Weigelt JA, Redlich PN, Anderson RC, Brasel KJ, Simpson D. Protected block curriculum enhances learning during general surgery residency training. Arch Surg. 2009;144(2):160–166.PubMedCrossRefGoogle Scholar
  11. 11.
    McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22:962–968.PubMedCrossRefGoogle Scholar
  12. 12.
    McDonald FS, Zeger SL, Kolars JC. Associations of conference attendance with Internal Medicine In-Training Examination Scores. Mayo Clin Proc. 2008;83(4):449–453.PubMedCrossRefGoogle Scholar
  13. 13.
    Accreditation Council for Graduate Medical Education. Educational Innovations Project. Available at: Accessed March 3, 2011.
  14. 14.
    Warm EJ, Schauer DP, Diers T, et al. The Ambulatory Long-Block: An Accreditation Council for Graduate Medical Education (ACGME) Educational Innovations Project (EIP). J Gen Intern Med. 2008;23(7):921–6.PubMedCrossRefGoogle Scholar
  15. 15.
    Warm EJ, Schauer DP, Revis B, Boex JR. Multi-source feedback in the ambulatory setting. J Grad Med Educ. 2010;2(2):269–277.PubMedGoogle Scholar

Copyright information

© Society of General Internal Medicine 2011

Authors and Affiliations

  • Bradley R. Mathis
    • 1
    Email author
  • Eric J. Warm
    • 1
  • Daniel P. Schauer
    • 1
  • Eric Holmboe
    • 2
  • Gregory W. Rouan
    • 1
  1. 1.Department of Internal MedicineUniversity of Cincinnati College of MedicineCincinnatiUSA
  2. 2.American Board of Internal MedicinePhiladelphiaUSA

Personalised recommendations