Abstract
Background
The US Medical Licensing Examination (USMLE) and Orthopaedic In-Training Examination (OITE) are commonly used to select medical students or residents, respectively. Knowing how well these examinations predict performance on the American Board of Orthopaedic Surgery (ABOS) Part I certifying examination is important to provide evaluations for medical students and residents. Previous studies comparing the OITE scores with the ABOS Part 1 scores have been limited to one program.
Questions/purposes
Therefore, we compared the scores on the USMLE Step 1 and OITE examinations with those on the ABOS Part I certifying examination using data from four ACGME approved residency programs.
Methods
We reviewed 202 resident files from 1996 to 2008 from four programs in the same geographic region. Of those, 181 (90%) had complete records. De-identified data were used to compare USMLE Step 1 scores, OITE percentile rank scores, and ABOS Part I percentile rank scores. Pearson coefficients and receiver operator curves were calculated to assess the relationships between tests.
Results
We found a correlation of 0.53 between the USMLE Step 1 and ABOS Part I, and an average correlation of 0.50 for postgraduate years (PGY) 2 through 5 OITE scores and ABOS Part I. There was a stepwise increase in correlation from PGY 2 through PGY 5 between the OITE scores and ABOS scores. Those who averaged in the 27th percentile or lower on the OITE had a 57% chance of failing the ABOS Part I examination.
Conclusion
USMLE Step 1 scores correlated with ABOS Part I certifying examination scores, and we therefore believe it may be used as one factor in resident selection. Use of the OITE scores in guiding education and feedback appears to be justified.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
ACGME. V.C.3. ACGME Program Requirements for Graduate Medical Education in Orthopaedic Surgery. Available at: http://www.acgme.org/acWebsite/downloads/RRC_progReq/260orthopaedicsurgery07012007.pdf. Accessed February 3, 2010.
Bernstein AD, Jazrawi LM, Elbeshbeshy B, Della Valle CJ, Zuckerman JD. Orthopaedic resident-selection criteria. J Bone Joint Surg Am. 2002;84:2090–2096.
Black KP, Abzug JM, Chinchilli VM. Orthopaedic in-training examination scores: a correlation with USMLE results. J Bone Joint Surg Am. 2006;88:671–676.
Carmichael KD, Westmoreland JB, Thomas JA, Patterson RM. Relation of residency selection factors to subsequent orthopaedic in-training examination performance. South Med J. 2005;98:528–532.
Case SM, Swanson DB. Validity of NBME Part I and Part II scores for selection of residents in orthopaedic surgery, dermatology, and preventive medicine. Acad Med. 1993;68(2 suppl):S51–S56.
Dirschl DR. Scoring of orthopaedic residency applicants: is a scoring system reliable? Clin Orthop Relat Res. 2002;399:260–264.
Dirschl DR, Adams GL. Reliability in evaluating letters of recommendation. Acad Med. 2000;75:1029.
Dirschl DR, Campion ER, Gilliam K. Resident selection and predictors of performance: can we be evidence based? Clin Orthop Relat Res. 2006;449:44–49.
Dirschl DR, Dahners LE, Adams GL, Crouch JH, Wilson FC. Correlating selection criteria with subsequent performance as residents. Clin Orthop Relat Res. 2002;399:265–271.
Evarts CM. Resident selection: a key to the future of orthopaedics. Clin Orthop Relat Res. 2006;449:39–43.
Godellas CCV, Hauge LLS, Huang RR. Factors affecting improvement on the American Board of Surgery In-Training Exam (ABSITE). J Surg Res. 2000;91:1–4.
Green M, Jones P, Thomas JX Jr. Selection criteria for residency: results of a national program directors survey. Acad Med. 2009;84:362–367.
Herndon JH, Allan BJ, Dyer G, Jawa A, Zurakowski D. Predictors of success on the American Board of Orthopaedic Surgery examination. Clin Orthop Relat Res. 2009;467:2436–2445.
Klein GR, Austin MS, Randolph S, Sharkey PF, Hilibrand AS. Passing the Boards: can USMLE and Orthopaedic in-Training Examination scores predict passage of the ABOS Part-I examination? J Bone Joint Surg Am. 2004;86:1092–1095.
Risner B, Nyland J, Crawford CH 3rd, Roberts CS, Johnson JR. Orthopaedic in-training examination performance: a nine-year review of a residency program database. South Med J. 2008;101:791–796.
Swanson DB, Sawhill A, Holtzman KZ, Bucak SD, Morrison C, Hurwitz S, DeRosa GP. Relationship between performance on part 1 of the American Board of Orthopaedic Surgery certifying examination and scores on USMLE Steps 1 and 2. Acad Med. 2009;84(10 suppl):S21–S24.
Thordarson DB, Ebramzadeh E, Sangiorgio SN, Schnall SB, Patzakis MJ. Resident selection: how we are doing and why? Clin Orthop Relat Res. 2007;459:255–259.
Turner NS, Shaughnessy WJ, Berg EJ, Larson DR, Hanssen AD. A quantitative composite scoring tool for orthopaedic residency screening and selection. Clin Orthop Relat Res. 2006;449:50–55.
Author information
Authors and Affiliations
Corresponding author
Additional information
One or more of the authors (HH and NW) has received funding from Stryker, Zimmer, and Biomet corporations.
Each author certifies that his or her institution has approved the human protocol for this investigation and that all investigations were conducted in conformity with ethical principles of research.
About this article
Cite this article
Dougherty, P.J., Walter, N., Schilling, P. et al. Do Scores of the USMLE Step 1 and OITE Correlate with the ABOS Part I Certifying Examination?: A Multicenter Study. Clin Orthop Relat Res 468, 2797–2802 (2010). https://doi.org/10.1007/s11999-010-1327-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11999-010-1327-3