Skip to main content
SpringerLink
Log in

Training Improves Interobserver Reliability for the Diagnosis of Scaphoid Fracture Displacement

  • Clinical Research
  • Published:
Clinical Orthopaedics and Related Research®

Abstract

Background

The diagnosis of displacement in scaphoid fractures is notorious for poor interobserver reliability.

Questions/purposes

We tested whether training can improve interobserver reliability and sensitivity, specificity, and accuracy for the diagnosis of scaphoid fracture displacement on radiographs and CT scans.

Methods

Sixty-four orthopaedic surgeons rated a set of radiographs and CT scans of 10 displaced and 10 nondisplaced scaphoid fractures for the presence of displacement, using a web-based rating application. Before rating, observers were randomized to a training group (34 observers) and a nontraining group (30 observers). The training group received an online training module before the rating session, and the nontraining group did not. Interobserver reliability for training and nontraining was assessed by Siegel’s multirater kappa and the Z-test was used to test for significance.

Results

There was a small, but significant difference in the interobserver reliability for displacement ratings in favor of the training group compared with the nontraining group. Ratings of radiographs and CT scans combined resulted in moderate agreement for both groups. The average sensitivity, specificity, and accuracy of diagnosing displacement of scaphoid fractures were, respectively, 83%, 85%, and 84% for the nontraining group and 87%, 86%, and 87% for the training group. Assuming a 5% prevalence of fracture displacement, the positive predictive value was 0.23 in the nontraining group and 0.25 in the training group. The negative predictive value was 0.99 in both groups.

Conclusions

Our results suggest training can improve interobserver reliability and sensitivity, specificity and accuracy for the diagnosis of scaphoid fracture displacement, but the improvements are slight. These findings are encouraging for future research regarding interobserver variation and how to reduce it further.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1A–C

Similar content being viewed by others

References

  1. Amadio PC, Berquist TH, Smith DK, Ilstrup DM, Cooney WP III, Linscheid RL. Scaphoid malunion. J Hand Surg Am. 1989;14:679–687.

    Article  PubMed  CAS  Google Scholar 

  2. Bain GI, Bennett JD, MacDermid JC, Slethaug GP, Richards RS, Roth JH. Measurement of the scaphoid humpback deformity using longitudinal computed tomography: intra- and interobserver variability using various measurement techniques. J Hand Surg Am. 1998;23:76–81.

    Article  PubMed  CAS  Google Scholar 

  3. Bankier AA, Fleischmann D, De Maertelaer V, Kontrus M, Zontsich T, Hittmair K, Mallek R. Subjective differentiation of normal and pathological bronchi on thin-section CT: impact of observer training. Eur Respir J. 1999;13:781–786.

    Article  PubMed  CAS  Google Scholar 

  4. Berg WA, D’Orsi CJ, Jackson VP, Bassett LW, Beam CA, Lewis RS, Crewson PE. Does training in the Breast Imaging Reporting and Data System (BI-RADS) improve biopsy recommendations or feature analysis agreement with experienced breast imagers at mammography? Radiology. 2002;224:871–880.

    Article  PubMed  Google Scholar 

  5. Bernard SA, Murray PM, Heckman MG. Validity of conventional radiography in determining scaphoid waist fracture displacement. J Orthop Trauma. 2010;24:448–451.

    Article  PubMed  Google Scholar 

  6. Bernstein J, Adler LM, Blank JE, Dalsey RM, Williams GR, Iannotti JP. Evaluation of the Neer system of classification of proximal humeral fractures with computerized tomographic scans and plain radiographs. J Bone Joint Surg Am. 1996;78:1371–1375.

    PubMed  CAS  Google Scholar 

  7. Brorson S, Bagger J, Sylvest A, Hrobjartsson A. Improved interobserver variation after training of doctors in the Neer system: a randomised trial. J Bone Joint Surg Br. 2002;84:950–954.

    Article  PubMed  CAS  Google Scholar 

  8. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.

    Article  Google Scholar 

  9. Cooney WP, Dobyns JH, Linscheid RL. Fractures of the scaphoid: a rational approach to management. Clin Orthop Relat Res. 1980;149:90–97.

    PubMed  Google Scholar 

  10. Dabezies EJ, Mathews R, Faust DC. Injuries to the carpus: fractures of the scaphoid. Orthopedics. 1982;5:1510–1515.

    Google Scholar 

  11. de Vet HC, Koudstaal J, Kwee WS, Willebrand D, Arends JW. Efforts to improve interobserver agreement in histopathological grading. J Clin Epidemiol. 1995;48:869–873.

    Article  PubMed  Google Scholar 

  12. Desai VV, Davis TR, Barton NJ. The prognostic value and reproducibility of the radiological features of the fractured scaphoid. J Hand Surg Br. 1999;24:586–590.

    PubMed  CAS  Google Scholar 

  13. Eddeland A, Eiken O, Hellgren E, Ohlsson NM. Fractures of the scaphoid. Scand J Plast Reconstr Surg. 1975;9:234–239.

    Article  PubMed  CAS  Google Scholar 

  14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174.

    Article  PubMed  CAS  Google Scholar 

  15. Lozano-Calderon S, Blazar P, Zurakowski D, Lee SG, Ring D. Diagnosis of scaphoid fracture displacement with radiography and computed tomography. J Bone Joint Surg Am. 2006;88:2695–2703.

    Article  PubMed  Google Scholar 

  16. Lujan ME, Chizen DR, Peppin AK, Kriegler S, Leswick DA, Bloski TG, Pierson RA. Improving inter-observer variability in the evaluation of ultrasonographic features of polycystic ovaries. Reprod Biol Endocrinol. 2008;6:30.

    Article  PubMed  Google Scholar 

  17. Magnan MA, Maklebust J. The effect of Web-based Braden Scale training on the reliability of Braden subscale ratings. J Wound Ostomy Continence Nurs. 2009;36:51–59.

    Article  PubMed  Google Scholar 

  18. Patel AB, Amin A, Sortey SZ, Athawale A, Kulkarni H. Impact of training on observer variation in chest radiographs of children with severe pneumonia. Indian Pediatr. 2007;44:675–681.

    PubMed  Google Scholar 

  19. Posner KL, Sampson PD, Caplan RA, Ward RJ, Cheney FW. Measuring interrater reliability among multiple raters: an example of methods for nominal data. Stat Med. 1990;9:1103–1115.

    Article  PubMed  CAS  Google Scholar 

  20. Ring D, Patterson JD, Levitz S, Wang C, Jupiter JB. Both scanning plane and observer affect measurements of scaphoid deformity. J Hand Surg Am. 2005;30:696–701.

    Article  PubMed  Google Scholar 

  21. Ripsweden J, Mir-Akbari H, Brolin EB, Brismar T, Nilsson T, Rasmussen E, Ruck A, Svensson A, Werner C, Winter R, Cederlund K. Is training essential for interpreting cardiac computed tomography? Acta Radiol. 2009;50:194–200.

    Article  PubMed  CAS  Google Scholar 

  22. Sanders WE. Evaluation of the humpback scaphoid by computed tomography in the longitudinal axial plane of the scaphoid. J Hand Surg Am. 1988;13:182–187.

    Article  PubMed  CAS  Google Scholar 

  23. Siegel S, Castellan NJ. Nonparametric Statistics for the Behavioral Sciences. 2nd ed. New York, NY: McGraw-Hill; 1988.

    Google Scholar 

  24. Stieber J, Quirno M, Cunningham M, Errico TJ, Bendo JA. The reliability of computed tomography and magnetic resonance imaging grading of lumbar facet arthropathy in total disc replacement patients. Spine (Phila Pa 1976). 2009;34:E833–840.

  25. Szabo RM, Manske D. Displaced fractures of the scaphoid. Clin Orthop Relat Res. 1988;230:30–38.

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Orthopaedic Research Center Amsterdam, Department of Orthopaedic Surgery, Academic Medical Centre, Amsterdam, The Netherlands

    Geert A. Buijze MD, Thierry G. Guitton PhD & C. Niek van Dijk MD, PhD

  2. Orthopaedic Hand and Upper Extremity Service, Massachusetts General Hospital, Harvard Medical School, Yawkey Center, Suite 2100, 55 Fruit Street, Boston, MA, 02114, USA

    David Ring MD, PhD

Authors
  1. Geert A. Buijze MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thierry G. Guitton PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Niek van Dijk MD, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Ring MD, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

The Science of Variation Group

Corresponding author

Correspondence to David Ring MD, PhD.

Additional information

The Science of Variation Group consists of the following members: A. Lee Osterman, Abhijeet L. Wahegaonkar, Amy Ladd, Antonio Barquet, Arie B. van Vugt, Ashok K. Shyam, Carrie Swigart, Chad P. Coles, Charalampos Zalavras, Charles A. Goldfarb, Charles Cassidy, Christopher Allan, Daphne Beingessner, David M. Kalainov, Denise Eygendaal, Parag Sancheti, Robert J. Feibel, Steve Rocha, Elena Grosso, Frede Frihagen, George S.M. Dyer, George S. Athwal, J. Carel Goslings, Gregory J. Della Rocca, Ian Harris, Jason C. Fanuele, Jeff Lawton, John Jiuliano, John McAuliffe, John T. Capo, Joseph M. Conflitti, Keith Segalman, Kenneth Egol, Kornelis J. Ponsen, Kyle Jeray, Lisa Lattanza, Louis Catalano III, Marc Swiontkowski, Martin Boyer, Martin Richardson, Maximillian Soong, Michael Baskies, Michael Prayson, Michael Mckee, Neal C. Chen, P.V. van Eerten, Peter R.G. Brink, Peter J. Evans, Peter Jebson, Peter Kloen, Philip Blazar, Steven J. Rhemrev, Richard S. Page, Rick Papandrea, Rob Nelissen, Robert D. Zura, Rodrigo Pesantez, Rudolf W. Poolman, Samir Sodha, Scott Duncan, Scott Wolfe, Taco Gosens, Thomas Wright, Tim Davis.

One of the authors (GAB) has received funding from the Netherlands Organisation for Scientific Research, The Hague, The Netherlands.

All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research editors and board members are on file with the publication and can be viewed on request.

Each author certifies that his or her institution approved the human protocol for this investigation, that all investigations were conducted in conformity with ethical principles of research, and that informed consent for participation in the study was obtained.

This work was performed at the Orthopaedic Hand and Upper Extremity Service of the Massachusetts General Hospital, Boston, MA, USA.

About this article

Cite this article

Buijze, G.A., Guitton, T.G., van Dijk, C.N. et al. Training Improves Interobserver Reliability for the Diagnosis of Scaphoid Fracture Displacement. Clin Orthop Relat Res 470, 2029–2034 (2012). https://doi.org/10.1007/s11999-012-2260-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11999-012-2260-4

Keywords

Search

Navigation