Abstract
Purpose
Assessments are fundamentally important for training surgeons. Currently, there are no formal means of assessing operative fracture fixation. An assessment tool has been developed which can be used by trainers to critique the quality of a trainee’s operative fracture fixation. The tool is based on the AO principles of fracture management. The reliability and validity of the assessment were tested in a prospective study.
Methods
The assessment tool comprises of 4 domains focusing on the different factors pertinent to fracture fixation (reduction, stability, implant and overall impression). Reliability and validity were evaluated by asking 10 consultant trauma and orthopaedic surgeons to score 20 test cases on two different occasions at least 7 weeks apart. Internal consistency was assessed by Cronbach’s alpha. Inter-rater reliability and test–retest reliability were assessed by the inter-class correlation coefficient (ICC) and content validity by the content validity ratio (CVR).
Results
Cronbach’s alpha was 0.976, with all component criteria correlating well with each other. Total score inter-rater reliability, for a single assessor, as given by the ICC, was 0.708. Overall test–retest reliability was 0.961. The CVR for the assessment tool was 0.65 (which is above the critical value for establishing validity with 10 assessors).
Conclusions
Internal consistency is demonstrated by the excellent Cronbach’s alpha with substantial single assessor and excellent test–retest reliability also shown. The CVR above the critical value illustrates that the assessment is valid. The assessment tool has a number of applications within training and service evaluation that could benefit the global orthopaedic community.
Similar content being viewed by others
References
Wilson T, Sahu A, Johnson D, Turner P (2010) The effect of trainee involvement on procedure and list times: a statistical analysis with discussion of current issues affecting orthopaedic training in UK. Surgeon 8:15–19
Halpin G, Halpin G (1982) Experimental investigation of the effects of study and testing on student learning, retention, and ratings of instruction. J Educ Psychol 74:32–38
Newble DI, Jaeger K (1983) The effects of assessments and examinations on the learning of medical students. Med Educ 17:165–171
Stillman PL, Haley HL, Regan MB, Philbin MM (1991) Positive effects of a clinical performance assessment program. Acad Med 66:481–483
Schatzker J (2000) AO philosophy and principles. In: Ruedi TP, Murphy WM (eds) AO principles of fracture management. Thieme, New York, pp 1–4
Survey Monkey. www.surveymonkey.co.uk/. Accessed 1 Nov 2016
Obuchowski NA (2000) Sample size tables for receiver operating characteristic studies. AJR Am J Roentgenol 175:603–608
Streiner DL (2003) Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess 80:99–103
Landis JR, Koch GC (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174
Lawshe CH (1975) A quantitative approach to content validity. Pers Psychol 28:563–575
Wilson FR, Pan W, Schumsky DA (2012) Recalculation of the critical values for Lawshe’s content validity ratio. Meas Eval Couns Dev 45:197–210
Acknowledgements
We would like to thank Sven Young, Tim Nunn, Jes Bates, Henry Wynn Jones, Rick Gardner, Wilfred Addo, Peter Kenyon, Linda Chokotho, Edward Wood and Nicholas Lubega for generously giving their time to assess the test cases. We would also like to thank Paul Chidothi for his help in providing the radiographs.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
David H. Hawkes and William J. Harrison declare that they have no conflict of interest.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Hawkes, D.H., Harrison, W.J. Critiquing operative fracture fixation: the development of an assessment tool. Eur J Orthop Surg Traumatol 27, 1083–1088 (2017). https://doi.org/10.1007/s00590-017-1943-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00590-017-1943-7