External validation of clinical decision rules for children with wrist trauma

An Erratum to this article is available

Abstract

Background

Clinical decision rules help to avoid potentially unnecessary radiographs of the wrist, reduce waiting times and save costs.

Objective

The primary aim of this study was to provide an overview of all existing non-validated clinical decision rules for wrist trauma in children and to externally validate these rules in a different cohort of patients. Secondarily, we aimed to compare the performance of these rules with the validated Amsterdam Pediatric Wrist Rules.

Materials and methods

We included all studies that proposed a clinical prediction or decision rule in children presenting at the emergency department with acute wrist trauma. We performed external validation within a cohort of 379 children. We also calculated the sensitivity, specificity, negative predictive value and positive predictive value of each decision rule.

Results

We included three clinical decision rules. The sensitivity and specificity of all clinical decision rules after external validation were between 94% and 99%, and 11% and 26%, respectively. After external validation 7% to 17% less radiographs would be ordered and 1.4% to 5.7% of all fractures would be missed. Compared to the Amsterdam Pediatric Wrist Rules only one of the three other rules had a higher sensitivity; however both the specificity and the reduction in requested radiographs were lower in the other three rules.

Conclusion

The sensitivity of the three non-validated clinical decision rules is high. However the specificity and the reduction in number of requested radiographs are low. In contrast, the validated Amsterdam Pediatric Wrist Rules has an acceptable sensitivity and the greatest reduction in radiographs, at 22%, without missing any clinically relevant fractures.

Introduction

In children, distal radius fractures comprise 25–36% of all fractures [1, 2] and are therefore the most common fractures in children [3]. This high prevalence is most likely a result of the relative weakness of the metaphyseal bone, which has not yet modelled in children [4]. For reasons not clarified, the incidence of distal forearm fractures has shown a significant increase over the last few decades, from 151 in Sweden and 309 in the USA per 100,000 person-years to respectively 240 and 409 per 100,000 person-years [1, 5]. This is accompanied by an increasing number of emergency department visits and requested radiographs, and consequently rising health care costs [1, 6, 7].

The decision whether to request a radiograph of the wrist can be difficult for physicians [8]. Slaar et al. [9] showed that 51% of 1,233 children with a trauma of the wrist who presented at the emergency department of three Dutch hospitals had sustained a wrist fracture. The remaining 49% of the radiographs did not reveal a fracture of the wrist and were potentially unnecessary [9]. The introduction of a clinical decision rule could help to avoid unnecessary radiographs of the wrist and therefore decrease costs and waiting times [10]. For ankle injuries, the Ottawa Ankle Rules was successfully introduced in 1992, showing a 7.2% to 16% reduction in radiographs of the ankle in children since validation [11, 12]. Moreover, this rule has shown a 36-min decrease in length of the emergency department visit in adults [13], along with cost savings of $614,226 to $3,145,910 USD per 100,000 patients [14].

The development of a clinical decision rule consists of three steps: (1) derivation of the rule; (2) (external) validation and (3) implementation of the rule in clinical practice to test its impact on the decision-making of physicians [15]. The second step, validation, is most reliable when it is performed external, in a different population and performed by a different research group than the group who developed the rule [16,17,18].

Several attempts have been made to generate a clinical decision rule for children with acute wrist trauma in an effort to support physicians in making a more validated decision on whether a wrist radiograph should be acquired. One of these decision rules is the Amsterdam Pediatric Wrist Rules [19]. The Amsterdam Pediatric Wrist Rules is a clinical decision rule that aids in determining the need for a radiograph of the wrist in children, based on age and variables visible deformation, swelling of the distal radius, bone tenderness of the distal radius and the anatomical snuff box, and painful supination. Up till now this is the only rule that has been externally validated, with a sensitivity of 95.9% (95% confidence interval [CI]: 91.7–98.0%) and a specificity of 37.3% (95% CI: 31.0–44.1%).

The primary aim of this study was to provide an overview of all existing non-validated decision rules for wrist trauma in children and to externally validate these rules in a different cohort of patients. Secondarily, we aimed to compare the performance of these rules with the validated Amsterdam Pediatric Wrist Rules.

Materials and methods

Selection of existing clinical decision rules for children with wrist trauma

We performed a systematic literature search in Medline (Pubmed) on Dec. 21, 2015, using the search strategy depicted in Table 1. This systematic search was conducted according to the MOOSE (Meta-analysis of observational studies in epidemiology) guidelines [20]. We restricted the languages to English and Dutch. We included all types of studies that proposed a clinical prediction rule or decision rule in children presenting at the emergency department with acute wrist trauma. After screening the title and abstract, we studied full-text articles as to whether the eligible criteria were met. Finally, we performed a cross-reference check. We used the CHARMS checklist for critical appraisal and data extraction [21]. The CHARMS (checklist for critical appraisal and data extraction for systemic reviews of predication modelling studies) checklist has been designed for data extraction and quality assessment for systematic reviews of decision rules. This checklist contains 11 parameters that could lead to bias or that affect the applicability of the results. Two independent authors conducted the data extraction, addressing disagreement with discussion and consensus.

Table 1 Search strategy

Validation cohort

For external validation we used a study population previously described in the development and external validation study of the Amsterdam Pediatric Wrist Rules [19]. The study included a total of 379 children between 3 years and 18 years old, presenting with 170 wrist fractures and 209 non-fractures between April 6, 2011, and April 15, 2014, at the emergency department of three non-university teaching hospitals in the Netherlands. All consecutive children between 3 years and 18 years old were included if they had pain or tenderness secondary to acute wrist trauma. Acute trauma was defined as wrist trauma sustained within 72 h before presentation at the emergency department [19].

Using a standardised case record form, the data collected comprised 18 variables including patient characteristics, physical examination, functional testing and grip strength measured with a Baseline Hydraulic Hand Dynamometer (Fabrications Enterprises Incorporated, White Plains, New York, USA) (Table 2). All included patients were physically examined before the radiographs were taken. A fracture of the wrist was defined as fracture or epiphysiolysis of the distal radius or the distal ulna, or both [19]. Because fractures of the carpal bones in children are rare and frequently occult on plain radiographs, these fractures were not taken into account [2, 22, 23]. A fracture was recorded if a disruption of one or more cortices of the bone were present. Buckle fractures of bowing fractures were also defined as a true fractures, as were fissures and avulsions.

Table 2 Clinical variables of validation cohort

Sample size and statistical analysis

The sample for a validation study should include at least 100 events (fractures) and 100 non-events to detect relevant differences [24]. Among such events, a missing value level of less than 5% is considered as an acceptable value to use complete case analysis [25]. Missing completely at random (MCAR) may be used if the missing data are a random sample of the original dataset [26]. During our validation, we used complete case analysis for each decision rule if data were missing completely at random. To determine this, we performed a Little’s MCAR test. If this test was not statistically significant, the data were missing completely at random and complete case analysis could be performed.

We calculated the sensitivity and specificity of each decision rule, as well as the negative and positive predictive values and the 95% confidence intervals. Additionally, we determined the reduction in radiographs requested and the missed fractures rates. We analysed data using SPSS version 22.0 (IBM, Armonk, NY).

Results

Study selection

The search yielded 67 articles. After title and abstract screening and full-text reading, four articles met all our inclusion criteria (Fig. 1). One of these studies was the previously validated Amsterdam Pediatric Wrist Rules study [19]. This resulted in three other decision rules for children with wrist trauma.

Fig. 1
figure1

Study selection

Study characteristics and results

The first study we included was a prospective blinded case series by Pershad et al. [27]. This study included a total of 48 patients ages 3–18 years who sustained acute wrist injury within the preceding 3 days. This study excluded children with gross deformity at presentation at the emergency department because they were extremely likely to have a fracture. In addition to the standard evaluation after injury, physical examination consisted of measurement of range of motion with a goniometer and measurement of the grip strength with a Martin vigorimeter (Elmed Inc., Addison, IL). Moreover, in each child the zone (distal radius, distal ulna, carpal bones and scaphoid) of maximal tenderness was recorded. A radiograph was obtained in all children after the initial clinical examination. The Wilk log likelihood ratio test was used for the selection of predictors for multivariable modelling [27].

Zone tenderness (P=0.005), functional grip strength compared to the uninjured hand (P=0.019) and the overall suspicion of a fracture judged by the investigator prior to the radiograph (P=0.0083) were all statistically significant predictors. However the authors included only zone tenderness and functional grip strength in their decision rule (Table 3) [27]. The sensitivity of this clinical decision rule in the study cohort was 79% and the specificity was 63% [27]. The negative and positive predictive values were respectively 75% and 68% [27].

The second study was a prospective cohort study of Webster et al. [28]. This study included a total of 227 children ages 3–16 years who presented within 72 h after blunt wrist trauma. Exclusion criteria were patients with gross deformity because of their high likelihood of having sustained a fracture, as well as altered mental status, bone disease and an open fracture. Physical examination included focal swelling; grip strength; zone tenderness; and the range of motion of supination, pronation, dorsiflexion and palmar flexion [28]. To determine zone tenderness, the same zones were used as by Pershad et al. [27]. Additionally, the time between injury and presentation to the emergency department (more or less than 6 h) was taken into account [28]. The decision to request a radiograph of the wrist was at the discretion of the treating physician. All the outcome variables were analysed with the Pearson chi-square test. Subsequently, all variables with P<0.2 were included in a multivariate logistic regression model [28].

Univariate analysis showed that 6 of the 10 clinical predictors were significantly associated with a fracture (at a significance level of P<0.2). In the multivariate analysis, only radial tenderness (P<0.01), reduced supination or pronation (P<0.05) and focal swelling (P<0.001) were significantly and independently associated with a fracture (Table 3) [28]. This rule had a sensitivity of 99.1% (95% CI: 94.8–100%) and a specificity of 24% (95% CI: 17.2–32.3%) when applied in the study’s own cohort [28].

The third study was a retrospective study by Rivara et al. [29]. This study included a total of 189 children younger than 16 years with an injury that occurred within 7 days prior to the emergency department visit. For each injury, data collected included bone deformity or bone instability, crepitance, pain or limited range of motion, swelling, point tenderness, decreased sensation and ecchymosis. In addition, age, race, sex, time of day and cause and mechanism of injury were recorded. The majority of physicians requested the radiograph before completing the data collection form. However there was no evidence that the timing of completing the data collection form influenced the findings [29].

Before entering variables in a linear discriminant model, odds ratios were calculated for each predictor (entered if p≤0.05). When it was impossible to estimate the relative discriminating power with an odds ratio, because all patients with this symptom had a fracture, a stepwise multivariable discriminant analysis was performed. Gross deformity and point tenderness showed the highest odds ratios, of respectively 16.1 (95% CI: 4.7 to 54.9) and 7.0 (95% CI: 3.2 to 15.6) and were consequently the best discriminators between the fracture and the non-fracture group (Table 3). The presence of ecchymosis was also a significant discriminator. However the presence of ecchymosis was not retained in the decision rule because, in the absence of point tenderness and gross deformity, it did not differ between the fracture and no-fracture groups indicated by the odds ratio [29]. This rule of Rivara et al. [29] showed a sensitivity and specificity of respectively 81% and 82% and negative predictive value of 75% in their own study cohort.

Table 3 Decision rules

Methodological quality of the studies

All three studies were single-centre studies. None of the studies clarified how missing data were handled. Only Rivara et al. [29] reported the number of missing data for each predictor. Both Pershad et al. [27] and Webster et al. [28] conducted a decision rule for the wrist only, whereas Rivara et al. made one for the whole upper extremity. Additionally, in the study of Rivara et al. not all physicians requested the radiograph after completing the data collection form. Although there was no difference in predictors of positive or negative radiograph findings between data collection forms that were finished after requesting a radiograph and those finished before requesting a radiograph, this could have led to inclusion bias. Last, except for Webster et al. [28], none of the studies mentioned a 95% confidence interval of the sensitivity or specificity, nor the discrimination and calibration curves. For a complete overview of the results of the CHARMS checklist, see Table 4.

Table 4 CHARMS checklist for quality assessment

External validation

Pershad et al. [27]

We evaluated the external performance of the clinical decision rule of Pershad et al. [27] in a cohort of 326 of our 379 subjects. We excluded a total of 53 patients: 28 patients because of the presence of gross deformity and 25 patients after complete case analysis (Little’s MCAR test; P=0.337).

The sensitivity and specificity after external validation were respectively 94% (95% CI: 89–97%) and 26% (95% CI: 20–33%). The negative predictive value was 86% (95% CI: 74–93%) and the positive predictive value was 49% (95% CI: 43–55%; Table 5). After applying this clinical decision rule to the validation cohort, 17% less radiographs would have been requested and 5.7% (8) of the fractures would have been missed (Table 6).

Table 5 Outcomes after external validation
Table 6 CHARMS (checklist for critical appraisal and data extraction for systemic reviews of predication modelling studies)

Webster et al. [28]

We evaluated the external performance of the clinical decision rule of Webster et al. [28] in a cohort of 351 of our 379 patients. The presence of gross deformity led to the exclusion of 28 subjects. No patients were excluded because of complete case analysis.

The sensitivity in the validation cohort was 99% (95% CI: 95–100%) and the specificity was 11% (95% CI: 7–17%). The negative and positive predictive values were respectively 92% (95% CI: 72–99%) and 44% (95% CI: 39–50%; Table 5). After applying this clinical decision rule to the validation cohort, 7% less radiographs would have been requested and 1.4% (2) of fractures would have been missed (Table 6).

Rivara et al. [29]

We evaluated the external performance of the clinical decision rule of Rivara et al. [29] in a cohort of 352 of our 379 patients. We excluded a total of 27 patients because they were 16 years or older. No patients were excluded because of complete case analysis.

The sensitivity and specificity were respectively 96% (95% CI: 91% to 98%) and 22% (95% CI: 16% to 28%). The negative predictive value was 85% (95% CI: 72% to 93%) and the positive predictive value was 51% (95% CI: 46% to 57%; Table 5). After applying this clinical decision rule on the validation cohort, 14% less radiographs would have been requested and 4.3% (7) fractures would have been missed (Table 6).

Comparison with Amsterdam Pediatric Wrist Rules [19]

The sensitivity of the Amsterdam Pediatric Wrist Rules was 96% (95% CI: 92–98%) and the specificity was 37% (95% CI: 31–44%). This specificity was higher compared to the specificity of the other three rules after external validation. In contrast, the sensitivity was lower compared to the sensitivity of the decision rule of Webster et al. [28] and comparable with the sensitivity of Rivara et al. [29]. However, the Amsterdam Pediatric Wrist Rules showed a 22% reduction of radiographic examinations after external validation, which was higher than the other three decision rules. Although 4.1% of fractures were missed, none of these was clinically relevant.

Discussion

We included three studies, each describing a non-validated clinical decision rule for children with wrist trauma, and externally validated these rules in the study population in which the Amsterdam Pediatric Wrist Rules was developed and externally validated. The sensitivity of these three clinical decision rules after external validation was high, ranging from 94% to 99%. However, besides a low specificity ranging 11% to 26%, the reduction in radiographs requested without missing any clinically relevant fractures was not of great significance.

In order for physicians to use a clinical decision rule in the emergency department, the sensitivity should be high. Stiell and Wells [30] suggested a sensitivity of at least 96%. This would mean that only the rule of Pershad et al. [27] would not qualify. Conversely, the reduction in radiographs, without missing any clinically relevant fractures, is determinative for the accepted sensitivity. Clinicians might be more willing to use the decision rule if they knew that no fractures would be missed or that any missed fractures would not be clinically relevant. The use of the rule by Webster et al. [28] was accompanied by only 1.4% missed fractures, in contrast with the 5.7% missed fractures in the rule by Pershad et al. [27] and 4.1% in the rule Rivara et al. [29]. However, the rule of Webster et al. [28] only had a reduction in radiograph requests of 7%, which is not of great significance compared to current practice.

Of all 24 missed fractures, 46% were buckle fractures. Buckle fractures are stable fractures and can be treated safely with a soft cast or bandage therapy with good functional outcomes [31,32,33]. This treatment is equal to the treatment of contusions or sprains of the wrist. Because treatment and prognosis would not have been influenced by a missed or delayed diagnosis [34], these fractures could be considered not clinically relevant. In contrast, after external validation of the other three decision rules, five epiphysiolysis injuries of the distal radius, four extra-articular distal radius fractures and three greenstick fractures were missed. For these missed fractures the clinical impact is considerable and treatment is necessary.

After applying the Ottawa Ankle Rules in children, a pooled reduction of radiographs of approximately 24.8% was observed [35], which is a higher reduction of radiographs than what we found when validating the decision rules for wrist trauma, except for the Amsterdam Pediatric Wrist Rules. This is possibly a result of the very low a priori probability for a fracture in ankle injuries of 14%, compared to 53% for wrist trauma [36, 37]. The higher probability limits the possibilities for improvement. This was confirmed by Van den Brand et al. [36], who investigated the need for a clinical decision rule for patients with blunt wrist trauma. They confirmed the high wrist fracture ratio and recommended radiographs in all patients with wrist trauma presenting at the emergency department. Additionally, they concluded that it is not feasible to develop a decision rule with a high sensitivity and specificity. Despite their conclusion, two fractures in children were missed without even using a clinical decision rule [36]. In contrast, Slaar et al. [9] stated that the development of a clinical decision rule for children with blunt wrist trauma is warranted [9]. Although the costs per radiograph are relatively low, the overall cost of negative radiographs in the three hospitals was approximately €28,608 (U.S. $30,000) per year [9]. A reduction in radiographs could therefore result in cost savings. Furthermore, a reduction in time spent at the emergency department could be realised, like that seen after implementation of the Ottawa Ankle Rules [13].

Compared to the three decision rules that were externally validated in this study, the externally validated Amsterdam Pediatric Wrist Rules showed a 22% reduction of radiographic examinations. At the same time, in external validation this rule missed 4.1% of fractures, although none of these was clinically relevant [19]. The sensitivity of the Amsterdam Pediatric Wrist Rules was 96% (95% CI: 92–98%) and the specificity was 37% (95% CI: 31–44%). This is higher than the specificity of the other three rules after validation. Although the sensitivity of the Amsterdam decision rule is lower than the sensitivity of the Webster et al. [28] decision rule, it is presumably high enough to persuade physicians to use the decision rule in the emergency department.

Looking at the different assessment criteria, except for the 20% decrease in grip strength used in the rule of Pershad et al. [27], the three decision rules used the same variables as in the Amsterdam Pediatric Wrist Rules (i.e. deformity, tenderness of the distal radius, and a reduction in supination). However a possible reason for the better performance of the Amsterdam Pediatric Wrist Rules compared to the other three decision rules is the difference in the derivation of the rule. The Amsterdam Pediatric Wrist Rules uses a linear predictor to calculate the probability of a fracture, whereas the other three rules use the presence or absence of clinical variables. Moreover, age is not taken into account in the three other decision rules.

This study has several limitations. First, we used complete case analysis because only prehensile grip strength had a missing value percentage of more than 5%, namely 12.5%. Because of this complete case analysis, the clinical decision rule of Pershad et al. [27], which contains the variable grip strength, was validated in a smaller cohort than the other two rules. However, the remaining sample was sufficiently large for validation, especially when compared to the size of the derivation cohorts (326 compared to 48 children), and therefore we believe that this does not limit the validity of our results.

Second, we excluded children with gross deformity during the validation of the clinical decision rules by Pershad et al. [27] and Webster et al. [28] because this was one of their exclusion criteria. They excluded these subjects because of their very high likelihood of having a fracture. In general only 86.4% of the children with gross deformity in our dataset, as noted by the emergency department physicians, had a fracture. Although it seems unlikely not to have a fracture with the presence of gross deformity, this could possibly be caused by the swelling that arises after the trauma and mimics a deformity. However, if you remove the evident cases of fractures, it becomes harder for a decision rule to identify fractures and it lowers the sensitivity.

Conclusion

The sensitivity of the three included clinical decision rules for wrist trauma ranges from 94% to 99%. However, the specificity and the reduction in requested radiographs of these three decision rules are low, and therefore it is doubtful whether these decision rules would supplement current practice. In contrast, the externally validated Amsterdam Pediatric Wrist Rules has been shown to have an acceptable sensitivity and a reduction in radiographs of 22% without missing any clinically relevant fractures.

References

  1. 1.

    Hedstrom EM, Svensson O, Bergstrom U et al (2010) Epidemiology of fractures in children and adolescents. Acta Orthop 81:148–153

    Article  PubMed  PubMed Central  Google Scholar 

  2. 2.

    Rennie L, Court-Brown CM, Mok JY et al (2007) The epidemiology of fractures in children. Injury 38:913–922

    Article  PubMed  Google Scholar 

  3. 3.

    Cooper C, Dennison EM, Leufkens HG et al (2004) Epidemiology of childhood fractures in Britain: a study using the general practice research database. J Bone Miner Res 19:1976–1981

    Article  PubMed  Google Scholar 

  4. 4.

    Parfitt AM (1994) The two faces of growth: benefits and risks to bone integrity. Osteoporos Int 4:382–398

    CAS  Article  PubMed  Google Scholar 

  5. 5.

    Khosla S, Melton LJ 3rd, Dekutoski MB et al (2003) Incidence of childhood distal forearm fractures over 30 years: a population-based study. JAMA 290:1479–1485

    CAS  Article  PubMed  Google Scholar 

  6. 6.

    Ryan LM, Teach SJ, Searcy K et al (2010) Epidemiology of pediatric forearm fractures in Washington, DC. J Trauma 69:S200–S205

    Article  PubMed  Google Scholar 

  7. 7.

    Frush DP, Frush KS (2008) The ALARA concept in pediatric imaging: building bridges between radiology and emergency medicine: consensus conference on imaging safety and quality for children in the emergency setting, Feb. 23–24, 2008, Orlando, FL — executive summary. Pediatr Radiol 38:S629–632

  8. 8.

    Walenkamp MM, Rosenwasser MP, Goslings JC et al (2015) A multicentre cross-sectional study to examine physicians’ ability to rule out a distal radius fracture based on clinical findings. Eur J Trauma Emerg Surg 42:185–190

    Article  PubMed  PubMed Central  Google Scholar 

  9. 9.

    Slaar A, Bentohami A, Kessels J et al (2012) The role of plain radiography in paediatric wrist trauma. Insights Imaging 3:513–517

    Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Gleadhill DN, Thomson JY, Simms P (1987) Can more efficient use be made of x ray examinations in the accident and emergency department? Br Med J (Clin Res Ed) 294:943–947

    CAS  Article  Google Scholar 

  11. 11.

    Libetta C, Burke D, Brennan P et al (1999) Validation of the Ottawa Ankle Rules in children. J Accid Emerg Med 16:342–344

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Plint AC, Bulloch B, Osmond MH et al (1999) Validation of the Ottawa Ankle Rules in children with ankle injuries. Acad Emerg Med 6:1005–1009

    CAS  Article  PubMed  Google Scholar 

  13. 13.

    Stiell IG, McKnight RD, Greenberg GH et al (1994) Implementation of the Ottawa Ankle Rules. JAMA 271:827–832

    CAS  Article  PubMed  Google Scholar 

  14. 14.

    Anis AH, Stiell IG, Stewart DG et al (1995) Cost-effectiveness analysis of the Ottawa Ankle Rules. Ann Emerg Med 26:422–428

    CAS  Article  PubMed  Google Scholar 

  15. 15.

    Adams ST, Leveson SH (2012) Clinical prediction rules. BMJ 344:d8312

    Article  PubMed  Google Scholar 

  16. 16.

    Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19:453–473

    CAS  Article  PubMed  Google Scholar 

  17. 17.

    Nieboer D, van der Ploeg T, Steyerberg EW (2016) Assessing discriminative performance at external validation of clinical prediction models. PLoS One 11, e0148820

    Article  PubMed  PubMed Central  Google Scholar 

  18. 18.

    Green SM, Schriger DL, Yealy DM (2014) Methodologic standards for interpreting clinical decision rules in emergency medicine: 2014 update. Ann Emerg Med 64:286–291

    Article  PubMed  Google Scholar 

  19. 19.

    Slaar A, Walenkamp MM, Bentohami A et al (2016) A clinical decision rule for the use of plain radiography in children after acute wrist injury: development and external validation of the Amsterdam Pediatric Wrist Rules. Pediatr Radiol 46:50–60

    Article  PubMed  Google Scholar 

  20. 20.

    Stroup DF, Berlin JA, Morton SC et al (2000) Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of observational studies in epidemiology (MOOSE) group. JAMA 283:2008–2012

    CAS  Article  PubMed  Google Scholar 

  21. 21.

    Moons KG, de Groot JA, Bouwmeester W et al (2014) Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med 11, e1001744

    Article  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Journeau P (2013) Carpal injuries in children. Chir Main 32:S16–S28

    Article  PubMed  Google Scholar 

  23. 23.

    Tiel-van Buul MM, van Beek EJ, Broekhuizen AH et al (1992) Diagnosing scaphoid fractures: radiographs cannot be used as a gold standard! Injury 23:77–79

    CAS  Article  PubMed  Google Scholar 

  24. 24.

    Vergouwe Y, Steyerberg EW, Eijkemans MJ et al (2005) Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol 58:475–483

    Article  PubMed  Google Scholar 

  25. 25.

    Tabachnick BG, Fidell LS (2007) Using multivariate statistics, 5th edn. Pearson Education, Boston

    Google Scholar 

  26. 26.

    Pigott TD (2001) A review of methods for missing data. Educ Res Eval 7:353–383

    Article  Google Scholar 

  27. 27.

    Pershad J, Monroe K, King W et al (2000) Can clinical parameters predict fractures in acute pediatric wrist injuries? Acad Emerg Med 7:1152–1155

    CAS  Article  PubMed  Google Scholar 

  28. 28.

    Webster AP, Goodacre S, Walker D et al (2006) How do clinical features help identify paediatric patients with fractures following blunt wrist trauma? Emerg Med J 23:354–357

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  29. 29.

    Rivara FP, Parish RA, Mueller BA (1986) Extremity injuries in children: predictive value of clinical findings. Pediatrics 78:803–807

    CAS  PubMed  Google Scholar 

  30. 30.

    Stiell IG, Wells GA (1999) Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 33:437–447

    CAS  Article  PubMed  Google Scholar 

  31. 31.

    Firmin F, Crouch R (2009) Splinting versus casting of ‘torus’ fractures to the distal radius in the paediatric patient presenting at the emergency department (ED): a literature review. Int Emerg Nurs 17:173–178

    Article  PubMed  Google Scholar 

  32. 32.

    Vernooij CM, Vreeburg ME, Segers MJ et al (2012) Treatment of torus fractures in the forearm in children using bandage therapy. J Trauma Acute Care Surg 72:1093–1097

    Article  PubMed  Google Scholar 

  33. 33.

    Plint AC, Perry JJ, Correll R et al (2006) A randomized, controlled trial of removable splinting versus casting for wrist buckle fractures in children. Pediatrics 117:691–697

    Article  PubMed  Google Scholar 

  34. 34.

    Bennett DL, Mencio GA, Hernanz-Schulman M et al (2009) Buckle fractures in children: is urgent treatment necessary? J Fam Pract 58:E1–E6

    PubMed  Google Scholar 

  35. 35.

    Dowling S, Spooner CH, Liang Y et al (2009) Accuracy of Ottawa Ankle Rules to exclude fractures of the ankle and midfoot in children: a meta-analysis. Acad Emerg Med 16:277–287

    Article  PubMed  Google Scholar 

  36. 36.

    van den Brand CL, van Leerdam RH, van Ufford JHMEQ et al (2013) Is there a need for a clinical decision rule in blunt wrist trauma? Injury 44:1615–1619

    Article  PubMed  Google Scholar 

  37. 37.

    Walenkamp MM, Bentohami A, Slaar A et al (2015) The Amsterdam wrist rules: the multicenter prospective derivation and external validation of a clinical decision rule for the use of radiography in acute wrist trauma. BMC Musculoskelet Disord 16:389

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Mrs. Faridi S. van Etten, clinical librarian, for assistance with the search strategy.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Marjolein A. M. Mulders.

Ethics declarations

Conflicts of interest

None

Additional information

An erratum to this article is available at http://dx.doi.org/10.1007/s00247-017-3841-x.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mulders, M.A.M., Walenkamp, M.M.J., Dubois, B.F.H. et al. External validation of clinical decision rules for children with wrist trauma. Pediatr Radiol 47, 590–598 (2017). https://doi.org/10.1007/s00247-017-3787-z

Download citation

Keywords

  • Adolescent
  • Child
  • Decision rule
  • Distal radius
  • Fracture
  • Trauma
  • Wrist