Abstract
Objective
The postoperative care regimes of ankle fractures are studied for over 30 years and recommendations have shifted only slightly in the last decades. However, study methodology might have evolved. The aim of this study was to evaluate the changes in time in the design, quality and outcome measures of studies investigating the postoperative care of ankle fractures.
Methods
The MEDLINE and EMBASE database were searched for both RCTs and cohort studies. The original studies were divided into decades of publication over the last 30 years. The methodological quality of the studies was assessed using the ‘traditional’ risk of bias assessment tool provided by The Cochrane Collaboration and the ‘newer’ MINORS criteria.
Results
The percentage of RCTs on this subject declined from 67 to 38 % in the last decades. According to the Cochrane tool, the reported quality of RCTs has improved in the last three decades whereas the reported quality of observational studies has remained unchanged. However, when quality was evaluated with the MINORS criteria, equal improvement was observed for both RCTs and observational studies. In the 80s, 67 % of all studies used the range of motion as the primary outcome measure, which decreased to 45 % in the 90s. In the 00s, none of the studies used the range of motion as the primary outcome.
Conclusion
For postoperative care of ankle fractures, results of this study showed a relative decrease in the published number of RCTs. The overall quality of the published articles did not decline. In addition, a gradual shift from physician measured to patient-reported outcome variables was observed. However, it should be borne in mind that the findings are based on a small sample (n = 25).
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
In evidence-based medicine, the value of observational studies, such as cohort and case–control studies, has generally been regarded as relatively insignificant [1]. Randomized controlled trials (RCTs) have long been considered the gold standard [2]. In the strive for optimal evidence-based medicine and subsequent patient care, the utility of conducting Randomized Controlled Trials (RCT) in trauma surgery has been questioned over the past years [3]. In addition, observational studies are published frequently, while the number of surgical RCTs from North America has decreased over the last decade [4]. This raises questions how the quality of studies evolved over the last years. In addition, as a result of evolving clinical experience and meeting patient demands, there is more emphasis on patient-reported outcome measurements. Patients are more involved in the decision-making process concerning their treatment, which should ultimately lead to shared decision making.
In trauma surgery, ankle fractures are among the most common fractures and the indication for surgery has not changed in the past decades [5]. Furthermore, the postoperative care regimes of ankle fractures are studied for over 30 years and recommendations have shifted only slightly in the last decades, as we demonstrated recently. As a consequence, the circumstances in which studies were performed did not change substantially in this field of medicine. Therefore, treatment of ankle fractures can be used as an example to investigate the evolution in study methodology. The goal of this study was to evaluate the changes in time in the design, quality and outcome measures of studies investigating the postoperative care of ankle fractures.
Methods
Selection
This study included the same studies as selected in our previous meta-analysis evaluating the postoperative care regimen of ankle fractures of twenty-five RCTs and cohort studies [6]. The studies included were retrieved by two reviewers (DS and FH), who independently searched the MEDLINE and EMBASE electronic databases. An extensive description of the search strategy and study selection with inclusion and exclusion criteria was published in the meta-analysis [6].
Data extraction
The original studies were divided into decades of publication, for the last 30 years. The decades covered the years from 1980 until 1989, 1990 until 1999 and 2000 and onwards. Subsequently, the studies were analysed for design, number of patients, quality, primary outcome variable and other outcomes including complications. The study designs were divided into a retrospective cohort, prospective cohort, randomised controlled trial or a combination of designs. Partly prospective and partly retrospective studies were defined as a mixed cohort study. The methodological quality of the studies was independently assessed by three reviewers (DS, FH and RH) using the risk of bias assessment tool provided by The Cochrane Collaboration and the MINORS criteria [7, 8]. The risk of bias assessment tool provided by The Cochrane Collaboration was developed to compare the quality of RCTs and the MINORS criteria were developed to compare the quality of nonrandomized studies. A higher score indicates a higher quality. In addition, we have checked the affiliations of the authors of the included articles to check if a methodologist cooperated in the research project. The primary outcome variables were the range of motion (ROM), functional scores (such as the Olerud Molander score; a patient-reported outcome) and return to work data [9]. Secondary outcomes ranged from objective physician measured outcomes to more subjective patient-dependent outcomes. If none of the above-mentioned outcome variables were used, the study was excluded from the analysis.
Descriptive analysis
SPSS software (version 20.0, Chicago, IL, USA) was used for descriptive analysis, both statistically and graphically. Additional analyses were not performed.
Results
A total of 25 studies were included, 6 of which from the 80s, 11 from the 90s and 8 from the twenty-first century (Table 1). From all studies, 12 (48 %) were RCTs.
Study design
Fewer RCTs were published in the last decades. The relative number of RCTs decreased. The percentage of RCTs on this subject declined from 67 to 38 % (Table 1; Fig. 1). The number of patients per study changed from 46 (±10) in the 80s to 51 (±13) in the 90s to 67 (±33) in the twenty-first century.
Quality
Over the past three decades, the overall quality of the published articles did not decline (Table 1). The mean score on the Cochrane risk of bias assessment tool was 2.2 (±1.3). The highest score was five out of seven points on the Cochrane risk of bias assessment tool, which was reached by two studies. Interestingly, the reported quality of RCTs improved in the last decade, from a mean of 2.3 (±0.5) to 4.0 (±1.0) (Fig. 2). The quality of prospective and mixed cohort studies remained unchanged. The mean score on the MINORS criteria was 16.3 (±2.9). When using the MINORS criteria, the reported quality of RCTs improved in the last decade, from a mean of 15.8 (±3.4) to 19.7 (±1.2). In addition, the quality of prospective and mixed cohort studies improved in the last decade, from a mean of 14.0 (±1.4) to 17.2 (±3.3) (Fig. 3).
Two articles had one or more authors that were affiliated with a research department [11, 18]. Both were RCTs and had a relative high score on the Cochrane risk of bias assessment tool and the MINORS criteria.
Primary outcome variable and the most patient-dependent outcome variable
A functional score was used as primary outcome in 13 (52 %) studies and the range of motion was used in 10 (40 %) studies as primary outcome. In the 80s, 67 % of all studies used the range of motion as the primary outcome measure, which decreased to 45 % in the 90s. In the 00s, none of the studies used the range of motion as the primary outcome. Less objective outcome measures and more functional scores were used over time (Table 1; Fig. 4).
The return to work was the most used patient-dependent outcome variable applied in 12 (48 %) studies (Fig. 5). In the 80s, all six studies used the range of motion as outcome compared to four studies (50 %) in the 00s. In the last decade, six studies (75 %) reported “return to work” data.
A total of two studies did not report on complications [14, 25]. Six other studies did report on complications, but without reporting the treatment group to which the complications belong or without reporting exact numbers [18–24, 26–29, 34].
Discussion
For postoperative care of ankle fractures, a change in the study design and presented outcome variables was observed over the last three decades. This study demonstrated an increase in the published number of cohort series on the treatment of ankle fractures, with increasing number of patients included. According to the Cochrane tool, the reported quality of RCTs has improved in the last three decades whereas the reported quality of observational studies has remained unchanged. However, when quality was evaluated with the MINORS criteria, equal improvement was observed for both RCTs and observational studies. Therefore, evaluation of quality is under influence of the quality assessment tool used. In addition, a gradual shift from physician measured to patient-reported outcome variables was observed.
There is a growing debate on the need of RCTs to evaluate the effectiveness of surgical procedures [35]. Results of this study indicate a relative decrease in the number of published RCTs over the past three decades in this field of research. Literature shows that one in five surgical RCTs is discontinued early, and one in three completed trials remains unpublished [35]. For (trauma) surgical research, such a trend might be explained by several factors.
First, including patients in a surgical RCT has become more and more challenging [36]. Both patient and surgeon have treatment preferences and surgeons may be better skilled in one of the study procedures. Ideally, randomisation distributes any unknown factors, thereby eliminating unforeseen confounders. However, patients may refuse or prefer the “new” technique and, therefore, do not wish to receive random treatment [37]. In addition, blinding of both patient and surgeon is frequently impossible and/or undesired by both patient and surgeon, which also neutralizes one of the advantages of an RCT over a cohort series.
Second, in the last decade the bureaucratic burden of ethical committee approval and monitoring has increased substantially, especially for conducting RCTs. This is accompanied by an increased financial demand. These procedures were developed to increase patient consent and safety, but sometimes might seem undue. For research projects in a field with limited financial resources, it is therefore increasingly difficult to meet the intensified quality demands [36]. Only a limited amount of funding went to surgical research in spite of the relatively large contribution of surgery to effective treatment [38]. These practical obstacles might have caused a substantial decrease in the quantity of RCTs in surgical research compared to other fields of medicine [4, 39].
However, as the goal of research remains to improve the quality of care for patients with an ankle fracture, alternatives are sought. This might reflect the subsequent increase in the number of published cohort series. An alternative explanation might be that due to the eagerness to produce papers in the highly competitive scientific world, where the number of publications counts heavily, there may be a bias towards writing more relatively inexpensive articles that describe cohort and retrospective studies. However, this was not demonstrated by the absolute number of published papers over the last three decades in this study, nor by a decrease in quality.
Reported quality of RCTs improved over time. This might be explained by several quality assessment tools that became available for RCTs in the 90s. The introduction of these tools provided guidance for constructing well-designed studies, with more standardised reporting and thereby improved reported research quality. Reporting of studies might have improved by reporting standard such as the CONSORT [2]. In the current study, we used the Cochrane risk of bias assessment tool to assess the quality of the studies included. This tool was primarily designed for RCTs, but is also recommended for quality assessment of observational studies [40, 41]. In addition, we used the MINORS criteria [8]. This tool is primarily designed for non-randomized studies. A combination of both tools can provide the overall quality of studies. Due to the design of the Cochrane risk of bias assessment tool, the design of a study is decisive for the results on a domain of the tool (e.g., a randomised trial has the potential to score positive on all domains contrary to observational studies). Therefore, observational studies will inevitably have a higher risk on bias in the Cochrane risk of bias assessment tool. In addition, we used the MINORS criteria [8]. This tool is especially designed for non-randomized studies. Other tools for observational studies were designed more recently, which are not frequently used [42]. It should always be kept in mind that evaluation of quality is under influence of the quality assessment tool used. Similarly to RCTs, the MINORS criteria might provide guidance for constructing well-designed observational studies and further improve observational research quality. Perhaps both RCTs and cohort series are needed to provide a complete image of a disease or treatment modality. RCTs are designed to compare specific items head to head, while large cohort series are more capable of exposing rare complications or the effect of a whole package [43].
A convincing finding of the present study is the increased use of patient-relevant outcomes. The next step would perhaps consist of consensus recommendations to define standard outcome measures for future ankle fracture studies. If all studies applied the same set of outcome measures at the same timepoints, this would make it much easier to compare and to use study results. In rheumatology, such an initiative (called OMERACT) was highly successful [44].
In the 80s, primary outcomes were focused on radiological results and objective variables. The attending physician or investigator could measure these objective variables. In the twentieth century, outcomes increasingly focused on the implications for the patient and were reported as primary end-point of several studies. In the last decade, over half of the included studies mentioned return to work as outcome measure which is, in our opinion, a legitimate patient-reported end-point.
The sample (n = 25 studies) is quite small. It is still possible that some of the observed results are attributable to the play of chance. This should be borne in mind when interpreting the results.
As ankle fractures are represented in the top five of most prevalent fractures and have an important socio-economic impact on both patient and society, the observations made in this study might be applicable to broader areas of (trauma) surgery research [45]. However, results of this study might not be applicable to other non-surgical specialties. This study can be seen as a case report and should be tested in further extent and on a larger scale in trauma surgical studies.
Results of this study indicate a relative decrease in the number of published RCTs, which might be a reflection of the practical difficulty to conduct RCTs in trauma surgery due to surgeon, patient and system-based factors. Furthermore, the desire for more patients orientated outcome variables is reflected in the changes seen in the last decades.
Conclusion
For postoperative care of ankle fractures, a change in study design and presented outcome variables was observed over the last three decades. RCTs were performed less frequently. According to the Cochrane tool, the reported quality of RCTs has improved whereas the reported quality of observational studies has remained unchanged. When quality was evaluated with the MINORS criteria, equal improvement was observed for both RCTs and observational studies. This shows that evaluation of quality is influenced by the quality assessment tool. In addition, a gradual shift from physician measured to patient-reported outcome variables was observed. However, it should be borne in mind that these findings are based on a small sample (n = 25).
References
Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ. 1996;312:1215–8.
Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285:1992–5.
Lonjon G, Boutron I, Trinquart L, et al. Comparison of treatment effect estimates from prospective nonrandomized studies with propensity score analysis and randomized controlled trials of surgical procedures. Ann Surg. 2014;259:18–25.
Ahmed AU, van der Sluis PC, Issa Y, et al. Trends in worldwide volume and methodological quality of surgical randomized controlled trials. Ann Surg. 2013;258:199–207.
Fogel GR, Morrey BF. Delayed open reduction and fixation of ankle fractures. Clin Orthop Relat Res. 1987;215:187–95.
Smeeing DP, Houwert RM, Briet JP, et al. Weight-bearing and mobilization in the postoperative care of ankle fractures: a systematic review and meta-analysis of randomized controlled trials and cohort studies. PLoS One. 2015;10:e0118320.
Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological index for non-randomized studies (minors): development and validation of a new instrument. ANZ J Surg. 2003;73:712–6.
Olerud C, Molander H. A scoring scale for symptom evaluation after ankle fracture. Arch Orthop Trauma Surg. 1984;103:190–4.
Gul A, Batra S, Mehmood S, Gillham N. Immediate unprotected weight-bearing of operatively treated ankle fractures. Acta Orthop Belg. 2007;73:360–5.
Honigmann P, Goldhahn S, Rosenkranz J, Audige L, Geissmann D, Babst R. After treatment of malleolar fractures following ORIF—functional compared to protected functional in a vacuum-stabilized orthesis: a randomized controlled trial. Arch Orthop Trauma Surg. 2007;127:195–203. doi: 10.1007/s00402-006-0255-x
Vioreanu M, Dudeney S, Hurson B, Kelly E, O'Rourke K, et al. Early mobilization in a removable cast compared with immobilization in a cast after operative treatment of ankle fractures: a prospective randomized study. Foot Ankle Int. 2007;28:13–9. doi:10.3113/FAI.2007.0003.
Simanski CJ, Maegele MG, Lefering R, Lehnen DM, Kawel N, et al. Functional treatment and early weightbearing after an ankle fracture: a prospective study. J Orthop Trauma. 2006;20:108–14. doi:10.1097/01.bot.0000197701.96954.8c.
Siddique A, Prasad C, Ö’Connor D. Early Active Mobilization versus Cast Immobilization in operatively treated ankle fractures. Eur J Trauma. 2005;31:398–400.
Lehtonen H, Jarvinen TL, Honkonen S, Nyman M, Vihtonen K, et al. Use of a cast compared with a functional ankle brace after operative treatment of an ankle fracture. A prospective, randomized study. J Bone Joint Surg Am. 2003;85-A: 205–211.
Egol KA, Dolan R, Koval KJ. Functional outcome of surgery for fractures of the ankle. A prospective, randomised comparison of management in a cast or a functional brace. J Bone Joint Surg Br. 2000;82:246–9.
Harager K, Hviid K, Jensen CM, Schantz K. Successful immediate weight-bearing of internal fixated ankle fractures in a general population. J Orthop Sci. 2000;5:552–4.
Dogra AS, Rangan A. Early mobilisation versus immobilisation of surgically treated ankle fractures. Prospective randomised control trial. Injury. 1999;30:417–9.
Laarhoven CJ, Meeuwis JD, van der Werken C. Postoperative treatment of internally fixed ankle fractures: a prospective randomised study. J Bone Jt Surg Br. 1996;78:395–9.
Richter J, Langer C, Hahn MP, Josten C, Muhr G. Ist die funktionell konservative Behandlung stabiler Aussenknöchelfrakturen gerechtfertigt? Chirurg. 1996;67:1255–60.
Tropp H, Norlin R. Ankle performance after ankle fracture: a randomized study of early mobilization. Foot Ankle Int. 1995;16:79–83.
DiStasio AJ, Jaggears FR, DePasquale LV, Frassica FJ, Turen CH. Protected early motion versus cast immobilization in postoperative management of ankle fractures. Contemp Orthop. 1994;29:273–7.
Hedstrom M, Ahl T, Dalen N. Early postoperative ankle exercise. A study of postoperative lateral malleolar fractures. Clin Orthop Relat Res. 1994;93–196.
Ahl T, Dalen N, Lundberg A, Bylund C. Early mobilization of operated on ankle fractures. Prospective, controlled study of 40 bimalleolar cases. Acta Orthop Scand. 1993;64:95–9.
Godsiff SP, Trakru S, Kefer G, Maniar RN, Flanagan JP, Tuite JD. A comparative study of early motion and immediate plaster splintage after internal fixation of unstable fractures of the ankle. Injury. 1993;24:529–30.
Cimino W, Ichtertz D, Slabaugh P. Early mobilization of ankle fractures after open reduction and internal fixation. Clin Orthop Relat Res. 1991;152–156.
Davies S. Effects of Continuous Passive Movement and Plaster of Paris after Internal Fixation of Ankle Fractures. Physiotherapy. 1991;77:516–20.
Wetzler M, Whitelaw G, Lee P. The post-operative management of ankle fractures with pneumatic braces versus short leg cast. Orthop Trans. 1991;15:719.
Finsen V, Saetermo R, Kibsgaard L, et al. Early postoperative weight-bearing and muscle activity in patients who have a fracture of the ankle. J Bone Jt Surg Am. 1989;71:23–7.
Ahl T, Dalen N, Selvik G. Mobilization after operation of ankle fractures. Good results of early motion and weight bearing. Acta Orthop Scand. 1988;59:302–6.
Ahl T, Dalen N, Holmberg S, Selvik G. Early weight bearing of displaced ankle fractures. Acta Orthop Scand. 1987;58:535–8.
Ahl T, Dalen N, Holmberg S, Selvik G. Early weight bearing of malleolar fractures. Acta Orthop Scand. 1986;57:526–9.
Søndenaa K, Hoigaard U, Smith D, Alho A. Immobilization of operated ankle fractures. Acta Orthop Scand. 1986;57:59–61.
Lund-Kristensen J, Greiff J, Riegels-Nielsen P. Malleolar fractures treated with rigid internal fixation and immediate mobilization. Injury. 1981;13:191–5.
Chapman SJ, Shelton B, Mahmood H, Fitzgerald JE, Harrison EM, Bhangu A. Discontinuation and non-publication of surgical randomised controlled trials: observational study. BMJ. 2014;349:g6870.
McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D. Randomised trials in surgery: problems and possible solutions. BMJ. 2002;324:1448–51.
Jacobs WC, Kruyt MC, Verbout AJ, Oner FC. Spine surgery research: on and beyond current strategies. Spine J. 2012;12:706–13.
McCulloch P. How to improve surgical research. BMJ. 2011;343:d4121.
Mundi R, Chaudhry H, Mundi S, Godin K, Bhandari M. Design and execution of clinical trials in orthopaedic surgery. Bone Jt Res. 2014;3:161–8.
Lin DM, Lin ES, Tran MH. Efficacy and safety of erythropoietin and intravenous iron in perioperative blood management: a systematic review. Transfus Med Rev. 2013;27:221–34.
Reeves BC, Deeks JJ, Higgins JP, Wells GA. Chapter 13: Including non-randomized studies. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). The Cochrane collaboration. Chichester: Wiley; 2011.
Zeng X, Zhang Y, Kwong JS, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med. 2015;8:2–10.
Herberts P, Malchau H. Long-term registration has improved the quality of hip replacement: a review of the Swedish THR Register comparing 160,000 cases. Acta Orthop Scand. 2000;71:111–21.
Fried BJ, Boers M, Baker PR. A method for achieving consensus on rheumatoid arthritis outcome measures: the OMERACT conference process. J Rheumatol. 1993;20:548–51.
Court-Brown CM, Caesar B. Epidemiology of adult fractures: a review. Injury. 2006;37:691–7.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
There are no potential conflicts of interests. The research does not directly include patients and, therefore, informed consent was not applicable.
Conflict of interest
Diederik Smeeing, Roderick Houwert, Moyo Kruyt, Olivier van der Meijden, and Falco Hietbrink have no conflict of interest.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Smeeing, D.P.J., Houwert, R.M., Kruyt, M.C. et al. Clinical research on postoperative trauma care: has the position of observational studies changed?. Eur J Trauma Emerg Surg 43, 43–51 (2017). https://doi.org/10.1007/s00068-016-0720-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00068-016-0720-3