Why critical appraisal is so important these days
- 985 Downloads
The paper by Donk et al. serves two purposes, none of which are intended by the authors themselves. It teaches us that (1) critical appraisal of studies is extremely important and (2) journals accepting randomized controlled trials should make a CONSORT checklist mandatory and refute studies like this initially without further review. The reason to publish this manuscript lies solely in the above, and by far not in its content, which would certainly not merit publication. This is especially true because this study here comes in the disguise of good clinical science, i.e., having the magic label of an RCT, luring the reader—or even the reviewer—into the belief of having reliable and valid data at his/her hands. In my opinion, the opposite is true here.
Before reading my comments further, I would like to encourage the readers to go to the citations [1, 2, 3, 4, 5], elaborating on what one should know about the CONSORT guidelines. I will only cite the first paragraph of reference , supporting my argument above. It states: "Overwhelming evidence shows the quality of reporting randomized controlled trials (RCTs) is not optimal. Without transparent reporting, readers cannot judge the reliability and validity of trial findings nor extract information for systematic reviews. Recent methodological analyses indicate that inadequate reporting and design are associated with biased estimates of treatment effects…" .
I would secondly encourage the reader to peruse reference , detailing the original design of the PROCON study, on which data of the current article are based and that should be read again after having digested all the information in the above references.
Let me start by detailing the most important discrepancies between the trial protocol as described in article  or the website (https://www.isrctn.com/ISRCTN41681847?q=ISRCTN41681847&filters=&sort=&offset=1&totalResults=1&page=1&pageSize=10&searchType=basic-search) and the data presented here, which are not appropriately acknowledged and revealed by the authors in the current publication. By this alone, the element of transparent reporting has been violated.
The PROCON trial was designed as a multicenter RCT comparing the three treatment modalities according to a primary clinical outcome parameter (done by an independent neurologist!) in sample sizes of 90 each with fixed clinical and radiological follow-up intervals at 6 weeks, 3 and 12 months and yearly thereafter until 60 months. None of this has happened, but instead in this single-center RCT one of several secondary outcomes (overall cervical sagittal alignment) was selected, which eventually did not show a statistical difference in an underpowered comparison of small groups, which were not examined according to protocol and had a complete lack of clinical data (see Fig. 1) to make a statement of equivalence.
The reasons given for these deviations are not acceptable. That patients prefer not to come for clinical follow-ups is a very poor reason to change the protocol, given the fact the clinical outcome is all that counts. Moreover, patients did not even adhere to the protocol for analysis of the surrogate parameters, i.e., radiographs. Although nicely hidden, it seems that only 5, 12 and 7 patients per group adhered to the radiological follow-up protocol. Furthermore, at the follow-up dates, reconstructed from the mean follow-up times, it seems (Table 2) that beyond approximately 1 year, overall only 83 and 27 data sets were available, without detailing their distribution among the three groups.
Having said this, I admit that the most irritating factor for me in this communication lies in the argument given for ending recruitment prematurely. In paragraph one of the methods section, the authors state, "however, inclusion was prematurely ended after publication of a metanalysis comparing ACDF and ACD, since we could not justify continuing the trial according to standards of good clinical practice."
The publication cited for this is a meta-analysis by Bartels published in the year 2010 in Neurosurgery , in which the available RCTs comparing cervical prosthesis with fusion—all of which were far superior to the one here by design and execution—were meticulously scrutinized and harshly criticized for flaws much less important than the ones in his trial. The meta-analysis concludes with the sentence that because of lack of superiority, disk arthroplasty should not be used in daily practice for reasons of cost effectiveness. Even if this expert opinion is considered true and discontinuitation in daily practice legitimate, it is quite clear that the assumed lack of superiority is the number one reason to test this under controlled conditions and thus to continue recruitment of an RCT like the PROCON.
I consider this very hypocritical, not to mention the fact that the same standards are not held up for the quality of one’s own research as for those of others. This is what I consider a major bias.
Apart from all this and the discussion about cage versus prosthesis in the cervical spine, it is quite clear that the most destructive procedure one can perform in a relatively young patient with a single-level soft disc is a discectomy without reconstruction of the disc space (not "without fusion," because it will fuse even without interposing material!). Even here, the local kyphosis can be seen, and the fact that the overall cervical lordosis is not significantly different (if at all true, keeping in mind the shortcomings) is simply not an argument in favor of this procedure. If clinical data were available, the authors would have shown what others have, i.e., a prolonged period of neck disability and pain in these patients.
This alone justifies the insertion of at least a stand-alone cage, which is very reasonably priced.
Overall differences in procedural outcome in the cervical spine are small and very hard to prove. To take this and/or the financial issue alone to completely refute every innovation is not wise. Clinical progress especially in surgical fields comes in tiny steps and can often only be appreciated if looking at a longer time scale. Refuting every single step in between would stall this process completely. We would then have still open knee surgeries, no microscopes for spinal and cranial procedures, no navigation or monitoring and probably keep our patients in the hospital for 2 weeks after lumbar disc surgery, etc.
In summary, I very much appreciate the author’s critical appraisal of new techniques and technologies and the respective clinical trials, but would like to see this attitude expanded to their own work.
- 1.Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG (2010) CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 340:c869. https://doi.org/10.1136/bmj.c869 CrossRefPubMedPubMedCentralGoogle Scholar
- 6.Bartels RH, Donk R, van der Wilt GJ, Grotenhuis JA, Venderink D (2006) Design of the PROCON trial: a prospective, randomized multi-center study comparing cervical anterior discectomy without fusion, with fusion or with arthroplasty. BMC Musculoskelet Disord 7:85CrossRefPubMedPubMedCentralGoogle Scholar
- 7.Bartels RH, Donk R, Verbeek AL (2010) No justification for cervical disk prostheses in clinical practice: a meta-analysis of randomized controlled trials. Neurosurgery 66(6):1153–1160; discussion 1160. https://doi.org/10.1227/01.NEU.0000369189.09182.5F CrossRefPubMedGoogle Scholar