A systematic review of the quality of reporting of interventions in the surgical treatment of Crohn’s anal fistula: an assessment using the TIDiER and Blencowe frameworks

Background Crohn’s anal fistula is a challenging condition, and may require multiple surgical procedures. To replicate successful procedures, these must be adequately reported in the literature. The aim of this study was to review the quality of reporting of components of surgical interventions for Crohn’s anal fistula. Methods A systematic review was conducted. It was registered with PROSPERO (CRD:42019135157). The Medline and EMBASE databases were searched for studies reporting interventions intended to close fistula in patients with Crohn’s disease, published between 1999 and August 2019. Abstracts and full texts were screened for inclusion by two reviewers. Dual extraction of data was performed to compare reporting to the TIDiER and Blencowe frameworks for reporting of interventions. Results Initial searches identified 207 unique studies; 38 full texts were screened for inclusion and 33 were included. The most common study design was retrospective cohort (17/33), and the most frequently reported interventions were anal fistula plug (n = 8) and fibrin glue (n = 6). No studies showed coverage of all domains of TIDieR. Reporting was poor among domains related to who provided an intervention, where it was provided, and how it was tailored. Reporting of domains in the Blencowe framework was poor; the majority of studies did not report the component steps of procedures or efforts to standardise them. Conclusions This study demonstrates that reporting on technical aspects of interventions for Crohn’s anal fistula is poor. Surgeons should aim to improve reporting to allow accurate reproduction of techniques both in clinical practice and in clinical trials. Supplementary Information The online version of this article (10.1007/s10151-020-02359-7) contains supplementary material, which is available to authorized users.


Introduction
Crohn's disease is a chronic inflammatory condition, and can have effects anywhere within the gastrointestinal tract. It is typically treated using a range of drugs that have effects on the immune system [1]. However, some presentations of Crohn's disease require significant elements of management by a surgeon. Prevalent among these is Crohn's anal fistula. This is a condition with considerable morbidity, and will require repeated surgical interventions as long-term remission is achieved in as few as 30% of patients [2].
There are a range of potential surgical techniques currently in use [3]. However, there is no clear front running intervention to achieve the 'Holy Grail' of fistula closure [4]. There are three main components within a study which assesses a surgical intervention; the population who were studied, the intervention or treatment(s) compared, and the outcomes reported. A mismatch between studies in any of these components has the potential to introduce heterogeneity into an analysis, potentially leading researchers to an incorrect conclusion. A particular perceived challenge to replication of research findings in practice is the fidelity with which one is able to reproduce interventions from a study [5].
One method to address limited reporting on the conduct of interventions is to use a reporting checklist. There are two such checklists available at present. The first of these is the Template for Intervention Description and Replication (TIDieR) checklist [6]. This is a 12-item checklist, which seeks details of interventions including mode and timing of delivery. TIDieR is a more generic framework, and is easily adaptable to medical and more complex interventions beyond surgery [7,8]. The Blencowe framework for surgical interventions is broadly broken into three domains: intervention description, standardisation of description, and monitoring the fidelity of the intervention [5]. As a tool designed specifically for surgical interventions, this identifies domains that might not be addressed through TIDieR. A summary of these tools is presented in Table 1.
The aim of this study was to provide evidence on the quality of surgical reporting for the treatment of fistula formation in patients with Crohn's disease using the TIDieR and Blencowe frameworks.

Overview and registration
This systematic review was conducted with reference to the Cochrane Handbook [9], and is reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses: (PRISMA) guidelines [9,10]. It was prospectively registered on the PROSPERO database (CRD:42019135157).

Inclusion and exclusion criteria
Studies reporting the use of surgical procedures to achieve fistula closure in Crohn's anal fistula were eligible for inclusion. Surgical procedures were defined as those performed by surgeons in an operating theatre. Eligible study designs included case series, cohort studies, and randomised-controlled trials. Studies had to be published after 1999, as this approximates the beginning of use of anti-TNF drugs in this setting [11]. Studies published until August 2019 were eligible for inclusion. This decision was made as anti-TNF drugs have moderate effects on remission and maintenance of fistulating disease [1]. They can conceivably change the surgical Deviation from steps decision-making in this setting based on likelihood of recurrence. No restrictions were applied based on reported outcomes, and studies had to be published in English. Studies reporting on procedures for cryptoglandular or traumatic fistulae were not eligible for inclusion, unless they reported approaches to Crohn's and other aetiologies separately. As a seton might be considered to be a drainage or temporising procedure rather than one aimed at closure, it was not eligible for inclusion in this study.

Search strategy and information sources
Scoping searches of Google Scholar and the National Center for Biotechnology Information were undertaken to identify relevant Medical Subject Heading (MeSH) terms. These were used to develop a search strategy which was deployed into EMBASE and MEDLINE databases via the OVID interface. Search strategies are presented in Online Appendix A. A secondary hand search of references of included papers was also performed.

Selection of studies
Candidate abstracts were assessed for inclusion against the eligibility criteria by two reviewers (ST and ML). Full manuscripts of eligible abstracts were retrieved and screened again by a single reviewer, with queries resolved by a second reviewer.

Data extraction
Data were extracted from the final selection of studies into a pre-designed proforma using Microsoft Excel (Microsoft, Redmond, VA, USA). Data extraction retrieved study descriptors, participant descriptors, and nature of outcome used. Intervention name was taken directly from the included study. Where a global name or label was not provided in a study, the intervention was reviewed by the research team to classify it.
The following items were assessed in the TIDieR framework, with reference to the source document [6] for definitions of each item: name of surgical intervention, why it was carried out, what materials were used for the intervention, what procedures were carried out, who provided each category of the intervention, how was the intervention carried out, when and how many times the intervention was performed, tailoring of the intervention, how the intervention was modified during the study, how well was the intervention planned, and how was adherence to the plan assessed?.
The following domains of the Blencowe framework were investigated, with reference to the source document [5] for guidance on their use: overall technical purpose of the intervention, identification of the intervention components, identification of individual steps of the intervention, types of standardisation, conditions relating to standardisation, flexibility of standardisation, and monitoring of the fidelity of the intervention. Data extraction was performed independently by two reviewers (ST and EC).

Risk of bias
Risk of bias was assessed using either the Cochrane Risk of Bias (ROB) tool [12] or the Risk of Bias In Non-randomised Studies of Interventions (ROBINS-I) tool [13]. Assessment was performed by two independent reviewers (ST and EC).

Synthesis of results
A descriptive synthesis of concordance with the two reporting frameworks was undertaken. A descriptive comparison of coverage of domains within each framework was made. Correlation between reporting coverage (total number of domains covered) in Blencowe and TIDieR frameworks was carried out using Spearman's correlation. Significance was set at 0.05 a priori. Analyses were conducted using R Studio software (RStudio team, Boston, MA) using Tidyverse [14].

Search results
Searches of databases identified 205 results, and four further studies were identified through hand searches of references. After removal of duplicates, this left 207 studies to be screened. Of these, 169 were excluded and full texts were retrieved for 38 studies. Of these 38, 5 were excluded at fulltext review (no surgical intervention performed n = 3, inappropriate study design n = 1, and non-Crohn's fistula n = 1). This left 33 studies for inclusion in the review. This is summarised in the PRISMA flowchart ( Fig. 1).

Risk of bias
Non-randomised studies: The assessment of bias using this tool is presented in Supplementary Table S1a. The 30 non-randomised studies demonstrated the issues including risks of bias due to confounding, selection of patients, outcome measurements, and reporting of results. Deviations from intervention were also not assessed. Bias in the 3 randomised trials was generally considered to be low, although there were some concerns related to blinding of those administering the intervention. Summaries of bias assessment are presented in Online Appendix B.

TIDieR framework
Assessment using the TIDieR checklist found that all studies reported intervention name (item 1), rationale (item 2), how the surgery was carried out (item 6), and when and how much (item 8). Item 3, the materials that were used in the intervention, showed higher levels of reporting with 31/33 (93.9%) of papers including a list of materials used. Though a large proportion of studies listed the items that were used, this was to a varying standard. An example is seen is van Koeperen et al. [37] and Davies et al. [38], which reported only general or locoregional anesthetic, and barium as items. Most other studies reported an extensive list of items used. The majority of studies also reported the procedure that was undertaken on the participants [31/33 (93.9%)]. The standard to which these were reported varied however. Some papers included the description of their procedure in other locations such as previous papers as in Kaminski et al. 2017 [27]. Other studies provided significantly greater detail within the paper such as in Mennigen et al. 2015 [31] and Wilhelm et al. 2017 [19].
Only 20/33 (60.6%) of the included manuscripts provided any description of who carried out the different stages of the all. Examples extracted include that seen in Papaconstantinou et al. [28] where the intervention used depended on the fistula anatomy among other characteristics such as presence of proctitis, providing a tailored approach. Item 10 is modifications of the intervention, which typically occurs with learning. Descriptions for this item were reported in 5/33 (15.1%). Whilst this was poorly reported, some studies reported this well. For example, Hyman et al. evidenced a modification in detail [26]. In this study, bowel confinement with opiates was omitted towards the end, having been present in the early stages. Item 11, strategies to improve fidelity/adherence, was reported in 6/33 (18.2%) studies. This was very poorly reported as the few studies that evidenced the inclusion of a plan were not detailed. An example of this is in Owen et al. [33], where it is stated that the 'Cook Surgisis AFP was inserted according to the Cook guidelines', but does not elaborate further. No studies reported the actual fidelity of intervention delivery (item 12). A summary of reporting within this framework is presented in Table 3.  [21]. Details on the performance of the video-assisted component (cautery settings, duration, etc.) were not presented. This pattern of incomplete reporting of standardisation was seen across studies. The most poorly reported components were the items that were included in Sect. 3 of the Blencowe framework, fidelity of the intervention. The levels of fidelity were poorly reported; deviation from the intervention (3.1) 2/33 (6.1%), deviation from the components reported (3.2) in 6/33 (18.2%), and deviation from the steps (3.3) in 0/33. Any form of deviation from the individual steps of the intervention was not reported in any of the studies. Nelson et al. were the only authors to report on both 3.1 and 3.2, noting that they abandoned the use of the combined surgical method and deviated from the intended intervention [43]. They also deviated from intended components by adopting mechanical bowel cleansing increasingly towards the later stages of the study. A summary of reporting is presented in Table 4.

Discussion
This study has assessed the reporting of the technical conduct of several surgical interventions aimed at definitive fistula treatment in Crohn's disease. It demonstrates a wide variation in the quality of reporting of interventions, with no intervention achieving 100% coverage in any study.
One of the more notable areas where information was missing was across the domain of tailoring and standardisation. There is potential for tailoring to occur in most scenarios; this could be the selection of suture size or material, the need to modify an incision due to sphincter proximity, or how to address any residual sepsis. There is also the potentially concerning lack of detail on the exact nature of the procedure received, i.e., fidelity. It is not clear if all participants received the intervention, they were expected to have, or whether other procedures were involved. It is easy to see how this could introduce bias in results. The TIDieR framework also points out the need to define who is delivering a procedure. In a surgical context, this is important in terms of learning curves, i.e., experience leads to improved outcomes [48]. Data from UK practice suggest that interventions in Crohn's fistula are typically delivered by consultant surgeons [3], and whilst this might be inferred in these reported studies, it is not explicit.
This review shows that coverage of higher level domains (name of procedure and broad aim of procedure) is frequently performed. More specific details such as fidelity and tailoring are poorly reported. It has also shown that there is variation in the quality of reporting, where retrospective studies report fewer operative details than prospective studies. This is not surprising, as the retrospective study dataset may be lacking contemporaneous detail. The finding of no difference in numbers reported according to other study designs might reflect low numbers of RCTs in this review, leading to false-negative results. We have attempted to identify comparable studies to the one presented here, but searches of citations of the original TIDieR and Blencowe documents revealed no similar surgically based work.
It is widely accepted that the rates of success of surgical interventions in Crohn's anal fistula can be disappointing. It is also notable that the rates of healing reported in surgical studies can vary widely. For example, reported ranges are from 15 to 86% for fistula plug and 29 to 79% for stem cell treatment [4]. The stem cell trial is worthy of specific comment [16]. Interventions were explained in detail with a robust attempt at standardisation. The addition of a standardised control arm is widely attributed to the relatively high efficacy of this arm compared with other trials. Broadly speaking, three main factors might be driving this variation. Either, there are patients with different disease characteristics going into studies [49], or there is a lack of consistency in the reporting of interventions, or there is variation in the measurement of outcomes [50]. The variation in quality of intervention reporting between a study such as Panes et al. [16] vs others with poorer reporting provides some insight into how limited reporting of interventions 1 3 might be influencing the relative effectiveness of interventions in this study. The challenges related to reporting quality of interventions is not limited to this setting, and have been seen across surgical and related specialties [8,51,52]. It is recognised that much of the surgical literature is made up of cohort studies. Where these are retrospective in nature, and particularly where they are large studies, it is likely to be difficult to extract all this information from historic records. If it is possible to retrieve relevant data, journal reporting restrictions (e.g., article length) might make it difficult to report all variations. It is, however, important to ensure coverage of key domains, so that 'active' ingredients of procedures can be appropriately replicated in wider practice. Some clinicians and researchers are averse to the idea of standardisation of procedures. There is debate about the level at which standardisation should happen [53], particularly as we move towards personalised or precision approaches to conditions [54]. Reporting of any and all deviations from treatment plans, no matter how minor, might lead to overload of potentially irrelevant information. Despite these challenges, accurate reporting is essential to allow the community to accurately determine the effectiveness (or not) of surgical treatments for this debilitating condition. The study does have several methodological strengths. The authors are reassured about the robustness of the screening process, and have compared the identified studies to other reviews and have not identified any key missing references. It was prospectively registered, used a robust search strategy, dual extraction, and bias assessment, and used two tools to assess quality of reporting. This study is not without limitations. Regarding the assessment tools themselves, TIDieR is designed to be used across multiple intervention types (medical, surgical, physiotherapy, etc.) [6]. As a result, it has broad domains with quite broad descriptors. This can lead to challenges in its use in settings such as specific surgical procedure and subsequent loss of clarity in reporting some aspects. The Blencowe framework was developed with the intention to use it in randomised trials [5]. The majority of surgical studies used here were cohorts and often retrospective in nature. Extracting the relevant items for all of the domains in this framework is challenging, and may be borderline impossible in some situations. Therefore, it may be more appropriate to limit the use of this tool to prospective cohort studies. The authors did consider the use of the CONSORT-NPT extension as an additional assessment framework [55]. This was ultimately dismissed as it was felt that the Blencowe and TIDieR tools were more relevant to the multiple non-randomised trial-based designs.
This study highlights some important findings for researchers. If colorectal surgeons wish to offer the best surgical interventions for their patients, the quality of reporting in studies should also improve. Efforts have already been made to standardise outcome reporting for this condition [56]. Whilst it is not fair to judge historic papers against standards that were developed after their completion, it is fair to apply these standards to the future. Surgeons should focus on capturing and reporting data relevant to the reporting frameworks to enable replication in the wider world, should their procedure suggest benefit. The comparison of domain coverage across the two frameworks studied leads us to recommend adherence to the Blencowe framework, as adherence to this is associated with good coverage of the TIDieR framework. This is further supported by the original intent of these tools-TIDieR can be applied across a range of settings including medical therapy and complex interventions, whereas the Blencowe framework is designed specifically for surgical procedures. Consequently, adherence to domains of the latter framework might lead to more applicable reporting for the surgical community. The authors also encourage journal editors to consider adherence to reporting frameworks when publishing future procedure-based studies. The study should also act as a caution to clinicians to consider reporting of techniques critically prior to introduction into practice.

Conclusions
This study shows that the quality of reporting of surgical procedures to treat Crohn's anal fistula is limited. Given the significant challenges associated with the treatment of this condition, surgeons must address this reporting to improve study quality, and hopefully outcomes.
Funding This study received no funding.

Compliance with ethical standards
Conflict of interest There are no conflicts of interest to report for any authors.
Ethical approval This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent No primary patient data was used in this study, therefore consent was not required.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.