Usability and sensitivity of the risk of bias assessment tool for randomized controlled trials of pharmacist interventions
- 23 Downloads
Background The Cochrane collaboration risk of bias assessment (RoB) tool is used in several fields to evaluate the methodological quality of studies. Its strengths and challenges are discussed. Objective To assess the sensitivity of the RoB tool in studies of pharmacist interventions. Setting DEPICT database was used to pool randomized controlled trials (RCTs) of complex interventions. Method A Guide for RoB Judgment in Pharmacy Services was created to help in the interpretation and judgment of bias criteria. The evaluation of bias (low, unclear, high risk) was performed by RCT. Sensitivity analyses were performed to assess the influence of different interpretations of eight elements of judgment in the RoB tool. Paired analysis and estimations of the effect size (95% confidence interval) of the criteria modifications compared to the original analyses were calculated. Main outcome measure Changes in the interpretations of judgment in the RoB tool. Results Overall, 8.3, 45.4, and 46.3% of the studies were determined to have low, unclear, and high risk of bias, respectively. High risk of bias was caused by attrition and detection domains. The number of studies classified with high risk of bias significantly increased for five of the eight interpretations, while unclear risk of bias increased for three interpretations (with a negligible effect size in all of them). Lack of blinding, loss of participants, and the use of subjective and self-reported outcomes were the main elements resulting in high risk of bias. Conclusion The RoB tool is useful for evaluating RCTs of pharmacist interventions if adapted criteria for judgment are used. Ignoring these adjustments produces a floor-effect with studies classified with high risk of bias.
KeywordsCochrane collaboration Methodology Outcome assessment Pharmacists Risk of bias RoB tool
This work was supported by Brazilian National Council of Technological and Scientific Development (CNPq), Coordination for the Improvement of Higher Education Personnel (CAPES). The funding sources had no role in the study design, data collection, data analyses, data interpretation, or writing of the report. The corresponding author had full access to all of the data in the study and was responsible for making the final decision to submit the manuscript for publication.
Conflicts of interest
All authors declare that they have no conflict of interest.
- 1.Armijo-Olivo S, Fuentes J, Ospina M, Saltaji H, Hartling L. Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: a descriptive analysis. BMC Med Res Methodol. 2013;13:116.CrossRefGoogle Scholar
- 4.Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters M, et al. Assessing the risk of bias of individual studies in systematic reviews of health care interventions. In: Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ: Rockville, 2008.Google Scholar
- 14.Armijo-Olivo S, Stiles CR, Hagen NA, Biondo PD, Cummings GG. Assessment of study quality for systematic reviews: a comparison of the Cochrane collaboration risk of bias tool and the effective public health practice project quality assessment tool: methodological research. J Eval Clin Pract. 2012;18:12–8.CrossRefGoogle Scholar
- 16.Higgins JPT, Green S. In: Cochrane Handbook for systematic reviews of interventions version 5.1.0. http://handbook-5-1.cochrane.org/. Accessed 09 Jan 2019.
- 19.Manchikanti L, Hirsch JA, Cohen SP, Heavner JE, Falco FJ, Diwan S, et al. Assessment of methodologic quality of randomized trials of interventional techniques: development of an interventional pain management specific instrument. Pain Physician. 2014;17:E263–90.Google Scholar
- 20.Jorgensen L, Paludan-Muller AS, Laursen DR, Savovic J, Boutron I, Sterne JA, et al. Evaluation of the Cochrane tool for assessing risk of bias in randomized clinical trials: overview of published comments and analysis of user practice in Cochrane and non-Cochrane reviews. Syst Rev. 2016;5:80.CrossRefGoogle Scholar
- 21.Higgins J, Sterne J, Savović J, Page M, Hróbjartsson A, Boutron I, et al. A revised tool for assessing risk of bias in randomized trials. Cochrane Database Syst Rev. 2016;10:30–1.Google Scholar
- 23.Tsuyuki RT. Designing pharmacy practice research trials. Can J Hosp Pharm. 2014;67:226–9.Google Scholar
- 29.Hoffman JI. The incorrect use of Chi square analysis for paired data. Clin Exp Immunol. 1976;24:227–9.Google Scholar
- 31.Dreier M, Borutta B, Stahmeyer J, Krauth C, Walter U. Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany. GMS Health Technol Assess. 2010;6:Doc07.Google Scholar