Skip to main content

Advertisement

SpringerLink
Log in

Report and Interpretation of Randomized Controlled Trials with Statistically Nonsignificant Results for Primary Outcomes in the Anesthesia Domain: A Systematic Review

  • Ambulatory Anesthesia (G Joshi, Section Editor)
  • Published:
Current Anesthesiology Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

Although the Declaration of Helsinki emphasizes the importance of accurate and transparent research communication, authors’ discretion in interpreting outcomes may spin their results and force a favorable outlook. This study investigated the use of “spin” in reporting randomized controlled trials (RCTs) in the anesthesia domain.

Recent Findings

We searched the PubMed database for articles reporting RCTs in the anesthesia domain published over 5 years (2014–2018). We excluded articles on cost-effectiveness, diagnostic test accuracy, and non-English language reports. The spin characteristics with reference to the location, extent, and strategy were assessed. We screened 799 articles and included 211 eligible articles. Spin reporting was found in 86 articles (40.8%), and was identified in the Results section in 75 articles, the results synthesis in the Discussion section in 52 articles, and the Conclusion section in 66 articles. The most common spin location was the Results section, and the type of spin was focused on statistically significant within-group comparisons. The most common spin strategy was language use that implied a benefit.

Summary

Spin reporting was highly prevalent in RCT reports in the anesthesia domain, and abstracts may effectively reflect the spin reporting status in articles. We scrutinized instances of spin within RCTs that failed to yield statistically significant results; however, the potential existence of spin in studies reporting statistically significant results may also exist. This warrants critical evaluation and consideration, and the application of a fragility index may be indispensable for this purpose.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability

All data to reproduce the results of this study are available at the following URL: https://github.com/bougtoir/spin_anaesthesia

References

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. Stanley K. Design of randomized controlled trials. Circulation. 2007;115:1164–9. https://doi.org/10.1161/CIRCULATIONAHA.105.594945.

    Article  PubMed  Google Scholar 

  2. Victora CG, Habicht JP, Bryce J. Evidence-based public health: moving beyond randomized trials. Am J Public Health. 2004;94:400–5. https://doi.org/10.2105/ajph.94.3.400.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Cokkinos DV, Pantos C. Myocardial protection in man–from research concept to clinical practice. Heart Fail Rev. 2007;12:345–62. https://doi.org/10.1007/s10741-007-9030-5.

    Article  CAS  PubMed  Google Scholar 

  4. Bellomo R, Bagshaw SM. Evidence-based medicine: classifying the evidence from clinical trials–the need to consider other dimensions. Crit Care. 2006;10:232. https://doi.org/10.1186/cc5045.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Dickersin K, Straus SE, Bero LA. Evidence based medicine: increasing, not dictating, choice. BMJ. 2007;334:s10. https://doi.org/10.1136/bmj.39062.639444.94.

    Article  PubMed  Google Scholar 

  6. Heneghan C, Mahtani KR, Goldacre B, Godlee F, Macdonald H, Jarvies D. Evidence based medicine manifesto for better healthcare. BMJ. 2017;357:j2973. https://doi.org/10.1136/bmj.j2973.

    Article  PubMed  Google Scholar 

  7. Horwitz RI, Hayes-Conroy A, Caricchio R, Singer BH. From evidence based medicine to medicine based evidence. Am J Med. 2017;130:1246–50. https://doi.org/10.1016/j.amjmed.2017.06.012.

    Article  PubMed  Google Scholar 

  8. Chapman G, Talbot N, McCartney D, Tippett V, Burch D. Evidence based medicine–older, but no better educated? Lancet. 2013;382:1484. https://doi.org/10.1016/S0140-6736(13)62286-2.

    Article  PubMed  Google Scholar 

  9. Greenhalgh T, Howick J, Maskrey N, Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725. https://doi.org/10.1136/bmj.g3725.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Montori VM, Guyatt GH. Progress in evidence-based medicine. JAMA. 2008;300:1814–6. https://doi.org/10.1001/jama.300.15.1814.

    Article  CAS  PubMed  Google Scholar 

  11. Haynes RB. What kind of evidence is it that evidence-based medicine advocates want health care providers and consumers to pay attention to? BMC Health Serv Res. 2002;2:3. https://doi.org/10.1186/1472-6963-2-3.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Declaration of Helsinki. https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. Accessed 7 April 2023.

  13. International clinical trials registry platform (ICTRP). https://www.who.int/clinical-trials-registry-platform. Accessed 7 April 2023.

  14. EU Clinical Trials Register. https://www.clinicaltrialsregister.eu/. Accessed 7 April 2023.

  15. Clinicaltrials.Gov. https://clinicaltrials.gov/. Accessed 7 April 2023.

  16. UMIN clinical trials registry (UMIN-CTR) https://www.umin.ac.jp/ctr/index-j.htm. Accessed 7 April 2023.

  17. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–9. https://doi.org/10.1001/jama.276.8.637.

    Article  CAS  PubMed  Google Scholar 

  18. •• Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically non-significant results for primary outcomes. JAMA. 2010;303:2058–64. https://doi.org/10.1001/jama.2010.651. Review that first mentioned the distorted interpretation of reporting in randomized controlled trials as spin.

  19. Gewandter JS, McKeown A, McDermott MP, et al. Data interpretation in analgesic clinical trials with statistically non-significant primary analyses: an ACTTION systematic review. J Pain. 2015;16:3–10. https://doi.org/10.1016/j.jpain.2014.10.003.

    Article  PubMed  Google Scholar 

  20. Latronico N, Metelli M, Turin M, Piva S, Rasulo FA, Minelli C. Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010. Intensive Care Med. 2013;39:1386–95. https://doi.org/10.1007/s00134-013-2947-3.

    Article  PubMed  Google Scholar 

  21. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–60. https://doi.org/10.1056/NEJMsa065779.

    Article  CAS  PubMed  Google Scholar 

  22. Turner EH, Cipriani A, Furukawa TA, Salanti G, de Vries YA. Selective publication of antidepressant trials and its influence on apparent efficacy: updated comparisons and meta-analyses of newer versus older trials. PLOS Med. 2022;19:e1003886. https://doi.org/10.1371/journal.pmed.1003886.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30:210–6. https://doi.org/10.1200/JCO.2011.37.0890.

    Article  PubMed  Google Scholar 

  24. Arunachalam L, Hunter IA, Killeen S. Reporting of randomized controlled trials with statistically non-significant primary outcomes published in high-impact surgical journals. Ann Surg. 2017;265:1141–5. https://doi.org/10.1097/SLA.0000000000001795.

    Article  PubMed  Google Scholar 

  25. Hemming K, Javid I, Taljaard M. A review of high impact journals found that misinterpretation of non-statistically significant results from randomized trials was common. J Clin Epidemiol. 2022;145:112–20. https://doi.org/10.1016/j.jclinepi.2022.01.014.

    Article  PubMed  Google Scholar 

  26. • Roberts WB, Cooper CM, Khattab M, et al. Evaluation of ‘spin’ in the abstracts of randomized controlled trial reports in cardiology. J Am Osteopath Assoc. 2020. https://doi.org/10.7556/jaoa.2020.133. Review regarding spin in abstracts of cardiology RCTs as clinicians often base their decisions solely on abstracts.

  27. • Kinder NC, Weaver MD, Wayant C, Vassar M. Presence of ‘spin’ in the abstracts and titles of anaesthesiology randomised controlled trials. Br J Anaesth. 2019;122:e13–4. https://doi.org/10.1016/j.bja.2018.10.023. Review based on spin in abstracts and titles for they may misrepresent RCT findings.

  28. Grolleau F, Collins GS, Smarandache A, et al. The fragility and reliability of conclusions of anesthesia and critical care randomized trials with statistically significant findings: a systematic review. Crit Care Med. 2019;47:456–62. https://doi.org/10.1097/CCM.0000000000003527.

    Article  PubMed  Google Scholar 

  29. Demarquette A, Perrault T, Alapetite T, et al. Spin and fragility in randomised controlled trials in the anaesthesia literature: a systematic review. Br J Anaesth. 2023;130:528–35. https://doi.org/10.1016/j.bja.2023.01.001.

    Article  CAS  PubMed  Google Scholar 

  30. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLOS Med. 2009;6:e1000100. https://doi.org/10.1371/journal.pmed.1000100.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. https://doi.org/10.2307/2529310.

    Article  CAS  PubMed  Google Scholar 

  32. • Wasserstein RL, Lazar NA. The ASA statement on p-values: context, process, and purpose. Am Stat. 2016;70:129–33. https://doi.org/10.1080/00031305.2016.1154108. Statement to review the attitude of biasing p < 0.05.

  33. Baker M. Statisticians issue warning over misuse of P values. Nature. 2016;531:151. https://doi.org/10.1038/nature.2016.19503.

    Article  ADS  CAS  PubMed  Google Scholar 

  34. Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias – an updated review. PLoS One. 2013;8:e66844. https://doi.org/10.1371/journal.pone.0066844.

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  35. Rennie D, Glass RM. Structuring abstracts to make them more informative. JAMA. 1991;266:116–7. https://doi.org/10.1001/jama.1991.03470010120043.

    Article  CAS  PubMed  Google Scholar 

  36. Eid T, vanSonnenberg E, Azar A, Mistry P, Eid K, Kang P. Analysis of the variability of abstract structures in medical journals. J Gen Intern Med. 2018;33:1013–4. https://doi.org/10.1007/s11606-018-4428-4.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Ibrahim AM. Seeing is believing: using visual abstracts to disseminate scientific research. Am J Gastroenterol. 2018;113:459–61. https://doi.org/10.1038/ajg.2017.268.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank Iwamuratea and Chiron for their assistance with environment management.

We also thank Onishi Kotoha and Onishi Ione for their assistance. This study would not have been possible without their assistance.

Author information

Authors and Affiliations

  1. Faculty of Data Science, Shiga University, Hikone, Japan

    Onishi Tatsuki

  2. Department of Anesthesiology and Pain Medicine, Juntendo University Shizuoka Hospital, Izunokuni, Japan

    Onishi Tatsuki

  3. Department of Anesthesiology, Kyowa Hospital, Kyoto, Japan

    Onishi Tatsuki

  4. Wellbeing Keiei LLC, Kyoto, Japan

    Onishi Tatsuki & Yoshika Onishi

Authors
  1. Onishi Tatsuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoshika Onishi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: O.T., Y.O.; Data curation: O.T., Y.O.; Writing—review & editing: O.T., Y.O.; Writing—original draft: O.T.. All authors contributed to the final draft of the manuscript.

Corresponding author

Correspondence to Onishi Tatsuki.

Ethics declarations

Competing Interests

The authors declare no competing interests.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tatsuki, O., Onishi, Y. Report and Interpretation of Randomized Controlled Trials with Statistically Nonsignificant Results for Primary Outcomes in the Anesthesia Domain: A Systematic Review. Curr Anesthesiol Rep (2024). https://doi.org/10.1007/s40140-024-00625-0

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s40140-024-00625-0

Keywords

Search

Navigation