Skip to main content
SpringerLink
Log in

Risk of bias for randomized controlled trials in Journal of Clinical Monitoring and Computing

  • Original Research
  • Published:
Journal of Clinical Monitoring and Computing Aims and scope Submit manuscript

Abstract

Purpose

Well-designed randomized controlled trials (RCTs) are considered to represent a high level of evidence and influence medical decision-making in evidence-based medicine. When biases occur in study design, processing, and reporting of RCTs, however, it is difficult to interpret results and judge the impact of interventions. Accordingly, we evaluate the quality of RCT reporting published in the Journal of Clinical Monitoring and Computing (JCMC) using three assessment tools.

Methods

Reporting quality of RCTs published in the JCMC was evaluated through December 31, 2020, using Jadad and van Tulder scales and the Cochrane Collaboration’s risk of bias tool (CCRBT). Stepwise regression analysis was performed to identify factors associated with reporting quality.

Results

Database searches confirmed 132 RCTs in 1,507 original articles. The numbers of RCTs meeting criteria for high reporting quality were 97 (73.5%) using the Jadad scale, 99 (75.0%) using the van Tulder scale, and 19 (14.4%) with the CCRBT. Jadad scores [median score (interquartile range) = 3.0 (2.0–5.0), coefficients (95% CI) = 0.08 (0.04, 0.11), p < 0.001], van Tulder scores [median score (interquartile range) = 7.0 (5.0–8.75), coefficients (95% CI) = 0.15 (0.11, 0.20), p < 0.001], and CCRBT assessment [coefficients (95% CI) = 0.04 (0.02, 0.06), p < 0.001] increased significantly with publication year. The median score (interquartile range) of the last 5 years were 4.0 (3.0–5.0) in Jadad scores, and 8.0 (6.0–9.0) in van Tulder scores. Only 33.3% and 37.1% of articles described detailed blinding and allocation methods, respectively.

Conclusions

Reporting quality increased over time, with consistently high reporting quality in recently published JCMC RCTs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet (London England). 2017;390(10092):415–23. doi:https://doi.org/10.1016/s0140-6736(16)31592-6.

    Article  Google Scholar 

  2. Uetani K, Nakayama T, Ikai H, Yonemoto N, Moher D. Quality of reports on randomized controlled trials conducted in Japan: evaluation of adherence to the CONSORT statement. Intern Med (Tokyo Japan). 2009;48(5):307–13. doi:https://doi.org/10.2169/internalmedicine.48.1358.

    Article  Google Scholar 

  3. Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Krebs Seida J, Klassen TP. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ (Clinical research ed). 2009;339:b4012. doi:https://doi.org/10.1136/bmj.b4012.

    Article  Google Scholar 

  4. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ (Clinical research ed). 2008;336(7644):601–5. doi:https://doi.org/10.1136/bmj.39465.451748.AD.

    Article  Google Scholar 

  5. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12. doi:https://doi.org/10.1001/jama.273.5.408.

    Article  CAS  Google Scholar 

  6. Estellat C, Torgerson DJ, Ravaud P. How to perform a critical analysis of a randomised controlled trial. Best Pract Res Clin Rheumatol. 2009;23(2):291–303. doi:https://doi.org/10.1016/j.berh.2009.03.003.

    Article  Google Scholar 

  7. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical research ed). 2010;340:c869. doi:https://doi.org/10.1136/bmj.c869.

    Article  Google Scholar 

  8. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet (London England). 2001;357(9263):1191–4.

    Article  CAS  Google Scholar 

  9. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, Pitkin R, Rennie D, Schulz KF, Simel D, Stroup DF. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276(8):637–9. doi:https://doi.org/10.1001/jama.276.8.637.

    Article  CAS  Google Scholar 

  10. Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A. A method for assessing the quality of a randomized control trial. Control Clin Trials. 1981;2(1):31–49. doi:https://doi.org/10.1016/0197-2456(81)90056-8.

    Article  CAS  Google Scholar 

  11. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ (1996) Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 17(1):1–12. doi:https://doi.org/10.1016/0197-2456(95)00134-4.

  12. van Tulder M, Furlan A, Bombardier C, Bouter L. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976). 2003;28(12):1290–9. doi:https://doi.org/10.1097/01.brs.0000065484.95996.af.

    Article  Google Scholar 

  13. Higgins JPT, Green S. (2011) Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011.

  14. Olivo SA, Macedo LG, Gadotti IC, Fuentes J, Stanton T, Magee DJ. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008;88(2):156–75. doi:https://doi.org/10.2522/ptj.20070147.

    Article  Google Scholar 

  15. Kim KS, Jo JK, Chung JH, Kim JH, Choi HY, Lee SW. Quality analysis of randomized controlled trials in the International Journal of Impotence Research: quality assessment and relevant clinical impact. Int J Impot Res. 2017;29(2):65–9. doi:https://doi.org/10.1038/ijir.2016.48.

    Article  CAS  Google Scholar 

  16. Hong SJ, Yoon DY, Cho YK, Yoon SJ, Moon JY, Baek S, Lim KJ. Characteristics and Quality of Radiologic Randomized Controlled Trials: A Bibliometric Analysis Between 1995 and 2014. AJR Am J Roentgenol. 2016;206(5):917–23. doi:https://doi.org/10.2214/ajr.15.15640.

    Article  Google Scholar 

  17. Lee JW, Chung JH, Jo JK, Lee SW. Analysis of randomized controlled trials in Rheumatology International from 1981 to 2012: methodological assessment. Rheumatol Int. 2014;34(9):1187–93. doi:https://doi.org/10.1007/s00296-014-2963-9.

    Article  Google Scholar 

  18. Chung JH, Kang DH, Jo JK, Lee SW. Assessing the quality of randomized controlled trials published in the Journal of Korean Medical Science from 1986 to 2011. J Korean Med Sci. 2012;27(9):973–80. doi:https://doi.org/10.3346/jkms.2012.27.9.973.

    Article  Google Scholar 

  19. Cho HJ, Chung JH, Jo JK, Kang DH, Cho JM, Yoo TK, Lee SW. Assessments of the quality of randomized controlled trials published in International Journal of Urology from 1994 to 2011. Int J urology: official J Japanese Urol Association. 2013;20(12):1212–9. doi:https://doi.org/10.1111/iju.12150.

    Article  Google Scholar 

  20. Blinding and allocation concealment. Sealed Envelope Ltd. (2019) https://www.sealedenvelope.com/randomisation/blinding/. Accessed 30 July 2019.

  21. Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet (London England). 2002;359(9306):614–8. doi:https://doi.org/10.1016/s0140-6736(02)07750-4.

    Article  Google Scholar 

  22. Lee JW, Chung JH, Jo JK, Lee SW. Assessing the quality of randomized controlled trials published in neurourology and urodynamics from 1993 to 2012. Neurourol Urodyn. 2014;33(5):472–4. doi:https://doi.org/10.1002/nau.22457.

    Article  Google Scholar 

  23. Bridoux V, Moutel G, Roman H, Kianifard B, Michot F, Herve C, Tuech JJ. Methodological and ethical quality of randomized controlled clinical trials in gastrointestinal surgery. J Gastrointest surgery: official J Soc Surg Aliment Tract. 2012;16(9):1758–67. doi:https://doi.org/10.1007/s11605-012-1952-0.

    Article  Google Scholar 

  24. Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic B. Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. BMJ (Clinical research ed). 2004;328(7430):22–4. doi:https://doi.org/10.1136/bmj.328.7430.22.

    Article  Google Scholar 

Download references

Acknowledgements

Since this manuscript is about the performance of this journal, it has been evaluated by an independent Editor from a different journal.

Funding

None.

Author information

Authors and Affiliations

  1. Department of Anesthesiology and Pain Medicine, College of Medicine, Hanyang University Hospital, 222-1, Wangsimni-ro, 04763, Seongdonggu, Seoul, Republic of Korea

    Jeong Min Sung, Ji Yoon Kim, Bo Seok Kwon & Kyu Nam Kim M.D., Ph.D.

Authors
  1. Jeong Min Sung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ji Yoon Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Seok Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kyu Nam Kim M.D., Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Study conception and design: Kyu Nam Kim. Data collection: Kyu Nam Kim, Jeong Min Sung, Ji Yoon Kim and Bo Seok Kwon. Data analysis: Kyu Nam Kim, Jeong Min Sung, Ji Yoon Kim, Bo Seok Kwon. Writing the draft of the manuscript: Kyu Nam Kim and Jeong Min Sung. Critical revision of the manuscript: Kyu Nam Kim, Jeong Min Sung, Ji Yoon Kim and Bo Seok Kwon. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kyu Nam Kim M.D., Ph.D..

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare.

Ethical approval

Not applicable.

Informed consent

Not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sung, J.M., Kim, J.Y., Kwon, B.S. et al. Risk of bias for randomized controlled trials in Journal of Clinical Monitoring and Computing. J Clin Monit Comput 37, 103–111 (2023). https://doi.org/10.1007/s10877-022-00864-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10877-022-00864-8

Keywords

Search

Navigation