Skip to main content

Advertisement

SpringerLink
Log in

Systematic review of sample size calculations and reporting in randomized controlled trials in ophthalmology over a 20-year period

  • Review
  • Published:
International Ophthalmology Aims and scope Submit manuscript

Abstract

Purpose

Randomized Controlled Trials (RCTs) are considered the gold standard for the practice of evidence-based medicine. The purpose of this study is to systematically assess the reporting of sample size calculations in ophthalmology RCTs in 5 leading journals over a 20-year period. Reviewing sample size calculations in ophthalmology RCTs will shed light on the methodological quality of RCTs and, by extension, on the validity of published results.

Methods

The MEDLINE database was searched to identify full reports of RCTs in the journals Ophthalmology, JAMA Ophthalmology, American Journal of Ophthalmology, Investigative Ophthalmology and Visual Science, and British Journal of Ophthalmology between January and December of the years 2000, 2010 and 2020. Screening identified 559 articles out of which 289 met the inclusion criteria for this systematic review. Data regarding sample size calculation reporting and trial characteristics was extracted for each trial by independent investigators.

Results

In 2020, 77.9% of the RCTs reported sample size calculations as compared with 37% in 2000 (p < 0.001) and 60.7% in 2010 (p = 0.012). Studies reporting all necessary parameters for sample size recalculation increased significantly from 17.2% in 2000 to 39.3% in 2010 and 43.0% in 2020 (p < 0.001). Reporting of funding was greater in 2020 (98.8%) compared with 2010 (89.3%) and 2000 (53.1%). Registration in a clinical trials database occurred more frequently in 2020 (94.2%) compared to 2000 (1.2%; p < 0.001) and 2010 (68%; p < 0.001). In 2020, 38.4% of studies reported different sample sizes in the online registry from the published article. Overall, the most studied area in 2000 was glaucoma (29.6% of RCTs), whereas in 2010 and 2020, it was retina (40.2 and 37.2% of the RCTs, respectively). The number of patients enrolled in a study and the number of eyes studied was significantly greater in 2020 compared to 2000 and 2010 (p < 0.001).

Conclusion

Sample size calculation reporting in ophthalmology RCTs has improved significantly between the years 2000 and 2020 and is comparable to other fields in medicine. However, reporting of certain parameters remains inconsistent with current publication guidelines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Calvert M, Blazeby J, Altman D, Revicki D, Moher D, Brundage M (2013) Reporting of patient-reported outcomes in randomized trials. JAMA 309(8):814. https://doi.org/10.1001/jama.2013.879

    Article  CAS  PubMed  Google Scholar 

  2. Moher D, Hopewell S, Schulz K et al (2010) CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomized trials. Br Med J 340:869. https://doi.org/10.1136/bmj.c869

    Article  Google Scholar 

  3. Chan A, Tetzlaff J, Altman D et al (2013) SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 158(3):200–207. https://doi.org/10.7326/0003-4819-158-3-201302050-00583

    Article  PubMed  PubMed Central  Google Scholar 

  4. Kane R, Wang J, Garrard J (2007) Reporting in randomized clinical trials improved after adoption of the CONSORT statement. J Clin Epidemiol 60(3):241–249. https://doi.org/10.1016/j.jclinepi.2006.06.016

    Article  PubMed  Google Scholar 

  5. Hopewell S, Dutton S, Yu L, Chan A, Douglas A (2010) The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. Br Med J. https://doi.org/10.1136/bmj.c723

    Article  Google Scholar 

  6. Abdul Latif L, Daud Amadera J, Pimentel D, Pimentel T, Fregni F (2011) Sample size calculation in physical medicine and rehabilitation: a systematic review of reporting, characteristics, and results in randomized controlled trials. Arch Phys Med Rehabil 92(2):306–315. https://doi.org/10.1016/j.apmr.2010.10.003

    Article  PubMed  Google Scholar 

  7. Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P (2009) Reporting of sample size calculation in randomised controlled trials: review. Br Med J. https://doi.org/10.1136/bmj.b1732

    Article  Google Scholar 

  8. Chan A, Asbjørn D, Haahr M, Gøtzsche P, Altman D (2004) Empirical evidence for selective reporting of outcomes in randomized trials comparison of protocols to published articles. JAMA 291(20):2457–2465. https://doi.org/10.1001/jama.291.20.2457

    Article  CAS  PubMed  Google Scholar 

  9. Bachir B, Shariat S, Zlotta A et al (2013) Demographic analysis of randomized controlled trials in bladder cancer. BJU Int 111(3):419–426. https://doi.org/10.1111/j.1464-410X.2012.11401.x

    Article  PubMed  Google Scholar 

  10. Mansouri A, Cooper B, Shin SM, Kondziolka D (2015) Randomized controlled trials and neurosurgery: the ideal fit or should alternative methodologies be considered? J Neurosurg 124(2):558–568. https://doi.org/10.3171/2014.12.JNS142465

    Article  PubMed  Google Scholar 

  11. Borrelli M, Farwana R, Andrew T et al (2019) Assessing the compliance of randomized controlled trials published in craniofacial surgery journals with the CONSORT statement. J Craniomaxillofac Surg 30(1):96–104. https://doi.org/10.1097/SCS.0000000000004900

    Article  Google Scholar 

  12. Copsey B, Thompson J, Vadher K et al (2018) Sample size calculations are poorly conducted and reported in many randomized trials of hip and knee osteoarthritis: results of a systematic review. J Clin Epidemiol 104:52–61. https://doi.org/10.1016/j.jclinepi.2018.08.013

    Article  PubMed  Google Scholar 

  13. Stevanovic A, Schmitz S, Rossaint R, Schürholz T, Coburn M (2015) CONSORT item reporting quality in the top ten ranked journals of critical care medicine in 2011: a retrospective analysis. PloS One. https://doi.org/10.1371/journal.pone.0128061

    Article  PubMed  PubMed Central  Google Scholar 

  14. Ahmed Ali U, ten Hove JR, Reiber BM, van der Sluis PC, Besselink MG (2018) Sample size of surgical randomized controlled trials: a lack of improvement over time. J Surg Res 228:1–7. https://doi.org/10.1016/j.jss.2018.02.014

    Article  PubMed  Google Scholar 

  15. Ayeni O, Dickson L, Ingacy T, Thoma A (2012) A systematic review of power and sample size reporting in randomized controlled trials within plastic surgery. Plast Reconstr Surg. https://doi.org/10.1097/PRS.0b013e318254b1d1

    Article  PubMed  Google Scholar 

  16. Alam M, Rauf M, Ali S, Nodzenski M, Minkis K (2014) A systematic review of reporting in randomized controlled trials in dermatologic surgery: jadad scores, power analysis, and sample size determination. Dermatol Surg 40(12):1299–1305. https://doi.org/10.1097/DSS.0000000000000166

    Article  CAS  PubMed  Google Scholar 

  17. Nikolakopoulos S, Roes K, van der Lee J, van der Tweel I (2014) Sample size calculations in pediatric clinical trials conducted in an ICU: A systematic review. Trials. https://doi.org/10.1186/1745-6215-15-274

    Article  PubMed  PubMed Central  Google Scholar 

  18. Chen X, Zhai X, Wang X, Su J, Ming L (2014) Methodological reporting quality of randomized controlled trials in three spine journals from 2010 to 2012. Eur Spine J 23(8):1606–1611. https://doi.org/10.1007/s00586-014-3283-1

    Article  PubMed  Google Scholar 

  19. Ahmed Ali U, van der Sluis PC, Issa Y et al (2013) Trends in worldwide volume and methodological quality of surgical randomized controlled trials. Ann Surg 258(2):199–207. https://doi.org/10.1097/SLA.0b013e31829c7795

    Article  PubMed  Google Scholar 

  20. Bai Y, Gao J, Zou D, Li Z (2009) Methodological reporting of randomized clinical trials in major gastroenterology and hepatology journals in 2006. Hepatology 49(6):2108–2112. https://doi.org/10.1002/hep.22861

    Article  PubMed  Google Scholar 

  21. Adie S, Harris I, Naylor J, Mittal R (2013) CONSORT compliance in surgical randomized trials: are we there yet? A systematic review. Ann Surg 258(6):872–878. https://doi.org/10.1097/SLA.0b013e31829664b9

    Article  PubMed  Google Scholar 

  22. Weinberg T, Wang G, Lam K, Kitchen J, Chan A (2019) Reporting of sample-size calculations for randomized trials in dermatology: comparison of publications with registries. Br J Dermatol 180(4):929–930. https://doi.org/10.1111/bjd.17332

    Article  CAS  PubMed  Google Scholar 

  23. Zhai X, Wang Y, Mu Q et al (2015) Methodological reporting quality of randomized controlled trials in 3 leading diabetes journals from 2011 to 2013 following CONSORT. Medicine. https://doi.org/10.1097/MD.0000000000001083

    Article  PubMed  PubMed Central  Google Scholar 

  24. Weaver C, Leonardi-Bee J, Bath- Hextall F, Bath P (2004) Sample size calculations in acute stroke trials: a systematic review of their reporting, characteristics, and relationship with outcome. Stroke 35(5):1216–1224. https://doi.org/10.1161/01.STR.0000125010.70652.93

    Article  PubMed  Google Scholar 

  25. Hebert R, Wright S, Dittus R, Elasy T (2002) Prominent medical journals often provide insufficient information to assess the validity of studies with negative results. J Negat Results Biomed 1(1):1–5. https://doi.org/10.1186/1477-5751-1-1

    Article  PubMed  PubMed Central  Google Scholar 

  26. Agha R, Cooper D, Muir G (2007) The reporting quality of randomised controlled trials in surgery: a systematic review. Int J Surg 5(6):413–422. https://doi.org/10.1016/j.ijsu.2007.06.002

    Article  PubMed  Google Scholar 

  27. Sanchez-Thorin J, Cortes MC, Montenegro M, Villate N (2001) The quality of reporting of randomized clinical trials published in ophthalmology. Ophthalmology 108(2):410–415. https://doi.org/10.1016/s0161-6420(00)00500-5

    Article  CAS  PubMed  Google Scholar 

  28. Lai T, Wong V, Lam R, Cheng A, Lam D, Leung G (2007) Quality of reporting of key methodological items of randomized controlled trials in clinical ophthalmic journals. Ophthalmic Epidemiol 14(6):390–398. https://doi.org/10.1080/09286580701344399

    Article  PubMed  Google Scholar 

  29. Yao A, Khajuria A, Camm C, Edison E, Agha R (2014) The reporting quality of parallel randomised controlled trials in ophthalmic surgery in 2011: a systematic review. Eye 28(11):1341–1349. https://doi.org/10.1038/eye.2014.206

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Shen C, Shamsudeen I, Farrokhyar F, Sabri K (2018) Fragility of results in ophthalmology randomized controlled trials: a systematic. Ophthalmology 215(5):642–648. https://doi.org/10.1016/j.ophtha.2017.11.015

    Article  Google Scholar 

  31. Krezel A, Hogg R, Krezel S, Fallis R, Azuara-Blanco A (2015) Design characteristic of randomised controlled trials for geographic atrophy in age-related macular degeneration: selection of outcomes and sample size. Eye 29(11):1458–1463. https://doi.org/10.1038/eye.2015.132

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Khan Z, Milko J, Iqbal M, Masri M, Almeida D (2016) Low power and type II errors in recent ophthalmology research. Can J Ophthalmol 51(5):368–372. https://doi.org/10.1016/j.jcjo.2016.02.002

    Article  PubMed  Google Scholar 

  33. Fung A, Palanki R, Bakri S, Depperschmidt E, Gibson A (2009) Applying the CONSORT and STROBE statements to evaluate the reporting quality of neovascular age-related macular degeneration studies. Ophthalmology. https://doi.org/10.1016/j.ophtha.2008.09.014

    Article  PubMed  Google Scholar 

  34. Abdulatif M, Mukhtar A, Obayah G (2015) Pitfalls in reporting sample size calculation in randomized controlled trials published in leading anaesthesia journals: a systematic review. Br J Anaesth 115(5):699–707. https://doi.org/10.1093/bja/aev166

    Article  CAS  PubMed  Google Scholar 

  35. Castellini G, Gianola S, Bonovas S, Moja L (2016) Improving power and sample size calculation in rehabilitation trial reports: a methodological assessment. Arch Phys Med Rehabil 97(7):1195–1201. https://doi.org/10.1016/j.apmr.2016.02.013

    Article  PubMed  Google Scholar 

  36. Kiehna EN, Starke RM, Pouratian N, Dumont A (2011) Standars for reporting randomized clinical trials in neurosurgery. J Neurosurg 114:280–285. https://doi.org/10.3171/2010.8.JNS091770

    Article  PubMed  Google Scholar 

  37. Clark T, Berger U, Mansmann U (2013) Sample size determinations in original research protocols for randomised clinical trials submitted to UK research ethics committees: review. Br Med J. https://doi.org/10.1136/bmj.f1135

    Article  Google Scholar 

  38. Kaido T (2006) Recent randomized controlled trials in pancreaticoduodenectomy. Pancreas 33(3):228–232

    Article  PubMed  Google Scholar 

  39. Schulz KF, Grimes DA (2005) Sample size calculations in randomised trials: mandatory and mystical. Lancet 365:1348–1353. https://doi.org/10.1016/S0140-6736(05)61034-3

    Article  PubMed  Google Scholar 

  40. Tavernier E, Giraudeau B (2015) Sample size calculation: inaccurate a priori assumptions for nuisance parameters can greatly affect the power of a randomized controlled trial. PloS One. https://doi.org/10.1371/journal.pone.0132578

    Article  PubMed  PubMed Central  Google Scholar 

  41. Chan A, Hróbjartsson A, Jørgensen K, Gøtzsche P, Douglas A (2008) Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. Br Med J 337(7683):1404–1407. https://doi.org/10.1136/bmj.a2299

    Article  Google Scholar 

  42. McDonald A, Knight R, Campbell M et al (2006) What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. https://doi.org/10.1186/1745-6215-7-9

    Article  PubMed  PubMed Central  Google Scholar 

  43. Barnett A, Glasziou P (2021) Target and actual sample sizes for studies from two trial registries from 1999 to 2020: an observational study. Br Med J. https://doi.org/10.1136/bmjopen-2021-053377

    Article  Google Scholar 

  44. Gazendam A, Slawaska-Eng D, Nucci N, Bhatt O, Ghert M (2022) The impact of industry funding on randomized controlled trials of biologic therapies. Medicines 9(3):18. https://doi.org/10.3390/medicines9030018

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Bhandari M, Busse J, Jackowski D et al (2004) Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 170(4):477–480

    PubMed  PubMed Central  Google Scholar 

  46. Peppercorn J, Blood E, Winer E, Partridge A (2007) Association between pharmaceutical involvement and outcomes in breast cancer clinical trials. Cancer 109(7):1239–1246. https://doi.org/10.1002/cncr.22528

    Article  PubMed  Google Scholar 

  47. Speich B, Odutayo A, Peckham N et al (2022) A longitudinal assessment of trial protocols approved by research ethics committees: the adherance to SPIrit REcommendations in the UK (ASPIRE-UK) study. Trials 23:601. https://doi.org/10.1186/s13063-022-06516-1

    Article  PubMed  PubMed Central  Google Scholar 

  48. Zhang X, Lam W, Liu F et al (2021) A Cross-sectional literature survey showed the reporting quality of multicenter randomized controlled trials should be improved. J Clin Edipemiol 137:250–261. https://doi.org/10.1016/j.jclinepi.2021.05.008

    Article  Google Scholar 

  49. Djurisic S, Rath A, Gaber S et al (2017) Barriers to the conduct of randomised clinical trials within all disease areas. Trials. https://doi.org/10.1186/s13063-017-2099-9

    Article  PubMed  PubMed Central  Google Scholar 

  50. Pocock S, Rossello X, Owen R, Collier T, Stone G, Rockhold F (2021) Primary and secondary outcome reporting in randomized trials: JACC state-of-the-art review. J Am Coll Cardiol 78(8):827–839. https://doi.org/10.1016/j.jacc.2021.06.024

    Article  PubMed  Google Scholar 

  51. Do-Yeop K, Sohee O, Hyun-Sun Y (2021) Complete and transparent reporting of primary endpoints of randomized trials in the dermatology literature: a comparison of registered and published primary endpoints. J Am Acad Dermatol 85(5):1201–1208. https://doi.org/10.1016/j.jaad.2020.04.067

    Article  Google Scholar 

Download references

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Medical School, National and Kapodistrian University of Athens, Athens, Greece

    Nefeli Eleni Kounatidou

  2. Department of Biostatistics, National and Kapodistrian University of Athens Medical School, Athens, Greece

    Chara Tzavara

  3. Department of Ophthalmology, University of Cyprus Medical School, Aglantzia, Cyprus

    Sotiria Palioura

Authors
  1. Nefeli Eleni Kounatidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chara Tzavara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sotiria Palioura
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Nefeli E Kounatidou, Chara Tzavara and Sotiria Palioura. The first draft of the manuscript was written by Nefeli E Kounatidou and all authors commented on previous versions of the manuscript. All authors read and approved the manuscript.

Corresponding author

Correspondence to Sotiria Palioura.

Ethics declarations

Conflicts of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kounatidou, N.E., Tzavara, C. & Palioura, S. Systematic review of sample size calculations and reporting in randomized controlled trials in ophthalmology over a 20-year period. Int Ophthalmol 43, 2999–3010 (2023). https://doi.org/10.1007/s10792-023-02687-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10792-023-02687-1

Keywords

Search

Navigation