Skip to main content
SpringerLink
Log in

Current Issues in Conduct and Reporting of Noninferiority Randomized Controlled Trials in Surgical Management of Cancer Patients

  • Health Services Research and Global Oncology
  • Published:
Annals of Surgical Oncology Aims and scope Submit manuscript

Abstract

Background

Serious concerns regarding quality of conduct and reporting of noninferiority trials (NITs) have been raised. Systematic analysis of the quality of the surgical NITs is lacking. Assessing the quality of conduct, reporting, and interpretation of surgical NITs in cancer patients is critical given their potential clinical impact. We aim to assess the quality of conduct, reporting, and interpretation of NITs that investigate the effects of surgical management in cancer patients.

Methods

A cross-sectional analysis of papers identified through a comprehensive literature database search was performed. Forty papers employing a phase III noninferiority (NI) randomized trial design to study effects of surgical methodology or sequencing of surgery in patients with solid cancers were included. Papers were assessed for type of analysis, justification of the noninferiority margin (NIM), consistency of type I error with confidence intervals (CIs), ability to achieve the predefined sample size, and interpretations regarding NI.

Results

Only half of the papers used both intention-to-treat and per protocol analyses; 62.5% provided no or poor justification for the NIM; 42.5% showed inconsistency of the type I error rate with CIs; 52.5% were deemed poor or fair quality, and 60.0% did not achieve the predefined sample size. One-fifth of the papers provided interpretation of the NI hypothesis that was not in concordance with the CONSORT guidelines.

Conclusions

The quality of conduct, reporting, and interpretation of surgical NITs is suboptimal, requiring further improvements through adherence to guidelines and rigorous assessment at the stages of the study approval, funding, and the peer-review process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Adapted from Piaggio et al.7

Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Huttner FJ, Capdeville L, Pianka F, et al. Systematic review of the quantity and quality of randomized clinical trials in pancreatic surgery. Br J Surg. Jan 2019;106(1):23–31.

    Article  CAS  Google Scholar 

  2. Huttner FJ, Doerr-Harim C, Probst P, Tenckhoff S, Knebel P, Diener MK. Study methods in evidence-based surgery: methodological impediments and suggested approaches for the creation and transfer of knowledge in surgery. Eur Surg Res. 2014;53(1–4):86–94.

    Article  Google Scholar 

  3. Huddart RA, Birtle A, Maynard L, et al. Clinical and patient-reported outcomes of SPARE—a randomised feasibility study of selective bladder preservation versus radical cystectomy. BJU Int. Nov 2017;120(5):639–50.

    Article  CAS  Google Scholar 

  4. Murthy VL, Desai NR, Vora A, Bhatt DL. Increasing proportion of clinical trials using noninferiority end points. Clin Cardiol. Sep 2012;35(9):522–3.

    Article  Google Scholar 

  5. Suda KJ, Hurley AM, McKibbin T, Motl Moroney SE. Publication of noninferiority clinical trials: changes over a 20-year interval. Pharmacotherapy. 2011;31(9):833–9.

    Article  Google Scholar 

  6. Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG, Group C. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594–604.

    Article  Google Scholar 

  7. Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ, Group C. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295(10):1152–60.

    Article  Google Scholar 

  8. FDA. Non-Inferiority Clinical Trials to Establish Effectiveness, Guidance for Industry. Food and Drug Administration, US Department of Health and Human Services. 2016.

  9. Aberegg SK, Hersh AM, Samore MH. Empirical consequences of current recommendations for the design and interpretation of noninferiority trials. J Gen Intern Med. 2018;33(1):88–96.

    Article  Google Scholar 

  10. Rehal S, Morris TP, Fielding K, Carpenter JR, Phillips PP. Non-inferiority trials: are they inferior? A systematic review of reporting in major medical journals. BMJ Open. Oct 7 2016;6(10):e012594.

    Article  Google Scholar 

  11. Turgeon RD, Reid EK, Rainkie DC. Design and interpretation of noninferiority trials. J Gen Intern Med. 2018;33(8):1215.

    Article  Google Scholar 

  12. Acuna SA, Dossa F, Baxter NN. Frequency of misinterpretation of inconclusive noninferiority trials: the case of the laparoscopic vs open resection for rectal cancer trials. JAMA Surg. 2019;154(1):90–2.

    Article  Google Scholar 

  13. Aberegg S. Reporting noninferiority trials. JAMA. 2013;309(15):1584–5.

    Article  CAS  Google Scholar 

  14. Paesmans M, Grigoriu B, Ocak S, et al. Systematic qualitative review of randomised trials conducted in nonsmall cell lung cancer with a noninferiority or equivalence design. Eur Respir J. 2015;45(2):511–24.

    Article  CAS  Google Scholar 

  15. Fueglistaler P, Adamina M, Guller U. Non-inferiority trials in surgical oncology. Ann Surg Oncol. 2007;14(5):1532–9.

    Article  Google Scholar 

  16. 2017 Journal Impact Factor, Journal Citation Reports, ClarivateAnalytics, 2018.

  17. Acuna SA, Chesney TR, Amarasekera ST, Baxter NN. Defining non-inferiority margins for quality of surgical resection for rectal cancer: a Delphi consensus study. Ann Surg Oncol. 2018;25(11):3171–8.

    Article  Google Scholar 

  18. Wyrwich KW, Spertus JA, Kroenke K, et al. Clinically important differences in health status for patients with heart disease: an expert consensus panel report. Am Heart J. 2004;147(4):615–22.

    Article  Google Scholar 

  19. Bellamy N, Carette S, Ford PM, et al. Osteoarthritis antirheumatic drug trials. III. Setting the delta for clinical trials–results of a consensus development (Delphi) exercise. J Rheumatol. 1992;19(3):451–7.

    CAS  PubMed  Google Scholar 

  20. McGlothlin AE, Lewis RJ. Minimal clinically important difference: defining what really matters to patients. JAMA. 2014;312(13):1342–3.

    Article  CAS  Google Scholar 

  21. Acuna SA, Chesney TR, Baxter NN. Incorporating patient preferences in noninferiority trials. JAMA. 2019;322(4):305–6.

    Article  Google Scholar 

  22. Llewellyn-Thomas HA, Williams JI, Levy L, Naylor CD. Using a trade-off technique to assess patients’ treatment preferences for benign prostatic hyperplasia. Med Decis Making. 1996;16(3):262–82.

    Article  CAS  Google Scholar 

  23. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.

    Article  Google Scholar 

  24. D’Agostino RB, Sr., Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med. 2003;22(2):169–86.

    Article  Google Scholar 

  25. Shrier I, Verhagen E, Stovitz SD. The intention-to-treat analysis is not always the conservative approach. Am J Med. 2017;130(7):867–71.

    Article  Google Scholar 

  26. Matsuyama Y. A comparison of the results of intent-to-treat, per-protocol, and g-estimation in the presence of non-random treatment changes in a time-to-event non-inferiority trial. Stat Med. 2010;29(20):2107–16.

    Article  Google Scholar 

  27. Head SJ, Kaul S, Bogers AJ, Kappetein AP. Non-inferiority study design: lessons to be learned from cardiovascular trials. Eur Heart J. 2012;33(11):1318–24.

    Article  Google Scholar 

  28. Macaya F, Ryan N, Salinas P, Pocock SJ. Challenges in the design and interpretation of noninferiority trials: insights from recent stent trials. J Am Coll Cardiol. 2017;70(7):894–903.

    Article  Google Scholar 

  29. Schmid SL. Five years post-DORA: promoting best practices for research assessment. Mol Biol Cell. 2017;28(22):2941–4.

    Article  CAS  Google Scholar 

  30. Zhang L, Rousseau R, Sivertsen G. Science deserves to be judged by its contents, not by its wrapping: revisiting Seglen’s work on journal impact and research evaluation. PLoS One. 2017;12(3):e0174205.

    Article  Google Scholar 

  31. DORA. San Francisco Declaration on Research Assessment. December 16, 2012, Accessed June 15, 2019.

  32. Fung EK, Lore JM, Jr. Randomized controlled trials for evaluating surgical questions. Arch Otolaryngol Head Neck Surg. 2002;128(6):631–4.

    Article  Google Scholar 

  33. McLeod RS. Issues in surgical randomized controlled trials. World J Surg. 1999;23(12):1210–4.

    Article  CAS  Google Scholar 

  34. Yu J, Chen W, Chen S, et al. Design, conduct, and analysis of surgical randomized controlled trials: a cross-sectional survey. Ann Surg. 2019;270(6):1065–9.

    Article  Google Scholar 

  35. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 Explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–37.

    Article  Google Scholar 

  36. Giuliano AE, Hunt KK, Ballman KV, et al. Axillary dissection vs no axillary dissection in women with invasive breast cancer and sentinel node metastasis: a randomized clinical trial. JAMA. 2011;305(6):569–75.

    Article  CAS  Google Scholar 

  37. Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol. 2014;15(12):1303–10.

    Article  Google Scholar 

  38. Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci USA. 2018;115(11):2613–9.

    Article  Google Scholar 

  39. Boutron I, Haneef R, Yavchitz A, et al. Three randomized controlled trials evaluating the impact of “spin” in health news stories reporting studies of pharmacologic treatments on patients’/caregivers’ interpretation of treatment benefit. BMC Med. 2019;17(1):105.

    Article  Google Scholar 

  40. Fleshman J, Branda M, Sargent DJ, et al. Effect of laparoscopic-assisted resection vs open resection of stage II or III rectal cancer on pathologic outcomes: the ACOSOG Z6051 randomized clinical trial. JAMA. 2015;314(13):1346–55.

    Article  CAS  Google Scholar 

  41. Stevenson AR, Solomon MJ, Lumley JW, et al. Effect of laparoscopic-assisted resection vs open resection on pathological outcomes in rectal cancer: the ALaCaRT randomized clinical trial. JAMA. 2015;314(13):1356–63.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

We thank Mr. Toni Panzarella, M.Sc. (Biostatistics Division, Dalla Lana School of Public Health, University of Toronto, Ontario, Canada) for valuable discussion regarding the design of this study.

Funding

This study was sponsored by the internal funds from the Department of Surgery, University of Toronto.

Author information

Author notes

  1. Armen Parsyan and Wanda Marini have contributed equally to this work.

Authors and Affiliations

  1. Department of Surgery and Oncology, London Regional Cancer Program, St Joseph’s Health Care and London Health Sciences Centre, Western University, London, ON, Canada

    Armen Parsyan MD, PhD, DrS, MPH, FRCSC

  2. Department of Surgery, University Health Network, Princess Margaret Hospital, University of Toronto, Toronto, Canada

    Wanda Marini MD, MSc, PhD & David McCready MD, FRCSC, FACS

  3. Library and Information Services, Princess Margaret Cancer Centre, University Health Network, Toronto, Canada

    Rouhi Fazelzad BSc, MISt

  4. Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada

    David Moher PhD

  5. School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada

    David Moher PhD

Authors
  1. Armen Parsyan MD, PhD, DrS, MPH, FRCSC
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wanda Marini MD, MSc, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rouhi Fazelzad BSc, MISt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Moher PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David McCready MD, FRCSC, FACS
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Armen Parsyan MD, PhD, DrS, MPH, FRCSC.

Ethics declarations

Disclosure

Authors do not have any commercial interests in the subject of study and the source of any financial or material support.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 191 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parsyan, A., Marini, W., Fazelzad, R. et al. Current Issues in Conduct and Reporting of Noninferiority Randomized Controlled Trials in Surgical Management of Cancer Patients. Ann Surg Oncol 28, 39–47 (2021). https://doi.org/10.1245/s10434-020-08575-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1245/s10434-020-08575-7

Search

Navigation