Skip to main content
SpringerLink
Log in

An Overview of Challenges and Approaches to Minimize Bias in Randomized Controlled Trials in Perioperative Medicine

  • Research Methods and Statistical Analyses (Y Le Manach, Section Editor)
  • Published:
Current Anesthesiology Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

Randomized controlled trials (RCT) are recognized as the most robust design to study the relationship between exposure and outcomes. The conventional RCT design is commonly used in pharmacological trials. Some surgical interventions are not be well suited to a conventional RCT design and may be associated with methodological challenges. Approaches have been proposed in non-pharmacological trials to overcome some of these challenges and minimize the risk of bias.

Recent Findings

Imbalance in prognostic factors between intervention groups, lack of allocation concealment, unblinding, non-intention-to-treat analysis, and losses to follow-ups can all threaten the validity of RCT results to various degrees. Procedure-based trials raise also specific challenges since physician expertise and training can affect the intervention, exposing to potential differential-expertise bias. Lack of statistical power can also affect the confidence in a trial’s result. Small sample sizes also usually mean small number of events for comparison between interventions, resulting in less statistically robust findings.

Summary

Minimizing risk of bias and achieving adequate statistical power are crucial to producing high quality and meaningful results. Non-pharmacological trials pose certain methodological challenges, and several approaches have been proposed to address the risk of bias. Large sample sizes are also usually required to achieve sufficient statistical power to provide answers to meaningful clinical questions. However, small perioperative trials remain frequent and result interpretation based solely on P values might not always appropriately inform on the confidence in a trial’s results. The Fragility Index can be used to further inform on the confidence of statistically significant result.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

Papers of particular interest, published recently, have been highlighted as • Of importance •• Of major importance

  1. Devereaux PJ, Yusuf S. The evolution of the randomized controlled trial and its role in evidence-based decision making. J Intern Med. 2003;254(2):105–13.

    Article  CAS  PubMed  Google Scholar 

  2. Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32(1–2):51–63.

    Article  CAS  PubMed  Google Scholar 

  3. Group GW. Grading quality of evidence and strength of recommendations. BMJ Br Med J. 2004;328(7454):1490.

    Article  Google Scholar 

  4. Farrokhyar F, Karanicolas PJ, Thoma A, Simunovic M, Bhandari M, Devereaux PJ, et al. Randomized controlled trials of surgical interventions. Ann Surg. 2010;251(3):409–16.

    Article  PubMed  Google Scholar 

  5. Schulz KF, Grimes DA. Generation of allocation sequences in randomised trials: chance, not choice. Lancet. 2002;359(9305):515–9.

    Article  PubMed  Google Scholar 

  6. Lachin JM. Properties of simple randomization in clinical trials. Control Clin Trials. 1988;9(4):312–26.

    Article  CAS  PubMed  Google Scholar 

  7. Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet. 2002;359(9306):614–8.

    Article  PubMed  Google Scholar 

  8. Schulz KF, Chalmers I, Grimes DA, Altman DG. Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA. 1994;272(2):125–8.

    Article  CAS  PubMed  Google Scholar 

  9. Altman DG, Dore CJ. Randomisation and baseline comparisons in clinical trials. Lancet. 1990;335(8682):149–53.

    Article  CAS  PubMed  Google Scholar 

  10. Meinert CL, Tonascia S. Clinical trials: design, conduct, and analysis. Oxford: Oxford University Press; 1986.

    Book  Google Scholar 

  11. Greenhalgh RM, Brown LC, Powell JT, Thompson SG, Epstein D, Sculpher MJ. Endovascular versus open repair of abdominal aortic aneurysm. N Engl J Med. 2010;362(20):1863–71.

    Article  PubMed  CAS  Google Scholar 

  12. Montenij L, de Waal E, Frank M, van Beest P, de Wit A, Kruitwagen C, et al. Influence of early goal-directed therapy using arterial waveform analysis on major complications after high-risk abdominal surgery: study protocol for a multicenter randomized controlled superiority trial. Trials. 2014;15:360.

    Article  PubMed Central  PubMed  Google Scholar 

  13. Buse GL, et al. Accelerated care versus standard care among patients with hip fracture: the HIP ATTACK pilot trial. Cmaj. 2014;186(1):52–60.

    Article  Google Scholar 

  14. Herbison P, Hay-Smith J, Gillespie WJ. Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study. J Clin Epidemiol. 2011;64(10):1070–5.

    Article  PubMed  Google Scholar 

  15. •• Evaniew N, Carrasco-Labra A, Devereaux PJ, Tikkinen KA, Fei Y, Bhandari M, et al. How to use a randomized clinical trial addressing a surgical procedure: users’ guide to the medical literature. JAMA Surg. 2016. - This publication of the Users Guide to the Medical Literature RCTs provides comprehensible review of use and misuse of RCTs in the surgical setting.

  16. Greenfield ML, Mhyre JM, Mashour GA, Blum JM, Yen EC, Rosenberg AL. Improvement in the quality of randomized controlled trials among general anesthesiology journals 2000 to 2006: a 6-year follow-up. Anesth Analg. 2009;108(6):1916–21.

    Article  PubMed  Google Scholar 

  17. Voineskos SH, Coroneos CJ, Ziolkowski NI, Kaur MN, Banfield L, Meade MO, et al. A systematic review of surgical randomized controlled trials: Part I. Risk of bias and outcomes: common pitfalls plastic surgeons can overcome. Plast Reconstr Surg. 2016;137(2):696–706.

    Article  CAS  PubMed  Google Scholar 

  18. Devereaux PJ, Choi PT, El-Dika S, Bhandari M, Montori VM, Schunemann HJ, et al. An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 2004;57(12):1232–6.

    Article  CAS  PubMed  Google Scholar 

  19. Moseley JB, O’Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002;347(2):81–8.

    Article  PubMed  Google Scholar 

  20. Wei JT, Nygaard I, Richter HE, Nager CW, Barber MD, Kenton K, et al. A midurethral sling to reduce incontinence after vaginal prolapse repair. N Engl J Med. 2012;366(25):2358–67.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  21. Koutsourelakis I, Georgoulopoulos G, Perraki E, Vagiakis E, Roussos C, Zakynthinos SG. Randomised trial of nasal surgery for fixed nasal obstruction in obstructive sleep apnoea. Eur Respir J. 2008;31(1):110–7.

    Article  CAS  PubMed  Google Scholar 

  22. Horng S, Miller FG. Ethical framework for the use of sham procedures in clinical trials. Crit Care Med. 2003;31(3 Suppl):S126–30.

    Article  PubMed  Google Scholar 

  23. Wolf BR, Buckwalter JA. Randomized surgical trials and “sham” surgery: relevance to modern orthopaedics and minimally invasive surgery. Iowa Orthop J. 2006;26:107–11.

    PubMed Central  PubMed  Google Scholar 

  24. Dowrick AS, Bhandari M. Ethical issues in the design of randomized trials: to sham or not to sham. J Bone Joint Surg Am. 2012;94(Suppl 1):7–10.

    Article  PubMed  Google Scholar 

  25. Sackett DL. Clinician-trialist rounds: 5. Cointervention bias–how to diagnose it in their trial and prevent it in yours. Clin Trials. 2011;8(4):440–2.

    Article  PubMed  Google Scholar 

  26. Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. 2014;43(4):1272–83.

    Article  PubMed Central  PubMed  Google Scholar 

  27. Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012;344:e1119.

    Article  PubMed  Google Scholar 

  28. Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ. 2013;185(4):E201–11.

    Article  PubMed Central  PubMed  Google Scholar 

  29. Poolman RW, Struijs PA, Krips R, Sierevelt IN, Marti RK, Farrokhyar F, et al. Reporting of outcomes in orthopaedic randomized trials: does blinding of outcome assessors matter? J Bone Joint Surg Am. 2007;89(3):550–8.

    PubMed  Google Scholar 

  30. Majeed AW, Troy G, Nicholl JP, Smythe A, Reed MW, Stoddard CJ, et al. Randomised, prospective, single-blind comparison of laparoscopic versus small-incision cholecystectomy. Lancet. 1996;347(9007):989–94.

    Article  CAS  PubMed  Google Scholar 

  31. Devereaux PJ, Mrkobrada M, Sessler DI, Leslie K, Alonso-Coello P, Kurz A, et al. Aspirin in patients undergoing noncardiac surgery. N Engl J Med. 2014;370(16):1494–503.

    Article  CAS  PubMed  Google Scholar 

  32. Vannabouathong C, Saccone M, Sprague S, Schemitsch EH, Bhandari M. Adjudicating outcomes: fundamentals. J Bone Joint Surg Am. 2012;94(Suppl 1):70–4.

    Article  PubMed  Google Scholar 

  33. Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ: Br Med J. 2001;322(7282):355–7.

    Article  CAS  Google Scholar 

  34. Cook JA, McCulloch P, Blazeby JM, Beard DJ, Marinac-Dabic D, Sedrakyan A. IDEAL framework for surgical innovation 3: randomised controlled trials in the assessment stage and evaluations in the long term study stage. BMJ. 2013;346:f2820.

    Article  PubMed Central  PubMed  Google Scholar 

  35. Devereaux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S, et al. Need for expertise based randomised controlled trials. BMJ. 2005;330(7482):88.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  36. • Cook JA, Elders A, Boachie C, Bassinga T, Fraser C, Altman DG, et al. A systematic review of the use of an expertise-based randomised controlled trial design. Trials. 2015;16:241. - A systematic review that informs on the current use of expertise-based design in RCTs. Expertise-based design has gained popularity in the last decade as a novel approach to conduct RCT, especially in non-pharmacological and surgical trials.

  37. Walter SD, Ismaila AS, Devereaux PJ. Statistical issues in the design and analysis of expertise-based randomized clinical trials. Stat Med. 2008;27(30):6583–96.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  38. Gupta SK. Intention-to-treat concept: a review. Perspect Clin Res. 2011;2(3):109–12.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Newell DJ. Intention-to-treat analysis: implications for quantitative and qualitative research. Int J Epidemiol. 1992;21(5):837–41.

    Article  CAS  PubMed  Google Scholar 

  40. Montori VM, Guyatt GH. Intention-to-treat principle. CMAJ. 2001;165(10):1339–41.

    PubMed Central  CAS  PubMed  Google Scholar 

  41. Abraha I, Montedori A. Modified intention to treat reporting in randomised controlled trials: systematic review. The BMJ. 2010;340:c2697.

    Article  PubMed Central  PubMed  Google Scholar 

  42. Abraha I, Cherubini A, Cozzolino F, De Florio R, Luchetta ML, Rimland JM, et al. Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study. BMJ. 2015;350:h2445

    Article  PubMed Central  PubMed  Google Scholar 

  43. Schulz KF, Grimes DA. Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet. 2002;359(9308):781–5.

    Article  PubMed  Google Scholar 

  44. Akl EA, Briel M, You JJ, Sun X, Johnston BC, Busse JW, et al. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review. BMJ. 2012;344:e2809.

    Article  PubMed  Google Scholar 

  45. Rerkasem K, Rothwell PM. Meta-analysis of small randomized controlled trials in surgery may be unreliable. Br J Surg. 2010;97(4):466–9.

    Article  CAS  PubMed  Google Scholar 

  46. Abdulatif M, Mukhtar A, Obayah G. Pitfalls in reporting sample size calculation in randomized controlled trials published in leading anaesthesia journals: a systematic review. Br J Anaesth. 2015;115(5):699–707.

    Article  CAS  PubMed  Google Scholar 

  47. Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Stat Med. 1984;3(4):409–22.

    Article  CAS  PubMed  Google Scholar 

  48. •• Walsh M, Srinathan SK, McAuley DF, Mrkobrada M, Levine O, Ribic C, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol. 2014;67(6):622–8. - This publication discusses the issue of fragility in trials and introduced the Fragility Index. The Fragility Index is an novel metric that is proposed to complement p-value in assessing statistically significant results reported in trials.

    Article  PubMed  Google Scholar 

  49. Ridgeon EE, Young PJ, Bellomo R, Mucchetti M, Lembo R, Landoni G. The fragility index in multicenter randomized controlled critical care trials. Crit Care Med. 2016;44(7):1278–84.

    Article  PubMed  Google Scholar 

  50. Evaniew N, Files C, Smith C, Bhandari M, Ghert M, Walsh M, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J. 2015;15(10):2188–97.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4K1, Canada

    Emmanuelle Duceppe & Emilie Belley-Coté

Authors
  1. Emmanuelle Duceppe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emilie Belley-Coté
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emmanuelle Duceppe.

Ethics declarations

Conflict of Interest

Emmanuelle Duceppe and Emilie Belley-Coté declare that they have no conflict of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

This article is part of the Topical collection on Research Methods and Statistical Analyses.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duceppe, E., Belley-Coté, E. An Overview of Challenges and Approaches to Minimize Bias in Randomized Controlled Trials in Perioperative Medicine. Curr Anesthesiol Rep 6, 276–282 (2016). https://doi.org/10.1007/s40140-016-0172-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40140-016-0172-7

Keywords

Search

Navigation