Current Anesthesiology Reports

, Volume 6, Issue 3, pp 276–282 | Cite as

An Overview of Challenges and Approaches to Minimize Bias in Randomized Controlled Trials in Perioperative Medicine

  • Emmanuelle DuceppeEmail author
  • Emilie Belley-Coté
Research Methods and Statistical Analyses (Y Le Manach, Section Editor)
Part of the following topical collections:
  1. Research Methods and Statistical Analyses


Purpose of Review

Randomized controlled trials (RCT) are recognized as the most robust design to study the relationship between exposure and outcomes. The conventional RCT design is commonly used in pharmacological trials. Some surgical interventions are not be well suited to a conventional RCT design and may be associated with methodological challenges. Approaches have been proposed in non-pharmacological trials to overcome some of these challenges and minimize the risk of bias.

Recent Findings

Imbalance in prognostic factors between intervention groups, lack of allocation concealment, unblinding, non-intention-to-treat analysis, and losses to follow-ups can all threaten the validity of RCT results to various degrees. Procedure-based trials raise also specific challenges since physician expertise and training can affect the intervention, exposing to potential differential-expertise bias. Lack of statistical power can also affect the confidence in a trial’s result. Small sample sizes also usually mean small number of events for comparison between interventions, resulting in less statistically robust findings.


Minimizing risk of bias and achieving adequate statistical power are crucial to producing high quality and meaningful results. Non-pharmacological trials pose certain methodological challenges, and several approaches have been proposed to address the risk of bias. Large sample sizes are also usually required to achieve sufficient statistical power to provide answers to meaningful clinical questions. However, small perioperative trials remain frequent and result interpretation based solely on P values might not always appropriately inform on the confidence in a trial’s results. The Fragility Index can be used to further inform on the confidence of statistically significant result.


Perioperative care Clinical trials Randomized controlled trial Research methods 


Compliance with Ethics Guidelines

Conflict of Interest

Emmanuelle Duceppe and Emilie Belley-Coté declare that they have no conflict of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.


Papers of particular interest, published recently, have been highlighted as • Of importance •• Of major importance

  1. 1.
    Devereaux PJ, Yusuf S. The evolution of the randomized controlled trial and its role in evidence-based decision making. J Intern Med. 2003;254(2):105–13.CrossRefPubMedGoogle Scholar
  2. 2.
    Sackett DL. Bias in analytic research. J Chronic Dis. 1979;32(1–2):51–63.CrossRefPubMedGoogle Scholar
  3. 3.
    Group GW. Grading quality of evidence and strength of recommendations. BMJ Br Med J. 2004;328(7454):1490.CrossRefGoogle Scholar
  4. 4.
    Farrokhyar F, Karanicolas PJ, Thoma A, Simunovic M, Bhandari M, Devereaux PJ, et al. Randomized controlled trials of surgical interventions. Ann Surg. 2010;251(3):409–16.CrossRefPubMedGoogle Scholar
  5. 5.
    Schulz KF, Grimes DA. Generation of allocation sequences in randomised trials: chance, not choice. Lancet. 2002;359(9305):515–9.CrossRefPubMedGoogle Scholar
  6. 6.
    Lachin JM. Properties of simple randomization in clinical trials. Control Clin Trials. 1988;9(4):312–26.CrossRefPubMedGoogle Scholar
  7. 7.
    Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet. 2002;359(9306):614–8.CrossRefPubMedGoogle Scholar
  8. 8.
    Schulz KF, Chalmers I, Grimes DA, Altman DG. Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA. 1994;272(2):125–8.CrossRefPubMedGoogle Scholar
  9. 9.
    Altman DG, Dore CJ. Randomisation and baseline comparisons in clinical trials. Lancet. 1990;335(8682):149–53.CrossRefPubMedGoogle Scholar
  10. 10.
    Meinert CL, Tonascia S. Clinical trials: design, conduct, and analysis. Oxford: Oxford University Press; 1986.CrossRefGoogle Scholar
  11. 11.
    Greenhalgh RM, Brown LC, Powell JT, Thompson SG, Epstein D, Sculpher MJ. Endovascular versus open repair of abdominal aortic aneurysm. N Engl J Med. 2010;362(20):1863–71.CrossRefPubMedGoogle Scholar
  12. 12.
    Montenij L, de Waal E, Frank M, van Beest P, de Wit A, Kruitwagen C, et al. Influence of early goal-directed therapy using arterial waveform analysis on major complications after high-risk abdominal surgery: study protocol for a multicenter randomized controlled superiority trial. Trials. 2014;15:360.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Buse GL, et al. Accelerated care versus standard care among patients with hip fracture: the HIP ATTACK pilot trial. Cmaj. 2014;186(1):52–60.CrossRefGoogle Scholar
  14. 14.
    Herbison P, Hay-Smith J, Gillespie WJ. Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study. J Clin Epidemiol. 2011;64(10):1070–5.CrossRefPubMedGoogle Scholar
  15. 15.
    •• Evaniew N, Carrasco-Labra A, Devereaux PJ, Tikkinen KA, Fei Y, Bhandari M, et al. How to use a randomized clinical trial addressing a surgical procedure: users’ guide to the medical literature. JAMA Surg. 2016. - This publication of the Users Guide to the Medical Literature RCTs provides comprehensible review of use and misuse of RCTs in the surgical setting. Google Scholar
  16. 16.
    Greenfield ML, Mhyre JM, Mashour GA, Blum JM, Yen EC, Rosenberg AL. Improvement in the quality of randomized controlled trials among general anesthesiology journals 2000 to 2006: a 6-year follow-up. Anesth Analg. 2009;108(6):1916–21.CrossRefPubMedGoogle Scholar
  17. 17.
    Voineskos SH, Coroneos CJ, Ziolkowski NI, Kaur MN, Banfield L, Meade MO, et al. A systematic review of surgical randomized controlled trials: Part I. Risk of bias and outcomes: common pitfalls plastic surgeons can overcome. Plast Reconstr Surg. 2016;137(2):696–706.CrossRefPubMedGoogle Scholar
  18. 18.
    Devereaux PJ, Choi PT, El-Dika S, Bhandari M, Montori VM, Schunemann HJ, et al. An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 2004;57(12):1232–6.CrossRefPubMedGoogle Scholar
  19. 19.
    Moseley JB, O’Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002;347(2):81–8.CrossRefPubMedGoogle Scholar
  20. 20.
    Wei JT, Nygaard I, Richter HE, Nager CW, Barber MD, Kenton K, et al. A midurethral sling to reduce incontinence after vaginal prolapse repair. N Engl J Med. 2012;366(25):2358–67.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Koutsourelakis I, Georgoulopoulos G, Perraki E, Vagiakis E, Roussos C, Zakynthinos SG. Randomised trial of nasal surgery for fixed nasal obstruction in obstructive sleep apnoea. Eur Respir J. 2008;31(1):110–7.CrossRefPubMedGoogle Scholar
  22. 22.
    Horng S, Miller FG. Ethical framework for the use of sham procedures in clinical trials. Crit Care Med. 2003;31(3 Suppl):S126–30.CrossRefPubMedGoogle Scholar
  23. 23.
    Wolf BR, Buckwalter JA. Randomized surgical trials and “sham” surgery: relevance to modern orthopaedics and minimally invasive surgery. Iowa Orthop J. 2006;26:107–11.PubMedPubMedCentralGoogle Scholar
  24. 24.
    Dowrick AS, Bhandari M. Ethical issues in the design of randomized trials: to sham or not to sham. J Bone Joint Surg Am. 2012;94(Suppl 1):7–10.CrossRefPubMedGoogle Scholar
  25. 25.
    Sackett DL. Clinician-trialist rounds: 5. Cointervention bias–how to diagnose it in their trial and prevent it in yours. Clin Trials. 2011;8(4):440–2.CrossRefPubMedGoogle Scholar
  26. 26.
    Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. 2014;43(4):1272–83.CrossRefPubMedPubMedCentralGoogle Scholar
  27. 27.
    Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012;344:e1119.CrossRefPubMedGoogle Scholar
  28. 28.
    Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ. 2013;185(4):E201–11.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Poolman RW, Struijs PA, Krips R, Sierevelt IN, Marti RK, Farrokhyar F, et al. Reporting of outcomes in orthopaedic randomized trials: does blinding of outcome assessors matter? J Bone Joint Surg Am. 2007;89(3):550–8.CrossRefPubMedGoogle Scholar
  30. 30.
    Majeed AW, Troy G, Nicholl JP, Smythe A, Reed MW, Stoddard CJ, et al. Randomised, prospective, single-blind comparison of laparoscopic versus small-incision cholecystectomy. Lancet. 1996;347(9007):989–94.CrossRefPubMedGoogle Scholar
  31. 31.
    Devereaux PJ, Mrkobrada M, Sessler DI, Leslie K, Alonso-Coello P, Kurz A, et al. Aspirin in patients undergoing noncardiac surgery. N Engl J Med. 2014;370(16):1494–503.CrossRefPubMedGoogle Scholar
  32. 32.
    Vannabouathong C, Saccone M, Sprague S, Schemitsch EH, Bhandari M. Adjudicating outcomes: fundamentals. J Bone Joint Surg Am. 2012;94(Suppl 1):70–4.CrossRefPubMedGoogle Scholar
  33. 33.
    Torgerson DJ. Contamination in trials: is cluster randomisation the answer? BMJ: Br Med J. 2001;322(7282):355–7.CrossRefGoogle Scholar
  34. 34.
    Cook JA, McCulloch P, Blazeby JM, Beard DJ, Marinac-Dabic D, Sedrakyan A. IDEAL framework for surgical innovation 3: randomised controlled trials in the assessment stage and evaluations in the long term study stage. BMJ. 2013;346:f2820.CrossRefPubMedPubMedCentralGoogle Scholar
  35. 35.
    Devereaux PJ, Bhandari M, Clarke M, Montori VM, Cook DJ, Yusuf S, et al. Need for expertise based randomised controlled trials. BMJ. 2005;330(7482):88.CrossRefPubMedPubMedCentralGoogle Scholar
  36. 36.
    • Cook JA, Elders A, Boachie C, Bassinga T, Fraser C, Altman DG, et al. A systematic review of the use of an expertise-based randomised controlled trial design. Trials. 2015;16:241. - A systematic review that informs on the current use of expertise-based design in RCTs. Expertise-based design has gained popularity in the last decade as a novel approach to conduct RCT, especially in non-pharmacological and surgical trials.Google Scholar
  37. 37.
    Walter SD, Ismaila AS, Devereaux PJ. Statistical issues in the design and analysis of expertise-based randomized clinical trials. Stat Med. 2008;27(30):6583–96.CrossRefPubMedPubMedCentralGoogle Scholar
  38. 38.
    Gupta SK. Intention-to-treat concept: a review. Perspect Clin Res. 2011;2(3):109–12.CrossRefPubMedPubMedCentralGoogle Scholar
  39. 39.
    Newell DJ. Intention-to-treat analysis: implications for quantitative and qualitative research. Int J Epidemiol. 1992;21(5):837–41.CrossRefPubMedGoogle Scholar
  40. 40.
    Montori VM, Guyatt GH. Intention-to-treat principle. CMAJ. 2001;165(10):1339–41.PubMedPubMedCentralGoogle Scholar
  41. 41.
    Abraha I, Montedori A. Modified intention to treat reporting in randomised controlled trials: systematic review. The BMJ. 2010;340:c2697.CrossRefPubMedPubMedCentralGoogle Scholar
  42. 42.
    Abraha I, Cherubini A, Cozzolino F, De Florio R, Luchetta ML, Rimland JM, et al. Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study. BMJ. 2015;350:h2445Google Scholar
  43. 43.
    Schulz KF, Grimes DA. Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet. 2002;359(9308):781–5.CrossRefPubMedGoogle Scholar
  44. 44.
    Akl EA, Briel M, You JJ, Sun X, Johnston BC, Busse JW, et al. Potential impact on estimated treatment effects of information lost to follow-up in randomised controlled trials (LOST-IT): systematic review. BMJ. 2012;344:e2809.CrossRefPubMedGoogle Scholar
  45. 45.
    Rerkasem K, Rothwell PM. Meta-analysis of small randomized controlled trials in surgery may be unreliable. Br J Surg. 2010;97(4):466–9.CrossRefPubMedGoogle Scholar
  46. 46.
    Abdulatif M, Mukhtar A, Obayah G. Pitfalls in reporting sample size calculation in randomized controlled trials published in leading anaesthesia journals: a systematic review. Br J Anaesth. 2015;115(5):699–707.CrossRefPubMedGoogle Scholar
  47. 47.
    Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Stat Med. 1984;3(4):409–22.CrossRefPubMedGoogle Scholar
  48. 48.
    •• Walsh M, Srinathan SK, McAuley DF, Mrkobrada M, Levine O, Ribic C, et al. The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. J Clin Epidemiol. 2014;67(6):622–8. - This publication discusses the issue of fragility in trials and introduced the Fragility Index. The Fragility Index is an novel metric that is proposed to complement p-value in assessing statistically significant results reported in trials. Google Scholar
  49. 49.
    Ridgeon EE, Young PJ, Bellomo R, Mucchetti M, Lembo R, Landoni G. The fragility index in multicenter randomized controlled critical care trials. Crit Care Med. 2016;44(7):1278–84.CrossRefPubMedGoogle Scholar
  50. 50.
    Evaniew N, Files C, Smith C, Bhandari M, Ghert M, Walsh M, et al. The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey. Spine J. 2015;15(10):2188–97.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science + Business Media New York 2016

Authors and Affiliations

  1. 1.Department of Clinical Epidemiology and BiostatisticsMcMaster UniversityHamiltonCanada

Personalised recommendations