International Journal of Colorectal Disease

, Volume 22, Issue 11, pp 1369–1376 | Cite as

Methodological quality of randomised controlled trials comparing short-term results of laparoscopic and conventional colorectal resection

  • Wolfgang Schwenk
  • Oliver Haase
  • Nina Günther
  • Jens Neudecker
Original Article



Randomised, controlled trials (RCT) and systematic reviews of RCT with meta-analysis are considered to be of highest methodological quality and therefore are given the highest level of evidence (Ia/b). Although, “low-quality” RCT may be downgraded to level of evidence IIb, the methodological quality of each individual RCT is not respected in detail in this classification of the level of evidence.

Materials and methods

Within a systematic Cochrane Review of RCT on short-term benefits of laparoscopic or conventional colorectal resections, the methodological quality of all included RCT was evaluated. All RCT were assessed by the Evans and Pollock questionnaire (E and P increasing quality from 0–100) and the Jadad score (increasing quality from 0–5).


Publications from 28 RCT printed from 1996 to 2005 were included in the analysis. Methodological quality of RCT was only moderate [E & P 55 (32–84); Jadad 2 (1–5)]. There was a significant correlation between the E & P and the Jadad score (r = 0.788; p < 0.001). Methodological quality of RCT slightly increased with increasing number of patients included (r = 0.494; p = 0.009) and year of publication (r = 0.427; p = 0.03). Meta-analysis of all RCT yielded clinically relevant differences for overall and local morbidity when compared to meta-analysis of “high-quality” (E & P > 70) RCT only.


The methodological quality of reports of RCT comparing laparoscopic and open colorectal resection varies considerably. In a systematic review, methodological quality of RCT should be assessed because meta-analysis of “high-quality” RCT may yield different results than meta-analysis of all RCT.


Randomised controlled trials Methodological quality Scores Evidence-based medicine 


Conflict of interest and funding

The authors state that there are no commercial or other associations that might pose a conflict of interest in connection with submitted material. There was no external funding of any kind supporting the work.


  1. 1.
    Sackett DL (1986) Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest 89(2):S2–S3CrossRefGoogle Scholar
  2. 2.
    Cook DJ, Guyatt GH, Laupacis A, Sackett DL (1992) Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest 102(4):S305–S311Google Scholar
  3. 3.
    Oxford Centre for Evidence-Based Medicine (2005) Levels of evidence and grades of recommendation. Centre for Evidence-Based Medicine;
  4. 4.
    Schwenk W, Haase O, Neudecker J, Muller JM (2005) Short term benefits for laparoscopic colorectal resection. Cochrane Database Syst Rev (3):CD003145Google Scholar
  5. 5.
    Basse L, Jakobsen DH, Bardram L, Billesbolle P, Lund C, Mogensen T et al (2005) Functional recovery after open versus laparoscopic colonic resection: a randomized, blinded study. Ann Surg 241(3):416–423CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Guillou PJ, Quirke P, Thorpe H, Walker J, Jayne DG, Smith A et al. (2005) Short-term endpoints of conventional versus laparoscopic assisted surgery in patients with colorectal cancer (MRC CLASICC trial): multicentre, randomised controlled trial. Lancet 365:1718–1726CrossRefPubMedGoogle Scholar
  7. 7.
    The Colon Cancer Open or Laparoscopic Study Group (COLOR) (2005) Laparoscopic surgery versus open surgery for colon cancer: short-term outcomes of a randomised trial. Lancet Oncol 6:477–484CrossRefGoogle Scholar
  8. 8.
    Evans M, Pollock AV (1985) A score system for evaluating random control clinical trials of prophylaxis of abdominal surgical wound infection. Br J Surg 72:256–260CrossRefPubMedGoogle Scholar
  9. 9.
    Jadad A, Moore A (1996) Assessing the quality of reports of randomized clinical trials: Is blindness necessary? Control Clin Trials 17:1–12CrossRefPubMedGoogle Scholar
  10. 10.
    Alderson P, Green S, Higgins JPT (2004) Cochrane Reviewer’s Handbook 4.2.2. Wiley, Chichester, UK, (updated March 2004)Google Scholar
  11. 11.
    DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7:177–188CrossRefPubMedGoogle Scholar
  12. 12.
    Spilker B (1991) Guide to clinical trials, 1st ed. Raven PressGoogle Scholar
  13. 13.
    Moher D, Jadad AR, Tugwell P (1996) Assessing the quality of randomized controlled trials. Current issues and future directions. Int J Technol Assess Health Care 12(2):195–208CrossRefPubMedGoogle Scholar
  14. 14.
    Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S (1995) Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 16(1):62–73CrossRefPubMedGoogle Scholar
  15. 15.
    Emerson JD, Burdick E, Hoaglin DC, Mosteller F, Chalmers TC (1990) An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Control Clin Trials 11(5):339–352CrossRefPubMedGoogle Scholar
  16. 16.
    Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273(5):408–412CrossRefPubMedGoogle Scholar
  17. 17.
    Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C et al (2006) Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 185(5):263–267PubMedGoogle Scholar
  18. 18.
    Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M et al. (1998) Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352(9128):609–613CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  • Wolfgang Schwenk
    • 1
  • Oliver Haase
    • 1
  • Nina Günther
    • 1
  • Jens Neudecker
    • 1
  1. 1.Universitätsklinik für Allgemein Visceral-, Gefäss- und ThoraxchirurgieCharité-Universitätsmedizin BerlinBerlinGermany

Personalised recommendations