Canadian Journal of Anesthesia

, Volume 54, Issue 5, pp 380–388 | Cite as

Publication bias in the medical literature: A review by a Canadian research ethics board

Special Article

Abstract

Background

We reviewed the publication record of all protocols submitted to the Capital District Health Authority Research Ethics Board (REB) in Halifax, Nova Scotia, for the period 1995–1996. Because of a heightened awareness of the issue, we hypothesized that there would be less publication bias (a failure to report negative results) and a higher publication rate from completed studies, than previously reported.

Methods

Closed studies were identified from the REB database. Publications were identified by the investigators, requests from sponsors, and a literature review. For each publication, we identified authors, title, journal, number of subjects enrolled, and whether or not the publication was a report of a randomized clinical trial. Comparisons were done using a Student’s t test, the Chi-square statistic, or Fisher’s exact test as appropriate.

Results

From the database of closed studies, 106 remained unpublished, while completed investigations resulted in 84 publications (44% publication rate). The median time to publication was 32.5 months. Publication of statistically significant results occurred in 71/84 trials. Publication of protocols submitted by departments ranged from 91% (anesthesia; 10/11) to 25% [nursing; 2/8 (P < 0.05)]. Trials investigating new drugs in Phase 3 or 4 studies were more likely to be published than trials investigating agents in Phase 1 or 2 (P < 0.05), and were less likely to be published if sponsored by a pharmaceutical company (P < 0.05).

Conclusions

Publication bias continues to be a problem, particularly for early phase investigative studies. Our results suggest that a different approach is required to reduce publication bias. The role that REBs and peer-reviewed journals might play requires further exploration.

Biais de publication dans la littérature médicale: Un compte rendu d’un Comité d’éthique de recherche Canadien

Résumé

Contexte

Nous avons passé en revue le fichier de publications de tous les protocoles soumis au Comité d’éthique de recherche (CER) du Capital District Health Authority à Halifax, Nouvelle Écosse, pour la période 1995–1996. Étant fortement sensibilisés à la question, nous avons émis l’hypothèse qu’il y aurait moins de biais de publication (le fait que des résultats négatifs ne soient pas rapportés) ainsi qu’un taux de publication plus élevé d’études achevées que ce qui avait été précédemment rapporté.

Méthode

Les études complétées ont été tirées de la base de données du CER. Les publications étaient identifiables par les chercheurs, les requêtes de commanditaires, et une revue de la littérature. Nous avons identifié les auteurs, le titre, le périodique, le nombre de patients ayant participé à l’étude pour chaque publication, et nous avons déterminé si la publication était le compterendu d’une étude clinique randomisée ou non. Les comparaisons ont été effectuées en utilisant un test t de Student, la statistique khi-carré ou le test exact de Fisher, selon leur pertinence.

Résultats

Dans la base de données des études terminées, 106 n’étaient pas publiées, alors que les études complétées ont abouti à 84 publications (taux de publication de 44 %). Le temps médian jusqu’à la publication était de 32,5 mois. Des résultats statistiquement significatifs ont été publiés dans 71/84 des études. La publication de protocoles soumis par des départements allait de 25 % (soins infirmiers; 2/8)] à 91 % [anesthésie ; 10/11(P < 0,05)]. La probabilité de publication était plus élevée pour des études au sujet de nouveaux médicaments en phase 3 ou 4 d’étude que pour celles examinant des agents en phase 1 ou 2 (P < 0,05), mais moins élevée si l’étude était commanditée par une société pharmaceutique (P < 0,05).

Conclusion

Le biais de publication demeure un problème, particulièrement pour les études de recherche en phase précoce. Nos résultats suggèrent la nécessité d’approche différente si l’on veut réduire le biais de publication. Le rôle que pourraient jouer les CER et les revues pratiquant l’évaluation par des pairs requiert une exploration plus approfondie.

References

  1. 1.
    Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991; 337: 867–72.PubMedCrossRefGoogle Scholar
  2. 2.
    Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990; 263: 1385–9.PubMedCrossRefGoogle Scholar
  3. 3.
    Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 1992; 267: 374–8.PubMedCrossRefGoogle Scholar
  4. 4.
    Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR. Empirical assessment of effect of publication bias on meta-analyses. BMJ 2000; 320: 1574–7.PubMedCrossRefGoogle Scholar
  5. 5.
    Egger M, Smith GD. Misleading meta-analysis. BMJ 1995; 310: 752–4.PubMedGoogle Scholar
  6. 6.
    Antes G, Chalmers I. Under-reporting of clinical trials is unethical. Lancet 2003; 361: 978–9.PubMedCrossRefGoogle Scholar
  7. 7.
    Chalmers I. Underreporting research is scientific misconduct. JAMA 1990; 263: 1405–8.PubMedCrossRefGoogle Scholar
  8. 8.
    Mann H. Research ethics committees and public dissemination of clinical trial results. Lancet 2002; 360: 406–8.PubMedCrossRefGoogle Scholar
  9. 9.
    Decullier E, Lheritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ 2005; 331: 19.PubMedCrossRefGoogle Scholar
  10. 10.
    Sune-Martin P, Montoro-Ronsano JB. Role of a research ethics committee in follow-up and publication of results. Lancet 2003; 361: 2245–6.PubMedCrossRefGoogle Scholar
  11. 11.
    Pich J, Carne X, Arnaiz JA, Gomez B, Trilla A, Rodes J. Role of a research ethics committee in follow-up and publication of results. Lancet 2003; 361: 1015–6.PubMedCrossRefGoogle Scholar
  12. 12.
    Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 1997; 315: 640–5.PubMedGoogle Scholar
  13. 13.
    De Angelis CD, Drazen JM, Frizelle FA, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA 2004; 292: 1363–4.CrossRefGoogle Scholar
  14. 14.
    De Angelis CD, Drazen JM, Frizelle FA, et al. Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors. JAMA 2005; 293: 2927–9.CrossRefGoogle Scholar
  15. 15.
    Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, and Social Sciences and Humanities Research Council of Canada. Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humanities Research Council of Canada, Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans. 1998 (with 2000, 2002 and 2005 amendments). Interagency Advisory Panel on Research Ethics (PRE) 2005. Available from URL; www.pre.ethics.gc.ca/english/ policystatement.Google Scholar
  16. 16.
    The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). E6(R1): Good Clinical Practice : Consolidated Guideline. The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) 2006. Available from URL; www.ich.org.Google Scholar
  17. 17.
    Government of Canada. Division 5 Drugs For Clinical Trials Involving Human Subjects. Food and Drugs Act ( R.S., 1985, c.F-27 ) 2006. Available from URL; http://laws.justice.gc.ca/en/F-27/C.R.C.-c.870/238148.html.Google Scholar
  18. 18.
    Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17: 1–12.PubMedCrossRefGoogle Scholar
  19. 19.
    Thomson ISI (Institute for Scientific Information). Journal Citation Report. Selected Scientific Journals 2001. Available from URL; www.genebee.msu.su/ journals.Google Scholar
  20. 20.
    Herxheimer A. Open access to industry’s clinically relevant data. BMJ 2004; 329: 64–5.PubMedCrossRefGoogle Scholar
  21. 21.
    Psaty BM, Furberg CD. COX-2 inhibitors--lessons in drug safety. N Engl J Med 2005; 352: 1133–5.PubMedCrossRefGoogle Scholar
  22. 22.
    Chopra SS. MS JAMA: industry funding of clinical trials: benefit or bias? JAMA. 2003; 290: 113–4.PubMedCrossRefGoogle Scholar
  23. 23.
    Goodyear M. Learning from the TGN1412 trial. BMJ 2006; 332: 677–8.PubMedCrossRefGoogle Scholar
  24. 24.
    Suntharalingam G, Perry MR, Ward S, et al. Cytokine storm in a phase 1 trial of the anti-CD28 monoclonal antibody TGN1412. N Engl J Med 2006; 355: 1018–28.PubMedCrossRefGoogle Scholar
  25. 25.
    Zarin DA, Tse T, Ide NC. Trial Registration at ClinicalTrials.gov between May and October 2005. N Engl J Med 2005; 353: 2779–87.PubMedCrossRefGoogle Scholar
  26. 26.
    Steinbrook R. Gag clauses in clinical-trial agreements. N Engl J Med 2005; 352: 2160–2.PubMedCrossRefGoogle Scholar
  27. 27.
    Olivieri NF, Brittenham GM, McLaren CE, et al. Long-term safety and effectiveness of iron-chelation therapy with deferiprone for thalassemia major. N Engl J Med 1998; 339: 417–23.PubMedCrossRefGoogle Scholar
  28. 28.
    Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004; 171: 735–40.PubMedGoogle Scholar
  29. 29.
    Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 2005; 330: 753.PubMedCrossRefGoogle Scholar
  30. 30.
    Garland EJ. Facing the evidence: antidepressant treatment in children and adolescents. CMAJ 2004; 170: 489–91.PubMedGoogle Scholar
  31. 31.
    Krzyzanowska MK, Pintilie M, Tannock IF. Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 2003; 290: 495–501.PubMedCrossRefGoogle Scholar
  32. 32.
    Thornton A, Lee P. Publication bias in meta-analysis: its causes and consequences. J Clin Epidemiol 2000; 53: 207–16.PubMedCrossRefGoogle Scholar
  33. 33.
    Gilbody SM, Song F, Eastwood AJ, Sutton A. The causes, consequences and detection of publication bias in psychiatry. Acta Psychiatr Scand 2000; 102: 241–9.PubMedCrossRefGoogle Scholar
  34. 34.
    Altman DG. Poor-quality medical research: what can journals do? JAMA 2002; 287: 2765–7.PubMedCrossRefGoogle Scholar
  35. 35.
    Massie BM. Negative, neutral, and discordant results: what should an editor do? J Card Fail 2003; 9: 77–9.PubMedCrossRefGoogle Scholar
  36. 36.
    Shields PG. Publication bias is a scientific problem with adverse ethical outcomes: the case for a section for null results. Cancer Epidemiol Biomarkers Prev 2000; 9: 771–2.PubMedGoogle Scholar
  37. 37.
    Yaphe J, Edman R, Knishkowy B, Herman J. The association between funding by commercial interests and study outcome in randomized controlled drug trials. Fam Pract 2001; 18: 565–8.PubMedCrossRefGoogle Scholar
  38. 38.
    Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine--selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 2003; 326: 1171–3.PubMedCrossRefGoogle Scholar
  39. 39.
    Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003; 326: 1167–70.PubMedCrossRefGoogle Scholar
  40. 40.
    Hailey D. Scientific harassment by pharmaceutical companies: time to stop. CMAJ 2000; 162: 212–3.PubMedGoogle Scholar
  41. 41.
    Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA 2003; 290: 921–8.PubMedCrossRefGoogle Scholar
  42. 42.
    Rennie D. Thyroid storm. JAMA 1997; 277: 1238–43.PubMedCrossRefGoogle Scholar
  43. 43.
    Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med 1996; 124: 485–9.PubMedGoogle Scholar

Copyright information

© Canadian Anesthesiologists 2007

Authors and Affiliations

  • Richard Hall
    • 1
    • 3
  • Cecilia de Antueno
    • 2
  • Adam Webber
    • 1
  1. 1.Dalhousie University and the Capital District Health AuthorityHalifaxCanada
  2. 2.Capital District Health Authority Research Ethics BoardHalifaxCanada
  3. 3.Department of AnesthesiaQueen Elizabeth II Health Sciences CentreHalifaxCanada

Personalised recommendations