Methods for Assessing the Credibility of Clinical Trial Outcomes

  • Robert A. J. MatthewsEmail author


Credibility—the believability of new findings in the light of current knowledge—is a key issue in the assessment of clinical trial outcomes. Yet, despite the growth of evidence-based medicine, credibility is usually dealt with in a broad-brush and qualitative fashion. This paper describes how Bayesian methods lead to quantitative credibility assessments that take explicit account of prior insights and experience. A simple technique based on the concept of the critical prior interval (CPI) is presented, which allows rapid credibility assessment of trial outcomes reported in the standard format of odds ratios and 95% confidence intervals. The critical prior interval is easily determined via a graph, and provides clinicians with an explicit and objective baseline on which to base their assessment of credibility. The use of the critical prior interval is demonstrated through several working examples.

Key Words

Credibility Critical prior interval Bayesian methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Egger M, Schneider M, Davey Smith G. Meta-analysis: Spurious precision? Br Med J. 1998:316:140–144.CrossRefGoogle Scholar
  2. 2.
    Freedman D, Pisani R, Purves R, Statistics. (3rd Ed) New York, NY: Norton; 1998: Chapter 29.Google Scholar
  3. 3.
    O’Hagan A. Kendall’s Advanced Theory of Statistics. Vol 2B: Bayesian Inference. London: Arnold; 1994.Google Scholar
  4. 4.
    Lilford RJ, Braunholtz D. The statistical basis of public policy: a paradigm shift is overdue. Br Med J. 1996;313:603–607.CrossRefGoogle Scholar
  5. 5.
    Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. An introduction to Bayesian methods in health technology assessment. Br Med J. 319:508-512Google Scholar
  6. 6.
    Matthews RAJ. Facts versus Factions: the use and abuse of subjectivity in scientific research. Cambridge: European Science and Environment Forum; 1998. Reprinted in Rethinking Risk and the Precautionary Principle. Morris, J, Ed. Oxford: Butter-worth; 2000: 247-282. Available online at: htm.Google Scholar
  7. 7.
    Matthews RAJ. Why should clinicians care about Bayesian methods? J Stat Inf Plan. 2001;94:43–58. See also discussion, 59-71.CrossRefGoogle Scholar
  8. 8.
    Pocock SJ, Spiegelhalter DJ. Letter (untitled). Br Med J. 1992;305:1015.CrossRefGoogle Scholar
  9. 9.
    GREAT Group. Feasibility, safety and efficacy of domiciliary thrombolysis by general practitioners: Grampian region early anistreplase trial. Br Med J. 1992;305:548.CrossRefGoogle Scholar
  10. 10.
    Morrison LJ, Verbeek R, McDonald AC, Sawadsky BV, Cook DJ. Mortality and prehospital thrombolysis for acute myocardial infarction: a meta-analysis. JAMA. 2000;283:2686–2692.CrossRefGoogle Scholar
  11. 11.
    The Subcutaneous Sumatriptan International Study Group. Treatment of migraine attacks with sumatriptan. N Engl J Med. 1991;325:316–321.CrossRefGoogle Scholar
  12. 12.
    Roberto Latini R, et al. Clinical effects of early angiotensin-converting enzyme inhibitor treatment for acute myocardial infarction are similar in the presence and absence of aspirin: Systematic over-view of individual data from 96,712 randomized patients. J Am Coll Cardio. 2000;35:1801–1807.CrossRefGoogle Scholar
  13. 13.
    Egger M, Davey Smith G, Schneider M, Minder CE. Bias in meta-analysis detected by a simple graphical test. Br Med J. 1997;315:629–634.CrossRefGoogle Scholar
  14. 14.
    Lee PM. Bayesian Statistics: An Introduction. 2nd Ed. London: Arnold; 1997.Google Scholar
  15. 15.
    Good U. Probability and the Weighing of Evidence. London: Griffin; 1950.Google Scholar

Copyright information

© Drug Information Association, Inc 2001

Authors and Affiliations

  1. 1.Department of Information EngineeringAston UniversityBirminghamEngland

Personalised recommendations