Methods for Assessing the Credibility of Clinical Trial Outcomes

Abstract

Credibility—the believability of new findings in the light of current knowledge—is a key issue in the assessment of clinical trial outcomes. Yet, despite the growth of evidence-based medicine, credibility is usually dealt with in a broad-brush and qualitative fashion. This paper describes how Bayesian methods lead to quantitative credibility assessments that take explicit account of prior insights and experience. A simple technique based on the concept of the critical prior interval (CPI) is presented, which allows rapid credibility assessment of trial outcomes reported in the standard format of odds ratios and 95% confidence intervals. The critical prior interval is easily determined via a graph, and provides clinicians with an explicit and objective baseline on which to base their assessment of credibility. The use of the critical prior interval is demonstrated through several working examples.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Egger M, Schneider M, Davey Smith G. Meta-analysis: Spurious precision? Br Med J. 1998:316:140–144.

    CAS  Article  Google Scholar 

  2. 2.

    Freedman D, Pisani R, Purves R, Statistics. (3rd Ed) New York, NY: Norton; 1998: Chapter 29.

    Google Scholar 

  3. 3.

    O’Hagan A. Kendall’s Advanced Theory of Statistics. Vol 2B: Bayesian Inference. London: Arnold; 1994.

    Google Scholar 

  4. 4.

    Lilford RJ, Braunholtz D. The statistical basis of public policy: a paradigm shift is overdue. Br Med J. 1996;313:603–607.

    CAS  Article  Google Scholar 

  5. 5.

    Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. An introduction to Bayesian methods in health technology assessment. Br Med J. 319:508-512

  6. 6.

    Matthews RAJ. Facts versus Factions: the use and abuse of subjectivity in scientific research. Cambridge: European Science and Environment Forum; 1998. Reprinted in Rethinking Risk and the Precautionary Principle. Morris, J, Ed. Oxford: Butter-worth; 2000: 247-282. Available online at: http://ourworld.compuserve.com/homepages/rajm/openesef. htm.

    Google Scholar 

  7. 7.

    Matthews RAJ. Why should clinicians care about Bayesian methods? J Stat Inf Plan. 2001;94:43–58. See also discussion, 59-71.

    Article  Google Scholar 

  8. 8.

    Pocock SJ, Spiegelhalter DJ. Letter (untitled). Br Med J. 1992;305:1015.

    CAS  Article  Google Scholar 

  9. 9.

    GREAT Group. Feasibility, safety and efficacy of domiciliary thrombolysis by general practitioners: Grampian region early anistreplase trial. Br Med J. 1992;305:548.

    Article  Google Scholar 

  10. 10.

    Morrison LJ, Verbeek R, McDonald AC, Sawadsky BV, Cook DJ. Mortality and prehospital thrombolysis for acute myocardial infarction: a meta-analysis. JAMA. 2000;283:2686–2692.

    CAS  Article  Google Scholar 

  11. 11.

    The Subcutaneous Sumatriptan International Study Group. Treatment of migraine attacks with sumatriptan. N Engl J Med. 1991;325:316–321.

    Article  Google Scholar 

  12. 12.

    Roberto Latini R, et al. Clinical effects of early angiotensin-converting enzyme inhibitor treatment for acute myocardial infarction are similar in the presence and absence of aspirin: Systematic over-view of individual data from 96,712 randomized patients. J Am Coll Cardio. 2000;35:1801–1807.

    Article  Google Scholar 

  13. 13.

    Egger M, Davey Smith G, Schneider M, Minder CE. Bias in meta-analysis detected by a simple graphical test. Br Med J. 1997;315:629–634.

    CAS  Article  Google Scholar 

  14. 14.

    Lee PM. Bayesian Statistics: An Introduction. 2nd Ed. London: Arnold; 1997.

    Google Scholar 

  15. 15.

    Good U. Probability and the Weighing of Evidence. London: Griffin; 1950.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Robert A. J. Matthews.

Additional information

Based on a presentation from the DIA Workshop “Statistical Methodology in Clinical R&D,” April 2–4, 2001, Vienna, Austria.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Matthews, R.A.J. Methods for Assessing the Credibility of Clinical Trial Outcomes. Ther Innov Regul Sci 35, 1469–1478 (2001). https://doi.org/10.1177/009286150103500442

Download citation

Key Words

  • Credibility
  • Critical prior interval
  • Bayesian methods