Analysis Issues – A Lot of Choices

In the statistical analysis of data from a clinical trial, one would expect the same set of data to lead to the same result. However analysts have many choices – what statistical test to use, what type of data to analyze, whether the underlying mathematical assumptions of the tests they use were met, etc. Depending on those decisions, different conclusions about a treatment's efficacy and safety could be supported. Furthermore, today the recommended research practice for analyzing a study is the intention-to-treat approach whereby all subjects the researcher intended to treat are included in the analysis, even if they didn't meet the protocol requirements or left the trial prematurely. This approach can cause a bias, but the alternative that allows researcher to decide who should be dropped and who should remain in an analysis can also introduce bias. The importance of statistical decisions is reflected in an astonishing report claiming that over half of all medical research findings are false. Too few subjects, small differences between treatments, the number of tests preformed and the incorrect interpretation of probabilities were used to support that premise.

Keywords

Intent-to-treat analysis multiple testing statistical assumptions statistical methods statistical tests 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Chapter 14 — Analysis Issues

Cited References

  1. Altman D. The scandal of poor medical research. Br Med J 1994:308;283–284.Google Scholar
  2. CONSORT Handbook. 12. Statistical methods. http://www.consort-statement.org/index. aspx?o=1029 Jan 3, 2003.
  3. Djulbegovic B, Iztok H. When should potentially false research findings be considered acceptable? PLoS Med 2007:4;e26.PubMedCrossRefGoogle Scholar
  4. Goodman S, Greenland S. Why most published research findings are false: problems in the analysis. PLoS Med 2007:4;e168.PubMedCrossRefGoogle Scholar
  5. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. Br Med J 1999:319;670–674.Google Scholar
  6. Ioannidis J. Why most published research findings are false. PLoS Med 2005:2;e124.PubMedCrossRefGoogle Scholar
  7. Ioannidis J. Evolution and translation of research findings: from bench to where? PLoS Clin Trials 2006:1;e36.PubMedCrossRefGoogle Scholar
  8. Johansen H, Gøtzsche P. Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. JAMA 1999:282;1752–1759.PubMedCrossRefGoogle Scholar
  9. Pauker S. The clinical interpretation of research. PLoS Med 2005:2;e395.PubMedCrossRefGoogle Scholar
  10. PLoS Medicine Editors. Minimizing mistakes and embracing uncertainty. PLoS Med 2005:2;e272.CrossRefGoogle Scholar
  11. Shaywitz D. Science and shams. Boston Globe. http://www.boston.com/news/science/arti-cles/2006/07/27/science_and_shams/ July 27, 2006.
  12. Shrier I. Power, reliability, and heterogeneous results. PLoS Med 2005:2;e386.PubMedCrossRefGoogle Scholar
  13. Wren J. Truth, probability, and frameworks. PLoS Med 2005:2;e361.PubMedCrossRefGoogle Scholar
  14. Yusuf S, Wittes J, Probstfield J, et al. Analysis and interpretation of treatment effects in subgroups of patient. JAMA 1991:266;93–98.PubMedCrossRefGoogle Scholar

General References

  1. Assmann S, Pocock S, Enos L, et al. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000:355;1064–1069.PubMedCrossRefGoogle Scholar
  2. Cochrane Handbook. 8.4 Intention to treat issues. http://www.cochrane.dk/cochrane/handbook/8_ analysing_and_presenting_results/8.4_intention_to_treat_issues.htm Sep 7, 2004.
  3. Ellenberg JH. Intent-to-treat analysis versus as-treated analysis., Newport Beach, CA: Drug Information Association, Jan 30, 1994.Google Scholar
  4. Ioannidis J. Author's reply. PLoS Med 2005:2;e398.CrossRefGoogle Scholar
  5. Johansen H, Gøtzsche P. Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. JAMA 1999:282;1752–1759.PubMedCrossRefGoogle Scholar
  6. Science Daily. Is most published research really false?http://72.14.205.104/search?q=cache: FEm9qpS9llcJ:www.sciencedaily.com/releases/2007/02/070227105745.htm + %22Is + most + published + research + really + false%3F%22&hl=en&ct=clnk&cd=1&gl=us Feb 27, 2007.
  7. Simon S. Statistical evidence in medical trials. Oxford: Oxford University Press, 2006.Google Scholar

Copyright information

© Springer Science + Business Media B.V 2009

Personalised recommendations