How to Interpret Results?

  • Laurence Collette


Every year, a mass of new information is published in the urologic literature that brings varied levels of scientific evidence to the medical community. This information is unfortunately not always objectively presented, despite the peer-review process implemented by most journals and congresses and the widespread adoption of reporting guidelines such as CONSORT. Intentional or more often unintentional misuse or misinterpretation of statistics, inadequate trial methodology, the natural tendency of humans to seek confirmation of their prior beliefs and to give more focus to the more statistically significant findings are the most commonly encountered flaws in the medical literature. In this chapter, we will illustrate some of these pitfalls, with the aim to exercise the reader to critical thinking in appraising published research. In these examples, we will focus on publications reporting comparisons between therapeutic interventions for prostate cancer, as modern clinical trials are becoming increasingly complex and therefore more difficult to appraise for the nonstatistically trained reader.


Prostate Cancer Radical Prostatectomy Androgen Deprivation Therapy Advanced Prostate Cancer Biochemical Failure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Anderson JR, Cain KC, Gelber RD (1983) Analysis of survival by tumor response. J Clin Oncol 1(11):710–719PubMedGoogle Scholar
  2. Anscher MS, Robertson CN, Prosnitz R (1995) Adjuvant radiotherapy for pathologic stage T3/4 adenocarcinoma of the prostate: ten-year update. Int J Radiat Oncol Biol Phys 33:37–43PubMedCrossRefGoogle Scholar
  3. Armstrong AJ, Febbo PG (2009) Using surrogate endpoints to predict clinical benefit in men with castration-­resistant prostate cancer: an update and review of the literature. Oncologist 14:816–827PubMedGoogle Scholar
  4. Bedard PL, Kryzanowska MK, Pintilie M et al (2007) Statistical power of negative randomized controlled trials presented at American society for clinical oncology annual meetings. J Clin Oncol 25(23):3482–3487PubMedCrossRefGoogle Scholar
  5. Bhattacharya S, Fyfe G, Gray RJ et al (2009) Role of ­sensitivity analyses in assessing progression-free survival in late-stage oncology trials. J Clin Oncol 27:5958–5964PubMedCrossRefGoogle Scholar
  6. Bolla M, van Poppel H, Collette L et al (2005) Postoperative radiotherapy after radical prostatectomy: a randomized controlled trial (EORTC trial 22911). Lancet 366:572–578PubMedCrossRefGoogle Scholar
  7. Bolla M, de Reijke THM, Van Tienhoven G et al (2009) Duration of androgen suppression in the treatment of prostate cancer. N Engl J Med 364(24):2516–2527CrossRefGoogle Scholar
  8. Bolla M, Van Tienhoven G, Warde P et al (2010) External irradiation with or without long-term androgen suppression for prostate cancer with high metastatic risk:10-year results of an EORTC randomised study. Lancet Oncol 11:1066–1073PubMedCrossRefGoogle Scholar
  9. Breau RH, Carnat TA, Gaboury I (2006) Inadequate statistical power of negative clinical trials in the urological literature. J Urol 176:263–266PubMedCrossRefGoogle Scholar
  10. Briganti A, Karnes JR, Da Pozzo LF et al (2009) Two positive nodes represent a significant cut-off value for cancer specific survival in patients with node positive prostate cancer. A new proposal based on a two-institution experience on 703 consecutive N + patients treated with radical prostatectomy, extended pelvic lymph node dissection and adjuvant therapy. Eur Urol 55:261–270PubMedCrossRefGoogle Scholar
  11. Budiharto T, Perneel C, Haustermans K et al (2010) A multi-institutional analysis comparing adjuvant and salvage radiation therapy for high-risk prostate cancer patients with undetectable PSA after prostatectomy. Radiother Oncol 97:474–479PubMedCrossRefGoogle Scholar
  12. Carroll KJ (2007) Analysis of progression-free survival in oncology trials: some common statistical issues. Pharm Stat 6:99–113PubMedCrossRefGoogle Scholar
  13. Catton C, Gospodarowicz M, Warde P et al (2001) Adjuvant and salvage radiation therapy after radical prostatectomy for adenocarcinoma of the prostate. Radiother Oncol 59:51–60PubMedCrossRefGoogle Scholar
  14. Chang SL, Freedland SJ, Terris MK et al (2010) Freedom from a detectable ultrasensitive prostate-specific antigen at Two years after radical prostatectomy predicts a favorable clinical outcome: analysis of the SEARCH database. Urology 75(2):439–444PubMedCrossRefGoogle Scholar
  15. Collette L (2008) Prostate-specific antigen (PSA) as a surrogate endpoint for survival in prostate cancer clinical trials. Eur Urol 53:6–9PubMedCrossRefGoogle Scholar
  16. Cookson MS, Aus G, Burnett AL et al (2007) Variation in the definition of biochemical recurrence in patients treated for localized prostate cancer: the American urological association prostate guidelines for localized prostate cancer update panel report and recommendations for a standard in the reporting of surgical outcomes. J Urol 177:540–545PubMedCrossRefGoogle Scholar
  17. D’Agostino RB Jr (1998) Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Stat Med 17(19):2265–2281PubMedCrossRefGoogle Scholar
  18. D’Amico AV, Denham JW, Bolla M et al (2007) Short- vs. long-term androgen suppression plus external beam radiation therapy and survival in men of advanced age with node negative high risk adenocarcinoma of the prostate. Cancer 109(10):2004–2010PubMedCrossRefGoogle Scholar
  19. Dancey JE, Dodd LE, Ford R et al (2009) Recommendations for the assessment of progression in randomized cancer treatment trials. Eur J Cancer 45:281–289PubMedCrossRefGoogle Scholar
  20. Daskivich TJ, Regan MM, Oh WK (2006) Prostate specific antigen doubling time calculation: not as easy as 1,2, 4. J Urol 176(5):1927–1937PubMedCrossRefGoogle Scholar
  21. Denham JW, Steigler A, Wilcox C et al (2008) Time to biochemical failure and prostate-specific antigen doubling time as surrogates for prostate cancer-specific mortality: evidence from the TROG 96.01 randomised controlled trial. Lancet Oncol 9(11):1058–1068PubMedCrossRefGoogle Scholar
  22. Engel J, Bastian PJ, Baur H et al (2010) Survival benefit of radical prostatectomy in lymph node positive patients with prostate cancer. Eur Urol 55:754–761CrossRefGoogle Scholar
  23. Freidlin B, Korn EL, Hunsberger S et al (2007) Proposal for the use of progression-free survival in unblinded randomized trials. J Clin Oncol 25:2122–2126PubMedCrossRefGoogle Scholar
  24. Gignac GA, Morris MJ, Heller G et al (2008) Assessing outcomes in prostate cancer clinical trials: a twenty-first century tower of Babel. Cancer 113(5):966–974PubMedCrossRefGoogle Scholar
  25. Howland RH (2011) What you see depends on where you’re looking and how you look at it: publication bias and outcome reporting bias. J Psychosoc Nurs Ment Health Serv 15:1–3Google Scholar
  26. Lin K, Lee SP, Steinberg ML (2010) Selection bias clouds apparent benefit of longer hormone duration. J Clin Oncol 28(5):e79PubMedCrossRefGoogle Scholar
  27. Mills JL (1993) Data torturing. N Engl J Med 329(16):1196–1199PubMedCrossRefGoogle Scholar
  28. Nielsen ME, Makarov DV, Humphreys E et al (2008) Is it possible to compare PSA recurrence-free survival after surgery and radiotherapy using revised ASTRO criterion “nadir + 2”? Urology 72:389–393PubMedCrossRefGoogle Scholar
  29. Ocana A, Tannock IF (2011) When are “positive” trials in oncology truly positive? J Natl Cancer Inst 103(1):16–20PubMedCrossRefGoogle Scholar
  30. Oei SG, Helmerhorst FM, Keirse MNC (1999) Postcoital test should be performed as routine infertility test. BMJ 318:1008–1009Google Scholar
  31. Panageas KS, Ben-Porat L, Dickler MN et al (2007) When you look matters: the effect of assessment schedule on progression-free survival. J Natl Cancer Inst 99(6):428–432PubMedCrossRefGoogle Scholar
  32. Patel AR, Stephenson AJ (2011) Radiation therapy for prostate cancer after radical prostatectomy: adjuvant or salvage? Nat Rev Urol 8:385–392PubMedCrossRefGoogle Scholar
  33. Pazdur R (2008) Endpoints for assessing drug activity in clinical trials. Oncologist 13(suppl 2):19–21PubMedCrossRefGoogle Scholar
  34. Ray ME, Bae K, Hussain MH, Hanks GE et al (2009) Potential surrogate endpoints for prostate cancer survival: analysis of a phase III randomized trial. J Natl Cancer Inst 101(4):228–236PubMedCrossRefGoogle Scholar
  35. Roach MI, Hanks G, Thames HJ et al (2006) Defining biochemical failure following radiotherapy with or without hormonal therapy in men with clinically localized prostate cancer: recommendations of the RTOG–ASTRO phoenix consensus conference. Int J Radiat Oncol Biol Phys 65:965–974PubMedCrossRefGoogle Scholar
  36. Rui Wang MS, Lagakos SW, Ware JH et al (2007) Statistics in medicine – reporting of subgroup analyses in clinical trials. N Engl J Med 357:2189–2194PubMedCrossRefGoogle Scholar
  37. Sandler HM, Pajak TF, Hanks GE et al (2003) Can biochemical failure (ASTRO definition) be used as a surrogate endpoint for prostate cancer survival in phase III localized prostate cancer clinical trials? Analysis of RTOG protocol 92–02. J Clin Oncol 22:381 (abstract 1529)CrossRefGoogle Scholar
  38. Scher HI, Halabi S, Tannock I et al (2008) Design and end points of clinical trials for patients with progressive prostate cancer and castrate levels of testosterone: recommendations of the prostate cancer clinical trials working group. J Clin Oncol 26:1148–1159PubMedCrossRefGoogle Scholar
  39. Schroeder FH, Whelan P, de Reijke TM et al (2004) Metastatic prostate cancer treated by flutamide versus cyproterone acetate: final analysis of the European organisation for research and treatment of cancer (EORTC) protocol 30892. Eur Urol 45(4):457–464CrossRefGoogle Scholar
  40. Schultz KF, Grimes DA (2005) Multiplicity in randomized trials II: subgroup and interim analyses. Lancet 365:1657–1661CrossRefGoogle Scholar
  41. Sharrock G, Graf C, Fitzpatrick JM (2011) The role of ethical publishing in promoting the evidence-based practice of urology. World J Urol 29:319–324PubMedCrossRefGoogle Scholar
  42. Souhami L, Bae K, Pilepich M et al (2009a) Impact of the duration of adjuvant hormonal therapy in patients with locally Advanced prostate cancer treated with radiotherapy: a secondary analysis of RTOG 85–31. J Clin Oncol 27(13):2137–2143PubMedCrossRefGoogle Scholar
  43. Souhami L, Bae K, Sandler H (2009b) Reply to Collette et al. and Tangen et al. J Clin Oncol 27(33):e204CrossRefGoogle Scholar
  44. Studer UE, Collette L, Sylvester R (2010) Can radical prostatectomy benefit patients despite the presence of regional metastases? Eur Urol 57:762–763PubMedCrossRefGoogle Scholar
  45. Tannock IF (1996) False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons. J Natl Cancer Inst 88:206–207PubMedCrossRefGoogle Scholar
  46. Tannock IF (2003) From evidence-based medicine to clinical practice: not always straightforward. Eur J Cancer Suppl 1(5):93–101CrossRefGoogle Scholar
  47. Taylor N, Kelly JF, Kuban DA (2003) Adjuvant and salvage radiotherapy after radical prostatectomy for prostate cancer. Int J Radiat Oncol Biol Phys 56:755–763PubMedCrossRefGoogle Scholar
  48. Thompson IM, Tangen CM, Paradelo J et al (2006) Adjuvant radiotherapy for pathologically advanced prostate cancer: a randomized trial. JAMA 296:2329–2335PubMedCrossRefGoogle Scholar
  49. Thompson I, Thrasher JB, Aus G (2007) Guideline for the management of clinically localized prostate cancer: 2007 update. J Urol 177(6):2106–2131PubMedCrossRefGoogle Scholar
  50. Trabulsi EJ, Valicenti RK, Hanlon AL et al (2008) A multi-institutional matched-control analysis of ­adjuvant and salvage postoperative radiation therapy for pT3–4N0 prostate cancer. Urology 72:1298–1304PubMedCrossRefGoogle Scholar
  51. U.S. Department of Health and Human Services Food and Drug Administration (2007) Guidance for industry clinical trial endpoints for the approval of cancer drugs and biologics. ( Accessed on 21 Jan 2012)
  52. Valicenti RK, Gomelia LG, Ismail M et al (1999) The efficacy of early adjuvant radiation therapy for pT3N0 prostate cancer: a matched-pair analysis. Int J Radiat Oncol Biol Phys 45:53–58PubMedCrossRefGoogle Scholar
  53. Valicenti R, Deslivio M, Hanks G et al (2006) Posttreatment prostatic-specific antigen doubling time as a surrogate endpoint for prostate cancer-specific survival: an ­analysis of radiation therapy oncology group proto­col 92–02. Int J Radiat Oncol Biol Phys 66(4):1064–1071PubMedCrossRefGoogle Scholar
  54. Wiegel T, Bottke D, Steiner U et al (2009) Phase III postoperative adjuvant radiotherapy after radical prostatectomy compared with radical prostatectomy alone in pT3 prostate cancer with postoperative undetectable prostate- specific antigen: ARO 96–02/AUO AP 09/95. J Clin Oncol 27:2924–2930PubMedCrossRefGoogle Scholar
  55. Wunsch H, Linde-Zwirble W, Angus DC (2006) Methods to adjust for bias and confounding in critical care health services research involving observational data. J Crit Care 21(1):1–7PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Statistics DepartmentEuropean Organisation for Research and Treatment of Cancer, HeadquartersBrusselsBelgium

Personalised recommendations