Skip to main content

Evaluating the PCPT risk calculator in ten international biopsy cohorts: results from the Prostate Biopsy Collaborative Group

World Journal of Urology volume 30, pages 181–187 (2012)Cite this article

Abstract

Objectives

To evaluate the discrimination, calibration, and net benefit performance of the Prostate Cancer Prevention Trial Risk Calculator (PCPTRC) across five European randomized study of screening for prostate cancer (ERSPC), 1 United Kingdom, 1 Austrian, and 3 US biopsy cohorts.

Methods

PCPTRC risks were calculated for 25,733 biopsies using prostate-specific antigen (PSA), digital rectal examination, family history, history of prior biopsy, and imputation for missing covariates. Predictions were evaluated using the areas underneath the receiver operating characteristic curves (AUC), discrimination slopes, chi-square tests of goodness of fit, and net benefit decision curves.

Results

AUCs of the PCPTRC ranged from a low of 56% in the ERSPC Goeteborg Rounds 2–6 cohort to a high of 72% in the ERSPC Goeteborg Round 1 cohort and were statistically significantly higher than that of PSA in 6 out of the 10 cohorts. The PCPTRC was well calibrated in the SABOR, Tyrol, and Durham cohorts. There was limited to no net benefit to using the PCPTRC for biopsy referral compared to biopsying all or no men in all five ERSPC cohorts and benefit within a limited range of risk thresholds in all other cohorts.

Conclusions

External validation of the PCPTRC across ten cohorts revealed varying degree of success highly dependent on the cohort, most likely due to different criteria for and work-up before biopsy. Future validation studies of new calculators for prostate cancer should acknowledge the potential impact of the specific cohort studied when reporting successful versus failed validation.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

References

  1. Thompson IM, Ankerst DP, Chi C, Goodman PJ, Tangen CM, Lucia MS, Feng Z, Parnes HL, Coltman CA Jr (2006) Assessing prostate cancer risk: results from the Prostate Cancer Prevention Trial. J Natl Cancer Inst 98:529–534

    PubMed  Article  Google Scholar 

  2. Vickers AJ, Cronin AM, Roobol MJ, Hugosson J, Jones JS, Kattan MW, Klein E, Hamdy F, Neal D, Donovan J, Parekh DJ, Ankerst D, Bartsch G, Klocker H, Horninger W, Benchikh A, Salama G, Villers A, Freedland SJ, Moreira DM, Schroeder FH, Lilja H (2010) The relationship between prostate-specific antigen and prostate cancer risk: the Prostate Biopsy Collaborative Group. Clin Cancer Res 16:4374–4381

    PubMed  Article  CAS  Google Scholar 

  3. van Buuren S (2007) Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res 16:219–242

    PubMed  Article  Google Scholar 

  4. Janssen KJM, Donders ART, Harrell FE Jr, Vergouwe Y, Chen Q, Grobbee DE, Moons KGM (2010) Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol 63:721–727

    PubMed  Article  Google Scholar 

  5. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845

    PubMed  Article  CAS  Google Scholar 

  6. Steyerberg EW (2010) Clinical prediction models. Springer, New York, pp 270–271

    Google Scholar 

  7. Vickers AJ, Elkin EB (2006) Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 26:565–574

    PubMed  Article  Google Scholar 

  8. Cavadas V, Osório L, Sabell F, Teves F, Branco F, Silva-Ramos M (2010) Prostate cancer prevention trial and European randomized study of screening for prostate cancer risk calculators: a performance comparison in a contemporary screened cohort. Eur Urol 58:551–558

    PubMed  Article  Google Scholar 

  9. Eyre SJ, Ankerst DP, Wei JT, Nair PV, Regan MM, Bueti G, Tang J, Rubin MA, Kearney M, Thompson IM, Sanda MG (2009) Validation in a multiple urology practice setting of the prostate cancer prevention trial calculator for predicting prostate cancer detection. J Urol 182:2653–2658

    PubMed  Article  Google Scholar 

  10. Hernandez DJ, Han M, Humphreys EB, Mangold LA, Taneja SS, Childs SJ, Bartsch G, Partin AW (2009) Predicting the outcome of prostate biopsy: comparison of a novel logistic regression-based model, the prostate cancer risk calculator, and prostate-specific antigen level alone. BJU Int 103:609–614

    PubMed  Article  Google Scholar 

  11. Nguyen CT, Yu C, Moussa A, Kattan MW, Jones JS (2010) Performance of prostate cancer prevention trial risk calculator in a contemporary cohort screened for prostate cancer and diagnosed by extended prostate biopsy. J Urol 183:529–533

    PubMed  Article  Google Scholar 

  12. Oliveira M, Marques V, Carvalho AP, Santos A (2011) Head-to-head comparison of two online nomograms for prostate biopsy outcome prediction. BJU Int 107:1780–1783

    PubMed  Article  Google Scholar 

  13. Parekh DJ, Ankerst DP, Higgins BA, Hernandez J, Canby-Hagino E, Brand T, Troyer DA, Leach RJ, Thompson IM (2006) External validation of the prostate cancer prevention trial risk calculator in a screened population. Urology 68:1153–1155

    Article  Google Scholar 

  14. van den Bergh RC, Roobol MJ, Wolters T, van Leeuwen PJ, Schroder FH (2008) The prostate cancer prevention trial and European randomized study of screening for prostate cancer risk calculators indicating a positive prostate biopsy: a comparison. BJU Int 102:1068–1073

    PubMed  Article  Google Scholar 

  15. Ankerst DP, Groskopf J, Day JR, Blase A, Rittenhouse H, Pollock BH, Tangen C, Parekh D, Leach RJ, Thompson I (2008) Predicting prostate cancer risk through incorporation of prostate cancer gene 3. J Urol 180:1303–1308

    PubMed  Article  Google Scholar 

  16. Prior C, Guillen-Grima F, Robles JE, Rosell D, Fernandez-Montero JM, Agirre X, Catena R, Calvo A (2010) Use of a combination of biomarkers in serum and urine to improve detection of prostate cancer. World J Urol 28:681–686

    PubMed  Article  CAS  Google Scholar 

  17. Vickers AJ, Cronin AM (2010) Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology 76:1298–1301

    PubMed  Article  Google Scholar 

  18. Kattan MW (2011) Factors affecting the accuracy of prediction models limit the comparison of rival prediction models when applied to separate data sets. Eur Urol 59:566–567

    PubMed  Article  Google Scholar 

Download references

Acknowledgments

Statistics supported in part by funds from David H. Koch provided through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers SPORE grant from the U.S. National Cancer Institute [P50-CA92629], and a Cancer Center Support Grant for the Cancer Therapy and Research Center at the University of Texas Health Science Center at San Antonio [P30-CA054174]. Grants to support the work of the ERSPC include: European Union Grants SOC 95 35109, SOC 96 201869 05F022, SOC 97 201329, SOC 98 32241, the 6th Framework Program of the EU: PMark:LSHC-CT-2004-503011; and The Dutch Cancer Society (KWF 94-869, 98-1657, 2002-277, 2006-3518); The Netherlands Organisation for Health Research and Development (ZonMW-002822820, 22000106, 50-50110-98-311); The Prostate Cancer Research Foundation of Rotterdam (SWOP); Beckman-Coulter-Hybritech Inc; Abbott Pharmaceuticals, Sweden; Af Jochnick’s foundation; Catarina and Sven Hagstroms family foundation; Gunvor and Ivan Svensson’s foundation; Johanniterorden, King Gustav V Jubilée Clinic Cancer Research Foundation; Sahlgrenska University Hospital; Schering Plough, Sweden, Swedish Cancer Society (Contract numbers 09 0107, 080315 and 083455); and Wallac Oy, Turkku, Finland. The Tyrol study is supported by the International Agency for Research on Cancer, Lyon and the Tyrolean Prostate Cancer Early Detection Group. The SABOR project is supported by the San Antonio Center of Biomarkers of Risk for Prostate Cancer CA086402. The ProtecT study is funded by the UK NIHR Health Technology Assessment Programme (projects 96/20/06, 96/20/99).

Conflict of interest

The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. Department of Urology, University of Texas Health Science Center at San Antonio, 7703 Floyd Curl Dr., San Antonio, TX, 78229, USA

    Donna P. Ankerst, Ian M. Thompson & Dipen J. Parekh

  2. Technische Universitaet Muenchen, Garching, Germany

    Donna P. Ankerst & Andreas Boeck

  3. Durham VA Medical Center and Duke University, Durham, NC, USA

    Stephen J. Freedland & Daniel M. Moreira

  4. Memorial Sloan-Kettering Cancer Center, New York, NY, USA

    Angel M. Cronin, Hans Lilja & Andrew J. Vickers

  5. Erasmus Medical Center, Rotterdam, The Netherlands

    Monique J. Roobol & Fritz H. Schröder

  6. Sahlgrenska University Hospital, Goteborg, Sweden

    Jonas Hugosson

  7. Cleveland Clinic, Cleveland, OH, USA

    J. Stephen Jones, Michael W. Kattan & Eric A. Klein

  8. Oxford University, Oxford, UK

    Freddie Hamdy

  9. Cambridge University, Cambridge, UK

    David Neal

  10. Bristol University, Bristol, UK

    Jenny Donovan

  11. Innsbruck Medical University, Innsbruck, Austria

    Helmut Klocker & Wolfgang Horninger

  12. Hôpital Bichat-Claude Bernard, Paris, France

    Amine Benchikh

  13. Centre Hospitalier Intercommunal Castres-Mazamet, Castres, France

    Gilles Salama

  14. Hôpital Huriez, CHRU Lille, Lille, France

    Arnauld Villers

Authors
  1. Donna P. Ankerst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Boeck
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephen J. Freedland
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ian M. Thompson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Angel M. Cronin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Monique J. Roobol
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jonas Hugosson
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. J. Stephen Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Michael W. Kattan
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Eric A. Klein
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Freddie Hamdy
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. David Neal
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Jenny Donovan
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Dipen J. Parekh
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Helmut Klocker
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Wolfgang Horninger
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Amine Benchikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Gilles Salama
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Arnauld Villers
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Daniel M. Moreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Fritz H. Schröder
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Hans Lilja
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Andrew J. Vickers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Donna P. Ankerst.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ankerst, D.P., Boeck, A., Freedland, S.J. et al. Evaluating the PCPT risk calculator in ten international biopsy cohorts: results from the Prostate Biopsy Collaborative Group. World J Urol 30, 181–187 (2012). https://doi.org/10.1007/s00345-011-0818-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00345-011-0818-5

Keywords

  • Receiver operating characteristic curve
  • Risk, prostate cancer
  • Calibration
  • Net benefit