Skip to main content
SpringerLink
Log in

Net reclassification improvement: a link between statistics and clinical practice

  • COMMENTARY
  • Published:
European Journal of Epidemiology Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. De Backer G, Ambrosioni E, Borch-Johnsen K, Brotons C, Cifkova R, Dallongeville J, et al. European guidelines on cardiovascular disease prevention in clinical practice. Third joint task force of european and other societies on cardiovascular disease prevention in clinical practice. Eur Heart J. 2003;24(17):1601–10.

    Article  PubMed  Google Scholar 

  2. Executive summary of the third report of the national cholesterol education program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III). JAMA. 2001;285(19):2486–97.

    Google Scholar 

  3. Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkind MS, et al. Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009;119(17):2408–16. doi:10.1161/CIRCULATIONAHA.109.192278.

    Article  PubMed  Google Scholar 

  4. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.

    PubMed  CAS  Google Scholar 

  5. Harrell FE Jr, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA. 1982;247(18):2543–6.

    Article  PubMed  Google Scholar 

  6. Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004;159(9):882–90.

    Article  PubMed  Google Scholar 

  7. Pencina MJ, D’Agostino RB, Pencina KM, Janssens AC, Greenland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol. 2012;176(6):473–81. doi:10.1093/aje/kws207.

    Article  PubMed  Google Scholar 

  8. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–35. doi:10.1161/CIRCULATIONAHA.106.672402.

    Article  PubMed  Google Scholar 

  9. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27(2):157–72. doi:10.1002/sim.2929.

    Article  PubMed  Google Scholar 

  10. Muhlenbruch K, Heraclides A, Steyerberg EW, Joost HG, Boeing H, Schulze MB. Assessing improvement in disease prediction using net reclassification improvement: impact of risk cut-offs and number of risk categories. Eur J Epidemiol. 2012. doi:10.1007/s10654-012-9744-0.

    PubMed  Google Scholar 

  11. Mealiffe ME, Stokowski RP, Rhees BK, Prentice RL, Pettinger M, Hinds DA. Assessment of clinical validity of a breast cancer risk model combining genetic and clinical information. J Natl Cancer Inst. 2010;102(21):1618–27. doi:10.1093/jnci/djq388.

    Article  PubMed  CAS  Google Scholar 

  12. Mihaescu R, van Zitteren M, van Hoek M, Sijbrands EJ, Uitterlinden AG, Witteman JC, et al. Improvement of risk prediction by genomic profiling: reclassification measures versus the area under the receiver operating characteristic curve. Am J Epidemiol. 2010;172(3):353–61. doi:10.1093/aje/kwq122.

    Article  PubMed  Google Scholar 

  13. Cook NR, Paynter NP. Comments on ‘Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers’ by M. J. Pencina, R. B. D’Agostino, Sr. and E. W. Steyerberg. Stat Med. 2012;31(1):93–5. doi:10.1002/sim.4209.

    Article  PubMed  Google Scholar 

  14. Pencina MJ, D’Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. doi:10.1002/sim.4085.

    Article  PubMed  Google Scholar 

  15. Cook NR, Ridker PM. Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures. Ann Intern Med. 2009;150(11):795–802.

    PubMed  Google Scholar 

  16. Cook NR. Clinically relevant measures of fit? A note of caution. Am J Epidemiol. 2012;176(6):488–91.

    Article  PubMed  Google Scholar 

  17. Hosmer DW, Lemeshow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat-Theory Methods. 1980;9(10):1043–69.

    Article  Google Scholar 

  18. Cook NR, Paynter NP. Performance of reclassification statistics in comparing risk prediction models. Biom J. 2011;53(2):237–58. doi:10.1002/bimj.201000078.

    Article  PubMed  Google Scholar 

  19. Vickers AJ, Cronin AM, Begg CB. One statistical test is sufficient for assessing new predictive markers. BMC Med Res Methodol. 2011;11:13. doi:10.1186/1471-2288-11-13.

    Article  PubMed  Google Scholar 

  20. Demler OV, Pencina MJ, D’Agostino RB Sr. Misuse of DeLong test to compare AUCs for nested models. Stat Med. 2012;31(23):2577–87. doi:10.1002/sim.5328.

    Article  PubMed  Google Scholar 

  21. Seshan VE, Gönen M, Begg CB. Comparing ROC curves derived from regression models. Stat Med. 2012. doi:10.1002/sim.5648.

    PubMed  Google Scholar 

  22. Steyerberg EW. Clinical prediction models: A practical approach to development, validation, and updating. New York: Springer; 2009.

    Google Scholar 

  23. Yeboah J, McClelland RL, Polonsky TS, Burke GL, Sibley CT, O’Leary D, et al. Comparison of novel risk markers for improvement in cardiovascular risk assessment in intermediate-risk individuals. JAMA. 2012;308(8):788–95. doi:10.1001/jama.2012.9624.

    Article  PubMed  CAS  Google Scholar 

  24. Cook NR. Comments on ‘Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929). Stat Med. 2008;27(2):191–5. doi:10.1002/sim.2987.

    Article  PubMed  Google Scholar 

  25. Paynter NP, Cook NR. A bias-corrected net reclassification improvement for clinical subgroups. Med Decis Mak. 2012. doi:10.1177/0272989X12461856.

    Google Scholar 

  26. Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302(20):1109–17. doi:10.1056/NEJM198005153022003.

    Article  PubMed  CAS  Google Scholar 

  27. Visvanathan K, Chlebowski RT, Hurley P, Col NF, Ropka M, Collyar D, et al. American society of clinical oncology clinical practice guideline update on the use of pharmacologic interventions including tamoxifen, raloxifene, and aromatase inhibition for breast cancer risk reduction. J Clin Oncol. 2009;27(19):3235–58. doi:10.1200/JCO.2008.20.5179.

    Article  PubMed  CAS  Google Scholar 

  28. Munshi NC, Anderson KC, Bergsagel PL, Shaughnessy J, Palumbo A, Durie B, et al. Consensus recommendations for risk stratification in multiple myeloma: report of the International Myeloma Workshop Consensus Panel 2. Blood. 2011;117(18):4696–700. doi:10.1182/blood-2010-10-300970.

    Article  PubMed  CAS  Google Scholar 

  29. Worth LJ, Worth LJ, Lingaratnam S, Taylor A, Hayward AM, Morrissey S, Cooney J, et al. Use of risk stratification to guide ambulatory management of neutropenic fever. Australian Consensus Guidelines 2011 Steering Committee. Intern Med J. 2011;41(1b):82–9. doi:10.1111/j.1445-5994.2010.02339.x.

    Article  PubMed  CAS  Google Scholar 

Download references

Conflict of interest

The authors declare that they have no conflicts of interest.

Author information

Authors and Affiliations

  1. Department of Epidemiology, Erasmus Medical Center, Rotterdam, The Netherlands

    Maarten J. G. Leening

  2. Department of Cardiology, Erasmus Medical Center, Rotterdam, The Netherlands

    Maarten J. G. Leening

  3. Division of Preventive Medicine, Brigham and Women’s Hospital, Harvard Medical School, 900 Commonwealth Ave. East, Boston, MA, 02215-1204, USA

    Nancy R. Cook

  4. Department of Epidemiology, Harvard School of Public Health, Boston, MA, USA

    Nancy R. Cook

Authors
  1. Maarten J. G. Leening
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nancy R. Cook
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nancy R. Cook.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Leening, M.J.G., Cook, N.R. Net reclassification improvement: a link between statistics and clinical practice. Eur J Epidemiol 28, 21–23 (2013). https://doi.org/10.1007/s10654-012-9759-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10654-012-9759-6

Keywords

Search

Navigation