Breast Cancer Research and Treatment

, Volume 133, Issue 1, pp 1–10 | Cite as

Risk prediction models of breast cancer: a systematic review of model performances

  • Thunyarat Anothaisintawee
  • Yot Teerawattananon
  • Chollathip Wiratkapun
  • Vijj Kasamesup
  • Ammarin ThakkinstianEmail author


The number of risk prediction models has been increasingly developed, for estimating about breast cancer in individual women. However, those model performances are questionable. We therefore have conducted a study with the aim to systematically review previous risk prediction models. The results from this review help to identify the most reliable model and indicate the strengths and weaknesses of each model for guiding future model development. We searched MEDLINE (PubMed) from 1949 and EMBASE (Ovid) from 1974 until October 2010. Observational studies which constructed models using regression methods were selected. Information about model development and performance were extracted. Twenty-five out of 453 studies were eligible. Of these, 18 developed prediction models and 7 validated existing prediction models. Up to 13 variables were included in the models and sample sizes for each study ranged from 550 to 2,404,636. Internal validation was performed in four models, while five models had external validation. Gail and Rosner and Colditz models were the significant models which were subsequently modified by other scholars. Calibration performance of most models was fair to good (expected/observe ratio: 0.87–1.12), but discriminatory accuracy was poor to fair both in internal validation (concordance statistics: 0.53–0.66) and in external validation (concordance statistics: 0.56–0.63). Most models yielded relatively poor discrimination in both internal and external validation. This poor discriminatory accuracy of existing models might be because of a lack of knowledge about risk factors, heterogeneous subtypes of breast cancer, and different distributions of risk factors across populations. In addition the concordance statistic itself is insensitive to measure the improvement of discrimination. Therefore, the new method such as net reclassification index should be considered to evaluate the improvement of the performance of a new develop model.


Breast cancer Risk prediction model Systematic review 



This study was supported by the Health Intervention and Technology Assessment Program, the Thai Health Promotion Foundation, the Health Systems Research Institute, the Bureau of Policy and Strategy of the Ministry of Public Health, and Thai Health-Global Link Initiative Project.

Conflict of interest

TA: received honorarium from the Health Intervention and Technology Assessment Program, Ministry of Public Health, Thailand; received travel grants for conference from the Health Intervention and Technology Assessment Program and Faculty of Graduate Studies, Mahidol University, Thailand. YT, CW, VK, and AT have no conflicts of interest to declare.


  1. 1.
    Parkin DM, Bray F, Ferlay J, Pisani P (2005) Global cancer statistics 2002. CA Cancer J Clinicians 55(2):74–108CrossRefGoogle Scholar
  2. 2.
    Ferlay J, Bray F, Parkin DM, Pisani P eds (2001) Globalcan 2000. IARC Press, LyonGoogle Scholar
  3. 3.
    VH BF (2002) IARC handbooks of cancer prevention. IARC Press, LyonGoogle Scholar
  4. 4.
    Ries LAG EM, Kosary CL, et al. (2004) SEER cancer statistics review, 1975–2001. National Cancer Institute. Accessed Dec 22 2004
  5. 5.
    Steyerberg EW (ed) (2009) Clinical prediction models a practical approach to development, validation, and updating. Springer Science + Business Media LLC, New YorkGoogle Scholar
  6. 6.
    Boyle P, Mezzetti M, La Vecchia C, Franceschi S, Decarli A, Robertson C (2004) Contribution of three components to individual cancer risk predicting breast cancer risk in Italy. Eur J Cancer Prev 13(3):183–191PubMedCrossRefGoogle Scholar
  7. 7.
    Chlebowski RT, Anderson GL, Lane DS, Aragaki AK, Rohan T, Yasmeen S, Sarto G, Rosenberg CA, Hubbell FA (2007) Predicting risk of breast cancer in postmenopausal women by hormone receptor status. J Natl Cancer Inst 99(22):1695–1705PubMedCrossRefGoogle Scholar
  8. 8.
    Colditz GA, Rosner B (2000) Cumulative risk of breast cancer to age 70 years according to risk factor status: data from the Nurses’ Health Study. Am J Epidemiol 152(10):950–964PubMedCrossRefGoogle Scholar
  9. 9.
    Colditz GA, Rosner BA, Chen WY, Holmes MD, Hankinson SE (2004) Risk factors for breast cancer according to estrogen and progesterone receptor status. J Natl Cancer Inst 96(3):218–228PubMedCrossRefGoogle Scholar
  10. 10.
    Decarli A, Calza S, Masala G, Specchia C, Palli D, Gail MH (2006) Gail model for prediction of absolute risk of invasive breast cancer: independent evaluation in the Florence-European prospective investigation into cancer and nutrition cohort. J Natl Cancer Inst 98(23):1686–1693PubMedCrossRefGoogle Scholar
  11. 11.
    Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, Mulvihill JJ (1989) Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst 81(24):1879–1886PubMedCrossRefGoogle Scholar
  12. 12.
    Lee EO, Ahn SH, You C, Lee DS, Han W, Choe KJ, Noh DY (2004) Determining the main risk factors and high-risk groups of breast cancer using a predictive model for breast cancer risk assessment in South Korea. Cancer Nurs 27(5):400–406PubMedCrossRefGoogle Scholar
  13. 13.
    Rosner B, Colditz GA (1996) Nurses’ health study: log-incidence mathematical model of breast cancer incidence. J Natl Cancer Inst 88(6):359–364PubMedCrossRefGoogle Scholar
  14. 14.
    Barlow WE, White E, Ballard-Barbash R, Vacek PM, Titus-Ernstoff L, Carney PA, Tice JA, Buist DS, Geller BM, Rosenberg R, Yankaskas BC, Kerlikowske K (2006) Prospective breast cancer risk prediction model for women undergoing screening mammography. J Natl Cancer Inst 98(17):1204–1214. doi: 10.1093/jnci/djj331 PubMedCrossRefGoogle Scholar
  15. 15.
    Chen J, Pee D, Ayyagari R, Graubard B, Schairer C, Byrne C, Benichou J, Gail MH (2006) Projecting absolute invasive breast cancer risk in white women with a model that includes mammographic density. J Natl Cancer Inst 98(17):1215–1226PubMedCrossRefGoogle Scholar
  16. 16.
    Gail MH, Costantino JP, Pee D, Bondy M, Newman L, Selvan M, Anderson GL, Malone KE, Marchbanks PA, McCaskill-Stevens W, Norman SA, Simon MS, Spirtas R, Ursin G, Bernstein L (2007) Projecting individualized absolute invasive breast cancer risk in African American women. J Natl Cancer Inst 99(23):1782–1792PubMedCrossRefGoogle Scholar
  17. 17.
    Novotny J, Pecen L, Petruzelka L, Svobodnik A, Dusek L, Danes J, Skovajsova M (2006) Breast cancer risk assessment in the Czech female population—an adjustment of the original Gail model. Breast Cancer Res Treat 95(1):29–35PubMedCrossRefGoogle Scholar
  18. 18.
    Rosner B, Colditz GA, Iglehart JD, Hankinson SE (2008) Risk prediction models with incomplete data with application to prediction of estrogen receptor-positive breast cancer: prospective data from the Nurses’ Health Study. Breast Cancer Res BCR 10(4):R55CrossRefGoogle Scholar
  19. 19.
    Tamimi RM, Rosner B, Colditz GA Evaluation of a breast cancer risk prediction model expanded to include category of prior benign breast disease lesion. Cancer. doi:  10.1002/cncr.25386
  20. 20.
    Tice JA, Cummings SR, Smith-Bindman R, Ichikawa L, Barlow WE, Kerlikowske K (2008) Using clinical factors and mammographic breast density to estimate breast cancer risk: development and validation of a new predictive model. Ann Intern Med 148(5):337–347PubMedGoogle Scholar
  21. 21.
    Tice JA, Cummings SR, Ziv E, Kerlikowske K (2005) Mammographic breast density and the Gail model for breast cancer risk prediction in a screening population. Breast Cancer Res Treat 94(2):115–122PubMedCrossRefGoogle Scholar
  22. 22.
    Tice JA, Miike R, Adduci K, Petrakis NL, King E, Wrensch MR (2005) Nipple aspirate fluid cytology and the Gail model for breast cancer risk assessment in a screening population. Cancer Epidemiol Biomarkers Prev 14(2):324–328PubMedCrossRefGoogle Scholar
  23. 23.
    Ueda K, Tsukuma H, Tanaka H, Ajiki W, Oshima A (2003) Estimation of individualized probabilities of developing breast cancer for Japanese women. Breast cancer (Tokyo, Japan) 10(1):54–62CrossRefGoogle Scholar
  24. 24.
    Bondy ML, Lustbader ED, Halabi S, Ross E, Vogel VG (1994) Validation of a breast cancer risk assessment model in women with a positive family history. J Natl Cancer Inst 86(8):620–625PubMedCrossRefGoogle Scholar
  25. 25.
    Costantino JP, Gail MH, Pee D, Anderson S, Redmond CK, Benichou J, Wieand HS (1999) Validation studies for models projecting the risk of invasive and total breast cancer incidence. J Natl Cancer Inst 91(18):1541–1548PubMedCrossRefGoogle Scholar
  26. 26.
    McKian KP, Reynolds CA, Visscher DW, Nassar A, Radisky DC, Vierkant RA, Degnim AC, Boughey JC, Ghosh K, Anderson SS, Minot D, Caudill JL, Vachon CM, Frost MH, Pankratz VS, Hartmann LC (2009) Novel breast tissue feature strongly associated with risk of breast cancer. J Clin Oncol 27(35):5893–5898PubMedCrossRefGoogle Scholar
  27. 27.
    Rockhill B, Byrne C, Rosner B, Louie MM, Colditz G (2003) Breast cancer risk prediction with a log-incidence model: evaluation of accuracy. J Clin Epidemiol 56(9):856–861PubMedCrossRefGoogle Scholar
  28. 28.
    Rockhill B, Spiegelman D, Byrne C, Hunter DJ, Colditz GA (2001) Validation of the Gail et al. Model of breast cancer risk prediction and implications for chemoprevention. J Natl Cancer Inst 93(5):358–366PubMedCrossRefGoogle Scholar
  29. 29.
    Schonfeld SJ, Pee D, Greenlee RT, Hartge P, Lacey JV, Jr., Park Y, Schatzkin A, Visvanathan K, Pfeiffer RM (2010) Effect of changing breast cancer incidence rates on the calibration of the Gail model. J Clin Oncol 28(14):2411–2417. doi:  10.1200/JCO.2009.25.2767 Google Scholar
  30. 30.
    Ulusoy C, Kepenekci I, Kose K, Aydintug S, Cam R (2010) Applicability of the Gail model for breast cancer risk assessment in Turkish female population and evaluation of breastfeeding as a risk factor. Breast Cancer Res Treat 120(2):419–424PubMedCrossRefGoogle Scholar
  31. 31.
    Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21(1):128–138. doi: 10.1097/EDE.0b013e3181c30fb2 PubMedCrossRefGoogle Scholar
  32. 32.
    Vickers AJ, Cronin AM (2010) Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol 37(1):31–38. doi: 10.1053/j.seminoncol.2009.12.004 PubMedCrossRefGoogle Scholar
  33. 33.
    Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS (2008) Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med 27(2):157–172 discussion 207–112PubMedCrossRefGoogle Scholar
  34. 34.
    Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19(4):453–473PubMedCrossRefGoogle Scholar
  35. 35.
    Peduzzi P, Concato J, Feinstein AR, Holford TR (1995) Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48(12):1503–1510PubMedCrossRefGoogle Scholar
  36. 36.
    Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV (2011) Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol 64(9):993–1000. doi: 10.1016/j.jclinepi.2010.11.012 PubMedCrossRefGoogle Scholar
  37. 37.
    Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P (2004) Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 159(9):882–890PubMedCrossRefGoogle Scholar
  38. 38.
    Haynes RB, Sackett DL, Guyatt GH (eds) (2006) Clinical epidemiology: how to do clinical practice research, 3rd edn. Lippincott Williams and Wilkins, PhiladelphiaGoogle Scholar
  39. 39.
    Breast cancer and hormone replacement therapy: collaborative reanalysis of data from 51 epidemiological studies of 52,705 women with breast cancer and 108,411 women without breast cancer. Collaborative Group on Hormonal Factors in Breast Cancer (1997). Lancet 350(9084):1047–1059Google Scholar
  40. 40.
    Ewertz M, Duffy SW, Adami HO, Kvale G, Lund E, Meirik O, Mellemgaard A, Soini I, Tulinius H (1990) Age at first birth, parity and risk of breast cancer: a meta-analysis of 8 studies from the Nordic countries. Int J Cancer 46(4):597–603PubMedCrossRefGoogle Scholar
  41. 41.
    Pharoah PD, Day NE, Duffy S, Easton DF, Ponder BA (1997) Family history and the risk of breast cancer: a systematic review and meta-analysis. Int J Cancer 71(5):800–809PubMedCrossRefGoogle Scholar
  42. 42.
    North RA, McCowan LME, Dekker GA, Poston L, Chan EHY, Stewart AW, Black MA, Taylor RS, Walker JJ, Baker PN, Kenny LC (2011) Clinical risk prediction for pre-eclampsia in nulliparous women: development of model in international prospective cohort. BMJ (Clinical research ed) 342:d1875s. doi:  10.1136/bmj.d1875
  43. 43.
    Tyrer J, Duffy SW, Cuzick J (2004) A breast cancer prediction model incorporating familial and personal risk factors. Stat Med 23(7):1111–1130PubMedCrossRefGoogle Scholar
  44. 44.
    Amir E, Evans DG, Shenton A, Lalloo F, Moran A, Boggis C, Wilson M, Howell A (2003) Evaluation of breast cancer risk assessment packages in the family history evaluation and screening programme. J Med Genet 40(11):807–814PubMedCrossRefGoogle Scholar
  45. 45.
    Cook NR (2007) Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 115(7):928–935. doi: 10.1161/CIRCULATIONAHA.106.672402 PubMedCrossRefGoogle Scholar
  46. 46.
    Cook NR, Buring JE, Ridker PM (2006) The effect of including C-reactive protein in cardiovascular risk prediction models for women. Ann Intern Med 145(1):21–29PubMedGoogle Scholar
  47. 47.
    Janes H, Pepe MS, Gu W (2008) Assessing the value of risk predictions by using risk stratification tables. Ann Intern Med 149(10):751–760PubMedGoogle Scholar
  48. 48.
    McShane LM, Altman DG, Sauerbrei W, Taube SE, Gion M, Clark GM (2005) Reporting recommendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst 97(16):1180–1184. doi: 10.1093/jnci/dji237 PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2011

Authors and Affiliations

  • Thunyarat Anothaisintawee
    • 1
  • Yot Teerawattananon
    • 2
  • Chollathip Wiratkapun
    • 3
  • Vijj Kasamesup
    • 4
  • Ammarin Thakkinstian
    • 5
    Email author
  1. 1.Section for Clinical Epidemiology and Biostatistics, Department of Family MedicineRamathibodi HospitalBangkokThailand
  2. 2.Health Intervention and Technology Assessment Program, Ministry of Public HealthNonthaburiThailand
  3. 3.Department of Radiology, Faculty of Medicine, Ramathibodi HospitalMahidol UniversityBangkokThailand
  4. 4.Department of Community Medicine, Faculty of Medicine, Ramathibodi HospitalMahidol UniversityBangkokThailand
  5. 5.Section for Clinical Epidemiology and Biostatistics, Faculty of Medicine, Ramathibodi HospitalMahidol UniversityBangkokThailand

Personalised recommendations