Abstract
Purpose
Recurrence of breast cancer leads to a high lifetime risk and a low 5 year survival rate. Researchers have used machine learning to predict the risk of recurrence in patients with breast cancer, but the predictive performance of machine learning remains controversial. Hence, this study aimed to explore the accuracy of machine learning in predicting breast cancer recurrence risk and aggregate predictive variables to provide guidance for the development of subsequent risk scoring systems.
Methods
We searched Pubmed, EMBASE, Cochrane, and Web of Science. The risk of bias in the included studies was evaluated using prediction model risk of bias assessment tool (PROBAST). Meta-regression was adopted to explore whether there was a significant difference in the recurrence time by machine learning.
Results
Thirty-four studies involving 67,560 subjects were included, among whom 8695 experienced breast cancer recurrence. The c-index of prediction models was 0.814 (95%CI 0.802–0.826) and 0.770 (95%CI 0.737–0.803) in the training and validation sets, respectively; the sensitivity and specificity were 0.69 (95% CI 0.64–0.74), 0.89 (95% CI 0.86–0.92) in the training, and 0.64 (95% CI 0.58–0.70), 0.88 (95% CI 0.82–0.92) in the validation, respectively. Age, histological grading, and lymph node status are the most commonly used variables in model construction. Attention should be paid to unhealthy lifestyles such as drinking, smoking and BMI as modeling variables. Risk prediction models based on machine learning have long-term monitoring value for breast cancer population, and subsequent studies should consider using large-sample and multi-center data to establish risk equations for verification.
Conclusion
Machine learning may be used as a predictive tool for breast cancer recurrence. Currently, there is a lack of effective and universally applicable machine learning models in clinical practice. We expect to incorporate multi-center studies in the future and attempt to develop tools for predicting breast cancer recurrence risk, so as to effectively identify populations at high risk of recurrence and develop personalized follow-up strategies and prognostic interventions to reduce the risk of recurrence.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Abbreviations
- TRIPOD:
-
Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis
- AUROC:
-
Area under the receiver operating characteristic curve
- GLCM:
-
Gray-level co-occurrence matrix
- GLRLM:
-
Gray-level run-length matrix
- WPB:
-
Wisconsin prognostic breast cancer
- FNA:
-
Fine needle aspiration biopsy digitized samples
References
Abreu DC, Henriques AM (2016) Male breast cancer: prognostic and predictive factors of response to therapy. Universidade do Porto (Portugal)
Abreu PH, Santos MS, Abreu MH, Andrade B, Silva DC (2016) Predicting breast cancer recurrence using machine learning techniques: a systematic review. ACM Comput Surveys. 49: 1–40. https://doi.org/10.1145/2988544
Bhardwaj D, Dasgupta A, Dicenzo D, Brade S, Fatima K, Quiaoit K, Trudeau M, Gandhi S, Eisen A, Wright F, Look-Hong N, Curpen B, Sannachi L, Czarnota GJ (2022) Early Changes in Quantitative Ultrasound Imaging Parameters during Neoadjuvant Chemotherapy to Predict Recurrence in Patients with Locally Advanced Breast Cancer. Cancers (Basel). https://doi.org/10.3390/cancers14051247
Breiman L (1996) Bagging predictors. Mach Learn 24:123–140. https://doi.org/10.1007/BF00058655
Buus R, Sestak I, Kronenwett R, Ferree S, Schnabel CA, Baehner FL, Mallon EA, Cuzick J, Dowsett M (2021) Molecular Drivers of Oncotype DX, Prosigna, EndoPredict, and the Breast Cancer Index: A TransATAC Study. J Clin Oncol 39:126–135. https://doi.org/10.1200/jco.20.00853
Chen X, Zhou Z, Thomas K, Folkert M, Kim N, Rahimi A, Wang J (2019) A Reliable Multi-classifier Multi-objective Model for Predicting Recurrence in Triple Negative Breast Cancer. Annu Int Conf IEEE Eng Med Biol Soc 2019:2182–2185. https://doi.org/10.1109/embc.2019.8857030
Choi WJ, Cha JH, Kim HH, Shin HJ, Chae EY, Jung KH, Ahn JH, Kim SB, Son BH, Ahn SH (2017) Long-term Survival Outcomes of Primary Breast Cancer in Women With or Without Preoperative Magnetic Resonance Imaging: A Matched Cohort Study. Clin Oncol (R Coll Radiol) 29:653–661. https://doi.org/10.1016/j.clon.2017.06.015
Colleoni M, Sun Z, Price KN, Karlsson P, Forbes JF, Thürlimann B, Gianni L, Castiglione M, Gelber RD, Coates AS, Goldhirsch A (2016) Annual Hazard Rates of Recurrence for Breast Cancer During 24 Years of Follow-Up: Results From the International Breast Cancer Study Group Trials I to V. J Clin Oncol 34:927–35. https://doi.org/10.1200/jco.2015.62.3504
Collins GS, Reitsma JB, Altman DG, Moons KG (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Bmj. 350:g7594. https://doi.org/10.1136/bmj.g7594
Comes MC, La Forgia D, Didonna V, Fanizzi A, Giotta F, Latorre A, Martinelli E, Mencattini A, Paradiso AV, Tamborra P, Terenzio A, Zito A, Lorusso V, Massafra R (2021) Early Prediction of Breast Cancer Recurrence for Patients Treated with Neoadjuvant Chemotherapy: A Transfer Learning Approach on DCE-MRIs. Cancers (Basel). https://doi.org/10.3390/cancers13102298
De La Rochefordiere A, Asselain B, Campana F, Scholl SM, Fenton J, Vilcoq JR, Durand JC, Pouillart P, Magdelenat H, Fourquet A (1993) Age as prognostic factor in premenopausal breast carcinoma. Lancet 341:1039–43. https://doi.org/10.1016/0140-6736(93)92407-k
Debray TP, Damen JA, Riley RD, Snell K, Reitsma JB, Hooft L, Collins GS, Moons KG (2019) A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes. Stat Methods Med Res. 28:2768–2786. https://doi.org/10.1177/0962280218785504
DI TANNA, GL, WIRTZ, H, BURROWS, KL, GLOBE, G (2020) Evaluating risk prediction models for adults with heart failure: A systematic literature review. PLoS One. 15: e0224135. https://doi.org/10.1371/journal.pone.0224135
Ehteshami Bejnordi, B, Veta, M, Johannes Van Diest, P, Van Ginneken, B, Karssemeijer, N, Litjens, G, Van Der Laak, J, Hermsen, M, Manson, QF, Balkenhol, M, Geessink, O, Stathonikos, N, Van Dijk, MC, Bult, P, Beca, F, Beck, AH, Wang, D, Khosla, A, Gargeya, R, Irshad, H, Zhong, A, Dou, Q, Li, Q, Chen, H, Lin, HJ, Heng, PA, Haß, C, Bruni, E, Wong, Q, Halici, U, Öner, M, Cetin-Atalay, R, Berseth, M, Khvatkov, V, Vylegzhanin, A, Kraus, O, Shaban, M, Rajpoot, N, Awan, R, Sirinukunwattana, K, Qaiser, T, Tsang, YW, Tellez, D, Annuscheit, J, Hufnagl, P, Valkonen, M, Kartasalo, K, Latonen, L, Ruusuvuori, P, Liimatainen, K, Albarqouni, S, Mungal, B, George, A, Demirci, S, Navab, N, Watanabe, S, Seno, S, Takenaka, Y, Matsuda, H, Ahmady Phoulady, H, Kovalev, V, Kalinovsky, A, Liauchuk, V, Bueno, G, Fernandez-Carrobles, MM, Serrano, I, Deniz, O, Racoceanu, D, Venâncio, R (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. Jama. 318: 2199-2210. https://doi.org/10.1001/jama.2017.14585
Eun NL, Kang D, Son EJ, Youk JH, Kim JA, Gweon HM (2021) Texture analysis using machine learning-based 3-T magnetic resonance imaging for predicting recurrence in breast cancer patients treated with neoadjuvant chemotherapy. Eur Radiol 31:6916–6928. https://doi.org/10.1007/s00330-021-07816-x
Gu D, Su K, Zhao H (2020) A case-based ensemble learning system for explainable breast cancer recurrence prediction. Artif Intell Med 107:101858. https://doi.org/10.1016/j.artmed.2020.101858
Hayashi Y, Satake H, Ishigaki S, Ito R, Kawamura M, Kawai H, Iwano S, Naganawa S (2020) Kinetic volume analysis on dynamic contrast-enhanced MRI of triple-negative breast cancer: associations with survival outcomes. Br J Radiol. 93:20190712. https://doi.org/10.1259/bjr.20190712
Holowatyj AN, Cote ML, Ruterbusch JJ, Ghanem K, Schwartz AG, Vigneau FD, Gorski DH, Purrington KS (2018) Racial differences in 21-gene recurrence scores among patients with hormone receptor-positive. Node-Negative Breast Cancer J Clin Oncol 36:652–658. https://doi.org/10.1200/JCO.2017.74.5448
Hoskins KF, Danciu OC, Ko NY, Calip GS (2021) Association of race/ethnicity and the 21-gene recurrence score with breast cancer-specific mortality among US women. JAMA Oncol 7:370–378. https://doi.org/10.1001/jamaoncol.2020.7320
Jerez, JM, Franco, L, Alba, E, Llombart-Cussac, A, Lluch, A, Ribelles, N, Munárriz, B, Martín, M (2005) Improvement of breast cancer relapse prediction in high risk intervals using artificial neural networks. Breast Cancer Res Treat. 94: 265-72. https://doi.org/10.1007/s10549-005-9013-y
Kim W, Kim KS, Park RW (2016) Nomogram of Naive Bayesian Model for Recurrence Prediction of Breast Cancer. Healthc Inform Res 22:89–94. https://doi.org/10.4258/hir.2016.22.2.89
Kim JY, Lee YS, Yu J, Park Y, Lee SK, Lee M, Lee JE, Kim SW, Nam SJ, Park YH, Ahn JS, Kang M, Im YH (2021) Deep Learning-Based Prediction Model for Breast Cancer Recurrence Using Adjuvant Breast Cancer Cohort in Tertiary Cancer Center Registry. Front Oncol. 11:596364. https://doi.org/10.3389/fonc.2021.596364
Koh, J, Lee, E, Han, K, Kim, S, Kim, DK, Kwak, JY, Yoon, JH, Moon, HJ (2020) Three-dimensional radiomics of triple-negative breast cancer: Prediction of systemic recurrence. Sci Rep. 10: 2976. https://doi.org/10.1038/s41598-020-59923-2
Kwan ML, Kushi LH, Weltzien E, Tam EK, Castillo A, Sweeney C, Caan BJ (2010) Alcohol consumption and breast cancer recurrence and survival among women with early-stage breast cancer: the life after cancer epidemiology study. J Clin Oncol. 28:4410–6. https://doi.org/10.1200/jco.2010.29.2730
Lê, MG, Arriagada, R, Spielmann, M, Guinebretière, JM, Rochard, F (2002) Prognostic factors for death after an isolated local recurrence in patients with early-stage breast carcinoma. Cancer. 94: 2813-20. https://doi.org/10.1002/cncr.10572
Li H, Liu RB, Long CM, Teng Y, Cheng L, Liu Y (2022) Development and Validation of a New Multiparametric Random Survival Forest Predictive Model for Breast Cancer Recurrence with a Potential Benefit to Individual Outcomes. Cancer Manag Res. 14:909–923. https://doi.org/10.2147/cmar.S346871
Ligibel JA, Strickler HD (2013) Obesity and its impact on breast cancer: tumor incidence, recurrence, survival, and possible interventions. Am Soc Clin Oncol Educ Book. https://doi.org/10.14694/EdBook_AM.2013.33.52
Lou SJ, Hou MF, Chang HT, Chiu CC, Lee HH, Yeh SJ, Shi HY (2020) Machine Learning Algorithms to Predict Recurrence within 10 Years after Breast Cancer Surgery: A Prospective Cohort Study. Cancers (Basel). https://doi.org/10.3390/cancers12123817
Magboo VPC, Magboo MSA (2021) Machine Learning Classifiers on Breast Cancer Recurrences. Proc Comput Sci 192:2742–2752. https://doi.org/10.1016/j.procs.2021.09.044
Mazo C, Barron S, Mooney C, Gallagher WM (2020) Multi-Gene Prognostic Signatures and Prediction of Pathological Complete Response to Neoadjuvant Chemotherapy in ER-positive, HER2-negative Breast Cancer Patients. Cancers (Basel). https://doi.org/10.3390/cancers12051133
Mehrotra J, Vali M, Mcveigh M, Kominsky SL, Fackler MJ, Lahti-Domenici J, Polyak K, Sacchi N, Garrett-Mayer E, Argani P, Sukumar S (2004) Very high frequency of hypermethylated genes in breast cancer metastasis to the bone, brain, and lung. Clin Cancer Res 10:3104–9. https://doi.org/10.1158/1078-0432.ccr-03-0118
Mohebian, MR, Marateb, HR, Mansourian, M, Mañanas, MA, Mokarian, F (2017) A Hybrid Computer-aided-diagnosis System for Prediction of Breast Cancer Recurrence (HPBCR) Using Optimized Ensemble Learning. Comput Struct Biotechnol J. 15: 75-85. https://doi.org/10.1016/j.csbj.2016.11.004
Moody SE, Perez D, Pan TC, Sarkisian CJ, Portocarrero CP, Sterner CJ, Notorfrancesco KL, Cardiff RD, Chodosh LA (2005) The transcriptional repressor Snail promotes mammary tumor recurrence. Cancer Cell 8:197–209. https://doi.org/10.1016/j.ccr.2005.07.009
Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, Topol EJ, Ioannidis JPA, Collins GS, Maruthappu M (2020) Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. Bmj. 368:m689. https://doi.org/10.1136/bmj.m689
Palazón-Bru, A, Mares-García, E, López-Bru, D, Mares-Arambul, E, Folgado-De La ROSA, DM, Carbonell-Torregrosa, M, Gil-Guillén, VF (2020) A critical appraisal of the clinical applicability and risk of bias of the predictive models for mortality and recurrence in patients with oropharyngeal cancer: Systematic review. Head Neck. 42: 763-773. https://doi.org/10.1002/hed.26025
Pickles MD, Lowry M, Gibbs P (2016) Pretreatment Prognostic Value of Dynamic Contrast-Enhanced Magnetic Resonance Imaging Vascular, Texture, Shape, and Size Parameters Compared With Traditional Survival Indicators Obtained From Locally Advanced Breast Cancer Patients. Invest Radiol. 51:177–85. https://doi.org/10.1097/rli.0000000000000222
Ries L, Melbert D, Krapcho M (2014) SEER cancer statistics review 1975–2012. NCI, Bethesda
Sadoughi F, Kazemy Z, Hamedan F, Owji L, Rahmanikatigari M, Azadboni TT (2018) Artificial intelligence methods for the diagnosis of breast cancer by image processing: a review. Breast Cancer (Dove Med Press). 10:219–230. https://doi.org/10.2147/bctt.S175311
Siegel RL, Miller KD, Fuchs HE, Jemal A (2022) Cancer statistics, 2022. CA Cancer J Clin 72:7–33. https://doi.org/10.3322/caac.21708
Sopik V, Sun P, Narod SA (2019) Predictors of time to death after distant recurrence in breast cancer patients. Breast Cancer Res Treat 173:465–474. https://doi.org/10.1007/s10549-018-5002-9
Su W, He B, Zhang YD, Yin G (2022) C-index regression for recurrent event data. Contemp Clin Trials. 118:106787. https://doi.org/10.1016/j.cct.2022.106787
Tseng YJ, Huang CE, Wen CN, Lai PY, Wu MH, Sun YC, Wang HY, Lu JJ (2019) Predicting breast cancer metastasis by using serum biomarkers and clinicopathological data with machine learning technologies. Int J Med Inform 128:79–86. https://doi.org/10.1016/j.ijmedinf.2019.05.003
Wolff RF, Moons KGM, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S, PROBAST Group† (2019) PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 170:51–58. https://doi.org/10.7326/M18-1376. (PMID: 30596875)
Wu X, Ye Y, Barcenas CH, Chow WH, Meng QH, Chavez-Macgregor M, Hildebrandt MA, Zhao H, Gu X, Deng Y, Wagar E, Esteva FJ, Tripathy D, Hortobagyi GN (2017) Personalized Prognostic Prediction Models for Breast Cancer Recurrence and Survival Incorporating Multidimensional Data. J Natl Cancer Inst. https://doi.org/10.1093/jnci/djw314
Yu Y, Tan Y, Xie C, Hu Q, Ouyang J, Chen Y, Gu Y, Li A, Lu N, He Z, Yang Y, Chen K, Ma J, Li C, Ma M, Li X, Zhang R, Zhong H, Ou Q, Zhang Y, He Y, Li G, Wu Z, Su F, Song E, Yao H (2020) Development and Validation of a Preoperative Magnetic Resonance Imaging Radiomics-Based Signature to Predict Axillary Lymph Node Metastasis and Disease-Free Survival in Patients With Early-Stage Breast Cancer. JAMA Netw Open. 3:e2028086. https://doi.org/10.1001/jamanetworkopen.2020.28086
Zeng Z, Yao L, Roy A, Li X, Espino S, Clare SE, Khan SA, Luo Y (2019) Identifying Breast Cancer Distant Recurrences from Electronic Health Records Using Machine Learning. J Healthc Inform Res 3:283–299. https://doi.org/10.1007/s41666-019-00046-3
Zhang Y, Li A, He J, Wang M (2020) A Novel MKL Method for GBM Prognosis Prediction by Integrating Histopathological Image and Multi-Omics Data. IEEE J Biomed Health Inform. 24:171–179. https://doi.org/10.1109/jbhi.2019.2898471
Zhijun L, Dapeng L, Xinrui W, Lisheng L, Quan Z (2018) Cancer diagnosis through IsomiR expression with machine learning method. Curr Bioinform 13:57–63. https://doi.org/10.2174/1574893611666160609081155
Funding
There was no project specific funding for this work.
Author information
Authors and Affiliations
Contributions
DL and XL contributed equally to this work. Conceptualization: SS and XL; methodology, formal analysis and investigation: DL and XL; writing—original draft preparation: DL and XL; writing—review and editing: DL, XL, WF and BL; resources: WF, BL and XZ; supervision: SS and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Ethical approval
Not applicable.
Consent to participate
Not applicable.
Consent to publish
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
432_2023_4967_MOESM5_ESM.docx
Supplementary file5 Summary of sensitivity and specificity of each model in the training and validation sets (DOCX 14 KB)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lu, D., Long, X., Fu, W. et al. Predictive value of machine learning for breast cancer recurrence: a systematic review and meta-analysis. J Cancer Res Clin Oncol 149, 10659–10674 (2023). https://doi.org/10.1007/s00432-023-04967-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00432-023-04967-w