We hypothesize that convolutional neural networks (CNN) can be used to predict neoadjuvant chemotherapy (NAC) response using a breast MRI tumor dataset prior to initiation of chemotherapy. An institutional review board-approved retrospective review of our database from January 2009 to June 2016 identified 141 locally advanced breast cancer patients who (1) underwent breast MRI prior to the initiation of NAC, (2) successfully completed adriamycin/taxane-based NAC, and (3) underwent surgical resection with available final surgical pathology data. Patients were classified into three groups based on their NAC response confirmed on final surgical pathology: complete (group 1), partial (group 2), and no response/progression (group 3). A total of 3107 volumetric slices of 141 tumors were evaluated. Breast tumor was identified on first T1 postcontrast dynamic images and underwent 3D segmentation. CNN consisted of ten convolutional layers, four max-pooling layers, and dropout of 50% after a fully connected layer. Dropout, augmentation, and L2 regularization were implemented to prevent overfitting of data. Non-linear functions were modeled by a rectified linear unit (ReLU). Batch normalization was used between the convolutional and ReLU layers to limit drift of layer activations during training. A three-class neoadjuvant prediction model was evaluated (group 1, group 2, or group 3). The CNN achieved an overall accuracy of 88% in three-class prediction of neoadjuvant treatment response. Three-class prediction discriminating one group from the other two was analyzed. Group 1 had a specificity of 95.1% ± 3.1%, sensitivity of 73.9% ± 4.5%, and accuracy of 87.7% ± 0.6%. Group 2 (partial response) had a specificity of 91.6% ± 1.3%, sensitivity of 82.4% ± 2.7%, and accuracy of 87.7% ± 0.6%. Group 3 (no response/progression) had a specificity of 93.4% ± 2.9%, sensitivity of 76.8% ± 5.7%, and accuracy of 87.8% ± 0.6%. It is feasible for current deep CNN architectures to be trained to predict NAC treatment response using a breast MRI dataset obtained prior to initiation of chemotherapy. Larger dataset will likely improve our prediction model.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Rastogi P, Anderson SJ, Bear HD, Geyer CE, Kahlenberg MS, Robidoux A, Margolese RG, Hoehn JL, Vogel VG, Dakhil SR, Tamkus D, King KM, Pajon ER, Wright MJ, Robert J, Paik S, Mamounas EP, Wolmark N: Preoperative chemotherapy: updates of National Surgical Adjuvant Breast and Bowel Project Protocols B-18 and B-27. J Clin Oncol 26:778–785, 2008
Gianni L, Pienkowski T, Im YH, Tseng LM, Liu MC, Lluch A, Starosławska E, de la Haba-Rodriguez J, Im SA, Pedrini JL, Poirier B, Morandi P, Semiglazov V, Srimuninnimit V, Bianchi GV, Magazzù D, McNally V, Douthwaite H, Ross G, Valagussa P: 5-year analysis of neoadjuvant pertuzumab and trastuzumab in patients with locally advanced, inflammatory, or early-stage HER2-positive breast cancer (NeoSphere): a multicentre, open-label, phase 2 randomised trial. Lancet Oncol 17:791–800, 2016
Cortazar P, Zhang L, Untch M, Mehta K, Costantino JP, Wolmark N, Bonnefoi H, Cameron D, Gianni L, Valagussa P, Swain SM, Prowell T, Loibl S, Wickerham DL, Bogaerts J, Baselga J, Perou C, Blumenthal G, Blohmer J, Mamounas EP, Bergh J, Semiglazov V, Justice R, Eidtmann H, Paik S, Piccart M, Sridhara R, Fasching PA, Slaets L, Tang S, Gerber B, Geyer, Jr CE, Pazdur R, Ditsch N, Rastogi P, Eiermann W, von Minckwitz G: Pathological complete response and long-term clinical benefit in breast cancer: the CTNeoBC pooled analysis. Lancet 384:164–172, 2014
Esserman LJ, Berry DA, DeMichele A, Carey L, Davis SE, Buxton M, Hudis C, Gray JW, Perou C, Yau C, Livasy C, Krontiras H, Montgomery L, Tripathy D, Lehman C, Liu MC, Olopade OI, Rugo HS, Carpenter JT, Dressler L, Chhieng D, Singh B, Mies C, Rabban J, Chen YY, Giri D, van 't Veer L, Hylton N: Pathologic complete response predicts recurrence-free survival more effectively by cancer subset: results from the I-SPY 1 TRIAL--CALGB 150007/150012, ACRIN 6657. J Clin Oncol 30:3242–3249, 2012
von Minckwitz G, Untch M, Blohmer JU, Costa SD, Eidtmann H, Fasching PA, Gerber B, Eiermann W, Hilfrich J, Huober J, Jackisch C, Kaufmann M, Konecny GE, Denkert C, Nekljudova V, Mehta K, Loibl S: Definition and impact of pathologic complete response on prognosis after neoadjuvant chemotherapy in various intrinsic breast cancer subtypes. J Clin Oncol 30:1796–1804, 2012
Wang-Lopez Q, Chalabi N, Abrial C, Radosevic-Robin N, Durando X, Mouret-Reynier MA, Benmammar KE, Kullab S, Bahadoor M, Chollet P, Penault-Llorca F, Nabholtz JM: Can pathologic complete response (pCR) be used as a surrogate marker of survival after neoadjuvant therapy for breast cancer? Crit Rev Oncol Hematol 95:88–104, 2015
Broglio KR, Quintana M, Foster M, Olinger M, McGlothlin A, Berry SM, Boileau JF, Brezden-Masley C, Chia S, Dent S, Gelmon K, Paterson A, Rayson D, Berry DA: Association of pathologic complete response to neoadjuvant therapy in HER2-positive breast cancer with long-term outcomes: a meta-analysis. JAMA Oncol 2:751–760, 2016
Carey LA: Neoadjuvant clinical trial designs: challenges of the genomic era. Breast 24(Suppl 2):S88–S90, 2015
Carey LA, Winer EP: I-SPY 2—toward more rapid progress in breast cancer treatment. N Engl J Med 375:83–84, 2016
Park JW, Liu MC, Yee D, Yau C, van 't Veer L, Symmans WF, Paoloni M, Perlmutter J, Hylton NM, Hogarth M, DeMichele A, Buxton MB, Chien AJ, Wallace AM, Boughey JC, Haddad TC, Chui SY, Kemmer KA, Kaplan HG, Isaacs C, Nanda R, Tripathy D, Albain KS, Edmiston KK, Elias AD, Northfelt DW, Pusztai L, Moulder SL, Lang JE, Viscusi RK, Euhus DM, Haley BB, Khan QJ, Wood WC, Melisko M, Schwab R, Helsten T, Lyandres J, Davis SE, Hirst GL, Sanil A, Esserman LJ, Berry DA, I-SPY 2 Investigators: Adaptive randomization of neratinib in early breast cancer. N Engl J Med 375:11–22, 2016
Rugo HS, Olopade OI, DeMichele A, Yau C, van 't Veer L, Buxton MB, Hogarth M, Hylton NM, Paoloni M, Perlmutter J, Symmans WF, Yee D, Chien AJ, Wallace AM, Kaplan HG, Boughey JC, Haddad TC, Albain KS, Liu MC, Isaacs C, Khan QJ, Lang JE, Viscusi RK, Pusztai L, Moulder SL, Chui SY, Kemmer KA, Elias AD, Edmiston KK, Euhus DM, Haley BB, Nanda R, Northfelt DW, Tripathy D, Wood WC, Ewing C, Schwab R, Lyandres J, Davis SE, Hirst GL, Sanil A, Berry DA, Esserman LJ, I-SPY 2 Investigators: Adaptive randomization of veliparib-carboplatin treatment in breast cancer. N Engl J Med 375:23–34, 2016
von Minckwitz G, Blohmer JU, Costa SD, Denkert C, Eidtmann H, Eiermann W, Gerber B, Hanusch C, Hilfrich J, Huober J, Jackisch C, Kaufmann M, Kümmel S, Paepke S, Schneeweiss A, Untch M, Zahm DM, Mehta K, Loibl S: Response-guided neoadjuvant chemotherapy for breast cancer. J Clin Oncol 31:3623–3630, 2013
Gu YL, Pan SM, Ren J, Yang ZX, Jiang GQ: Role of magnetic resonance imaging in detection of pathologic complete remission in breast cancer patients treated with neoadjuvant chemotherapy: a meta-analysis. Clin Breast Cancer 17:245–255, 2017
Li W, Arasu V, Newitt DC, Jones EF, Wilmes L, Gibbs J, Kornak J, Joe BN, Esserman LJ, Hylton NM, ACRIN 6657 Trial Team and I-SPY Investigators Network: Effect of MR imaging contrast thresholds on prediction of neoadjuvant chemotherapy response in breast cancer subtypes: a subgroup analysis of the ACRIN 6657/I-SPY 1 TRIAL. Tomography 2:378–387, 2016
Hylton NM, Blume JD, Bernreuter WK, Pisano ED, Rosen MA, Morris EA, Weatherall PT, Lehman CD, Newstead GM, Polin S, Marques HS, Esserman LJ, Schnall MD, ACRIN 6657 Trial Team and I-SPY 1 TRIAL Investigators: Locally advanced breast cancer: MR imaging for prediction of response to neoadjuvant chemotherapy—results from ACRIN 6657/I-SPY TRIAL. Radiology 263:663–672, 2012
Hylton NM, Gatsonis CA, Rosen MA, Lehman CD, Newitt DC, Partridge SC, Bernreuter WK, Pisano ED, Morris EA, Weatherall PT, Polin SM, Newstead GM, Marques HS, Esserman LJ, Schnall MD, For the ACRIN 6657 Trial Team and I-SPY 1 TRIAL Investigators: Neoadjuvant chemotherapy for breast cancer: functional tumor volume by MR imaging predicts recurrence-free survival-results from the ACRIN 6657/CALGB 150007 I-SPY 1 TRIAL. Radiology 279:44–55, 2016
Weis JA, Miga MI, Yankeelov TE: Three-dimensional image-based mechanical modeling for predicting the response of breast cancer to neoadjuvant therapy. Comput Methods Appl Mech Eng 314:494–512, 2017
Yankeelov TE: Integrating imaging data into predictive biomathematical and biophysical models of cancer. ISRN Biomath 2012:1–12, 2012
LeCun Y, Bengio Y, Hinton G: Deep learning. Nature 521:436–444, 2015
Hammond ME, Hayes DF, Dowsett M et al.: American Society of Clinical Oncology/College of American Pathologists guideline recommendations for immunohistochemical testing of estrogen and progesterone receptors in breast cancer. J Clin Oncol 28:2784–2795, 2010
Wolff AC, Hammond ME, Hicks DG, Dowsett M, McShane L, Allison KH, Allred DC, Bartlett JM, Bilous M, Fitzgibbons P, Hanna W, Jenkins RB, Mangu PB, Paik S, Perez EA, Press MF, Spears PA, Vance GH, Viale G, Hayes DF, American Society of Clinical Oncology, College of American Pathologists: Recommendations for human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology/College of American Pathologists clinical practice guideline update. Arch Pathol Lab Med 138:241–256, 2014
Pieper S, Halle M, Kikinis R. 3D Slicer. In: 2004 2nd IEEE International Symposium on Biomedical Imaging: Macro to Nano (IEEE Cat No 04EX821). IEEE; p. 632–635.
Kingma DP, Ba J. Adam: a method for stochastic optimization. arXiv:1412.6980
Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R: Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 15:1929–1958, 2014
Therasse P, Arbuck SG, Eisenhauer EA, Wanders J, Kaplan RS, Rubinstein L, Verweij J, van Glabbeke M, van Oosterom AT, Christian MC, Gwyther SG: New guidelines to evaluate the response to treatment in solid tumors. J Natl Cancer Inst 92:205–216, 2000
Li X, Arlinghaus LR, Ayers GD, Chakravarthy AB, Abramson RG, Abramson VG, Atuegwu N, Farley J, Mayer IA, Kelley MC, Meszoely IM, Means-Powell J, Grau AM, Sanders M, Bhave SR, Yankeelov TE: DCE-MRI analysis methods for predicting the response of breast cancer to neoadjuvant chemotherapy: pilot study findings. Magn Reson Med 71:1592–1602, 2014
Li X, Abramson RG, Arlinghaus LR, Kang H, Chakravarthy AB, Abramson VG, Farley J, Mayer IA, Kelley MC, Meszoely IM, Means-Powell J, Grau AM, Sanders M, Yankeelov TE: Multiparametric magnetic resonance imaging for predicting pathological response after the first cycle of neoadjuvant chemotherapy in breast cancer. Invest Radiol 50:195–204, 2015
Ah-See ML, Makris A, Taylor NJ et al.: Early changes in functional dynamic magnetic resonance imaging predict for pathologic response to neoadjuvant chemotherapy in primary breast cancer. Clin Cancer Res 14:6580–6589, 2008
Atuegwu NC, Arlinghaus LR, Li X, Chakravarthy AB, Abramson VG, Sanders ME, Yankeelov TE: Parameterizing the logistic model of tumor growth by DW-MRI and DCE-MRI data to predict treatment response and changes in breast cancer cellularity during neoadjuvant chemotherapy. Transl Oncol 6:256–264, 2013
Weis JA, Miga MI, Arlinghaus LR, Li X, Abramson V, Chakravarthy AB, Pendyala P, Yankeelov TE: Predicting the response of breast cancer to neoadjuvant therapy using a mechanically coupled reaction-diffusion model. Cancer Res 75:4697–4707, 2015
Ravichandran K, Braman N, Janowczyk A, Madabushi A: A deep learning classifier for prediction of pathological complete response to neoadjuvant chemotherapy from baseline breast DCE-MRI. SPIE medical imaging 2018: computer-aided diagnosis 105750C, 2018
Houssami N, Macaskill P, von Minckwitz G, Marinovich ML, Mamounas E: Meta-analysis of the association of breast cancer subtype and pathologic complete response to neoadjuvant chemotherapy. Eur J Cancer 48:3342–3352, 2012
Sorlie T, Perou CM, Tibshirani R, Aas T, Geisler S, Johnsen H, Hastie T, Eisen MB, van de Rijn M, Jeffrey SS, Thorsen T, Quist H, Matese JC, Brown PO, Botstein D, Lonning PE, Borresen-Dale AL: Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications. Proc Natl Acad Sci U S A 98:10869–10874, 2001
Carey LA, Perou CM, Livasy CA, Dressler LG, Cowan D, Conway K, Karaca G, Troester MA, Tse CK, Edmiston S, Deming SL, Geradts J, Cheang MCU, Nielsen TO, Moorman PG, Earp HS, Millikan RC: Race, breast cancer subtypes, and survival in the Carolina Breast Cancer Study. JAMA 295:2492–2502, 2006
Nguyen PL, Taghian AG, Katz MS, Niemierko A, Abi Raad RF, Boon WL, Bellon JR, Wong JS, Smith BL, Harris JR: Breast cancer subtype approximated by estrogen receptor, progesterone receptor, and HER-2 is associated with local and distant recurrence after breast-conserving therapy. J Clin Oncol. 26(14):2373–2378, 2008
Conflict of Interest
The authors declare that they have no conflict of interest.
About this article
Cite this article
Ha, R., Chin, C., Karcich, J. et al. Prior to Initiation of Chemotherapy, Can We Predict Breast Tumor Response? Deep Learning Convolutional Neural Networks Approach Using a Breast MRI Tumor Dataset. J Digit Imaging 32, 693–701 (2019). https://doi.org/10.1007/s10278-018-0144-1
- Breast MRI
- Chemotherapy treatment response
- Convolutional neural network