Skip to main content

Advertisement

SpringerLink
Log in

Artificial intelligence applied to breast pathology

  • Review
  • Published:
Virchows Archiv Aims and scope Submit manuscript

Abstract

The convergence of digital pathology and computer vision is increasingly enabling computers to perform tasks performed by humans. As a result, artificial intelligence (AI) is having an astoundingly positive effect on the field of pathology, including breast pathology. Research using machine learning and the development of algorithms that learn patterns from labeled digital data based on “deep learning” neural networks and feature-engineered approaches to analyze histology images have recently provided promising results. Thus far, image analysis and more complex AI-based tools have demonstrated excellent success performing tasks such as the quantification of breast biomarkers and Ki67, mitosis detection, lymph node metastasis recognition, tissue segmentation for diagnosing breast carcinoma, prognostication, computational assessment of tumor-infiltrating lymphocytes, and prediction of molecular expression as well as treatment response and benefit of therapy from routine H&E images. This review critically examines the literature regarding these applications of AI in the area of breast pathology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

source software QuPath. Red = cancer, purple = tumor infiltrating lymphocytes (TIL), green = fibrocyte, and yellow = other

Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Hamet P, Tremblay J (2017) Artificial intelligence in medicine. Metabolism 69:S36–S40. https://doi.org/10.1016/j.metabol.2017.01.011

    Article  CAS  Google Scholar 

  2. Holzinger A (2019) Introduction to MAchine Learning & Knowledge Extraction (MAKE). Mach Learn Knowl Extr 1:1–20. https://doi.org/10.3390/make1010001

    Article  Google Scholar 

  3. Basavanhally A, Feldman M, Shih N et al (2011) Multi-field-of-view strategy for image-based outcome prediction of multi-parametric estrogen receptor-positive breast cancer histopathology: comparison to Oncotype DX. J Pathol Inform 2:S1. https://doi.org/10.4103/2153-3539.92027

    Article  PubMed  Google Scholar 

  4. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539

    Article  CAS  PubMed  Google Scholar 

  5. Janowczyk A, Madabhushi A (2016) Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J Pathol Inform 7:29. https://doi.org/10.4103/2153-3539.186902

    Article  PubMed  PubMed Central  Google Scholar 

  6. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  7. Bejnordi BE, Zuidhof G, Balkenhol M et al (2017) Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. J Med Imaging Bellingham Wash 4:044504. https://doi.org/10.1117/1.JMI.4.4.044504

    Article  Google Scholar 

  8. Madabhushi A, Agner S, Basavanhally A et al (2011) Computer-aided prognosis: predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data. Comput Med Imaging Graph Off J Comput Med Imaging Soc 35:506–514. https://doi.org/10.1016/j.compmedimag.2011.01.008

  9. Lu MY, Williamson DFK, Chen TY et al (2021) Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng 5:555–570. https://doi.org/10.1038/s41551-020-00682-w

  10. Tellez D, Balkenhol M, Otte-Höller I et al (2018) Whole-slide mitosis detection in H E breast histology using PHH3 as a reference to train distilled stain-invariant convolutional networks. IEEE Trans Med Imaging 37:2126–2136. https://doi.org/10.1109/TMI.2018.2820199

    Article  Google Scholar 

  11. Romo-Bucheli D, Janowczyk A, Gilmore H et al (2017) A deep learning based strategy for identifying and associating mitotic activity with gene expression derived risk categories in estrogen receptor positive breast cancers. Cytom Part J Int Soc Anal Cytol 91:566–573. https://doi.org/10.1002/cyto.a.23065

    Article  CAS  Google Scholar 

  12. Lu C, Romo-Bucheli D, Wang X et al (2018) Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers. Lab Investig J Tech Methods Pathol 98:1438–1448. https://doi.org/10.1038/s41374-018-0095-7

    Article  CAS  Google Scholar 

  13. Allison KH, Hammond MEH, Dowsett M et al (2020) Estrogen and progesterone receptor testing in breast cancer: ASCO/CAP guideline update. J Clin Oncol Off J Am Soc Clin Oncol 38:1346–1366. https://doi.org/10.1200/JCO.19.02309

    Article  Google Scholar 

  14. Peck M, Moffat D, Latham B, Badrick T (2018) Review of diagnostic error in anatomical pathology and the role and value of second opinions in error prevention. J Clin Pathol 71:995–1000. https://doi.org/10.1136/jclinpath-2018-205226

    Article  PubMed  Google Scholar 

  15. Rimm DL, Leung SCY, McShane LM et al (2019) An international multicenter study to evaluate reproducibility of automated scoring for assessment of Ki67 in breast cancer. Mod Pathol Off J U S Can Acad Pathol Inc 32:59–69. https://doi.org/10.1038/s41379-018-0109-4

    Article  CAS  Google Scholar 

  16. Rexhepaj E, Brennan DJ, Holloway P et al (2008) Novel image analysis approach for quantifying expression of nuclear proteins assessed by immunohistochemistry: application to measurement of oestrogen and progesterone receptor levels in breast cancer. Breast Cancer Res BCR 10:R89. https://doi.org/10.1186/bcr2187

    Article  CAS  PubMed  Google Scholar 

  17. Tuominen VJ, Ruotoistenmäki S, Viitanen A et al (2010) ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67. Breast Cancer Res BCR 12:R56. https://doi.org/10.1186/bcr2615

    Article  PubMed  Google Scholar 

  18. Masmoudi H, Hewitt SM, Petrick N et al (2009) Automated quantitative assessment of HER-2/neu immunohistochemical expression in breast cancer. IEEE Trans Med Imaging 28:916–925. https://doi.org/10.1109/TMI.2009.2012901

  19. Hall BH, Ianosi-Irimie M, Javidian P et al (2008) Computer-assisted assessment of the human epidermal growth factor receptor 2 immunohistochemical assay in imaged histologic sections using a membrane isolation algorithm and quantitative analysis of positive controls. BMC Med Imaging 8:11. https://doi.org/10.1186/1471-2342-8-11

    Article  PubMed  PubMed Central  Google Scholar 

  20. Bolton KL, Garcia-Closas M, Pfeiffer RM et al (2010) Assessment of automated image analysis of breast cancer tissue microarrays for epidemiologic studies. Cancer Epidemiol Prev Biomark 19:992–999. https://doi.org/10.1158/1055-9965.EPI-09-1023

    Article  CAS  Google Scholar 

  21. Hartage R, Li A, Hammond S, Parwani A (2020) A validation study of human epidermal growth factor receptor 2 immunohistochemistry digital imaging analysis and its correlation with human epidermal growth factor receptor 2 fluorescence In situ hybridization results in breast carcinoma. J Pathol Inform 11:2. https://doi.org/10.4103/jpi.jpi_52_19

    Article  PubMed  PubMed Central  Google Scholar 

  22. Pantanowitz L, Liu C, Huang Y et al (2017) Impact of altering various image parameters on human epidermal growth factor receptor 2 image analysis data quality. J Pathol Inform 8:39. https://doi.org/10.4103/jpi.jpi_46_17

    Article  PubMed  PubMed Central  Google Scholar 

  23. Bui MM, Riben MW, Allison KH et al (2019) Quantitative image analysis of human epidermal growth factor receptor 2 immunohistochemistry for breast cancer: guideline from the College of American Pathologists. Arch Pathol Amp Lab Med 143:1180–1196. https://doi.org/10.5858/arpa.2018-0378-CP

    Article  Google Scholar 

  24. Longacre TA, Ennis M, Quenneville LA et al (2006) Interobserver agreement and reproducibility in classification of invasive breast carcinoma: an NCI breast cancer family registry study. Mod Pathol 19:195–207. https://doi.org/10.1038/modpathol.3800496

    Article  PubMed  Google Scholar 

  25. Elmore JG, Longton GM, Carney PA et al (2015) Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA 313:1122–1132. https://doi.org/10.1001/jama.2015.1405

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. van Baardwijk A, Bosmans G, Boersma L et al (2007) PET-CT–based auto-contouring in non–small-cell lung cancer correlates with pathology and reduces interobserver variability in the delineation of the primary tumor and involved nodal volumes. Int J Radiat Oncol Biol Phys 68:771–778. https://doi.org/10.1016/j.ijrobp.2006.12.067

    Article  PubMed  Google Scholar 

  27. Weaver DL (2010) Pathology evaluation of sentinel lymph nodes in breast cancer: protocol recommendations and rationale. Mod Pathol Off J U S Can Acad Pathol Inc 23(Suppl 2):S26–32. https://doi.org/10.1038/modpathol.2010.36

  28. Aresta G, Araújo T, Kwok S et al (2019) BACH: grand challenge on breast cancer histology images. Med Image Anal 56:122–139. https://doi.org/10.1016/j.media.2019.05.010

    Article  PubMed  Google Scholar 

  29. Polónia A, Campelos S, Ribeiro A et al (2020) Artificial Intelligence improves the accuracy in histologic classification of breast lesions. Am J Clin Pathol. https://doi.org/10.1093/ajcp/aqaa151

    Article  Google Scholar 

  30. Bejnordi BE, Balkenhol M, Litjens G et al (2016) Automated detection of DCIS in whole-slide H E stained breast histopathology images. IEEE Trans Med Imaging 35:2141–2150. https://doi.org/10.1109/TMI.2016.2550620

    Article  Google Scholar 

  31. Radiya-Dixit E, Zhu D, Beck AH (2017) Automated classification of benign and malignant proliferative breast lesions. Sci Rep 7:9900. https://doi.org/10.1038/s41598-017-10324-y

    Article  PubMed  PubMed Central  Google Scholar 

  32. Dong F, Irshad H, Oh E-Y et al (2014) Computational pathology to discriminate benign from malignant intraductal proliferations of the breast. PLoS ONE 9:e114885. https://doi.org/10.1371/journal.pone.0114885

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Wetstein SC, Stathonikos N, Pluim JPW et al (2021) Deep learning-based grading of ductal carcinoma in situ in breast histopathology images. Lab Investig J Tech Methods Pathol 101:525–533. https://doi.org/10.1038/s41374-021-00540-6

    Article  CAS  Google Scholar 

  34. Cruz-Roa A, Gilmore H, Basavanhally A et al (2017) Accurate and reproducible invasive breast cancer detection in whole-slide images: a deep learning approach for quantifying tumor extent. Sci Rep 7:46450. https://doi.org/10.1038/srep46450

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. van Rijthoven M, Balkenhol M, Siliņa K et al (2021) HookNet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images. Med Image Anal 68:101890. https://doi.org/10.1016/j.media.2020.101890

    Article  PubMed  Google Scholar 

  36. Kashyap A, Jain M, Shukla S, Andley M (2017) Study of nuclear morphometry on cytology specimens of benign and malignant breast lesions: a study of 122 cases. J Cytol 34:10–15. https://doi.org/10.4103/0970-9371.197591

    Article  PubMed  PubMed Central  Google Scholar 

  37. Osareh A, Shadgar B (2010) Machine learning techniques to diagnose breast cancer. In: 2010 5th International Symposium on Health Informatics and Bioinformatics. IEEE, Ankara, pp 114–120. https://doi.org/10.1109/HIBIT.2010.5478895

  38. Dey P, Logasundaram R, Joshi K (2013) Artificial neural network in diagnosis of lobular carcinoma of breast in fine-needle aspiration cytology. Diagn Cytopathol 41:102–106. https://doi.org/10.1002/dc.21773

    Article  PubMed  Google Scholar 

  39. Subbaiah RM, Dey P, Nijhawan R (2014) Artificial neural network in breast lesions from fine-needle aspiration cytology smear: ANN of Breast Carcinoma. Diagn Cytopathol 42:218–224. https://doi.org/10.1002/dc.23026

    Article  CAS  PubMed  Google Scholar 

  40. Filipczuk P, Fevens T, Krzyzak A, Monczak R (2013) Computer-aided breast cancer diagnosis based on the analysis of cytological images of fine needle biopsies. IEEE Trans Med Imaging 32:2169–2178. https://doi.org/10.1109/TMI.2013.2275151

    Article  PubMed  Google Scholar 

  41. Bloom HJ, Richardson WW (1957) Histological grading and prognosis in breast cancer; a study of 1409 cases of which 359 have been followed for 15 years. Br J Cancer 11:359–377. https://doi.org/10.1038/bjc.1957.43

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Genestie C, Zafrani B, Asselain B et al (1998) Comparison of the prognostic value of Scarff-Bloom-Richardson and Nottingham histological grades in a series of 825 cases of breast cancer: major importance of the mitotic count as a component of both grading systems. Anticancer Res 18:571–576

    CAS  PubMed  Google Scholar 

  43. Gurcan MN, Boucheron LE, Can A et al (2009) Histopathological image analysis: a review. IEEE Rev Biomed Eng 2:147–171. https://doi.org/10.1109/RBME.2009.2034865

    Article  PubMed  PubMed Central  Google Scholar 

  44. Roux L, Racoceanu D, Loménie N et al (2013) Mitosis detection in breast cancer histological images An ICPR 2012 contest. J Pathol Inform 4.https://doi.org/10.4103/2153-3539.112693

  45. Wang H, Cruz-Roa A, Basavanhally A et al (2014) Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J Med Imaging Bellingham Wash 1:034003. https://doi.org/10.1117/1.JMI.1.3.034003

    Article  Google Scholar 

  46. Veta M, Heng YJ, Stathonikos N et al (2019) Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge. Med Image Anal 54:111–121. https://doi.org/10.1016/j.media.2019.02.012

    Article  PubMed  Google Scholar 

  47. Veta M, van Diest PJ, Willems SM et al (2015) Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal 20:237–248. https://doi.org/10.1016/j.media.2014.11.010

    Article  PubMed  Google Scholar 

  48. Balkenhol MCA, Tellez D, Vreuls W et al (2019) Deep learning assisted mitotic counting for breast cancer. Lab Investig J Tech Methods Pathol 99:1596–1606. https://doi.org/10.1038/s41374-019-0275-0

    Article  Google Scholar 

  49. Pantanowitz L, Hartman D, Qi Y et al (2020) Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses. Diagn Pathol 15:80. https://doi.org/10.1186/s13000-020-00995-z

    Article  PubMed  PubMed Central  Google Scholar 

  50. Criscitiello C, Disalvatore D, De Laurentiis M et al (2014) High Ki-67 score is indicative of a greater benefit from adjuvant chemotherapy when added to endocrine therapy in luminal B HER2 negative and node-positive breast cancer. Breast Edinb Scotl 23:69–75. https://doi.org/10.1016/j.breast.2013.11.007

    Article  Google Scholar 

  51. Polley M-YC, Leung SCY, Gao D et al (2015) An international study to increase concordance in Ki67 scoring. Mod Pathol Off J U S Can Acad Pathol Inc 28:778–786. https://doi.org/10.1038/modpathol.2015.38

    Article  Google Scholar 

  52. Nielsen TO, Leung SCY, Rimm DL et al (2021) Assessment of Ki67 in breast cancer: updated recommendations from the International Ki67 in Breast Cancer Working Group. JNCI J Natl Cancer Inst 113:808–819. https://doi.org/10.1093/jnci/djaa201

    Article  PubMed  Google Scholar 

  53. Leung SCY, Nielsen TO, Zabaglo L et al (2016) Analytical validation of a standardized scoring protocol for Ki67: phase 3 of an international multicenter collaboration. NPJ Breast Cancer 2:16014. https://doi.org/10.1038/npjbcancer.2016.14

    Article  PubMed  PubMed Central  Google Scholar 

  54. Koopman T, Buikema HJ, Hollema H et al (2018) Digital image analysis of Ki67 proliferation index in breast cancer using virtual dual staining on whole tissue sections: clinical validation and inter-platform agreement. Breast Cancer Res Treat 169:33–42. https://doi.org/10.1007/s10549-018-4669-2

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Acs B, Pelekanou V, Bai Y et al (2019) Ki67 reproducibility using digital image analysis: an inter-platform and inter-operator study. Lab Investig J Tech Methods Pathol 99:107–117. https://doi.org/10.1038/s41374-018-0123-7

    Article  CAS  Google Scholar 

  56. Plancoulaine B, Laurinaviciene A, Herlin P et al (2015) A methodology for comprehensive breast cancer Ki67 labeling index with intra-tumor heterogeneity appraisal based on hexagonal tiling of digital image analysis data. Virchows Arch Int J Pathol. https://doi.org/10.1007/s00428-015-1865-x

    Article  Google Scholar 

  57. Zilenaite D, Rasmusson A, Augulis R et al (2020) Independent prognostic value of intratumoral heterogeneity and immune response features by automated digital immunohistochemistry analysis in early hormone receptor-positive breast carcinoma. Front Oncol 10:950. https://doi.org/10.3389/fonc.2020.00950

    Article  PubMed  PubMed Central  Google Scholar 

  58. Laurinavicius A, Plancoulaine B, Rasmusson A et al (2016) Bimodality of intratumor Ki67 expression is an independent prognostic factor of overall survival in patients with invasive breast carcinoma. Virchows Arch 468:493–502. https://doi.org/10.1007/s00428-016-1907-z

    Article  CAS  PubMed  Google Scholar 

  59. TNM Classification of Malignant Tumours, 7th edn, Wiley. In: Wiley.com. https://www.wiley.com/en-us/TNM+Classification+of+Malignant+Tumours%2C+7th+Edition-p-9781444358964. Accessed 19 Feb 2021

  60. Vestjens JHMJ, Pepels MJ, de Boer M et al (2012) Relevant impact of central pathology review on nodal classification in individual breast cancer patients. Ann Oncol Off J Eur Soc Med Oncol 23:2561–2566. https://doi.org/10.1093/annonc/mds072

    Article  CAS  Google Scholar 

  61. Litjens G, Sánchez CI, Timofeeva N et al (2016) Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep 6:26286. https://doi.org/10.1038/srep26286

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Ehteshami Bejnordi B, Veta M, Johannes van Diest P et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318:2199–2210. https://doi.org/10.1001/jama.2017.14585

    Article  PubMed  PubMed Central  Google Scholar 

  63. Liu Y, Kohlberger T, Norouzi M et al (2019) Artificial intelligence-based breast cancer nodal metastasis detection: insights into the black box for pathologists. Arch Pathol Lab Med 143:859–868. https://doi.org/10.5858/arpa.2018-0147-OA

    Article  CAS  PubMed  Google Scholar 

  64. Bandi P, Geessink O, Manson Q et al (2019) From detection of individual metastases to classification of lymph node status at the patient level: The CAMELYON17 challenge. IEEE Trans Med Imaging 38:550–560. https://doi.org/10.1109/TMI.2018.2867350

    Article  PubMed  Google Scholar 

  65. Steiner DF, MacDonald R, Liu Y et al (2018) Impact of deep learning assistance on the histopathologic review of lymph nodes for metastatic breast cancer. Am J Surg Pathol 42:1636–1646. https://doi.org/10.1097/PAS.0000000000001151

    Article  PubMed  PubMed Central  Google Scholar 

  66. Wiegand T, Krishnamurthy R, Kuglitsch M et al (2019) WHO and ITU establish benchmarking process for artificial intelligence in health. The Lancet 394:9–11. https://doi.org/10.1016/S0140-6736(19)30762-7

    Article  Google Scholar 

  67. Savas P, Salgado R, Denkert C et al (2016) Clinical relevance of host immunity in breast cancer: from TILs to the clinic. Nat Rev Clin Oncol 13:228–241. https://doi.org/10.1038/nrclinonc.2015.215

    Article  CAS  PubMed  Google Scholar 

  68. Adams S, Diamond JR, Hamilton EP et al (2016) Phase Ib trial of atezolizumab in combination with nab-paclitaxel in patients with metastatic triple-negative breast cancer (mTNBC). J Clin Oncol 34:1009–1009. https://doi.org/10.1200/JCO.2016.34.15_suppl.1009

    Article  Google Scholar 

  69. Nanda R, Chow LQM, Dees EC et al (2016) Pembrolizumab in patients with advanced triple-negative breast cancer: phase Ib KEYNOTE-012 study. J Clin Oncol Off J Am Soc Clin Oncol 34:2460–2467. https://doi.org/10.1200/JCO.2015.64.8931

    Article  CAS  Google Scholar 

  70. Denkert C, Loibl S, Noske A et al (2010) Tumor-associated lymphocytes as an independent predictor of response to neoadjuvant chemotherapy in breast cancer. J Clin Oncol Off J Am Soc Clin Oncol 28:105–113. https://doi.org/10.1200/JCO.2009.23.7370

    Article  CAS  Google Scholar 

  71. Amgad M, Elfandy H, Hussein H et al (2019) Structured crowdsourcing enables convolutional segmentation of histology images. Bioinformatics 35:3461–3467. https://doi.org/10.1093/bioinformatics/btz083

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Amgad M, Sarkar A, Srinivas C et al (2019) Joint region and nucleus segmentation for characterization of tumor infiltrating lymphocytes in breast cancer. Proc SPIE-- Int Soc Opt Eng 10956:109560M. https://doi.org/10.1117/12.2512892

  73. Amgad M, Stovgaard ES, Balslev E et al (2020) Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group. NPJ Breast Cancer 6:16. https://doi.org/10.1038/s41523-020-0154-2

    Article  PubMed  PubMed Central  Google Scholar 

  74. Basavanhally AN, Ganesan S, Agner S et al (2010) Computerized image-based detection and grading of lymphocytic infiltration in HER2+ breast cancer histopathology. IEEE Trans Biomed Eng 57:642–653. https://doi.org/10.1109/TBME.2009.2035305

    Article  PubMed  Google Scholar 

  75. Luen S, Virassamy B, Savas P et al (2016) The genomic landscape of breast cancer and its interaction with host immunity. Breast Edinb Scotl 29:241–250. https://doi.org/10.1016/j.breast.2016.07.015

    Article  Google Scholar 

  76. Rasmusson A, Zilenaite D, Nestarenkaite A et al (2020) Immunogradient indicators for antitumor response assessment by automated tumor-stroma interface zone detection. Am J Pathol 190:1309–1322. https://doi.org/10.1016/j.ajpath.2020.01.018

    Article  CAS  PubMed  Google Scholar 

  77. Saltz J, Gupta R, Hou L et al (2018) Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep 23:181-193.e7. https://doi.org/10.1016/j.celrep.2018.03.086

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  78. Heindl A, Sestak I, Naidoo K et al (2018) Relevance of spatial heterogeneity of immune infiltration for predicting risk of recurrence after endocrine therapy of ER+ breast cancer. JNCI J Natl Cancer Inst 110:166–175. https://doi.org/10.1093/jnci/djx137

    Article  CAS  Google Scholar 

  79. Maley CC, Koelble K, Natrajan R et al (2015) An ecological measure of immune-cancer colocalization as a prognostic factor for breast cancer. Breast Cancer Res BCR 17:131. https://doi.org/10.1186/s13058-015-0638-4

  80. Klein ME, Dabbs DJ, Shuai Y et al (2013) Prediction of the Onco type DX recurrence score: use of pathology-generated equations derived by linear regression analysis. Mod Pathol 26:658–664. https://doi.org/10.1038/modpathol.2013.36

    Article  PubMed  PubMed Central  Google Scholar 

  81. Basavanhally A, Jun Xu, Madabhushi A, Ganesan S (2009) Computer-aided prognosis of ER+ breast cancer histopathology and correlating survival outcome with Oncotype DX assay. In: 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro. IEEE, Boston, MA, pp 851–854. https://doi.org/10.1109/ISBI.2009.5193186

  82. Romo-Bucheli D, Janowczyk A, Gilmore H et al (2016) Automated tubule nuclei quantification and correlation with oncotype DX risk categories in ER+ Breast Cancer Whole Slide Images. Sci Rep 6:32706. https://doi.org/10.1038/srep32706

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  83. Whitney J, Corredor G, Janowczyk A et al (2018) Quantitative nuclear histomorphometry predicts oncotype DX risk categories for early stage ER+ breast cancer. BMC Cancer 18:610. https://doi.org/10.1186/s12885-018-4448-9

    Article  PubMed  PubMed Central  Google Scholar 

  84. Xu Z, Verma A, Naveed U et al (2021) Deep learning predicts chromosomal instability from histopathology images. iScience 24:102394. https://doi.org/10.1016/j.isci.2021.102394

  85. Meti N, Saednia K, Lagree A et al (2021) Machine learning frameworks to predict neoadjuvant chemotherapy response in breast cancer using clinical and pathological features. JCO Clin Cancer Inform 5:66–80. https://doi.org/10.1200/CCI.20.00078

  86. Qu Y, Zhu H, Cao K et al (2020) Prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer using a deep learning (DL) method. Thorac Cancer 11:651–658. https://doi.org/10.1111/1759-7714.13309

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  87. Samek W, Müller K-R (2019) Towards explainable artificial intelligence. In: Samek W, Montavon G, Vedaldi A et al (eds) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer International Publishing, Cham, pp 5–22

    Chapter  Google Scholar 

  88. Binder A, Bockmayr M, Hägele M et al (2021) Morphological and molecular breast cancer profiling through explainable machine learning. Nat Mach Intell 3:355–366. https://doi.org/10.1038/s42256-021-00303-4

    Article  Google Scholar 

  89. Ja R, Ml G, Pl F et al (2013) A call to standardize preanalytic data elements for biospecimens. Arch Pathol Lab Med 138:526–537. https://doi.org/10.5858/arpa.2013-0250-cp

    Article  Google Scholar 

  90. Rawat RR, Ortega I, Roy P et al (2020) Deep learned tissue “fingerprints” classify breast cancers by ER/PR/Her2 status from H&E images. Sci Rep 10:7275. https://doi.org/10.1038/s41598-020-64156-4

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  91. Naik N, Madani A, Esteva A et al (2020) Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains. Nat Commun 11:5727. https://doi.org/10.1038/s41467-020-19334-3

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  92. Anand D, Kurian N, Dhage S et al (2020) Deep learning to estimate human epidermal growth factor receptor 2 status from hematoxylin and eosin-stained breast tissue images. J Pathol Inform 11:19. https://doi.org/10.4103/jpi.jpi_10_20

    Article  PubMed  PubMed Central  Google Scholar 

  93. Shamai G, Binenbaum Y, Slossberg R et al (2019) Artificial intelligence algorithms to assess hormonal status from tissue microarrays in patients with breast cancer. JAMA Netw Open 2:e197700. https://doi.org/10.1001/jamanetworkopen.2019.7700

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank Yalai Bai for help with Figures 6 and 7, as well as Haojia Li, Yuli Chen, and Daniel Shao for help with Figures 1, 6, and 9; and Benoit Plancoulaine and Allan Rasmusson for help with Figures 3 and 8.

Author information

Authors and Affiliations

  1. Department of Pathology, University of Michigan, Ann Arbor, MI, USA

    Mustafa Yousif & Liron Pantanowitz

  2. Department of Pathology, Vanderbilt University Medical Center, Nashville, TN, USA

    Mustafa Yousif

  3. Department of Pathology, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands

    Paul J. van Diest

  4. Department of Pathology, Pharmacology and Forensic Medicine, Faculty of Medicine, Vilnius University, and National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Klinikos, Vilnius, Lithuania

    Arvydas Laurinavicius

  5. Department of Pathology, Yale University School of Medicine, New Haven, CT, USA

    David Rimm

  6. Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands

    Jeroen van der Laak

  7. Department of Biomedical Engineering, Case Western Reserve University, and Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA

    Anant Madabhushi

  8. Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA

    Anant Madabhushi

  9. Department of Pathology, Brigham and Women’s Hospital and Harvard Medical School, Breast Oncology Program, Dana-Farber/Brigham and Women’s Cancer Center, Boston, MA, USA

    Stuart Schnitt

Authors
  1. Mustafa Yousif
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul J. van Diest
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arvydas Laurinavicius
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Rimm
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeroen van der Laak
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anant Madabhushi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stuart Schnitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Liron Pantanowitz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The paper was conceived by Stuart Schnitt, outline developed by Liron Pantanowitz, first draft written by Mustafa Yousif, and all authors reviewed the draft and contributed equally to the final version of the paper.

No IRB required.

Corresponding author

Correspondence to Mustafa Yousif.

Ethics declarations

Conflict of interest

Paul J. van Diest serves on the scientific advisory of Sectra (non-paid). Arvydas Laurinavicius is an independent scientific advisor (non-paid) to the portal https://pathologynews.com/, and a co-author on international patent application (no commercial interest). Anant Madabhushi is an equity holder in Elucid Bioimaging and in Inspirata Inc. In addition, Madabhushi has served as a scientific advisory board member for Inspirata Inc., Astrazeneca, Bristol Meyers-Squibb, and Merck. Currently, he serves on the advisory board of Aiforia Inc., has sponsored research agreements with Philips, AstraZeneca, Boehringer-Ingelheim, and Bristol Meyers-Squibb. Madabhushi’s technology has been licensed to Elucid Bioimaging and he is also involved in a NIH U24 grant with PathCore Inc, and 3 different R01 grants with Inspirata Inc. Liron Pantanowitz is on the scientific advisory board for Ibex and NTP and serves as a consultant for Hamamatsu. David L. Rimm has served as an advisor for Astra Zeneca, Agendia, Amgen, BMS, Cell Signaling Technology, Cepheid, Danaher, Daiichi Sankyo, Konica Minolta, Merck, NanoString, PAIGE.AI, Perkin Elmer, Roche, Sanofi, Ventana, and Ultivue. Amgen, Cepheid, NavigateBP, NextCure, and Konica Minolta fund research in David L. Rimm’s lab. Stuart J. Schnitt is on the scientific advisory boards of PathAI and Ibex. Jeroen van der Laak is a member of the advisory boards of Philips, The Netherlands, and ContextVision, Sweden, and received research funding from Philips, The Netherlands; ContextVision, Sweden; and Sectra, Sweden in the last five years.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yousif, M., van Diest, P.J., Laurinavicius, A. et al. Artificial intelligence applied to breast pathology. Virchows Arch 480, 191–209 (2022). https://doi.org/10.1007/s00428-021-03213-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00428-021-03213-3

Keywords

Search

Navigation