Skip to main content

Advertisement

SpringerLink
Log in

Ultrasound-based deep learning in the establishment of a breast lesion risk stratification system: a multicenter study

  • Breast
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To establish a breast lesion risk stratification system using ultrasound images to predict breast malignancy and assess Breast Imaging Reporting and Data System (BI-RADS) categories simultaneously.

Methods

This multicenter study prospectively collected a dataset of ultrasound images for 5012 patients at thirty-two hospitals from December 2018 to December 2020. A deep learning (DL) model was developed to conduct binary categorization (benign and malignant) and BI-RADS categories (2, 3, 4a, 4b, 4c, and 5) simultaneously. The training set of 4212 patients and the internal test set of 416 patients were from thirty hospitals. The remaining two hospitals with 384 patients were used as an external test set. Three experienced radiologists performed a reader study on 324 patients randomly selected from the test sets. We compared the performance of the DL model with that of three radiologists and the consensus of the three radiologists.

Results

In the external test set, the DL model achieved areas under the receiver operating characteristic curve (AUCs) of 0.980 and 0.945 for the binary categorization and six-way categorizations, respectively. In the reader study set, the DL BI-RADS categories achieved a similar AUC (0.901 vs. 0.933, p = 0.0632), sensitivity (90.98% vs. 95.90%, p = 0.1094), and accuracy (83.33% vs. 79.01%, p = 0.0541), but higher specificity (78.71% vs. 68.81%, p = 0.0012) than those of the consensus of the three radiologists.

Conclusions

The DL model performed well in distinguishing benign from malignant breast lesions and yielded outcomes similar to experienced radiologists. This indicates the potential applicability of the DL model in clinical diagnosis.

Key Points

• The DL model can achieve binary categorization for benign and malignant breast lesions and six-way BI-RADS categorizations for categories 2, 3, 4a, 4b, 4c, and 5, simultaneously.

• The DL model showed acceptable agreement with radiologists for the classification of breast lesions.

• The DL model performed well in distinguishing benign from malignant breast lesions and had promise in helping reduce unnecessary biopsies of BI-RADS 4a lesions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

ACR:

American College of Radiology

AI:

Artificial intelligence

AUC:

Area under the receiver operating characteristic curve

BI-RADS:

Breast Imaging Reporting and Data System

CAD:

Computer-aided diagnosis

CNN:

Convolutional neural network

DL:

Deep learning

ETC:

External test cohort

ITC:

Internal test cohort

ML:

Machine learning

NPV:

Negative predictive value

PPV:

Positive predictive value

TC:

Training cohort

US:

Ultrasound

References

  1. Harbeck N, Gnant M (2017) Breast cancer. Lancet 389:1134–1150

    Article  PubMed  Google Scholar 

  2. Lei S, Zheng R, Zhang S et al (2021) Breast cancer incidence and mortality in women in China: temporal trends and projections to 2030. Cancer Biol Med 18:900–909

    Article  PubMed  PubMed Central  Google Scholar 

  3. Zheng C, Yu ZG, Chinese Society of Breast S (2021) Clinical practice guidelines for pre-operative evaluation of breast cancer: Chinese Society of Breast Surgery (CSBrS) practice guidelines 2021. Chin Med J 134:2147–2149

    Article  PubMed  PubMed Central  Google Scholar 

  4. Hooley RJ, Scoutt LM, Philpotts LE (2013) Breast ultrasonography: state of the art. Radiology 268:642–659

    Article  PubMed  Google Scholar 

  5. Chang JM, Leung JWT, Moy L, Ha SM, Moon WK (2020) Axillary nodal evaluation in breast cancer: state of the art. Radiology 295:500–515

    Article  PubMed  Google Scholar 

  6. D’Orsi C, Sickles E, Mendelson E, Morris E et al (2013) ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System. 5th ed. American College of Radiology, Reston, VA

    Google Scholar 

  7. Raza S, Chikarmane SA, Neilsen SS, Zorn LM, Birdwell RL (2008) BI-RADS 3, 4, and 5 lesions: value of US in management--follow-up and outcome. Radiology 248:773–781

    Article  PubMed  Google Scholar 

  8. Raza S, Goldkamp AL, Chikarmane SA, Birdwell RL (2010) US of breast masses categorized as BI-RADS 3, 4, and 5: pictorial review of factors influencing clinical management. Radiographics 30:1199–1213

    Article  PubMed  Google Scholar 

  9. Berg WA, Cosgrove DO, Dore CJ et al (2012) Shear-wave elastography improves the specificity of breast US: the BE1 multinational study of 939 masses. Radiology 262:435–449

    Article  PubMed  Google Scholar 

  10. Lazarus E, Mainiero MB, Schepps B, Koelliker SL, Livingston LS (2006) BI-RADS lexicon for US and mammography: interobserver variability and positive predictive value. Radiology 239:385–391

    Article  PubMed  Google Scholar 

  11. Abdullah N, Mesurolle B, El-Khoury M, Kao E (2009) Breast imaging reporting and data system lexicon for US: interobserver agreement for assessment of breast masses. Radiology 252:665–672

    Article  PubMed  Google Scholar 

  12. Menezes GLG, Pijnappel RM, Meeuwis C et al (2018) Downgrading of breast masses suspicious for cancer by using optoacoustic breast imaging. Radiology 288:355–365

    Article  PubMed  Google Scholar 

  13. Berg WA, Blume JD, Cormack JB et al (2008) Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer. JAMA 299:2151–2163

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Nothacker M, Duda V, Hahn M et al (2009) Early detection of breast cancer: benefits and risks of supplemental breast ultrasound in asymptomatic women with mammographically dense breast tissue. A systematic review. BMC Cancer 9:335

    Article  PubMed  PubMed Central  Google Scholar 

  15. Berg WA (2020) Reducing unnecessary biopsy and follow-up of benign cystic breast lesions. Radiology 295:52–53

    Article  PubMed  Google Scholar 

  16. Han S, Kang HK, Jeong JY et al (2017) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62:7714–7728

    Article  PubMed  Google Scholar 

  17. Zhuang Z, Yang Z, Zhuang S, Joseph Raj AN, Yuan Y, Nersisson R (2021) Multi-features-based automated breast tumor diagnosis using ultrasound image and support vector machine. Comput Intell Neurosci 2021:9980326

    Article  PubMed  PubMed Central  Google Scholar 

  18. Shia WC, Lin LS, Chen DR (2021) Classification of malignant tumours in breast ultrasound using unsupervised machine learning approaches. Sci Rep 11:1418

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Shia WC, Chen DR (2021) Classification of malignant tumors in breast ultrasound using a pretrained deep residual network model and support vector machine. Comput Med Imaging Graph 87:101829

    Article  PubMed  Google Scholar 

  20. Romeo V, Cuocolo R, Apolito R et al (2021) Clinical value of radiomics and machine learning in breast ultrasound: a multicenter study for differential diagnosis of benign and malignant lesions. Eur Radiol 31:9511–9519

    Article  PubMed  PubMed Central  Google Scholar 

  21. Huo L, Tan Y, Wang S et al (2021) Machine learning models to improve the differentiation between benign and malignant breast lesions on ultrasound: a multicenter external validation study. Cancer Manag Res 13:3367–3379

    Article  PubMed  PubMed Central  Google Scholar 

  22. Kalafi EY, Jodeiri A, Setarehdan SK et al (2021) Classification of breast cancer lesions in ultrasound images by using attention layer and loss ensemble in deep convolutional neural networks. Diagn (Basel) 11:1859

    Google Scholar 

  23. Qian X, Pei J, Zheng H et al (2021) Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning. Nat Biomed Eng 5:522–532

    Article  PubMed  Google Scholar 

  24. Shen Y, Shamout FE, Oliver JR et al (2021) Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams. Nat Commun 12:5645

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Xing J, Chen C, Lu Q et al (2021) Using BI-RADS stratifications as auxiliary information for breast masses classification in ultrasound images. IEEE J Biomed Health Inform 25:2058–2070

    Article  PubMed  Google Scholar 

  26. Shen WC, Chang RF, Moon WK (2007) Computer aided classification system for breast ultrasound based on Breast Imaging Reporting and Data System (BI-RADS). Ultrasound Med Biol 33:1688–1698

    Article  PubMed  Google Scholar 

  27. Huang Y, Han L, Dou H et al (2019) Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images. Biomed Eng Online 18:8

    Article  PubMed  PubMed Central  Google Scholar 

  28. Ciritsis A, Rossi C, Eberhard M, Marcon M, Becker AS, Boss A (2019) Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. Eur Radiol 29:5458–5468

    Article  PubMed  Google Scholar 

  29. Qian X, Zhang B, Liu S et al (2020) A combined ultrasonic B-mode and color Doppler system for the classification of breast masses using neural network. Eur Radiol 30:3023–3033

    Article  PubMed  Google Scholar 

  30. Zhang H, Han L, Chen K, Peng Y, Lin J (2020) Diagnostic efficiency of the breast ultrasound computer-aided prediction model based on convolutional neural network in breast cancer. J Digit Imaging 33:1218–1223

    Article  PubMed  PubMed Central  Google Scholar 

  31. Qi X, Zhang L, Chen Y et al (2019) Automated diagnosis of breast ultrasonography images using deep neural networks. Med Image Anal 52:185–198

    Article  PubMed  Google Scholar 

  32. Liu J, Li W, Zhao N et al (2018) Integrate domain knowledge in training CNN for ultrasonography breast cancer diagnosis. Springer International Publishing, Cham, pp 868–875

    Google Scholar 

  33. Al-Dhabyani W, Gomaa M, Khaled H, Fahmy A (2020) Dataset of breast ultrasound images. Data Brief 28:104863

    Article  PubMed  Google Scholar 

  34. Ding X, Zhang X, Ma N, Han J, Ding G, Sun J (2021) RepVGG: Making VGG-style ConvNets Great Again. https://arxiv.org/abs/2101.03697. Accessed 19 Apr 2021

  35. Spinelli VMA, Teixeira DCJ, Rauber A, Varella IS, Fleck JF, Moreira LF (2018) Role of BI-RADS ultrasound subcategories 4A to 4C in predicting breast cancer. Clin Breast Cancer 18:e507–e511

    Article  Google Scholar 

  36. Stavros AT, Freitas AG, deMello GGN et al (2017) Ultrasound positive predictive values by BI-RADS categories 3-5 for solid masses: an independent reader study. Eur Radiol 27:4307–4315

    Article  PubMed  Google Scholar 

  37. Fu CY, Hsu HH, Yu JC et al (2011) Influence of age on PPV of sonographic BI-RADS categories 3, 4, and 5. Ultraschall Med 32(Suppl 1):S8–S13

  38. Yoon JH, Kim MJ, Moon HJ, Kwak JY, Kim EK (2011) Subcategorization of ultrasonographic BI-RADS category 4: positive predictive value and clinical factors affecting it. Ultrasound Med Biol 37:693–699

    Article  PubMed  Google Scholar 

  39. Lee HJ, Kim EK, Kim MJ et al (2008) Observer variability of Breast Imaging Reporting and Data System (BI-RADS) for breast ultrasound. Eur J Radiol 65:293–298

    Article  PubMed  Google Scholar 

  40. Jales RM, Sarian LO, Torresan R, Marussi EF, Alvares BR, Derchain S (2013) Simple rules for ultrasonographic subcategorization of BI-RADS(R)-US 4 breast masses. Eur J Radiol 82:1231–1235

    Article  PubMed  Google Scholar 

  41. He P, Cui LG, Chen W, Yang RL (2019) Subcategorization of ultrasonographic BI-RADS category 4: assessment of diagnostic accuracy in diagnosing breast lesions and influence of clinical factors on positive predictive value. Ultrasound Med Biol 45:1253–1258

    Article  PubMed  Google Scholar 

  42. Hu Y, Yang Y, Gu R et al (2018) Does patient age affect the PPV3 of ACR BI-RADS Ultrasound categories 4 and 5 in the diagnostic setting? Eur Radiol 28:2492–2498

    Article  PubMed  Google Scholar 

  43. Lee YJ, Choi SY, Kim KS, Yang PS (2016) Variability in observer performance between faculty members and residents using Breast Imaging Reporting and Data System (BI-RADS)-Ultrasound, Fifth Edition (2013). Iran J Radiol 13:e28281

    Article  PubMed  PubMed Central  Google Scholar 

  44. Park CS, Kim SH, Jung NY, Choi JJ, Kang BJ, Jung HS (2015) Interobserver variability of ultrasound elastography and the ultrasound BI-RADS lexicon of breast lesions. Breast Cancer 22:153–160

    Article  PubMed  Google Scholar 

Download references

Funding

This work is supported by the Beijing Natural Science Foundation (7202156), and the Foundation of International Health Exchange and Cooperation Center NHC PRC (ihecc2018C0032-2).

Author information

Author notes

  1. Hongyan Wang and Yuxin Jiang contributed equally to this work.

Authors and Affiliations

  1. Department of Ultrasound, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, No.1 Shuai Fu Yuan, Dong Cheng District, Beijing, 100730, China

    Yang Gu, Wen Xu, Jianchu Li, Hongyan Wang & Yuxin Jiang

  2. Department of Medical Imaging Advanced Research, Beijing Research Institute, Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Beijing, China

    Ting Liu, Xing An & Longfei Cong

  3. Department of Ultrasound, The Second Affiliated Hospital of Harbin Medical University, Harbin, China

    Jiawei Tian

  4. Department of Ultrasound, The Second Affiliated Hospital of Chongqing Medical University & Chongqing Key Laboratory of Ultrasound Molecular Imaging, Chongqing, China

    Haitao Ran

  5. Department of Ultrasound, Shengjing Hospital of China Medical University, Shenyang, China

    Weidong Ren

  6. Department of Medical Ultrasound, Fudan University Shanghai Cancer Center, Fudan University, Shanghai, China

    Cai Chang

  7. Department of Ultrasonography, Henan Provincial People′s Hospital, Zhengzhou, China

    Jianjun Yuan

  8. Department of Ultrasound, Shanxi Bethune Hospital, Shanxi Academy of Medical Sciences, Taiyuan, China

    Chunsong Kang

  9. Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College of Huazhong University of Science and Technology, Wuhan, China

    Youbin Deng

  10. Department of Ultrasound, China-Japan Union Hospital of Jilin University, Changchun, China

    Hui Wang

  11. Department of Ultrasound, The Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China

    Baoming Luo

  12. Department of Ultrasonography, First Affiliated Hospital of Guangxi Medical University, Nanning, China

    Shenglan Guo

  13. Department of Medical Ultrasound, The Second Affiliated Hospital, School of Medicine, Xi’an Jiaotong University, Xi’an, China

    Qi Zhou

  14. Department of Ultrasound, Union Hospital of Fujian Medical University, Fujian Institute of Ultrasound Medicine, Fuzhou, China

    Ensheng Xue

  15. Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University, School of Medicine, Shanghai, China

    Weiwei Zhan

  16. Department of Ultrasonography, Renmin Hospital of Wuhan University, Wuhan, China

    Qing Zhou

  17. Department of Ultrasound, Qilu Hospital, Shandong University, Jinan, China

    Jie Li

  18. Department of Ultrasound, The Third Xiangya Hospital of Central South University, Changsha, China

    Ping Zhou

  19. Department of Ultrasound Medicine, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China

    Man Chen

  20. Department of Ultrasonography, The Affiliated Hospital of Guizhou Medical University, Guiyang, China

    Ying Gu

  21. Department of Ultrasound, The First Hospital of Shanxi Medical University, Taiyuan, China

    Wu Chen

  22. Department of Ultrasound, The Second Hospital of Dalian Medical University, Dalian, China

    Yuhong Zhang

  23. Department of Medical Imaging Advanced Research, Shenzhen Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China

    Lei Zhu

Authors
  1. Yang Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ting Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xing An
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiawei Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haitao Ran
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Weidong Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Cai Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jianjun Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Chunsong Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Youbin Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Baoming Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Shenglan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Qi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Ensheng Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Weiwei Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Qing Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Jie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Ping Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Man Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Ying Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Wu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Yuhong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Jianchu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Longfei Cong
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Lei Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Hongyan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  29. Yuxin Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Hongyan Wang or Yuxin Jiang.

Ethics declarations

Guarantor

The scientific guarantors of this publication are Hongyan Wang and Yuxin Jiang.

Conflict of interest

Four of the authors are engineers in Shenzhen Mindray Bio-Medical Electronics Co., Ltd, which provides the ultrasound system and technical support to our research.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was obtained from all patients before they underwent US.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• prospective

• diagnostic study

• multicenter study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(DOCX 736 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gu, Y., Xu, W., Liu, T. et al. Ultrasound-based deep learning in the establishment of a breast lesion risk stratification system: a multicenter study. Eur Radiol 33, 2954–2964 (2023). https://doi.org/10.1007/s00330-022-09263-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-022-09263-8

Keywords

Search

Navigation