Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A combined ultrasonic B-mode and color Doppler system for the classification of breast masses using neural network



To develop a dual-modal neural network model to characterize ultrasound (US) images of breast masses.

Materials and methods

A combined US B-mode and color Doppler neural network model was developed to classify US images of the breast. Three datasets with breast masses were originally detected and interpreted by 20 experienced radiologists according to Breast Imaging-Reporting and Data System (BI-RADS) lexicon ((1) training set, 103212 masses from 45,433 + 12,519 patients. (2) held-out validation set, 2748 masses from 1197 + 395 patients. (3) test set, 605 masses from 337 + 78 patients). The neural network was first trained on training set. Then, the trained model was tested on a held-out validation set to evaluate agreement on BI-RADS category between the model and the radiologists. In addition, the model and a reader study of 10 radiologists were applied to the test set with biopsy-proven results. To evaluate the performance of the model in benign or malignant classifications, the receiver operating characteristic curve, sensitivities, and specificities were compared.


The trained dual-modal model showed favorable agreement with the assessment performed by the radiologists (κ = 0.73; 95% confidence interval, 0.71–0.75) in classifying breast masses into four BI-RADS categories in the validation set. For the binary categorization of benign or malignant breast masses in the test set, the dual-modal model achieved the area under the ROC curve (AUC) of 0.982, while the readers scored an AUC of 0.948 in terms of the ROC convex hull.


The dual-modal model can be used to assess breast masses at a level comparable to that of an experienced radiologist.

Key Points

• A neural network model based on ultrasonic imaging can classify breast masses into different Breast Imaging-Reporting and Data System categories according to the probability of malignancy.

• A combined ultrasonic B-mode and color Doppler neural network model achieved a high level of agreement with the readings of an experienced radiologist and has the potential to automate the routine characterization of breast masses.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5



Area under the ROC curve


Breast Imaging-Reporting and Data System


Confidence interval


Convolutional neural network


Receiver operating characteristic curve




  1. 1.

    American Cancer Society (2017) Cancer Facts & Figures 2017. American Cancer Society, Atlanta.

  2. 2.

    Nothacker M, Duda V, Hahn M et al (2009) Early detection of breast cancer: benefits and risks of supplemental breast ultrasound in asymptomatic women with mammographically dense breast tissue. A systematic review. BMC Cancer 9:335

  3. 3.

    Mendelson E, Böhm-Vélez M, Berg W et al (2013) ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System. American College of Radiology, Reston

  4. 4.

    Lazarus E, Mainiero MB, Schepps B, Koelliker SL, Livingston LS (2006) BI-RADS lexicon for US and mammography: interobserver variability and positive predictive value. Radiology 239:385–391

  5. 5.

    Abdullah N, Mesurolle B, El-Khoury M, Kao E (2009) Breast imaging reporting and data system lexicon for US: interobserver agreement for assessment of breast masses. Radiology 252:665–672

  6. 6.

    Choi J-H, Kang BJ, Baek JE, Lee HS, Kim SH (2018) Application of computer-aided diagnosis in breast ultrasound interpretation: improvements in diagnostic performance according to reader experience. Ultrasonography 37:217

  7. 7.

    Gulshan V, Peng L, Coram M et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316:2402–2410

  8. 8.

    Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118

  9. 9.

    Fujioka T, Kubota K, Mori M et al (2019) Distinction between benign and malignant breast masses at breast ultrasound using deep learning method with convolutional neural network. Jpn J Radiol:1–7

  10. 10.

    Becker AS, Mueller M, Stoffel E, Marcon M, Ghafoor S, Boss A (2018) Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study. Br J Radiol 91:20170576

  11. 11.

    Ciritsis A, Rossi C, Eberhard M, Marcon M, Becker AS, Boss A (2019) Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. Eur Radiol 29:5458–5468

  12. 12.

    Han S, Kang H-K, Jeong J-Y et al (2017) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62:7714

  13. 13.

    Cheng J-Z, Ni D, Chou Y-H et al (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6:24454

  14. 14.

    Stead WW (2018) Clinical implications and challenges of artificial intelligence and deep learning. JAMA 320:1107–1108

  15. 15.

    Adler DD, Carson PL, Rubin JM, Quinn-Reid D (1990) Doppler ultrasound color flow imaging in the study of breast cancer: preliminary findings. Ultrasound Med Biol 16:553–559

  16. 16.

    Itoh A, Ueno E, Tohno E et al (2006) Breast disease: clinical application of US elastography for diagnosis. Radiology 239:341–350

  17. 17.

    Qian X, Ma T, Yu M, Chen X, Shung KK, Zhou Q (2017) Multi-functional ultrasonic micro-elastography imaging system. Sci Rep 7:1230

  18. 18.

    Xian M, Zhang Y, Cheng H-D (2015) Fully automatic segmentation of breast ultrasound images based on breast characteristics in space and frequency domains. Pattern Recogn 48:485–497

  19. 19.

    Gómez-Flores W, Ruiz-Ortega BA (2016) New fully automated method for segmentation of breast lesions on ultrasound based on texture analysis. Ultrasound Med Biol 42:1637–1650

  20. 20.

    Shen W-C, Chang R-F, Moon WK, Chou Y-H, Huang C-S (2007) Breast ultrasound computer-aided diagnosis using BI-RADS features. Acad Radiol 14:928–939

  21. 21.

    Gómez W, Pereira W, Infantosi AFC (2012) Analysis of co-occurrence texture statistics as a function of gray-level quantization for classifying breast ultrasound. IEEE Trans Med Imaging 31:1889–1899

  22. 22.

    Moon WK, Lo C-M, Chang JM, Huang C-S, Chen J-H, Chang R-F (2013) Quantitative ultrasound analysis for classification of BI-RADS category 3 breast masses. J Digit Imaging 26:1091–1098

  23. 23.

    Min-Chun Yang, Woo Kyung Moon, Wang YC et al (2013) Robust texture analysis using multi-resolution gray-scale invariant features for breast sonographic tumor diagnosis. IEEE Trans Med Imaging 32:2262–2273

  24. 24.

    Cho N, Jang M, Lyou CY, Park JS, Choi HY, Moon WK (2012) Distinguishing benign from malignant masses at breast US: combined US elastography and color Doppler US—influence on radiologist accuracy. Radiology 262:80–90

  25. 25.

    Raza S, Baum JK (1997) Solid breast lesions: evaluation with power Doppler US. Radiology 203:164–168

  26. 26.

    Saito T, Rehmsmeier M (2015) The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 10:e0118432

  27. 27.

    Moore SF, Barraclough K, Hamilton W (2018) Measuring health and illness: development and validation of tools. In: Jones R (Ed) Critical Appraisal for Primary Care, pp 24

  28. 28.

    Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605

  29. 29.

    Berg WA, Zhang Z, Lehrer D et al (2012) Detection of breast cancer with addition of annual screening ultrasound or a single screening MRI to mammography in women with elevated breast cancer risk. JAMA 307:1394–1404

  30. 30.

    Berg WA, Bandos AI, Mendelson EB, Lehrer D, Jong RA, Pisano ED (2015) Ultrasound as the primary screening test for breast cancer: analysis from ACRIN 6666. J Natil Cancer Inst 108:djv367

  31. 31.

    Jalalian A, Mashohor SB, Mahmud HR, Saripan MIB, Ramli ARB, Karasfi B (2013) Computer-aided detection/diagnosis of breast cancer in mammography and ultrasound: a review. Clin Imaging 37:420–426

  32. 32.

    Shi X, Cheng H-D, Hu L, Ju W, Tian J (2010) Detection and classification of masses in breast ultrasound images. Digital Signal Process 20:824–836

  33. 33.

    Shan J, Alam SK, Garra B, Zhang Y, Ahmed T (2016) Computer-aided diagnosis for breast ultrasound using computerized BI-RADS features and machine learning methods. Ultrasound Med Biol 42:980–988

  34. 34.

    Castelvecchi D (2016) Can we open the black box of AI? Nature News 538:20

Download references


The authors state that this work has not received any funding.

Author information

Correspondence to Zeyu Chen.

Ethics declarations


The scientific guarantor of this publication is Zeyu Chen.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.


• Retrospective

• Multicenter study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Qian, X., Zhang, B., Liu, S. et al. A combined ultrasonic B-mode and color Doppler system for the classification of breast masses using neural network. Eur Radiol (2020). https://doi.org/10.1007/s00330-019-06610-0

Download citation


  • B-mode ultrasound
  • Color Doppler
  • Breast mass
  • Neural network
  • Breast Imaging-Reporting and Data System categories