To develop a dual-modal neural network model to characterize ultrasound (US) images of breast masses.
Materials and methods
A combined US B-mode and color Doppler neural network model was developed to classify US images of the breast. Three datasets with breast masses were originally detected and interpreted by 20 experienced radiologists according to Breast Imaging-Reporting and Data System (BI-RADS) lexicon ((1) training set, 103212 masses from 45,433 + 12,519 patients. (2) held-out validation set, 2748 masses from 1197 + 395 patients. (3) test set, 605 masses from 337 + 78 patients). The neural network was first trained on training set. Then, the trained model was tested on a held-out validation set to evaluate agreement on BI-RADS category between the model and the radiologists. In addition, the model and a reader study of 10 radiologists were applied to the test set with biopsy-proven results. To evaluate the performance of the model in benign or malignant classifications, the receiver operating characteristic curve, sensitivities, and specificities were compared.
The trained dual-modal model showed favorable agreement with the assessment performed by the radiologists (κ = 0.73; 95% confidence interval, 0.71–0.75) in classifying breast masses into four BI-RADS categories in the validation set. For the binary categorization of benign or malignant breast masses in the test set, the dual-modal model achieved the area under the ROC curve (AUC) of 0.982, while the readers scored an AUC of 0.948 in terms of the ROC convex hull.
The dual-modal model can be used to assess breast masses at a level comparable to that of an experienced radiologist.
• A neural network model based on ultrasonic imaging can classify breast masses into different Breast Imaging-Reporting and Data System categories according to the probability of malignancy.
• A combined ultrasonic B-mode and color Doppler neural network model achieved a high level of agreement with the readings of an experienced radiologist and has the potential to automate the routine characterization of breast masses.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Area under the ROC curve
Breast Imaging-Reporting and Data System
Convolutional neural network
Receiver operating characteristic curve
American Cancer Society (2017) Cancer Facts & Figures 2017. American Cancer Society, Atlanta.
Nothacker M, Duda V, Hahn M et al (2009) Early detection of breast cancer: benefits and risks of supplemental breast ultrasound in asymptomatic women with mammographically dense breast tissue. A systematic review. BMC Cancer 9:335
Mendelson E, Böhm-Vélez M, Berg W et al (2013) ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System. American College of Radiology, Reston
Lazarus E, Mainiero MB, Schepps B, Koelliker SL, Livingston LS (2006) BI-RADS lexicon for US and mammography: interobserver variability and positive predictive value. Radiology 239:385–391
Abdullah N, Mesurolle B, El-Khoury M, Kao E (2009) Breast imaging reporting and data system lexicon for US: interobserver agreement for assessment of breast masses. Radiology 252:665–672
Choi J-H, Kang BJ, Baek JE, Lee HS, Kim SH (2018) Application of computer-aided diagnosis in breast ultrasound interpretation: improvements in diagnostic performance according to reader experience. Ultrasonography 37:217
Gulshan V, Peng L, Coram M et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316:2402–2410
Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118
Fujioka T, Kubota K, Mori M et al (2019) Distinction between benign and malignant breast masses at breast ultrasound using deep learning method with convolutional neural network. Jpn J Radiol:1–7
Becker AS, Mueller M, Stoffel E, Marcon M, Ghafoor S, Boss A (2018) Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study. Br J Radiol 91:20170576
Ciritsis A, Rossi C, Eberhard M, Marcon M, Becker AS, Boss A (2019) Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. Eur Radiol 29:5458–5468
Han S, Kang H-K, Jeong J-Y et al (2017) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62:7714
Cheng J-Z, Ni D, Chou Y-H et al (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6:24454
Stead WW (2018) Clinical implications and challenges of artificial intelligence and deep learning. JAMA 320:1107–1108
Adler DD, Carson PL, Rubin JM, Quinn-Reid D (1990) Doppler ultrasound color flow imaging in the study of breast cancer: preliminary findings. Ultrasound Med Biol 16:553–559
Itoh A, Ueno E, Tohno E et al (2006) Breast disease: clinical application of US elastography for diagnosis. Radiology 239:341–350
Qian X, Ma T, Yu M, Chen X, Shung KK, Zhou Q (2017) Multi-functional ultrasonic micro-elastography imaging system. Sci Rep 7:1230
Xian M, Zhang Y, Cheng H-D (2015) Fully automatic segmentation of breast ultrasound images based on breast characteristics in space and frequency domains. Pattern Recogn 48:485–497
Gómez-Flores W, Ruiz-Ortega BA (2016) New fully automated method for segmentation of breast lesions on ultrasound based on texture analysis. Ultrasound Med Biol 42:1637–1650
Shen W-C, Chang R-F, Moon WK, Chou Y-H, Huang C-S (2007) Breast ultrasound computer-aided diagnosis using BI-RADS features. Acad Radiol 14:928–939
Gómez W, Pereira W, Infantosi AFC (2012) Analysis of co-occurrence texture statistics as a function of gray-level quantization for classifying breast ultrasound. IEEE Trans Med Imaging 31:1889–1899
Moon WK, Lo C-M, Chang JM, Huang C-S, Chen J-H, Chang R-F (2013) Quantitative ultrasound analysis for classification of BI-RADS category 3 breast masses. J Digit Imaging 26:1091–1098
Min-Chun Yang, Woo Kyung Moon, Wang YC et al (2013) Robust texture analysis using multi-resolution gray-scale invariant features for breast sonographic tumor diagnosis. IEEE Trans Med Imaging 32:2262–2273
Cho N, Jang M, Lyou CY, Park JS, Choi HY, Moon WK (2012) Distinguishing benign from malignant masses at breast US: combined US elastography and color Doppler US—influence on radiologist accuracy. Radiology 262:80–90
Raza S, Baum JK (1997) Solid breast lesions: evaluation with power Doppler US. Radiology 203:164–168
Saito T, Rehmsmeier M (2015) The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 10:e0118432
Moore SF, Barraclough K, Hamilton W (2018) Measuring health and illness: development and validation of tools. In: Jones R (Ed) Critical Appraisal for Primary Care, pp 24
Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9:2579–2605
Berg WA, Zhang Z, Lehrer D et al (2012) Detection of breast cancer with addition of annual screening ultrasound or a single screening MRI to mammography in women with elevated breast cancer risk. JAMA 307:1394–1404
Berg WA, Bandos AI, Mendelson EB, Lehrer D, Jong RA, Pisano ED (2015) Ultrasound as the primary screening test for breast cancer: analysis from ACRIN 6666. J Natil Cancer Inst 108:djv367
Jalalian A, Mashohor SB, Mahmud HR, Saripan MIB, Ramli ARB, Karasfi B (2013) Computer-aided detection/diagnosis of breast cancer in mammography and ultrasound: a review. Clin Imaging 37:420–426
Shi X, Cheng H-D, Hu L, Ju W, Tian J (2010) Detection and classification of masses in breast ultrasound images. Digital Signal Process 20:824–836
Shan J, Alam SK, Garra B, Zhang Y, Ahmed T (2016) Computer-aided diagnosis for breast ultrasound using computerized BI-RADS features and machine learning methods. Ultrasound Med Biol 42:980–988
Castelvecchi D (2016) Can we open the black box of AI? Nature News 538:20
The authors state that this work has not received any funding.
The scientific guarantor of this publication is Zeyu Chen.
Conflict of interest
The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.
Statistics and biometry
One of the authors has significant statistical expertise.
Written informed consent was waived by the Institutional Review Board.
Institutional Review Board approval was obtained.
• Multicenter study
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Qian, X., Zhang, B., Liu, S. et al. A combined ultrasonic B-mode and color Doppler system for the classification of breast masses using neural network. Eur Radiol (2020). https://doi.org/10.1007/s00330-019-06610-0
- B-mode ultrasound
- Color Doppler
- Breast mass
- Neural network
- Breast Imaging-Reporting and Data System categories