Skip to main content

Advertisement

SpringerLink
Log in

The diagnostic performance of ultrasound computer-aided diagnosis system for distinguishing breast masses: a prospective multicenter study

  • Breast
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To evaluate the diagnostic value of computer-aided diagnosis (CAD) software on ultrasound in distinguishing benign and malignant breast masses and avoiding unnecessary biopsy.

Methods

This prospective, multicenter study included patients who were scheduled for pathological diagnosis of breast masses between April 2019 and November 2020. Ultrasound images, videos, CAD analysis, and BI-RADS were obtained. The AUC, accuracy, sensitivity, specificity, PPV, and NPV were calculated and compared with radiologists.

Results

Overall, 901 breast masses in 901 patients were enrolled in this study. The accuracy, sensitivity, specificity, PPV and NPV of CAD software were 89.6%, 94.2%, 87.0%, 80.4%, and 96.3, respectively, in the long-axis section; 89.0%, 91.4%, 87.7%, 80.8%, and 94.7%, respectively, in the short-axis section. With BI-RADS 4a as the cut-off value, CAD software has a higher AUC (0.906 vs 0.734 vs 0.696, all p < 0.001) than both experienced and less experienced radiologists. With BI-RADS 4b as the cut-off value, CAD software showed better AUC than less experienced radiologists (0.906 vs 0.874, p < 0.001), but not superior to experienced radiologists (0.906 vs 0.883, p = 0.057). After the application of CAD software, the unnecessary biopsy rate of BI-RADS categories 4 and 5 was significantly decreased (33.0% vs 11.9%, 37.8% vs 14.5%), and the malignant rate of biopsy in category 4a was significantly increased (11.6% vs 40.7%, 7.4% vs 34.9%, all p < 0.001).

Conclusions

CAD software on ultrasound can be used as an effective auxiliary diagnostic tool for differential diagnosis of benign and malignant breast masses and reducing unnecessary biopsy.

Clinical trial registration

ClinicalTrials.gov (NCT 03887598)

Key Points

• Prospective multicenter study showed that computer-aided diagnosis software provides greater diagnostic confidence for differentiating benign and malignant breast masses.

• Computer-aided diagnosis software can help radiologists reduce unnecessary biopsy.

• The management of patients with breast masses becomes more appropriate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

AUC:

Area under the receiver operating characteristic curve

BI-RADS:

Breast Imaging Reporting and Data System

CAD:

Computer-aided diagnosis

NPV:

Negative predictive value

PPV:

Positive predictive value

US:

Ultrasound

References

  1. Sung H, Ferlay J, Siegel RL et al (2021) Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. https://doi.org/10.3322/caac.21660

    Article  PubMed  Google Scholar 

  2. World Health Organization (2014) WHO Position Paper on Mammography Screening. World Health Organization, Geneva. Available via https://www.ncbi.nlm.nih.gov/books/NBK269545. Accessed 25 Apr 2021

  3. Oeffinger KC, Fontham ET, Etzioni R et al (2015) Breast Cancer Screening for Women at Average Risk: 2015 Guideline Update From the American Cancer Society. JAMA 314:1599–1614

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Kolb TM, Lichy J, Newhouse JH (2002) Comparison of the performance of screening mammography, physical examination, and breast US and evaluation of factors that influence them: an analysis of 27,825 patient evaluations. Radiology 225:165–175

    Article  PubMed  Google Scholar 

  5. Schäfgen B, Juskic M, Radicke M et al (2021) Evaluation of the FUSION-X-US-II prototype to combine automated breast ultrasound and tomosynthesis. Eur Radiol 31:3712–3720

    Article  PubMed  Google Scholar 

  6. Lin X, Jia M, Zhou X et al (2021) The diagnostic performance of automated versus handheld breast ultrasound and mammography in symptomatic outpatient women: a multicenter, cross-sectional study in China. Eur Radiol 31:947–957

    Article  PubMed  Google Scholar 

  7. Hooley RJ, Scoutt LM, Philpotts LE (2013) Breast ultrasonography: state of the art. Radiology 268:642–659

    Article  PubMed  Google Scholar 

  8. D’Orsi C, Sickles E, Mendelson E, Morris E (2013) ACR BI-RADS® Atlas, breast imaging reporting and data system, 5th edn. American College of Radiology, Reston

    Google Scholar 

  9. Elverici E, Barça AN, Aktaş H et al (2015) Nonpalpable BI-RADS 4 breast lesions: sonographic findings and pathology correlation. Diagn Interv Radiol 21:189–194

    Article  PubMed  PubMed Central  Google Scholar 

  10. Choi JS, Han BK, Ko ES et al (2019) Effect of a deep learning framework-based computer-aided diagnosis system on the diagnostic performance of radiologists in differentiating between malignant and benign masses on breast ultrasonography. Korean J Radiol 20:749–758

    Article  PubMed  PubMed Central  Google Scholar 

  11. Wei Q, Zeng SE, Wang LP et al (2020) The value of S-Detect in improving the diagnostic performance of radiologists for the differential diagnosis of thyroid nodules. Med Ultrason 22:415–423

    Article  PubMed  Google Scholar 

  12. Kim K, Song MK, Kim EK, Yoon JH (2017) Clinical application of S-Detect to breast masses on ultrasonography: a study evaluating the diagnostic performance and agreement with a dedicated breast radiologist. Ultrasonography 36:3–9

    Article  PubMed  Google Scholar 

  13. Cho E, Kim EK, Song MK, Yoon JH (2018) Application of computer-aided diagnosis on breast ultrasonography: evaluation of diagnostic performances and agreement of radiologists according to different levels of experience. J Ultrasound Med 37:209–216

    Article  PubMed  Google Scholar 

  14. Di Segni M, de Soccio V, Cantisani V et al (2018) Automated classification of focal breast lesions according to S-detect: validation and role as a clinical and teaching tool. J Ultrasound 21:105–118

    Article  PubMed  PubMed Central  Google Scholar 

  15. Zhao C, Xiao M, Liu H et al (2020) Reducing the number of unnecessary biopsies of US-BI-RADS 4a lesions through a deep learning method for residents-in-training: a cross-sectional study. BMJ Open 10:e035757

    Article  PubMed  PubMed Central  Google Scholar 

  16. Wang XY, Cui LG, Feng J, Chen W (2021) Artificial intelligence for breast ultrasound: an adjunct tool to reduce excessive lesion biopsy. Eur J Radiol 138:109624

    Article  PubMed  Google Scholar 

  17. Lee SH, Chung J, Choi HY et al (2017) Evaluation of screening us-detected breast masses by combined use of elastography and color Doppler US with B-Mode US in women with dense breasts: a multicenter prospective study. Radiology 285:660–669

    Article  PubMed  Google Scholar 

  18. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845

    Article  CAS  PubMed  Google Scholar 

  19. Jalalian A, Mashohor S, Mahmud R, Karasfi B, Saripan MIB, Ramli ARB (2017) Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection. EXCLI J 16:113–137

  20. Choi JH, Kang BJ, Baek JE, Lee HS, Kim SH (2018) Application of computer-aided diagnosis in breast ultrasound interpretation: improvements in diagnostic performance according to reader experience. Ultrasonography 37:217–225

    Article  PubMed  Google Scholar 

  21. Zhao C, Xiao M, Jiang Y et al (2019) Feasibility of computer-assisted diagnosis for breast ultrasound: the results of the diagnostic performance of S-detect from a single center in China. Cancer Manag Res 11:921–930

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Wu JY, Zhao ZZ, Zhang WY et al (2019) Computer-aided diagnosis of solid breast lesions with ultrasound: factors associated with false-negative and false-positive results. J Ultrasound Med 38:3193–3202

    Article  PubMed  Google Scholar 

  23. Xiao M, Zhao C, Li J et al (2020) Diagnostic value of breast lesions between deep learning-based computer-aided diagnosis system and experienced radiologists: comparison the performance between symptomatic and asymptomatic patients. Front Oncol 10:1070

    Article  PubMed  PubMed Central  Google Scholar 

  24. Lee J, Kim S, Kang BJ, Kim SH, Park GE (2019) Evaluation of the effect of computer aided diagnosis system on breast ultrasound for inexperienced radiologists in describing and determining breast lesions. Med Ultrason 21:239–245

    Article  PubMed  Google Scholar 

  25. Yongping L, Juan Z, Zhou P, Yongfeng Z, Liu W, Shi Y (2020) Evaluation of the Quadri-planes method in computer-aided diagnosis of breast lesions by ultrasonography: prospective single-center study. JMIR Med Inform 8:e18251

    Article  PubMed  PubMed Central  Google Scholar 

  26. Xiao M, Zhao C, Zhu Q et al (2019) An investigation of the classification accuracy of a deep learning framework-based computer-aided diagnosis system in different pathological types of breast lesions. J Thorac Dis 11:5023–5031

    Article  PubMed  PubMed Central  Google Scholar 

  27. Menezes GLG, Pijnappel RM, Meeuwis C et al (2018) Downgrading of breast masses suspicious for cancer by using optoacoustic breast imaging. Radiology 288:355–365

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank all radiologists of the hospitals for assisting with the collection of the imaging data used in this study.

Funding

This study has received funding from the Tongji Hospital (HUST) Foundation for Excellent Young Scientist (Grant No. 2020YQ01), National Natural Science Foundation of China (Grant No. 82071953), and Key R&D Projects of Science and Technology of Hubei Province in 2020 (Grant No. 2020BCB022).

Author information

Authors and Affiliations

  1. Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, No. 1095, Jiefang Avenue, Wuhan, 430030, Hubei Province, China

    Qi Wei, Yu-Jing Yan, Ge-Ge Wu & Xin-Wu Cui

  2. Department of Medical Ultrasound, The Central Hospital of EDong Healthcare, Huangshi, 435000, Hubei Province, China

    Xi-Rong Ye

  3. Department of Medical Ultrasound, The Second Hospital of Anhui Medical University, Hefei, 230601, Anhui Province, China

    Fan Jiang

  4. Department of Medical Ultrasound, Yichang General Hospital, Renmin Hospital of Three Gorges University, Hubei Province, Yichang, 443099, China

    Jie Liu

  5. Department of Medical Ultrasound, Taizhou Hospital of Zhejiang Province, Linhai, 318000, Zhejiang Province, China

    Gang Wang & Chao-Ying Jin

  6. Department of Medical Ultrasound, Macheng People’s Hospital, Macheng, 438300, Hubei Province, China

    Yi Wang & Xiang Wang

  7. Department of Medical Ultrasound, Xiangyang No. 1 People’s Hospital, Affiliated Hospital of Hubei University of Medicine, Xiangyang, 441000, Hubei Province, China

    Juan Song

  8. Department of Medical Ultrasound, Yixing Traditional Chinese Medicine Hospital, Yixing, 214200, Jiangsu Province, China

    Zhi-Ping Pan

  9. Department of Medical Ultrasound, Anqing First People’s Hospital of Anhui Medical University, Anqing, 246052, Anhui Province, China

    Jin-Hua Hu

  10. Department of Internal Medicine, Hirslanden Clinic, Schänzlihalde 11, 3013, Bern, Switzerland

    Christoph F. Dietrich

Authors
  1. Qi Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu-Jing Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ge-Ge Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xi-Rong Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fan Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Juan Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Zhi-Ping Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jin-Hua Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Chao-Ying Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Xiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Christoph F. Dietrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Xin-Wu Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin-Wu Cui.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Xin-Wu Cui.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was obtained from all subjects (patients) in this study.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• prospective

• diagnostic or prognostic study

• multicenter study

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, Q., Yan, YJ., Wu, GG. et al. The diagnostic performance of ultrasound computer-aided diagnosis system for distinguishing breast masses: a prospective multicenter study. Eur Radiol 32, 4046–4055 (2022). https://doi.org/10.1007/s00330-021-08452-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-021-08452-1

Keywords

Search

Navigation