Skip to main content
SpringerLink
Log in

Application of Artificial Intelligence Computer-Assisted Diagnosis Originally Developed for Thyroid Nodules to Breast Lesions on Ultrasound

Journal of Digital Imaging volume 35, pages 1699–1707 (2022)Cite this article

Abstract

As thyroid and breast cancer have several US findings in common, we applied an artificial intelligence computer-assisted diagnosis (AI-CAD) software originally developed for thyroid nodules to breast lesions on ultrasound (US) and evaluated its diagnostic performance. From January 2017 to December 2017, 1042 breast lesions (mean size 20.2 ± 11.8 mm) of 1001 patients (mean age 45.9 ± 12.9 years) who underwent US-guided core-needle biopsy were included. An AI-CAD software that was previously trained and validated with thyroid nodules using the convolutional neural network was applied to breast nodules. There were 665 benign breast lesions (63.0%) and 391 breast cancers (37.0%). The area under the receiver operating characteristic curve (AUROC) of AI-CAD to differentiate breast lesions was 0.678 (95% confidence interval: 0.649, 0.707). After fine-tuning AI-CAD with 1084 separate breast lesions, the diagnostic performance of AI-CAD markedly improved (AUC 0.841). This was significantly higher than that of radiologists when the cutoff category was BI-RADS 4a (AUC 0.621, P < 0.001), but lower when the cutoff category was BI-RADS 4b (AUC 0.908, P < 0.001). When applied to breast lesions, the diagnostic performance of an AI-CAD software that had been developed for differentiating malignant and benign thyroid nodules was not bad. However, an organ-specific approach guarantees better diagnostic performance despite the similar US features of thyroid and breast malignancies.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Availability of Data and Materials

Additional documents related to this study are available on request to the corresponding author. However, the datasets from Severance Hospital were used under license for the current study and are not publicly available.

Code Availability

The homepage of SERA is “http://seracse.yonsei.ac.kr,” and it is only available through membership registration.

References

  1. Frates MC, Benson CB, Charboneau JW, Cibas ES, Clark OH, Coleman BG, Cronan JJ, Doubilet PM, Evans DB, Goellner JR: Management of thyroid nodules detected at US: Society of Radiologists in Ultrasound consensus conference statement. Radiology 237(3):794-800, 2005.

    Article  PubMed  Google Scholar 

  2. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, Pacini F, Randolph GW, Sawka AM, Schlumberger M: 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1):1-133, 2016.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Kim E-K, Park CS, Chung WY, Oh KK, Kim DI, Lee JT, Yoo HS: New sonographic criteria for recommending fine-needle aspiration biopsy of nonpalpable solid nodules of the thyroid. American Journal of Roentgenology 178(3):687-691, 2002.

    Article  PubMed  Google Scholar 

  4. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim E-K: Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology 260(3):892-899, 2011.

    Article  PubMed  Google Scholar 

  5. Moon W-J, Jung SL, Lee JH, Na DG, Baek J-H, Lee YH, Kim J, Kim HS, Byun JS, Lee DH: Benign and malignant thyroid nodules: US differentiation—multicenter retrospective study. Radiology 247(3):762-770, 2008.

    Article  PubMed  Google Scholar 

  6. Mendelson EB B-VM, Berg WA, et al.: ACR BI-RADS® Ultrasound. In: ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System. 5th ed. VA: Reston, 2013.

  7. Kim SH, Park CS, Jung SL, Kang BJ, Kim JY, Choi JJ, Kim YI, Oh JK, Oh JS, Kim H: Observer variability and the performance between faculties and residents: US criteria for benign and malignant thyroid nodules. Korean journal of radiology 11(2):149-155, 2010.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Ko SY, Lee JH, Yoon JH, Na H, Hong E, Han K, Jung I, Kim EK, Moon HJ, Park VY: Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound. Head & neck 41(4):885-891, 2019.

    Article  Google Scholar 

  9. Park VY, Han K, Seong YK, Park MH, Kim E-K, Moon HJ, Yoon JH, Kwak JY: Diagnosis of thyroid nodules: performance of a Deep Learning convolutional neural network Model vs. Radiologists. Scientific reports 9(1):1-9, 2019.

    Google Scholar 

  10. Lee E, Ha H, Kim HJ, Moon HJ, Byon JH, Huh S, Son J, Yoon J, Han K, Kwak JY: Differentiation of thyroid nodules on US using features learned and extracted from various convolutional neural networks. Scientific Reports 9(1):1-11, 2019.

    Article  Google Scholar 

  11. Gao L, Liu R, Jiang Y, Song W, Wang Y, Liu J, Wang J, Wu D, Li S, Hao A: Computer‐aided system for diagnosing thyroid nodules on ultrasound: a comparison with radiologist‐based clinical assessments. Head & neck 40(4):778-783, 2018.

    Article  Google Scholar 

  12. Ma J, Wu F, Zhu J, Xu D, Kong D: A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics 73:221-230, 2017.

    Article  PubMed  Google Scholar 

  13. Chi J, Walia E, Babyn P, Wang J, Groot G, Eramian M: Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. Journal of digital imaging 30(4):477-486, 2017.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Kim K, Song MK, Kim EK, Yoon JH: Clinical application of S-Detect to breast masses on ultrasonography: a study evaluating the diagnostic performance and agreement with a dedicated breast radiologist. Ultrasonography 36(1):3–9, 2017. https://doi.org/10.14366/usg.16012

  15. Cho E, Kim EK, Song MK, Yoon JH: Application of computer-aided diagnosis on breast ultrasonography: evaluation of diagnostic performances and agreement of radiologists according to different levels of experience. J Ultrasound Med 37(1):209-216, 2018. https://doi.org/10.1002/jum.14332

    Article  PubMed  Google Scholar 

  16. Choi JS, Han BK, Ko ES, Bae JM, Ko EY, Song SH, Kwon MR, Shin JH, Hahn SY: Effect of a deep learning framework-based computer-aided diagnosis system on the diagnostic performance of radiologists in differentiating between malignant and benign masses on breast ultrasonography. Korean J Radiol 20(5):749-758, 2019. https://doi.org/10.3348/kjr.2018.0530

    Article  PubMed  PubMed Central  Google Scholar 

  17. Koh J, Lee E, Han K, Kim EK, Son EJ, Sohn YM, Seo M, Kwon MR, Yoon JH, Lee JH, Park YM, Kim S, Shin JH, Kwak JY: Diagnosis of thyroid nodules on ultrasonography by a deep convolutional neural network. Sci Rep 10(1):15245, 2020. https://doi.org/10.1038/s41598-020-72270-6

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Houssami N, Ciatto S, Bilous M, Vezzosi V, Bianchi S: Borderline breast core needle histology: predictive values for malignancy in lesions of uncertain malignant potential (B3). British journal of cancer 96(8):1253-1257, 2007.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Lee SE, Park VY, Yoon JH, Moon HJ, Kim Mj, Kim E-K: Sonographically guided 14-guage core needle biopsy: medical audit for one year of 2017 Journal of Korean Society of Breast Screening 16:70–76, 2019.

  20. Lee SE, Park VY, Yoon JH, Moon HJ, Kim Mj, Kim E-K: Positive predictive value of breast ultrasonography BI-RADS category 4 and 5 lesions in one institution at 2017 Journal of Korean Society of Breast Screening 16:53–59, 2019.

  21. Lee SE, Han K, Kim E-K: Application of artificial intelligence–based computer-assisted diagnosis on synthetic mammograms from breast tomosynthesis: comparison with digital mammograms. European Radiology:1–9, 2021.

  22. Baba AI, Catoi C. Comparative oncology. Bucharest (RO): The Publishing House of the Romanian Academy; 2007

    Google Scholar 

  23. Cho N: Molecular subtypes and imaging phenotypes of breast cancer. Ultrasonography 35(4):281, 2016.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Park JW, Ko KH, Kim E-K, Kuzmiak CM, Jung HK: Non-mass breast lesions on ultrasound: final outcomes and predictors of malignancy. Acta Radiol 58(9):1054-1060, 2017.

    Article  PubMed  Google Scholar 

  25. Park HJ, Kim SM, La Yun B, Jang M, Kim B, Jang JY, Lee JY, Lee SH: A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of breast masses on ultrasound: added value for the inexperienced breast radiologist. Medicine 98(3):e14146-e14146, 2019. https://doi.org/10.1097/MD.0000000000014146

    Article  PubMed  PubMed Central  Google Scholar 

  26. Lee J, Kim S, Kang BJ, Kim SH, Park GE: Evaluation of the effect of computer aided diagnosis system on breast ultrasound for inexperienced radiologists in describing and determining breast lesions. Medical ultrasonography 21(3):7, 2019. https://doi.org/10.11152/mu-1889

Download references

Acknowledgements

The authors thank Hanpyo Hong, PhD for providing the expertise needed to perform the additional statistical analysis for revision and Medical Illustration and Design, part of the Medical Research Support Services of Yonsei University College of Medicine for their help in designing the figures.

Funding

This study was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (2019R1A2C1002375 and 2021R1A2C2007492).

Author information

Authors and Affiliations

  1. Department of Radiology, Yongin Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, Seoul, Korea

    Si Eun Lee & Eun-Kyung Kim

  2. Department of Computational Science and Engineering, Yonsei University, Seoul, Korea

    Eunjung Lee

  3. Department of Radiology, Severance Hospital, Research Institute of Radiological Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Korea

    Jung Hyun Yoon, Vivian Youngjean Park & Jin Young Kwak

  4. Department of Radiology, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul, Korea

    Ji Hyun Youk

Authors
  1. Si Eun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eunjung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eun-Kyung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jung Hyun Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vivian Youngjean Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ji Hyun Youk
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jin Young Kwak
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SEL and JYK contributed to the design and implementation of the research, to the analysis of the results, and to the writing of the manuscript. EL developed the AI-CAD program and obtained the results from the program. E-KK, JHY, VYP, and JHY participated in revision of the manuscript.

Corresponding author

Correspondence to Jin Young Kwak.

Ethics declarations

Disclaimer

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Ethics Approval

This retrospective study was approved by the institutional review board of Severance Hospital, Seoul, Korea, with a waiver for informed consent.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, S.E., Lee, E., Kim, EK. et al. Application of Artificial Intelligence Computer-Assisted Diagnosis Originally Developed for Thyroid Nodules to Breast Lesions on Ultrasound. J Digit Imaging 35, 1699–1707 (2022). https://doi.org/10.1007/s10278-022-00680-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-022-00680-1

Keywords

search

Navigation