Skip to main content

Advertisement

SpringerLink
Log in

Mammographic Density Assessment by Artificial Intelligence-Based Computer-Assisted Diagnosis: A Comparison with Automated Volumetric Assessment

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

We evaluated and compared the mammographic density assessment of an artificial intelligence-based computer-assisted diagnosis (AI-CAD) program using inter-rater agreements between radiologists and an automated density assessment program. Between March and May 2020, 488 consecutive mammograms of 488 patients (56.2 ± 10.9 years) were collected from a single institution. We assigned four classes of mammographic density based on BI-RADS (Breast Imaging Reporting and Data System) using commercial AI-CAD (Lunit INSIGHT MMG), and compared inter-rater agreements between radiologists, AI-CAD, and another commercial automated density assessment program (Volpara®). The inter-rater agreement between AI-CAD and the reader consensus was 0.52 with a matched rate of 68.2% (333/488). The inter-rater agreement between Volpara® and the reader consensus was similar to AI-CAD at 0.50 with a matched rate of 62.7% (306/488). The inter-rater agreement between AI-CAD and Volpara® was 0.54 with a matched rate of 61.5% (300/488). In conclusion, density assessments by AI-CAD showed fair agreement with those of radiologists, similar to the agreement between the commercial automated density assessment program and radiologists.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Availability of Data and Material

Additional documents related to this study are available on request to the corresponding author. However, the datasets from Yongin Severance Hospital were used under license for the current study and are not publicly available. The AI algorithm developed from this study is available through a commercial product, Lunit INSIGHT MMG, and can be freely experienced through an online demo (https://www.lunit.io/ko/ product/insight_mmg).

Code Availability

We used commercial software.

References

  1. Marmot MG, Altman DG, Cameron DA, Dewar JA, Thompson SG, Wilcox M: The benefits and harms of breast cancer screening: an independent review. Br J Cancer 108(11):2205-2240, 2013. https://doi.org/10.1038/bjc.2013.177

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Tabar L, Vitak B, Chen TH, Yen AM, Cohen A, Tot T, Chiu SY, Chen SL, Fann JC, Rosell J, Fohlin H, Smith RA, Duffy SW: Swedish two-county trial: impact of mammographic screening on breast cancer mortality during 3 decades. Radiology 260(3):658-663, 2011. https://doi.org/10.1148/radiol.11110469

    Article  PubMed  Google Scholar 

  3. Boyd NF, Byng J, Jong R, Fishell E, Little L, Miller A, Lockwood G, Tritchler D, Yaffe MJ: Quantitative classification of mammographic densities and breast cancer risk: results from the Canadian National Breast Screening Study. JNCI: Journal of the National Cancer Institute 87(9):670–675, 1995.

  4. Carney PA, Miglioretti DL, Yankaskas BC, Kerlikowske K, Rosenberg R, Rutter CM, Geller BM, Abraham LA, Taplin SH, Dignan M: Individual and combined effects of age, breast density, and hormone replacement therapy use on the accuracy of screening mammography. Annals of internal medicine 138(3):168-175, 2003.

    Article  Google Scholar 

  5. Sprague BL, Conant EF, Onega T, Garcia MP, Beaber EF, Herschorn SD, Lehman CD, Tosteson AN, Lacson R, Schnall MD: Variation in mammographic breast density assessments among radiologists in clinical practice: a multicenter observational study. Annals of internal medicine 165(7):457-464, 2016.

    Article  Google Scholar 

  6. Spayne MC, Gard CC, Skelly J, Miglioretti DL, Vacek PM, Geller BM: Reproducibility of BI-RADS breast density measures among community radiologists: a prospective cohort study. The breast journal 18(4):326-333, 2012. https://doi.org/10.1111/j.1524-4741.2012.01250.x

    Article  PubMed  PubMed Central  Google Scholar 

  7. Youk JH, Gweon HM, Son EJ, Kim JA: Automated Volumetric Breast Density Measurements in the Era of the BI-RADS Fifth Edition: A Comparison With Visual Assessment. AJR Am J Roentgenol 206(5):1056-1062, 2016. https://doi.org/10.2214/ajr.15.15472

    Article  PubMed  Google Scholar 

  8. Irshad A, Leddy R, Ackerman S, Cluver A, Pavic D, Abid A, Lewis MC: Effects of Changes in BI-RADS Density Assessment Guidelines (Fourth Versus Fifth Edition) on Breast Density Assessment: Intra- and Interreader Agreements and Density Distribution. American Journal of Roentgenology 207(6):1366–1371, 2016. https://doi.org/10.2214/AJR.16.16561

  9. Flack VF, Afifi A, Lachenbruch P, Schouten H: Sample size determinations for the two rater kappa statistic. Psychometrika 53(3):321-325, 1988.

    Article  Google Scholar 

  10. Landis JR, Koch GG: The measurement of observer agreement for categorical data. biometrics:159–174, 1977.

  11. Singh T, Sharma M, Singla V, Khandelwal N: Breast Density Estimation with Fully Automated Volumetric Method: Comparison to Radiologists' Assessment by BI-RADS Categories. Acad Radiol 23(1):78-83, 2016. https://doi.org/10.1016/j.acra.2015.09.012

    Article  PubMed  Google Scholar 

  12. Brandt KR, Scott CG, Ma L, Mahmoudzadeh AP, Jensen MR, Whaley DH, Wu FF, Malkov S, Hruska CB, Norman AD, Heine J, Shepherd J, Pankratz VS, Kerlikowske K, Vachon CM: Comparison of Clinical and Automated Breast Density Measurements: Implications for Risk Prediction and Supplemental Screening. Radiology 279(3):710-719, 2016. https://doi.org/10.1148/radiol.2015151261

    Article  PubMed  Google Scholar 

  13. Moshina N, Roman M, Sebuødegård S, Waade GG, Ursin G, Hofvind S: Comparison of subjective and fully automated methods for measuring mammographic density. Acta Radiol 59(2):154-160, 2018. https://doi.org/10.1177/0284185117712540

    Article  PubMed  Google Scholar 

  14. Eom HJ, Cha JH, Kang JW, Choi WJ, Kim HJ, Go E: Comparison of variability in breast density assessment by BI-RADS category according to the level of experience. Acta Radiol 59(5):527-532, 2018. https://doi.org/10.1177/0284185117725369

    Article  PubMed  Google Scholar 

  15. Lehman CD, Yala A, Schuster T, Dontchos B, Bahl M, Swanson K, Barzilay R: Mammographic Breast Density Assessment Using Deep Learning: Clinical Implementation. Radiology 290(1):52-58, 2019. https://doi.org/10.1148/radiol.2018180694

    Article  PubMed  Google Scholar 

  16. Mohamed AA, Berg WA, Peng H, Luo Y, Jankowitz RC, Wu S: A deep learning method for classifying mammographic breast density categories. Med Phys 45(1):314-321, 2018. https://doi.org/10.1002/mp.12683

    Article  PubMed  Google Scholar 

  17. Wu N, Phang J, Park J, Shen Y, Huang Z, Zorin M, Jastrzębski S, Févry T, Katsnelson J, Kim E: Deep neural networks improve radiologists’ performance in breast cancer screening. IEEE transactions on medical imaging 39(4):1184-1194, 2019.

    Article  Google Scholar 

  18. Matthews TP, Singh S, Mombourquette B, Su J, Shah MP, Pedemonte S, Long A, Maffit D, Gurney J, Hoil RM, Ghare N, Smith D, Moore SM, Marks SC, Wahl RL: A Multisite Study of a Breast Density Deep Learning Model for Full-Field Digital Mammography and Synthetic Mammography. Radiology: Artificial Intelligence 3(1):e200015, 2021. https://doi.org/10.1148/ryai.2020200015

  19. Chang K, Beers AL, Brink L, Patel JB, Singh P, Arun NT, Hoebel KV, Gaw N, Shah M, Pisano ED, Tilkin M, Coombs LP, Dreyer KJ, Allen B, Agarwal S, Kalpathy-Cramer J: Multi-Institutional Assessment and Crowdsourcing Evaluation of Deep Learning for Automated Classification of Breast Density. Journal of the American College of Radiology 17(12):1653–1662, 2020. https://doi.org/10.1016/j.jacr.2020.05.015

  20. Kim H-E, Kim HH, Han B-K, Kim KH, Han K, Nam H, Lee EH, Kim E-K: Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. The Lancet Digital Health 2(3):e138-e148, 2020. https://doi.org/10.1016/S2589-7500(20)30003-0

    Article  PubMed  Google Scholar 

  21. El-Bastawissi AY, White E, Mandelson MT, Taplin S: Variation in Mammographic Breast Density by Race. Annals of Epidemiology 11(4):257–263, 2001. https://doi.org/10.1016/S1047-2797(00)00225-8

  22. del Carmen MG, Halpern EF, Kopans DB, Moy B, Moore RH, Goss PE, Hughes KS: Mammographic Breast Density and Race. American Journal of Roentgenology 188(4):1147-1150, 2007. https://doi.org/10.2214/AJR.06.0619

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yongin Severance Hospital, Yonsei University College of Medicine, Yongin, Gyeonggi-do, Korea

    Si Eun Lee, Myung Hyun Kim & Eun-Kyung Kim

  2. Division of Biostatistics, Yongin Severance Hospital, Yonsei University College of Medicine, Gyeonggi-do, Yongin, Republic of Korea

    Nak-Hoon Son

Authors
  1. Si Eun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nak-Hoon Son
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Myung Hyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eun-Kyung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

E-K.K. and S.E.L. contributed to the design and implementation of the research, to the analysis of the results, and to the writing of the manuscript. N.-H.S. performed statistical analysis. M.H.K. participated in reader study and revised the manuscript.

Corresponding author

Correspondence to Eun-Kyung Kim.

Ethics declarations

Ethics Approval

This retrospective study was approved by the institutional review board of Yongin Severance Hospital, Yongin, Gyeonggi-do, South Korea, with a waiver for informed consent.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 85 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, S.E., Son, NH., Kim, M.H. et al. Mammographic Density Assessment by Artificial Intelligence-Based Computer-Assisted Diagnosis: A Comparison with Automated Volumetric Assessment. J Digit Imaging 35, 173–179 (2022). https://doi.org/10.1007/s10278-021-00555-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00555-x

Keywords

Search

Navigation