Mammographic density: Comparison of visual assessment with fully automatic calculation on a multivendor dataset

Abstract

Objectives

To compare breast density (BD) assessment provided by an automated BD evaluator (ABDE) with that provided by a panel of experienced breast radiologists, on a multivendor dataset.

Methods

Twenty-one radiologists assessed 613 screening/diagnostic digital mammograms from nine centers and six different vendors, using the BI-RADS a, b, c, and d density classification. The same mammograms were also evaluated by an ABDE providing the ratio between fibroglandular and total breast area on a continuous scale and, automatically, the BI-RADS score. A panel majority report (PMR) was used as reference standard. Agreement (κ) and accuracy (proportion of cases correctly classified) were calculated for binary (BI-RADS a-b versus c-d) and 4-class classification.

Results

While the agreement of individual radiologists with the PMR ranged from κ = 0.483 to κ = 0.885, the ABDE correctly classified 563/613 mammograms (92 %). A substantial agreement for binary classification was found for individual reader pairs (κ = 0.620, standard deviation [SD] = 0.140), individual versus PMR (κ = 0.736, SD = 0.117), and individual versus ABDE (κ = 0.674, SD = 0.095). Agreement between ABDE and PMR was almost perfect (κ = 0.831).

Conclusions

The ABDE showed an almost perfect agreement with a 21-radiologist panel in binary BD classification on a multivendor dataset, earning a chance as a reproducible alternative to visual evaluation.

Key Points

Individual BD assessment differs from PMR with κ as low as 0.483.

An ABDE correctly classified 92 % of mammograms with almost perfect agreement (κ = 0.831).

An ABDE can be a valid alternative to subjective BD assessment.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Abbreviations

ABDE:

Automated breast density evaluator

BD:

Breast density

PMR:

Panel majority report

SD:

Standard deviation

References

  1. 1.

    Ursin G, Qureshi SA (2009) Mammographic density – a useful biomarker for breast cancer risk in epidemiologic studies. Norsk Epidemiol 19:59–68

    Google Scholar 

  2. 2.

    Colin C, Schott AM, Valette PJ (2014) Mammographic density is not a worthwhile examination to distinguish high cancer risk women in screening. Eur Radiol 24:2412–2416

    Article  PubMed  Google Scholar 

  3. 3.

    Buist DS, Porter PL, Lehman C, Taplin SH, White E (2004) Factors contributing to mammography failure in women aged 40–49 years. J Natl Cancer Inst 96:1432–1440

    Article  PubMed  Google Scholar 

  4. 4.

    Kavanagh AM, Byrnes GB, Nickson C et al (2008) English DR. Using mammographic density to improve breast cancer screening outcomes. Cancer Epidemiol Biomarkers Prev 17:2818–2824

    Article  PubMed  Google Scholar 

  5. 5.

    Kolb TM, Lichy J, Newhouse JH (2002) Comparison of the performance of screening mammography, physical examination, and breast US and evaluation of factors that influence them: an analysis of 27,825 patient evaluations. Radiology 225:165–175

    Article  PubMed  Google Scholar 

  6. 6.

    Nothacker M, Duda V, Hahn M et al (2009) Early detection of breast cancer: benefits and risks of supplemental breast ultrasound in asymptomatic women with mammographically dense breast tissue. A systematic review. BMC Cancer 9:335–341

    PubMed Central  Article  PubMed  Google Scholar 

  7. 7.

    Ciatto S, Houssami N, Bernardi D et al (2013) Integration of 3D digital mammography with tomosynthesis for population breast-cancer screening (STORM): a prospective comparison study. Lancet Oncol 14:583–589

    Article  PubMed  Google Scholar 

  8. 8.

    Skaane P, Bandos AI, Gullien R et al (2013) Comparison of digital mammography alone and digital mammography plus tomosynthesis in a population-based screening program. Radiology 267(1):47–56

    Article  PubMed  Google Scholar 

  9. 9.

    Nickson C, Watson R, Kavanagh AM. A microsimulation model of the Breast Screen Australia program, Proceedings of the 18th World IMACS Congress/MODSIM Congress, Cairns, Australia 13-17 July 2009, 164–170

  10. 10.

    American College of Radiology (ACR). ACR BI-RADS® Mammography. 5th Edition. In: Breast Imaging Reporting and Data System, Breast Imaging Atlas. Reston, VA: American College of Radiology; 2013

  11. 11.

    Ciatto S, Bernardi D, Calabrese M et al (2012) A first evaluation of breast radiological density assessment by QUANTRA software as compared to visual classification. The Breast 21:503–506

    Article  PubMed  Google Scholar 

  12. 12.

    Redondo A, Comas M, Macià F et al (2012) Inter- and intraradiologist variability in the BI-RADS assessment and breast density categories for screening mammograms. Br J Radiol 85:1465–1470

    PubMed Central  CAS  Article  PubMed  Google Scholar 

  13. 13.

    Bernardi D, Pellegrini M, Di Michele S et al (2012) Interobserver agreement in breast radiological density attribution according to BI-RADS quantitative classification. Radiol Med 17:519–528

    Article  Google Scholar 

  14. 14.

    Cohen J (1960) A coefficient of agreement for nominal scales. Educat Psychol Meas 20:37–46

    Article  Google Scholar 

  15. 15.

    Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307–310

    CAS  Article  PubMed  Google Scholar 

  16. 16.

    Gweon HM, Youk JH, Kim JA, Son EJ (2013) Radiologist assessment of breast density by BI-RADS categories versus fully automated volumetric assessment. AJR 201:692–697

    Article  PubMed  Google Scholar 

  17. 17.

    Kundel HL, Polansky M (2003) Measurement of observer agreement. Radiology 228:303–308

    Article  PubMed  Google Scholar 

  18. 18.

    Tagliafico AS, Tagliafico G, Cavagnetto F, Calabrese M, Houssami N (2013) Estimation of percentage breast tissue density: comparison between digital mammography (2D full field digital mammography) and digital breast tomosynthesis according to different BI-RADS categories. Br J Radiol 86:20130255

    PubMed Central  CAS  Article  PubMed  Google Scholar 

Download references

Acknowledgement

The authors would like to thank all the centres and the professionals who provided mammography exams for this study: in alphabetical order, APSS (Trento), A.O.U. Città della Salute e della Scienza (Turin), IRCCS Policlinico San Donato (Milan), ISPO (Florence), Ospedale Maggiore della Carità (Novara), Ospedale Regionale (Bolzano), Ospedale S.Andrea (Vercelli), and San Giovanni Bosco ASLTO2Nord (Turin).

The authors would like to acknowledge Dr. Stefano Ciatto for the helpful discussions and suggestions he provided for this study and, in general, to the im3D research team: even after his passing, his teachings continue to drive breast imaging scientific research.

The scientific guarantor of this publication is Francesco Sardanelli (IRCCS Policlinico San Donato, Milan, Italy and Università degli Studi di Milano, Department of Biomedical Sciences for Health, Milan, Italy). Some authors of this manuscript (L. Morra, D. Sacchetto, S. Agliozzo, L. Correale, A. Bert and L.A. Carbonaro) declare relationships with the company im3D. The authors state that this work has not received any funding. Two of the authors (L. Correale and F. Sardanelli) have significant statistical expertise. Institutional review board approval and written informed consent were not required because the study retrospectively evaluated a dataset of fully anonymized images acquired within routine diagnostic procedures. Methodology: retrospective, multicenter study.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Daniela Sacchetto.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 277 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sacchetto, D., Morra, L., Agliozzo, S. et al. Mammographic density: Comparison of visual assessment with fully automatic calculation on a multivendor dataset. Eur Radiol 26, 175–183 (2016). https://doi.org/10.1007/s00330-015-3784-2

Download citation

Keywords

  • Breast density
  • Digital mammography
  • Automated system
  • BI-RADS density classification
  • Multireader/multivendor