Abstract
Objectives
We developed a computer-aided detection (CAD) system aimed at decision support for detection of malignant masses and architectural distortions in mammograms. The effect of this system on radiologists' performance depends strongly on its standalone performance. The purpose of this study was to compare the standalone performance of this CAD system to that of radiologists.
Methods
In a retrospective study, nine certified screening radiologists and three residents read 200 digital screening mammograms without the use of CAD. Performances of the individual readers and of CAD were computed as the true-positive fraction (TPF) at a false-positive fraction of 0.05 and 0.2. Differences were analysed using an independent one-sample t-test.
Results
At a false-positive fraction of 0.05, the performance of CAD (TPF = 0.487) was similar to that of the certified screening radiologists (TPF = 0.518, P = 0.17). At a false-positive fraction of 0.2, CAD performance (TPF = 0.620) was significantly lower than the radiologist performance (TPF = 0.736, P <0.001). Compared to the residents, CAD performance was similar for all false-positive fractions.
Conclusions
The sensitivity of CAD at a high specificity was comparable to that of human readers. These results show potential for CAD to be used as an independent reader in breast cancer screening.
Key points
• Computer-aided detection (CAD) systems are used to detect malignant masses in mammograms
• Current CAD systems operate at low specificity to avoid perceptual oversight
• A CAD system has been developed that operates at high specificity
• The performance of the CAD system is approaching that of trained radiologists
• CAD has the potential to be an independent reader in screening
Similar content being viewed by others
Abbreviations
- FFDM:
-
full field digital mammograms
- k-NN:
-
k-nearest neighbour
References
Rao VM, Levin DC, Parker L, Cavanaugh B, Frangos AJ, Sunshine JH (2010) How widely is computer-aided detection used in screening and diagnostic mammography? J Am Coll Radiol 7:802–805
Houssami N, Given-Wilson R, Ciatto S (2009) Early detection of breast cancer: overview of the evidence on computer-aided detection in mammography screening. J Med Imaging Radiat Oncol 53:171–176
Alberdi E, Povyakalo A, Strigini L, Ayton P, Given-Wilson R (2008) CAD in mammography: lesion-level versus case-level analysis of the effects of prompts on human decisions. Int J Comput Assist Radiol Surg 3:115–122
Fenton JJ, Abraham L, Taplin SH et al (2011) Effectiveness of computer-aided detection in community mammography practice. J Natl Cancer Inst 103:1152–1161
Taylor P, Potts HWW (2008) Computer aids and human second reading as interventions in screening mammography: two systematic reviews to compare effects on cancer detection and recall rate. Eur J Cancer 44:798–807
Fenton JJ, Taplin SH, Carney PA et al (2007) Inuence of computer-aided detection on performance of screening mammography. N Engl J Med 356:1399–1409
Gur D, Stalder JS, Hardesty LA et al (2004) Computer-aided detection performance in mammographic examination of masses: assessment. Radiology 233:418–423
Blanks RG, Wallis MG, Given-Wilson RM (1999) Observer variability in cancer detection during routine repeat (incident) mammographic screening in a study of two versus one view mammography. J Med Screen 6:152–158
Mello-Thoms C (2003) Perception of breast cancer: eye-position analysis of mammogram interpretation. Acad Radiol 10:4–12
Samulski M, Hupse R, Boetes C, Mus R, den Heeten G, Karssemeijer N (2010) Using computer aided detection in mammography as a decision support. Eur Radiol 20(10):2323–2330
Karssemeijer N, Otten JD, Verbeek AL et al (2003) Computer-aided detection versus independent double reading of masses on mammograms. Radiology 227(1):192–200
Hupse R, Karssemeijer N (2009) The use of contextual information for computer aided detection of masses in mammograms. Proc SPIE 7260:72600Q
Samulski M, Karssemeijer N (2011) Optimizing case-based detection performance in a multiview CAD system for mammography. IEEE Trans Med Imaging 30(4):1001–1009
Karssemeijer N, Bluekens AM, Beijerinck D et al (2009) Breast cancer screening results 5 years after introduction of digital mammography in a population-based screening program. Radiology 253:353–358
Kallenberg M, Karssemeijer N (2008) Computer-aided detection of masses in full-field digital mammography using screen-film mammograms for training. Phys Med Biol 53(23):6879–6891
Fracheboud J, de Koning HJ, Beemsterboer PM et al (1998) Nation-wide breast cancer screening in The Netherlands: results of initial and subsequent screening 1990-1995. National Evaluation Team for Breast Cancer Screening. Int J Cancer 75(5):694–698
Holland R, Rijken H, Hendriks J (2007) The Dutch population-based mammography screening: 30-year experience. Breast Care 2:12–18
Timp S, Varela C, Karssemeijer N (2007) Temporal change analysis for characterization of mass lesions in mammography. IEEE Trans Med Imaging 26:945–953
Kamitani T, Yabuuchi H, Soeda H et al (2007) Detection of masses and micro-calcifications of breast cancer on digital mammograms: comparison among hard-copy film, 3-megapixel liquid crystal display (LCD) monitors and 5-megapixel LCD monitors: an observer performance study. Eur Radiol 17:1365–1371
Gur D, Bandos AI, Cohen CS et al (2008) The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 249:47–53
Thurfjell EL, Lernevall KA, Taube AA (1994) Benefit of independent double reading in a population-based mammography screening program. Radiology 191:241–244
Anderson ED, Muir BB, Walsh JS, Kirkpatrick AE (1994) The efficacy of double reading mammograms in breast screening. Clin Radiol 49:248–251
Brown J, Bryan S, Warren R (1996) Mammography screening: an incremental cost effectiveness analysis of double versus single reading of mammograms. Br Med J 312:809–812
Blanks RG, Wallis MG, Moss SM (1998) A comparison of cancer detection rates achieved by breast cancer screening programmes by number of readers, for one and two view mammography: results from the UK National Health Service breast screening program. J Med Screen 5:195–201
Karssemeijer N, Otten JD, Roelofs AAJ, van Woudenberg S, Hendriks JHCL (2004) Effect of independent multiple reading of mammograms on detection performance. Proc SPIE 5372:82–89
Acknowledgements
This work was funded by grant no. KUN 2006-3655 of the Dutch Cancer Society. The authors gratefully acknowledge the participation of C.N.A. Frotscher, E. Ghazi, S. Gommers, U.C. Lalji, R.M. Mann and R.D. Mus in the observer performance study.
Author information
Authors and Affiliations
Corresponding author
Additional information
Carla Boetes deceased in May 2011.
Rights and permissions
About this article
Cite this article
Hupse, R., Samulski, M., Lobbes, M. et al. Standalone computer-aided detection compared to radiologists’ performance for the detection of mammographic masses. Eur Radiol 23, 93–100 (2013). https://doi.org/10.1007/s00330-012-2562-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-012-2562-7