European Radiology

, Volume 23, Issue 1, pp 93–100 | Cite as

Standalone computer-aided detection compared to radiologists’ performance for the detection of mammographic masses

  • Rianne Hupse
  • Maurice Samulski
  • Marc Lobbes
  • Ard den Heeten
  • Mechli W. Imhof-Tas
  • David Beijerinck
  • Ruud Pijnappel
  • Carla Boetes
  • Nico Karssemeijer



We developed a computer-aided detection (CAD) system aimed at decision support for detection of malignant masses and architectural distortions in mammograms. The effect of this system on radiologists' performance depends strongly on its standalone performance. The purpose of this study was to compare the standalone performance of this CAD system to that of radiologists.


In a retrospective study, nine certified screening radiologists and three residents read 200 digital screening mammograms without the use of CAD. Performances of the individual readers and of CAD were computed as the true-positive fraction (TPF) at a false-positive fraction of 0.05 and 0.2. Differences were analysed using an independent one-sample t-test.


At a false-positive fraction of 0.05, the performance of CAD (TPF = 0.487) was similar to that of the certified screening radiologists (TPF = 0.518, P = 0.17). At a false-positive fraction of 0.2, CAD performance (TPF = 0.620) was significantly lower than the radiologist performance (TPF = 0.736, P <0.001). Compared to the residents, CAD performance was similar for all false-positive fractions.


The sensitivity of CAD at a high specificity was comparable to that of human readers. These results show potential for CAD to be used as an independent reader in breast cancer screening.

Key points

Computer-aided detection (CAD) systems are used to detect malignant masses in mammograms

Current CAD systems operate at low specificity to avoid perceptual oversight

A CAD system has been developed that operates at high specificity

The performance of the CAD system is approaching that of trained radiologists

CAD has the potential to be an independent reader in screening


Mammography Computer-assisted diagnosis Breast Mass screening Neoplasms 

Abbreviations and acronyms


full field digital mammograms


k-nearest neighbour



This work was funded by grant no. KUN 2006-3655 of the Dutch Cancer Society. The authors gratefully acknowledge the participation of C.N.A. Frotscher, E. Ghazi, S. Gommers, U.C. Lalji, R.M. Mann and R.D. Mus in the observer performance study.


  1. 1.
    Rao VM, Levin DC, Parker L, Cavanaugh B, Frangos AJ, Sunshine JH (2010) How widely is computer-aided detection used in screening and diagnostic mammography? J Am Coll Radiol 7:802–805PubMedCrossRefGoogle Scholar
  2. 2.
    Houssami N, Given-Wilson R, Ciatto S (2009) Early detection of breast cancer: overview of the evidence on computer-aided detection in mammography screening. J Med Imaging Radiat Oncol 53:171–176PubMedCrossRefGoogle Scholar
  3. 3.
    Alberdi E, Povyakalo A, Strigini L, Ayton P, Given-Wilson R (2008) CAD in mammography: lesion-level versus case-level analysis of the effects of prompts on human decisions. Int J Comput Assist Radiol Surg 3:115–122CrossRefGoogle Scholar
  4. 4.
    Fenton JJ, Abraham L, Taplin SH et al (2011) Effectiveness of computer-aided detection in community mammography practice. J Natl Cancer Inst 103:1152–1161PubMedCrossRefGoogle Scholar
  5. 5.
    Taylor P, Potts HWW (2008) Computer aids and human second reading as interventions in screening mammography: two systematic reviews to compare effects on cancer detection and recall rate. Eur J Cancer 44:798–807PubMedCrossRefGoogle Scholar
  6. 6.
    Fenton JJ, Taplin SH, Carney PA et al (2007) Inuence of computer-aided detection on performance of screening mammography. N Engl J Med 356:1399–1409PubMedCrossRefGoogle Scholar
  7. 7.
    Gur D, Stalder JS, Hardesty LA et al (2004) Computer-aided detection performance in mammographic examination of masses: assessment. Radiology 233:418–423PubMedCrossRefGoogle Scholar
  8. 8.
    Blanks RG, Wallis MG, Given-Wilson RM (1999) Observer variability in cancer detection during routine repeat (incident) mammographic screening in a study of two versus one view mammography. J Med Screen 6:152–158PubMedGoogle Scholar
  9. 9.
    Mello-Thoms C (2003) Perception of breast cancer: eye-position analysis of mammogram interpretation. Acad Radiol 10:4–12PubMedCrossRefGoogle Scholar
  10. 10.
    Samulski M, Hupse R, Boetes C, Mus R, den Heeten G, Karssemeijer N (2010) Using computer aided detection in mammography as a decision support. Eur Radiol 20(10):2323–2330PubMedCrossRefGoogle Scholar
  11. 11.
    Karssemeijer N, Otten JD, Verbeek AL et al (2003) Computer-aided detection versus independent double reading of masses on mammograms. Radiology 227(1):192–200PubMedCrossRefGoogle Scholar
  12. 12.
    Hupse R, Karssemeijer N (2009) The use of contextual information for computer aided detection of masses in mammograms. Proc SPIE 7260:72600QCrossRefGoogle Scholar
  13. 13.
    Samulski M, Karssemeijer N (2011) Optimizing case-based detection performance in a multiview CAD system for mammography. IEEE Trans Med Imaging 30(4):1001–1009PubMedCrossRefGoogle Scholar
  14. 14.
    Karssemeijer N, Bluekens AM, Beijerinck D et al (2009) Breast cancer screening results 5 years after introduction of digital mammography in a population-based screening program. Radiology 253:353–358PubMedCrossRefGoogle Scholar
  15. 15.
    Kallenberg M, Karssemeijer N (2008) Computer-aided detection of masses in full-field digital mammography using screen-film mammograms for training. Phys Med Biol 53(23):6879–6891PubMedCrossRefGoogle Scholar
  16. 16.
    Fracheboud J, de Koning HJ, Beemsterboer PM et al (1998) Nation-wide breast cancer screening in The Netherlands: results of initial and subsequent screening 1990-1995. National Evaluation Team for Breast Cancer Screening. Int J Cancer 75(5):694–698PubMedCrossRefGoogle Scholar
  17. 17.
    Holland R, Rijken H, Hendriks J (2007) The Dutch population-based mammography screening: 30-year experience. Breast Care 2:12–18CrossRefGoogle Scholar
  18. 18.
    Timp S, Varela C, Karssemeijer N (2007) Temporal change analysis for characterization of mass lesions in mammography. IEEE Trans Med Imaging 26:945–953PubMedCrossRefGoogle Scholar
  19. 19.
    Kamitani T, Yabuuchi H, Soeda H et al (2007) Detection of masses and micro-calcifications of breast cancer on digital mammograms: comparison among hard-copy film, 3-megapixel liquid crystal display (LCD) monitors and 5-megapixel LCD monitors: an observer performance study. Eur Radiol 17:1365–1371PubMedCrossRefGoogle Scholar
  20. 20.
    Gur D, Bandos AI, Cohen CS et al (2008) The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 249:47–53PubMedCrossRefGoogle Scholar
  21. 21.
    Thurfjell EL, Lernevall KA, Taube AA (1994) Benefit of independent double reading in a population-based mammography screening program. Radiology 191:241–244PubMedGoogle Scholar
  22. 22.
    Anderson ED, Muir BB, Walsh JS, Kirkpatrick AE (1994) The efficacy of double reading mammograms in breast screening. Clin Radiol 49:248–251PubMedCrossRefGoogle Scholar
  23. 23.
    Brown J, Bryan S, Warren R (1996) Mammography screening: an incremental cost effectiveness analysis of double versus single reading of mammograms. Br Med J 312:809–812CrossRefGoogle Scholar
  24. 24.
    Blanks RG, Wallis MG, Moss SM (1998) A comparison of cancer detection rates achieved by breast cancer screening programmes by number of readers, for one and two view mammography: results from the UK National Health Service breast screening program. J Med Screen 5:195–201PubMedGoogle Scholar
  25. 25.
    Karssemeijer N, Otten JD, Roelofs AAJ, van Woudenberg S, Hendriks JHCL (2004) Effect of independent multiple reading of mammograms on detection performance. Proc SPIE 5372:82–89CrossRefGoogle Scholar

Copyright information

© European Society of Radiology 2012

Authors and Affiliations

  • Rianne Hupse
    • 1
  • Maurice Samulski
    • 1
  • Marc Lobbes
    • 2
  • Ard den Heeten
    • 3
    • 4
  • Mechli W. Imhof-Tas
    • 1
  • David Beijerinck
    • 5
  • Ruud Pijnappel
    • 3
    • 6
  • Carla Boetes
    • 2
  • Nico Karssemeijer
    • 1
  1. 1.Department of RadiologyRadboud University Nijmegen Medical CentreNijmegenThe Netherlands
  2. 2.Department of RadiologyMaastricht University Medical CentreMaastrichtThe Netherlands
  3. 3.National Expert and Training Centre for Breast Cancer ScreeningNijmegenThe Netherlands
  4. 4.Department of RadiologyAcademic Medical Centre AmsterdamAmsterdamThe Netherlands
  5. 5.Screening Program Early Detection of Breast Cancer in the Centre/Mid-West Part of the NetherlandsUtrechtThe Netherlands
  6. 6.Department of RadiologyUniversity Medical Centre UtrechtUtrechtThe Netherlands

Personalised recommendations