CAD in mammography: lesion-level versus case-level analysis of the effects of prompts on human decisions

  • Eugenio Alberdi
  • Andrey A. Povyakalo
  • Lorenzo Strigini
  • Peter Ayton
  • Rosalind Given-Wilson
Original Article

Abstract

Object

To understand decision processes in CAD-supported breast screening by analysing how prompts affect readers’ judgements of individual mammographic features (lesions). To this end we analysed hitherto unexamined details of reports completed by mammogram readers in an earlier evaluation of a CAD tool.

Material and methods

Assessments of lesions were extracted from 5,839 reports for 59 cancer cases. Statistical analyses of these data focused on what features readers considered when recalling a cancer case and how readers reacted to CAD prompts.

Results

About 13.5% of recall decisions were found to be caused by responses to features other than those indicating actual cancer. Effects of CAD: lesions were more likely to be examined if prompted; the presence of a prompt on a cancer increased the probability of both detection and recall especially for less accurate readers in subtler cases; lack of prompts made cancer features less likely to be detected; false prompts made non-cancer features more likely to be classified as cancer.

Conclusion

The apparent lack of impact reported for CAD in some studies is plausibly due to CAD systematically affecting readers’ identification of individual features, in a beneficial way for certain combinations of readers and features and a damaging way for others. Mammogram readers do not ignore prompts. Methodologically, assessing CAD by numbers of recalled cancer cases may be misleading.

Keywords

Computer-assisted diagnosis Mammography Screening Computer system evaluation Decision making 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Astley SM, Gilbert FJ (2004) Computer-aided detection in mammography. Clin Radiol 59: 390–399PubMedCrossRefGoogle Scholar
  2. 2.
    Bazzocchi M, Mazzarella F, Del Frate C, Girometti R, Zuiani C (2007) CAD systems for mammography: a real opportunity? A review of the literature. Radiol Med 112: 329–353CrossRefPubMedGoogle Scholar
  3. 3.
    Freer TW, Ulissey MJ (2001) Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology 220: 781–786PubMedCrossRefGoogle Scholar
  4. 4.
    Cupples TE, Cunningham JE, Reynolds JC (2005) Impact of computer-aided detection in a regional screening mammography program. Am J Roentgenol 185: 944–950CrossRefGoogle Scholar
  5. 5.
    Gur D, Sumkin JH, Rockette HE, Ganott M, Hakim C, Hardesty L, Poller WR, Shah R, Wallace L (2004) Changes in breast cancer detection and mammography recall rates after the introduction of a computer-aided detection system. J Natl Cancer Inst 96: 185–190PubMedCrossRefGoogle Scholar
  6. 6.
    Taylor PM, Champness J, Given-Wilson RM, Potts HWE, Johnston K (2004) An evaluation of the impact of computer-based prompts on screen readers’ interpretation of mammograms. Br J Radiol 77: 21–27PubMedCrossRefGoogle Scholar
  7. 7.
    Khoo LAL, Taylor P, Given-Wilson RM (2005) Computer-aided detection in the United Kingdom national breast screening programme: prospective study. Radiology 237: 444–449PubMedCrossRefGoogle Scholar
  8. 8.
    Fenton JJ, Taplin SH, Carney PA, Abraham L, Sickles EA, D’Orsi C, Berns EA, Cutter G, Hendrick RE, Barlow WE, Elmore JG (2007) Influence of computer-aided detection on performance of screening mammography. N Engl J Med 356: 1399–1409PubMedCrossRefGoogle Scholar
  9. 9.
    Alberdi E, Povyakalo AA, Strigini L, Ayton P (2004) Effects of incorrect CAD output on human decision making in mammography. Acad Radiol 11: 909–918PubMedCrossRefGoogle Scholar
  10. 10.
    Taplin SH, Rutter CM, Lehman CD (2006) Testing the effect of computer-assisted detection on interpretive performance in screening mammography. Am J Roentgenol 187: 1475–1482CrossRefGoogle Scholar
  11. 11.
    Houssami N, Given-Wilson R (2007) Incorporating new technologies into clinical practice without evidence of effectiveness in prospective studies: computer-aided detection (CAD) in breast screening reinforces the need for better initial evaluation. Breast 16: 219–221PubMedCrossRefGoogle Scholar
  12. 12.
    Povyakalo AA, Alberdi E, Strigini L, Ayton P (2004) Evaluating ‘Human + Advisory computer’ system: a case study. In: Dearden A, Watts L (eds) HCI2004, 18th British HCI Group Annual Conf. British HCI Group, Leeds, pp 93–96Google Scholar
  13. 13.
    Strigini L, Povyakalo AA, Alberdi E (2003) Human-machine diversity in the use of computerised advisory systems: a case study. In: 2003 Int. Conf. on Dependable Systems and Networks (DSN’03). IEEE, San FranciscoGoogle Scholar
  14. 14.
    Alberdi E, Povyakalo AA, Strigini L, Ayton P, Hartswood M, Procter R, Slack R (2005) Use of computer-aided detection (CAD) tools in screening mammography: a multidisciplinary investigation. Br J Radiol 78: S31–S40PubMedCrossRefGoogle Scholar
  15. 15.
    Skitka LJ, Mosier K, Burdick MD (1999) Does automation bias decision making?. Int J Human-Comput Stud 51: 991–1006CrossRefGoogle Scholar
  16. 16.
    Hartswood M, Procter R, Rouncefield M, Slack R, Soutter J, Voss A (2003) ‘Repairing’ the Machine: a Case study of the evaluation of computer-aided detection tools in breast screening. In: Kuutti K Karsten EH, Fitzpatrick G, Dourish P, Schmidt K (ed) Eighth European Conference on Computer Supported Cooperative Work (ECSCW 2003). Helsinki, FinlandGoogle Scholar
  17. 17.
    Parasuraman R, Molloy R, Singh IL (1993) Performance consequences of automation-induced “complacency”. Int J Aviat Psychol 3: 1–23CrossRefGoogle Scholar

Copyright information

© CARS 2008

Authors and Affiliations

  • Eugenio Alberdi
    • 1
  • Andrey A. Povyakalo
    • 1
  • Lorenzo Strigini
    • 1
  • Peter Ayton
    • 2
  • Rosalind Given-Wilson
    • 3
  1. 1.Centre for Software Reliability, Northampton SquareCity UniversityLondonUK
  2. 2.Psychology DepartmentCity UniversityLondonUK
  3. 3.Department of RadiologySt. George’s HospitalLondonUK

Personalised recommendations