European Radiology

, Volume 26, Issue 10, pp 3654–3659 | Cite as

Reporting instructions significantly impact false positive rates when reading chest radiographs

  • John W. RobinsonEmail author
  • Patrick C. Brennan
  • Claudia Mello-Thoms
  • Sarah J. Lewis



To determine the impact of specific reporting tasks on the performance of radiologists when reading chest radiographs.


Ten experienced radiologists read a set of 40 postero-anterior (PA) chest radiographs: 21 nodule free and 19 with a proven solitary nodule. There were two reporting conditions: an unframed task (UFT) to report any abnormality and a framed task (FT) reporting only lung nodule/s. Jackknife free-response operating characteristic (JAFROC) figure of merit (FOM), specificity, location sensitivity and number of true positive (TP), false positive (FP), true negative (TN) and false negative (FN) decisions were used for analysis.


JAFROC FOM for tasks showed a significant reduction in performance for framed tasks (P = 0.006) and an associated decrease in specificity (P = 0.011) but no alteration to the location sensitivity score. There was a significant increase in number of FP decisions made during framed versus unframed tasks for nodule-containing (P = 0.005) and nodule-free (P = 0.011) chest radiographs. No significant differences in TP were recorded.


Radiologists report more FP decisions when given specific reporting instructions to search for nodules on chest radiographs. The relevance of clinical history supplied to radiologists is called into question and may induce a negative effect.

Key Points

• Framed reporting tasks increases false positive rates when searching for pulmonary nodules

• False positive results were observed in both nodule-containing and nodule-free cases

• Radiologist’s decision-making may be influenced by clinical history in thoracic imaging


Pulmonary nodules Thoracic imaging Decision-making Radiologists Observer performance 



Chest radiograph


Interquartile range


Jackknife free-response operating characteristic


Figure of merit


False positive in nodule-containing image


False positive in nodule-free image


Framed task

Loc sens

location sensitivity




Receiver operator characteristics




Unframed task



The scientific guarantor of this publication is Sarah Lewis. The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article. The authors state that this work has not received any funding. One of the authors has significant statistical expertise. Institutional review board approval was obtained. Written informed consent was obtained from all subjects in this study. Methodology: experimental.


  1. 1.
    Cancer Research UK (2011) Lung cancer. Accessed 25 Aug 2011
  2. 2.
    Quekel LGBA, Kessels AGH, Goei R, van Engelshoven JMA (1999) Miss rate of lung cancer on the chest radiograph in clinical practice. Chest 115:720–724CrossRefPubMedGoogle Scholar
  3. 3.
    Tan BB, Flaherty KR, Kazerooni EE, Iannettoni MD (2003) The solitary pulmonary nodule. Chest 123:89S–96SCrossRefPubMedGoogle Scholar
  4. 4.
    Donald JJ, Barnard SA (2012) Common patterns in 558 diagnostic radiology errors. J Med Imaging Radiat Oncol 56:173–178CrossRefPubMedGoogle Scholar
  5. 5.
    Berlin L (1996) Malpractice issues in radiology–perceptual errors. AJR Am J Roentgenol 167:587–590CrossRefPubMedGoogle Scholar
  6. 6.
    Berlin L, Hendrix RW (1998) Malpractice issues in radiology–perceptual errors and negligence. AJR Am J Roentgenol 170:863–867CrossRefPubMedGoogle Scholar
  7. 7.
    Renfrew DL, Franken EA Jr, Berbaum KS, Weigelt FH, Abu-Yousef MM (1992) Errors in radiology: classification and lessons in 182 cases presented at a problem case conference. Radiology 183:145–150CrossRefPubMedGoogle Scholar
  8. 8.
    Robinson JW, Ryan JT, McEntee MF et al (2013) Grey-scale inversion improves detection of lung nodules. Br J Radiol 86. doi: 10.1259/bjr/27961545
  9. 9.
    Håkansson M, Båth M, Börjesson S et al (2005) Nodule detection in digital chest radiography: effect of nodule location. Radiat Prot Dosim 114(1–3):92–96CrossRefGoogle Scholar
  10. 10.
    Brascamp JW, Blake R, Kristjánsson A (2011) Deciding where to attend: priming of pop-out drives target selection. J Exp Psychol Hum Percept Perform 37(6):1700–1707CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Gunderman RB (2009) Biases in radiologic reasoning. AJR Am J Roentgenol 192:561–564CrossRefPubMedGoogle Scholar
  12. 12.
    Black WC, Dwyer AJ (1990) Knowledge of clinical history and the accuracy of interpretation of chest radiographs. AJR Am J Roentgenol 155:1342–1343CrossRefPubMedGoogle Scholar
  13. 13.
    Shiraishi J, Katsuragawa S, Ikezoe J et al (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. AJR Am J Roentgenol 174(1):71–74CrossRefPubMedGoogle Scholar
  14. 14.
    Singh SP, Gierada DS, Pinsky P et al (2012) Reader variability in identifying pulmonary nodules on chest radiographs from the National Lung Screening. J Thorac Imaging 27:249–254CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Potchen EJ, Cooper TG, Sierra AE et al (2000) Measuring performance in chest radiography. Radiology 217:456–45917CrossRefPubMedGoogle Scholar
  16. 16.
    Berbaum KS, Franken EA Jr, Dorfman DD et al (1986) Tentative diagnoses facilitate the detection of diverse lesions in chest radiographs. Investig Radiol 21:532–539CrossRefGoogle Scholar
  17. 17.
    Swensson RG, Hessel SJ, Herman PG (1977) Omissions in radiology: faulty search or stringent reporting criteria? Radiology 123:563–567CrossRefPubMedGoogle Scholar
  18. 18.
    Swensson RG, Hessel SJ, Herman PG (1982) Radiographic interpretation with and without search: visual search aids the recognition of chest pathology. Investig Radiol 17:145–151CrossRefGoogle Scholar
  19. 19.
    Swensson RG, Hessel SJ, Herman PG (1985) The value of searching films without specific preconceptions. Investig Radiol 20:100–107CrossRefGoogle Scholar
  20. 20.
    Patz EJ (2006) Lung cancer screening, overdiagnosis bias, and reevaluation of Mayo Lung Project. J Natl Cancer Inst 98(11):724–725CrossRefPubMedGoogle Scholar
  21. 21.
    Chakraborty DP, Berbaum KS (2004) Jackknife free-response ROC methodology. Proc SPIE 5372. doi: 10.1117/12.533319
  22. 22.
    Chakraborty DV, Berbaum KS (2005) Observer studies involving detection and localisation: modelling, analysis and validation. Med Phys 31(8):2313–2330CrossRefGoogle Scholar
  23. 23.
    Berbaum KS, Franken EA, Dorfman DD, Barloon TJ (1988) Influence of clinical history upon detection of nodules and other lesions. Investig Radiol 23:48–55CrossRefGoogle Scholar
  24. 24.
    Vikgren J, Zachrisson S, Svalkvist A et al (2008) Comparison of chest tomosynthesis and chest radiography for detection of pulmonary nodules: human observer study of clinical cases. Radiology 249:1034–1041CrossRefPubMedGoogle Scholar

Copyright information

© European Society of Radiology 2016

Authors and Affiliations

  • John W. Robinson
    • 1
    Email author
  • Patrick C. Brennan
    • 1
  • Claudia Mello-Thoms
    • 1
  • Sarah J. Lewis
    • 1
  1. 1.Medical Image Optimisation and Perception Group, Discipline of Medical Radiation Sciences, Faculty of Health SciencesThe University of SydneyLidcombeAustralia

Personalised recommendations