Reporting instructions significantly impact false positive rates when reading chest radiographs
- 323 Downloads
To determine the impact of specific reporting tasks on the performance of radiologists when reading chest radiographs.
Ten experienced radiologists read a set of 40 postero-anterior (PA) chest radiographs: 21 nodule free and 19 with a proven solitary nodule. There were two reporting conditions: an unframed task (UFT) to report any abnormality and a framed task (FT) reporting only lung nodule/s. Jackknife free-response operating characteristic (JAFROC) figure of merit (FOM), specificity, location sensitivity and number of true positive (TP), false positive (FP), true negative (TN) and false negative (FN) decisions were used for analysis.
JAFROC FOM for tasks showed a significant reduction in performance for framed tasks (P = 0.006) and an associated decrease in specificity (P = 0.011) but no alteration to the location sensitivity score. There was a significant increase in number of FP decisions made during framed versus unframed tasks for nodule-containing (P = 0.005) and nodule-free (P = 0.011) chest radiographs. No significant differences in TP were recorded.
Radiologists report more FP decisions when given specific reporting instructions to search for nodules on chest radiographs. The relevance of clinical history supplied to radiologists is called into question and may induce a negative effect.
• Framed reporting tasks increases false positive rates when searching for pulmonary nodules
• False positive results were observed in both nodule-containing and nodule-free cases
• Radiologist’s decision-making may be influenced by clinical history in thoracic imaging
KeywordsPulmonary nodules Thoracic imaging Decision-making Radiologists Observer performance
Jackknife free-response operating characteristic
Figure of merit
False positive in nodule-containing image
False positive in nodule-free image
- Loc sens
Receiver operator characteristics
The scientific guarantor of this publication is Sarah Lewis. The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article. The authors state that this work has not received any funding. One of the authors has significant statistical expertise. Institutional review board approval was obtained. Written informed consent was obtained from all subjects in this study. Methodology: experimental.
- 1.Cancer Research UK (2011) Lung cancer. http://www.cancerresearchuk.org/cancer-info/cancerstats/world/lung-cancer-world/. Accessed 25 Aug 2011
- 8.Robinson JW, Ryan JT, McEntee MF et al (2013) Grey-scale inversion improves detection of lung nodules. Br J Radiol 86. doi: 10.1259/bjr/27961545
- 21.Chakraborty DP, Berbaum KS (2004) Jackknife free-response ROC methodology. Proc SPIE 5372. doi: 10.1117/12.533319