Emergent Diagnoses from a Collective of Radiologists: Algorithmic versus Social Consensus Strategies

  • Daniel W. Palmer
  • David W. Piraino
  • Nancy A. Obuchowski
  • Jennifer A. Bullen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8667)

Abstract

Twelve radiologists independently diagnosed 74 medical images. We use two approaches to combine their diagnoses: a collective algorithmic strategy and a social consensus strategy using swarm techniques. The algorithmic strategy uses weighted averages and a geometric approach to automatically produce an aggregate diagnosis. The social consensus strategy used visual cues to quickly impart the essence of the diagnoses to the radiologists as they produced an emergent diagnosis. Both strategies provide access to additional useful information from the original diagnoses. The mean number of correct diagnoses from the radiologists was 50 and the best was 60. The algorithmic strategy produced 63 correct diagnoses and the social consensus produced 67. The algorithm’s accuracy in distinguishing normal vs. abnormal patients (0.919) was significantly higher than the radiologists’ mean accuracy (0.861; p = 0.047). The social consensus’ accuracy (0.951; p = 0.007) showed further improvement.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Brutschy, A., Scheidler, A., Merkle, D., Middendorf, M.: Learning from house-hunting ants: collective decision-making in organic computing systems. In: Dorigo, M., Birattari, M., Blum, C., Clerc, M., Stützle, T., Winfield, A.F.T. (eds.) ANTS 2008. LNCS, vol. 5217, pp. 96–107. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  2. 2.
    DeLong, E., DeLong, D., Clarke-Pearson, D.: Comparing the areas under two of more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44, 837–845 (1988)CrossRefMATHGoogle Scholar
  3. 3.
    Drew, T., Vo, M., Wolfe, J.: The invisible gorilla strikes again sustained inattentional blindness in expert observers. Psychological Science 24(9), 1848–1853 (2013)CrossRefGoogle Scholar
  4. 4.
    Obuchowski, N., Goske, M., Applegate, K.: Assessing physicians’ accuracy in diagnosing pediatric patients with acute abdominal pain: measuring accuracy for multiple diseases. Statistics in Medicine 20, 3261–3278 (2001)CrossRefGoogle Scholar
  5. 5.
    Obuchowski, N., Rockette, H.: Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an anova approach with dependent observations. Communication in Statistics – Simulation 24, 285–308 (1995)CrossRefMATHGoogle Scholar
  6. 6.
    Surowiecki, J.: The Wisdom of Crowds. Doubleday, New York (2005)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Daniel W. Palmer
    • 1
  • David W. Piraino
    • 2
  • Nancy A. Obuchowski
    • 2
  • Jennifer A. Bullen
    • 2
  1. 1.John Carroll UniversityUniversity HeightsUSA
  2. 2.Cleveland ClinicClevelandUSA

Personalised recommendations