Skip to main content

Advertisement

SpringerLink
Log in

Impact of a Computer-Aided Detection (CAD) System Integrated into a Picture Archiving and Communication System (PACS) on Reader Sensitivity and Efficiency for the Detection of Lung Nodules in Thoracic CT Exams

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

The objective of this study is to assess the impact on nodule detection and efficiency using a computer-aided detection (CAD) device seamlessly integrated into a commercially available picture archiving and communication system (PACS). Forty-eight consecutive low-dose thoracic computed tomography studies were retrospectively included from an ongoing multi-institutional screening study. CAD results were sent to PACS as a separate image series for each study. Five fellowship-trained thoracic radiologists interpreted each case first on contiguous 5 mm sections, then evaluated the CAD output series (with CAD marks on corresponding axial sections). The standard of reference was based on three-reader agreement with expert adjudication. The time to interpret CAD marking was automatically recorded. A total of 134 true-positive nodules, measuring 3 mm and larger were included in our study; with 85 ≥ 4 and 50 ≥ 5 mm in size. Readers detection improved significantly in each size category when using CAD, respectively, from 44 to 57 % for ≥3 mm, 48 to 61 % for ≥4 mm, and 44 to 60 % for ≥5 mm. CAD stand-alone sensitivity was 65, 68, and 66 % for nodules ≥3, ≥4, and ≥5 mm, respectively, with CAD significantly increasing the false positives for two readers only. The average time to interpret and annotate a CAD mark was 15.1 s, after localizing it in the original image series. The integration of CAD into PACS increases reader sensitivity with minimal impact on interpretation time and supports such implementation into daily clinical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Summers RM: Road maps for advancement of radiologic computer-aided detection in the 21st century. Radiology 229(1):11–13, 2003

    Article  PubMed  Google Scholar 

  2. Brown MS, Goldin JG, Rogers S, et al: Computer-aided lung nodule detection in CT: results of large-scale observer test. Acad Radiol 12:681–686, 2005

    Article  PubMed  Google Scholar 

  3. McNitt-Gray MF, Armato SG, Meyer CR, Reeves AP, McLennan G, Pais RC, Freymann J, Brown MS, Engelmann RM, Bland PH, Laderach GE, Piker C, Guo J, Towfic Z, Qing DPY, Yankelevitz DF, Aberle DR, van Beek EJR, MacMahon H, Kazerooni EA, Croft BY, Clarke LP: The lung image database consortium (LIDC) data collection process for nodule detection and annotation. Acad Radiol 14(12):1464–1474, 2007

    Article  PubMed  Google Scholar 

  4. Suzuki K, Li F, Sone S, Doi K: Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose CT by use of massive training artificial neural network. IEEE Trans Med Imaging 24(9):1138–1150, 2005

    Article  PubMed  Google Scholar 

  5. Goldin JG, Brown MS, Petkovska I: Computer-aided diagnosis in lung nodule assessment. J Thorac Imaging 23:97–104, 2008

    Article  PubMed  Google Scholar 

  6. Marten K, Engelke C, Seyfarth T, et al: Computer-aided detection of pulmonary nodules: influence of nodule characteristics on detection performance. Clin Radiol 60:196–206, 2005

    Article  PubMed  CAS  Google Scholar 

  7. Way T, Chan HP, Hadjiiski L, et al: Computer-aided diagnosis of lung nodules on CT scans: ROC study on its effect on radiologists’ performance. Acad Radiol 17:323–332, 2010

    Article  PubMed  Google Scholar 

  8. White CS, Pugatch R, Koonce T, et al: Lung nodule CAD software as a second reader: a multicenter study. Acad Radiol 15:326–333, 2008

    Article  PubMed  Google Scholar 

  9. Hein PA, Rogalla P, Klessen C, et al: Computer-aided pulmonary nodule detection—performance of two CAD systems at different CT dose levels. Rofo 181:1056–1064, 2009

    Article  PubMed  CAS  Google Scholar 

  10. Das M, Muhlenbruch G, Heinen S, et al: Performance evaluation of a computer-aided detection algorithm for solid pulmonary nodules in low-dose and standard-dose MDCT chest examinations and its influence on radiologists. Br J Radiol 81:841–847, 2008

    Article  PubMed  CAS  Google Scholar 

  11. Kim JS, Kim JH, Cho GS, et al: Automated detection of pulmonary nodules on CT images: effect of section thickness and reconstruction interval—initial results. Radiology 236:295–299, 2005

    Article  PubMed  Google Scholar 

  12. Teague SD, Trilikis G, Dharaiya E: Lung nodule computer-aided detection as a second reader: influence on radiology residents. J Comput Assist Tomogr 34:35–39, 2010

    Article  PubMed  Google Scholar 

  13. Doi K: Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 31:198–211, 2007

    Article  PubMed  Google Scholar 

  14. van Ginneken B, Schaefer-Prokop CM, Prokop M: Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology 261(3):719–732, 2011 Dec

    Article  PubMed  Google Scholar 

  15. Welter P, Hocken C, Deserno TM, Grouls C, Günther RW: Workflow management of content-based image retrieval for CAD support in PACS environments based on IHE. Int J Comput Assist Radiol Surg 5(4):393–400, 2010

    Article  PubMed  Google Scholar 

  16. Faggioni L, Neri E, Castellana C, Caramella D, Bartolozzi C: The Future of PACS in healthcare enterprises. Eur J Radiol 78(2):253–258, 2011

    Article  PubMed  Google Scholar 

  17. Erickson BJ, Bartholmai B: Computer-aided diagnosis at the start of the Third Millenium. J Digit Imag 15(2):59–68, June 2002

    Article  Google Scholar 

  18. Boone JM: Radiological interpretation 2020: toward quantitative image assessment. Med Phys 34(11):4173–4179, 2007

    Article  PubMed  Google Scholar 

  19. Anh H, Le T, Liu B, Huang K: Integration of a computer-aided Diagnosis/Detection (CAD) results in a PACS environment using CAD-PACS toolkit and DICOM SR. Int J Comput Assist Radiol Surg 4(4):317–329, 2007

    Google Scholar 

  20. Huang K, Liu BJ, Anh H et al, Chapter 18: PACS-based computer aided detection and diagnosis. Biomedical Image Processing (Biological and Medical Physics, Biomedical Engineering), 455–470, DOI: 10.1007/978-3-642-15816-2_18

  21. National Lung Screening Trial Research Team, Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, et al: Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 365:395–409, 2011

    Article  PubMed  Google Scholar 

  22. Roos JE, Paik D, Olsen D, et al: Computer-aided detection (CAD) of lung nodules in CT scans. Eur Radiol 10:549–557, 2010

    Article  Google Scholar 

  23. Naidich DP, Ko JP, Stockel J, et al: Computer aided diagnosis: impact on nodule detection among community level radiologists, a multi-reader study. Int J Comput Assist Radiol Surg 1268:902–907, 2004

    Google Scholar 

  24. Godoy M, Kim TJ, Ko J, Florin CH, et al, Computer-aided detection of pulmonary nodules on CT: evaluation of a new prototype for detection of ground-glass and part-solid nodules, SSK04-07 RSNA 2008, p.517.

  25. Das M, Honnef, D, O’Dell D et al, Prospective Evaluation of a CAD Sever for Computer-aided Detection in Daily Routine Chest CT Examination: Evaluation of 234 Patients, SSK-08 RSNA 2008, p. 517.

  26. Sakai S, Sod Y, Takahashi N, et al: Computer-aided nodule detection on digital chest radiolography: validation test on consecutive T1 cases of resectable lung cancer. J Digit Imag 19(4):376–382, Dec. 2006

    Article  Google Scholar 

  27. Pietka E, Pospiech-Kurkowska S, Gertych A: Integration of computer assisted bone age assessment with clinical PACS. Comp Med Img Graph 27(2):217–228, Mar. 2003

    Article  Google Scholar 

  28. Sakai S, Yabuuchi H, Matsuo Y, et al: Integration of temporal subtraction and nodule detection system for digital chest radiographs into picture archiving and communication system (PACS): four-year experience. J Digit Imag 21(1):91–98, Mar 2008

    Article  Google Scholar 

  29. Ko JP, Betke M: Chest CT: automated nodule detection and assessment of change over time-preliminary experience. Radiology 218(1):267–273, 2001

    PubMed  CAS  Google Scholar 

  30. Tam M, Deklerck R, Jansen B, et al: A novel computer-aided lung nodule detection system for CT images. Med Phys 38(10):5630–5645, 2011

    Article  Google Scholar 

  31. Armato III, S, Giger M, Moran C, Blackburn J, Doi K, MacMahon H: Computerized detection of pulmonary nodules on CT scans. Radiographics 19:1303–1311, 1999

    PubMed  Google Scholar 

  32. Lee Y, Hara T, Fujita H, Itoh S, Ishigaki T: Automated detection of pulmonary nodules in helical CT images based on an improved template-matching algorithm technique. IEEE Trans Med Imaging 20(7):595–604, 2001

    Article  PubMed  CAS  Google Scholar 

  33. Bogoni L, Bi J, Florin C, et al: Lung nodule detection. In: Müller H, Clough P, Deselaers T, Caputo B Eds. ImageCLEF—experimental evaluation in visual information retrieval series: the information retrieval series. Springer, Berlin, 2010, pp 415–451

    Google Scholar 

  34. Periaswamy S, and Bogoni L, System and method for filtering and automatic detection of candidate anatomical structures in medical images. US Patent 7,912,292.

  35. Liang J and Bogoni L, Toboggan-based shape characterization. US Patent 7,480,412.

  36. Jerebko A, Bogoni L, Lakare S, Segmentation of structures based on curvature slope. US Patent 7,634,133.

  37. Okada K, Comaniciu D, Krishnan A: Robust anisotriopic Gaussian fitting for volumetric characterization of pulmonary nodules in multislice CT. IEEE Trans Med Imaging 24(3):409–423, Mar 2005

    Article  PubMed  Google Scholar 

  38. Kubota T, Estimation of solitary pulmonary nodule diameters with reaction-diffusion segmentation. US Patent 7,720,271.

  39. V, Krshmapuram B, Bi J, et al. Bayesian multiple instance learning: automatic feature section and inductive transfer. In: Proc. 25th Intr Conf Mach. Learning, 2008, pp 808–815.

  40. Raykar VC, Yu S, Zhao LH, Hermosillo G, Florin CH, Bogoni L, Moy L: Learning from crowds. J Mach Learn Res 11:1297–1322, 2010

    Google Scholar 

  41. Armato SG, Roberts RY, Kocherginsky M, et al: Assessment of radiologist performance in the detection of lung nodules: dependence on the definition of “truth”. Acad Radiol 16:28–38, 2009

    Article  PubMed  Google Scholar 

  42. Ochs RA, Kim HJ, Angel E, et al. Forming a reference standard from LIDC data: impact of LIDC reader agreement on the reference dataset and reported CAD performance. In: Proc. SPIE, 30 Mar 2007, vol. 6514, p 82, DOI: 10.1117/12.707916

  43. MacMahon H, Austin JH, Gamsu G, et al: Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society. Radiology 237:395–400, 2005

    Article  PubMed  Google Scholar 

  44. Beigelman-Aubry C, Hill C, Boulanger X, et al: Evaluation of a computer aided detection system for lung nodules with groundglass opacity component on multidetector-row CT. J Radiol 90:1843–1849, 2009

    Article  PubMed  CAS  Google Scholar 

  45. Kim KG, Goo JM, Kim JH, et al: Computer-aided diagnosis of localized ground-glass opacity in the lung at CT: initial experience. Radiology 237:657–661, 2005

    Article  PubMed  Google Scholar 

  46. Lee JW, Jeong JW, Lee S, et al: The GGO lesions detected by computer-aided detection system on chest MDCT images. Conf Proc IEEE Eng Med Biol Soc 1:1983–1985, 2006

    PubMed  Google Scholar 

  47. Okada T, Iwano S, Ishigaki T, et al: Computer-aided diagnosis of lung cancer: definition and detection of ground-glass opacity type of nodules by high-resolution computed tomography. Jpn J Radiol 27:91–99, 2009

    Article  PubMed  Google Scholar 

  48. Hein PA, Romano VC, Rogalla P, et al: Variability of semiautomated lung nodule volumetry on ultralow-dose CT: comparison with nodule volumetry on standard-dose CT. J Digit Imag 23:8–17, 2009

    Article  Google Scholar 

  49. Park EA, Goo JM, Lee JW, et al: Efficacy of computer-aided detection system and thin-slab maximum intensity projection technique in the detection of pulmonary nodules in patients with resected metastases. Invest Radiol 44:105–113, 2009

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Siemens Healthcare, Malvern, PA, 19355, USA

    Luca Bogoni, Vikram Anand, Charles H. Florin & Marcos Salganicoff

  2. Department of Radiology, New York University-Langone Medical Center, Tisch Hospital, 560 1st Avenue, New York City, NY, 10016, USA

    Jane P. Ko, Jeffrey Alpert, John Fantauzzi, Chi Wan Koo, Derek Mason, Maria Shiau & David P. Naidich

  3. The Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, New York University School of Medicine, 560 1st Avenue, New York City, NY, USA

    William Rom

Authors
  1. Luca Bogoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jane P. Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeffrey Alpert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vikram Anand
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John Fantauzzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Charles H. Florin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chi Wan Koo
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Derek Mason
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. William Rom
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Maria Shiau
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Marcos Salganicoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. David P. Naidich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca Bogoni.

Additional information

This study was supported by the National Institute of Health/National Cancer Institute-UO-1, CA, 86137 and NYU Biomarker, Clinical and Epidemiologic Center—Early Detection Research Network

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bogoni, L., Ko, J.P., Alpert, J. et al. Impact of a Computer-Aided Detection (CAD) System Integrated into a Picture Archiving and Communication System (PACS) on Reader Sensitivity and Efficiency for the Detection of Lung Nodules in Thoracic CT Exams. J Digit Imaging 25, 771–781 (2012). https://doi.org/10.1007/s10278-012-9496-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-012-9496-0

Keywords

Search

Navigation