Skip to main content
SpringerLink
Log in

Efficiency of a computer-aided diagnosis (CAD) system with deep learning in detection of pulmonary nodules on 1-mm-thick images of computed tomography

  • Original Article
  • Published:
Japanese Journal of Radiology Aims and scope Submit manuscript

Abstract

Purpose

To evaluate the performance of a deep learning-based computer-aided diagnosis (CAD) system at detecting pulmonary nodules on CT by comparing radiologists’ readings with and without CAD.

Materials and methods

A total of 120 chest CT images were randomly selected from patients with suspected lung cancer. The gold standard of nodules ≥ 3 mm was established by a panel of three expert radiologists. Two less experienced radiologists read the images without and afterward with CAD system. Their reading times were recorded.

Results

The radiologists’ sensitivity increased from 20.9% to 38.0% with the introduction of CAD. The positive predictive value (PPV) decreased from 70.5% to 61.8%, and the F1-score increased from 32.2% to 47.0%. The sensitivity significantly increased from 13.7% to 32.4% for small nodules (3–6 mm) and from 33.3% to 47.6% for medium nodules (6–10 mm). CAD alone showed a sensitivity of 70.3%, a PPV of 57.9%, and an F1-score of 63.5%. Reading time decreased by 11.3% with the use of CAD.

Conclusion

CAD improved the less experienced radiologists’ sensitivity in detecting pulmonary nodules of all sizes, especially including a significant improvement in the detection of clinically important-sized medium nodules (6–10 mm) as well as small nodules (3–6 mm) and reduced their reading time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global Cancer Statistics 2018. CA Cancer J Clin. 2018;68:394–424.

    Article  PubMed  Google Scholar 

  2. El-Baz A, Suri J. Lung imaging and computer aided diagnosis. 1st ed. Abingdon: Taylor and Francis; 2011.

    Google Scholar 

  3. Li Q. Recent progress in computer-aided diagnosis of lung nodules on thin-section CT. Computed Imaging Graph. 2007;31(4–5):248–57.

    Article  Google Scholar 

  4. Centers for Medicare & Medicaid Services (CMS). Decision memo for screening for lung cancer with low dose computed tomography (LDCT) (CAG-00439N). 2015. Available via https://www.cms.gov/medicare-coverage-database/details/nca-decision-memo.aspx?NCAId=274. Accessed 25 Jun 2019.

  5. Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, National Lung Screening Trial Research Team. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365(5):395–409.

    Article  PubMed  Google Scholar 

  6. Aberle DR, DeMello S, Berg CD, Black WC, Brewer B, Church TR, et al. Results of the two incidence screenings in the national lung screening trial. N Engl J Med. 2013;369(10):920–31.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Lo SCB, Freedman MT, Gillis LB, White CS, Mun SK. Computer-aided detection of lung nodules on CT with a computerized pulmonary vessel suppressed function. Am J Roentgenol. 2018;210:480–8.

    Article  Google Scholar 

  8. Al Mohammad B, Brennan PC, Mello-Thoms C. A review of lung cancer screening and the role of computer-aided detection. Clin Radiol. 2017;72(6):433–42.

    Article  CAS  PubMed  Google Scholar 

  9. Li F, Sone S, Abe H, MacMahon H, Armato SG, Doi K. Lung cancers missed at low-dose helical CT screening in a general population: comparison of clinical, histopathologic, and imaging findings. Radiology. 2002;225:673–83.

    Article  PubMed  Google Scholar 

  10. Torres EL, Fiorina E, Pennazio F, Peroni C, Saletta M, Camarlinghi N, et al. Large scale validation of the M5L lung CAD on heterogeneous CT datasets. Med Phys. 2015;42:1477–89.

    Article  PubMed  Google Scholar 

  11. Goo JM, Kim HY, Lee JW, Lee HJ, Lee CH, Lee KW, et al. Is the computer-aided detection scheme for lung nodule also useful in detecting lung cancer? J Comput Assist Tomogr. 2008;32(4):570–5.

    Article  PubMed  Google Scholar 

  12. Marten K, Seyfarth T, Auer F, Wiener E, Grillhösl A, Obenauer S, et al. Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists. Eur Radiol. 2004;14:1930–8.

    Article  PubMed  Google Scholar 

  13. Sahiner B, Pezeshk A, Hadjiiski LM, Wang X, Drukker K, Cha KH, et al. Deep learning in medical imaging and radiation therapy. Med Phys. 2019;46(1):e1–36.

    Article  PubMed  Google Scholar 

  14. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44.

    Article  CAS  PubMed  Google Scholar 

  15. Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw. 2015;61:85–117.

    Article  PubMed  Google Scholar 

  16. da Silva GLF, Valente TLA, Silva AC, de Paiva AC, Gattass M. Convolutional neural network-based PSO for lung nodule false positive reduction on CT images. Comput Methods Programs Biomed. 2018;162:109–18.

    Article  PubMed  Google Scholar 

  17. Dou Q, Chen H, Yu L, Qin J, Heng PA. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng. 2017;64:1558–677.

    Article  PubMed  Google Scholar 

  18. Li W, Cao P, Zhao D, Wang J. Pulmonary nodule classification with deep convolutional neural networks on computed tomography images. Comput Math Methods Med. 2016;2016:7 (6215085).

    Google Scholar 

  19. Ren S, He K, Girshick R, Sun J. Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence. Faster R-CNN. 2017;39(6):1137–49.

    Google Scholar 

  20. MacMahon H, Bankier AA, Naidich DP. Lung cancer screening: what is the effect of using a larger nodule threshold size to determine who is assigned to short-term CT follow-up? Radiology. 2014;273:326–7.

    Article  PubMed  Google Scholar 

  21. Vassallo L, Traverso A, Agnello M, Bracco C, Campanella D, Chiara G, et al. A cloud-based computer-aided detection system improves identification of lung nodules on computed tomography scans of patients with extra-thoracic malignancies. Eur Radiol. 2019;29(1):144–52.

    Article  PubMed  Google Scholar 

  22. Li L, Liu Z, Huang H, Lin M, Luo D. Evaluating the performance of a deep learning-based computer-aided diagnosis (DL-CAD) system for detecting and characterizing lung nodules: comparison with the performance of double reading by radiologists. Thoracic Cancer. 2019;10:183–92.

    Article  CAS  PubMed  Google Scholar 

  23. Armato SG III, Li F, Giger ML, MacMahon H, Sone S, Doi K. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology. 2002;225:685–92.

    Article  PubMed  Google Scholar 

  24. Yuan R, Vos PM, Cooperberg PL. Computer-aided detection in screening CT for pulmonary nodules. Am J Roentgenol. 2006;186:1280–7.

    Article  Google Scholar 

  25. Lee IJ, Gamsu G, Czum J, Wu N, Johnson R, Chakrapani S. Lung nodule detection on chest CT: evaluation of a computer-aided detection (CAD) system. Korean J Radiol. 2005;6:89–93.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Jacobs C, van Rikxoort EM, Murphy K, Prokop M, Schaefer-Prokop CM, van Ginneken B. Computer-aided detection of pulmonary nodules: a comparative study using the public LIDC/IDRI database. Eur Radiol. 2016;26:2139–47.

    Article  PubMed  Google Scholar 

  27. Zhao Y, de Bock GH, Vliegenthart R, van Klaveren RJ, Wang Y, Bogoni L, et al. Performance of computer-aided detection of pulmonary nodules in low-dose CT: comparison with double reading by nodule volume. Eur Radiol. 2012;22:2076–84.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Rubin GD, Lyo JK, Paik DS, Sherbondy AJ, Chow LC, Leung AN, et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology. 2005;234(1):274–83.

    Article  PubMed  Google Scholar 

  29. Retico A. Computer-aided detection for pulmonary nodule identification: improving the radiologist's performance? Imaging Med. 2013;5:249–63.

    Article  CAS  Google Scholar 

  30. Beigelman-Aubry C, Raffy P, Yang W, Castellino RA, Grenier PA. Computed-aided detection of solid lung nodules on followup MDCTscreening: evaluation of detection, tracking, and reading time. Am J Roentgenol. 2007;189:948–55.

    Article  Google Scholar 

  31. Godoy MCB, Kim TJ, White CS, Bogoni L, de Groot P, Florin C, et al. Benefit of computer-aided detection analysis for the detection of subsolid and solid lung nodules on thin- and thick-section CT. Am J Roentgenol. 2013;200:74–83.

    Article  Google Scholar 

  32. Wahidi MM, Govert JA, Goudar RK, Gould MK, McCrory DC. Evidence for the treatment of patients with pulmonary nodules: when is it lung cancer? ACCP evidence-based clinical practice guidelines (2nd edition). Chest. 2007;132:94S–107S.

    Article  PubMed  Google Scholar 

  33. Scholten ET, Horeweg N, de Koning HJ, Vliegenthart R, Oudkerk M, Mali WP, et al. Computed tomographic characteristics of interval and post screen carcinomas in lung cancer screening. Eur Radiol. 2015;25:81–8.

    Article  PubMed  Google Scholar 

  34. Henschke CI, Yankelevitz DF, Yip R, Reeves AP, Farooqi A, Xu D, et al. Lung cancers diagnosed at annual CT screening: volume doubling times. Radiology. 2012;263:578–83.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Henschke CI, Yankelevitz DF, Mirtcheva R, McGuinness G, McCauley D, Miettinen OS. CT screening for lung cancer: frequency and significance of part-solid and nonsolid nodules. Am J Roentgenol. 2002;178(5):1053–7.

    Article  Google Scholar 

  36. Armato SG 3rd, McNitt-Gray MF, Reeves AP, Meyer CR, McLennan G, Aberle DR, et al. The lung image database consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Acad Radiol. 2007;14:1409–21.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This paper is based on results obtained from research commissioned by Infervision Japan, Inc. provided support with equipment.

Author information

Authors and Affiliations

  1. Department of Radiology, Kindai University Faculty of Medicine, 377-2 Ohno-Higashi, Osaka-Sayama, Osaka, 589-8511, Japan

    Takenori Kozuka, Yuko Matsukubo, Tomoya Kadoba, Teruyoshi Oda, Ayako Suzuki, Tomoko Hyodo, SungWoon Im, Hayato Kaida, Yukinobu Yagyu, Masakatsu Tsurusaki, Mitsuru Matsuki & Kazunari Ishii

Authors
  1. Takenori Kozuka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuko Matsukubo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomoya Kadoba
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Teruyoshi Oda
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ayako Suzuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomoko Hyodo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. SungWoon Im
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hayato Kaida
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yukinobu Yagyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Masakatsu Tsurusaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Mitsuru Matsuki
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Kazunari Ishii
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Takenori Kozuka.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kozuka, T., Matsukubo, Y., Kadoba, T. et al. Efficiency of a computer-aided diagnosis (CAD) system with deep learning in detection of pulmonary nodules on 1-mm-thick images of computed tomography. Jpn J Radiol 38, 1052–1061 (2020). https://doi.org/10.1007/s11604-020-01009-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11604-020-01009-0

Keywords

Search

Navigation