Skip to main content
SpringerLink
Log in

Lung nodule malignancy classification in chest computed tomography images using transfer learning and convolutional neural networks

  • Computer aided Medical Diagnosis
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Lung cancer accounts for more than 1.5 million deaths worldwide, and it corresponded to 26% of all deaths due to cancer in 2017. However, lung computer-aided diagnosis systems developed to identify lung cancer at early stages are increasing survival rates. This study explores the performance of deep transfer learning from non-medical images on lung nodule malignancy classification tasks in order to improve such systems. Initially, the 1018 chest computed tomography (CT) examinations and medical annotations from the LIDC/IDRI were processed. Then, several convolutional neural networks (VGG16, VGG19, MobileNet, Xception, InceptionV3, ResNet50, InceptionResNetV2, DenseNet169, DenseNet201, NASNetMobile and NASNetLarge) were built, trained on the ImageNet dataset, converted into feature extractors and applied on the LIDC/IDRI nodule images. Following this, each set of deep features was submitted to 10-fold cross-validations with naive Bayes, multilayer perceptron, support vector machine (SVM), K-nearest neighbors KNN and random forest classifiers. Finally, the evaluation metrics accuracy (ACC), area under the curve (AUC), true positive rate (TPR), precision (PPV) and F1-score of each cross-validation average result were computed and compared. The results showed that the deep feature extractor based on the ResNet50 and the SVM RBF classifier, achieved an AUC metric of 93.1% (the highest value not only among the evaluated combinations, but also among the related works in the literature evaluated), a TPR of 85.38%, an ACC of 88.41%, a PPV of 73.48% and an F1-score of 78.83%. Based on these results, deep transfer learning proves to be a relevant strategy to extract representative features from lung nodule CT images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, Bussink J, Monshouwer R, Haibe-Kains B, Rietveld D et al (2014) Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006

    Article  Google Scholar 

  2. de Albuquerque VHC, Damaševičius R, Garcia NM, Pinheiro PR, Filho PPR (2017) Brain computer interface systems for neurorobotics: methods and applications. BioMed Res Int 2017:1–2. https://doi.org/10.1155/2017/2505493

    Article  Google Scholar 

  3. Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA, Kazerooni EA, MacMahon H, van Beek EJR, Yankelevitz D, Biancardi AM, Bland PH, Brown MS, Engelmann RM, Laderach GE, Max D, Pais RC, Qing DPY, Roberts RY, Smith AR, Starkey A, Batra P, Caligiuri P, Farooqi A, Gladish GW, Jude CM, Munden RF, Petkovska I, Quint LE, Schwartz LH, Sundaram B, Dodd LE, Fenimore C, Gur D, Petrick N, Freymann J, Kirby J, Hughes B, Casteele AV, Gupte S, Sallam M, Heath MD, Kuhn MH, Dharaiya E, Burns R, Fryd DS, Salganicoff M, Anand V, Shreter U, Vastagh S, Croft BY, Clarke LP (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 38(2):915–931. https://doi.org/10.1118/1.3528204

    Article  Google Scholar 

  4. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2017.195

  5. Christodoulidis S, Anthimopoulos M, Ebner L, Christe A, Mougiakakou S (2017) Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J Biomed Health Inform 21(1):76–84

    Article  Google Scholar 

  6. Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  Google Scholar 

  7. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009. , pp. 248–255. IEEE

  8. Dhara AK, Mukhopadhyay S, Dutta A, Garg M, Khandelwal N (2016) A combination of shape and texture features for classification of pulmonary nodules in lung ct images. J Digit Imaging 29(4):466–475

    Article  Google Scholar 

  9. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118. https://doi.org/10.1038/nature21056

    Article  Google Scholar 

  10. Filho PPR, Cortez PC, Holanda MA (2011) Modelo de contorno ativo crisp: nova técnica de segmentação dos pulmões em imagens de TC. Rev Bras Eng Bioméd 27(4):259–272. https://doi.org/10.4322/rbeb.2011.021

    Article  Google Scholar 

  11. Filho PPR, Cortez PC, da Silva Barros AC, Albuquerque VHC, Tavares JMRS (2017) Novel and powerful 3d adaptive crisp active contour method applied in the segmentation of CT lung images. Med Image Anal 35:503–516. https://doi.org/10.1016/j.media.2016.09.002

    Article  Google Scholar 

  12. Filho PPR, Sarmento RM, Holanda GB, de Alencar Lima D (2017) New approach to detect and classify stroke in skull CT images via analysis of brain tissue densities. Comput Methods Programs Biomed 148:27–43. https://doi.org/10.1016/j.cmpb.2017.06.011

    Article  Google Scholar 

  13. Gupta D, Julka A, Jain S, Aggarwal T, Khanna A, Arunkumar N, de Albuquerque VHC (2018) Optimized cuttlefish algorithm for diagnosis of parkinson’s disease. Cogn Syst Res 52:36–48. https://doi.org/10.1016/j.cogsys.2018.06.006

    Article  Google Scholar 

  14. Han F, Wang H, Zhang G, Han H, Song B, Li L, Moore W, Lu H, Zhao H, Liang Z (2015) Texture feature analysis for computer-aided diagnosis on pulmonary nodules. J Digit Imaging 28(1):99–115

    Article  Google Scholar 

  15. Haykin S (2008) Neural networks and learning machines. Prentice Hall, McMaster University, Upper Saddle River, Hamilton

    Google Scholar 

  16. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. CoRR arXiv:1704.04861

  17. Hsu CW, Chang CC, Lin CJ et al (2003) A practical guide to support vector classification

  18. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2017.243

  19. Hussein S, Cao K, Song Q, Bagci, (2017) Risk stratification of lung nodules using 3d cnn-based multi-task learning. In: International conference on information processing in medical imaging, pp. 249–260. Springer

  20. Huynh BQ, Li H, Giger ML (2016) Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. J Med Imaging 3(3):034,501. https://doi.org/10.1117/1.jmi.3.3.034501

    Article  Google Scholar 

  21. Hwang S, Kim HE, Jeong J, Kim HJ (2016) A novel approach for tuberculosis screening based on deep convolutional neural networks. In: Tourassi GD, Armato SG (eds) Medical imaging 2016: computer-aided diagnosis. SPIE, Bellingham. https://doi.org/10.1117/12.2216198

    Chapter  Google Scholar 

  22. Kang G, Liu K, Hou B, Zhang N (2017) 3d multi-view convolutional neural networks for lung nodule classification. PloS ONE 12(11):e0188,290

    Article  Google Scholar 

  23. Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L (2014) Large-scale video classification with convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1725–1732 (2014)

  24. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. CoRR arXiv:1412.6980

  25. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105

  26. Längkvist M, Karlsson L, Loutfi A (2014) A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognit Lett 42:11–24. https://doi.org/10.1016/j.patrec.2014.01.008

    Article  Google Scholar 

  27. LeCun Y, Boser BE, Denker JS, Henderson D, Howard RE, Hubbard WE, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems, pp 396–404

  28. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  29. Liaw A, Wiener M et al (2002) Classification and regression by randomforest. R news 2(3):18–22

    Google Scholar 

  30. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JA, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005

    Article  Google Scholar 

  31. Ma J, Wang Q, Ren Y, Hu H, Zhao J (2016) Automatic lung nodule classification with radiomics approach. In: SPIE medical imaging, pp 978,906–978,906. International Society for Optics and Photonics

  32. Marcus PM, Doria-Rose VP, Gareen IF, Brewer B, Clingan K, Keating K, Rosenbaum J, Rozjabek HM, Rathmell J, Sicks J et al (2016) Did death certificates and a death review process agree on lung cancer cause of death in the national lung screening trial? Clin Trials 13(4):434–438

    Article  Google Scholar 

  33. Näppi JJ, Hironaka T, Regge D, Yoshida H (2016) Deep transfer learning of virtual endoluminal views for the detection of polyps in CT colonography. In: Tourassi GD, Armato SG (eds) Medical imaging 2016: computer-aided diagnosis. SPIE, Bellingham. https://doi.org/10.1117/12.2217260

    Chapter  Google Scholar 

  34. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359. https://doi.org/10.1109/tkde.2009.191

    Article  Google Scholar 

  35. Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238. https://doi.org/10.1109/tpami.2005.159

    Article  Google Scholar 

  36. Rodrigues MB, Nóbrega RVMD, Alves SSA, Filho PPR, Duarte JBF, Sangaiah AK, Albuquerque VHCD (2018) Health of things algorithms for malignancy level classification of lung nodules. IEEE Access 6:18592–18601. https://doi.org/10.1109/ACCESS.2018.2817614

    Article  Google Scholar 

  37. Rodrigues MB, Nobrega RVMD, Alves SSA, Filho PPR, Duarte JBF, Sangaiah AK, Albuquerque VHCD (2018) Health of things algorithms for malignancy level classification of lung nodules. IEEE Access 6:18592–18601. https://doi.org/10.1109/access.2018.2817614

    Article  Google Scholar 

  38. Sergeeva M, Ryabchikov I, Glaznev M, Gusarova N (2016) Classification of pulmonary nodules on computed tomography scans. evaluation of the effectiveness of application of textural features extracted using wavelet transform of image. In: 2016 18th conference of open innovations association and seminar on information security and protection of information technology (FRUCT-ISPIT). IEEE. https://doi.org/10.1109/fruct-ispit.2016.7561541

  39. Shankar K, Lakshmanaprabu SK, Gupta D, Maseleno A, de Albuquerque VHC (2018) Optimal feature-based multi-kernel SVM approach for thyroid disease classification. J Supercomput. https://doi.org/10.1007/s11227-018-2469-4

  40. Shen W, Zhou M, Yang F, Yang C, Tian J (2015) Multi-scale convolutional neural networks for lung nodule classification. In: International conference on information processing in medical imaging, pp. 588–599. Springer

  41. Shen W, Zhou M, Yang F, Yu D, Dong D, Yang C, Zang Y, Tian J (2017) Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit 61:663–673

    Article  Google Scholar 

  42. Shie CK, Chuang CH, Chou CN, Wu MH, Chang EY (2015) Transfer representation learning for medical image analysis. In: 2015 37th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 711–714. IEEE

  43. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298. https://doi.org/10.1109/tmi.2016.2528162

    Article  Google Scholar 

  44. Siegel RL, Miller KD, Jemal A (2017) Cancer statistics, 2017. CA A Cancer J Clin 67(1):7–30. https://doi.org/10.3322/caac.21387

    Article  Google Scholar 

  45. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. CoRR arXiv:1409.1556

  46. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR). IEEE. https://doi.org/10.1109/cvpr.2015.7298594

  47. Theodoridis S, Koutroumbas K (2008) Pattern recognition, 4th edn. Academic Press, Cambridge

    MATH  Google Scholar 

  48. Tiwari P, Qian J, Li Q, Wang B, Gupta D, Khanna A, Rodrigues JJ (2018) Detection of subtype blood cells using deep learning. Cognit Syst Res. https://doi.org/10.1016/j.cogsys.2018.08.022

  49. Vapnik VN (1998) Statistical learning theory. Wiley, Hoboken

    MATH  Google Scholar 

  50. Wei G, Cao H, Ma H, Qi S, Qian W, Ma Z (2017) Content-based image retrieval for lung nodule classification using texture features and learned distance metric. J Med Syst 42(1):13. https://doi.org/10.1007/s10916-017-0874-5

    Article  Google Scholar 

  51. Wei G, Ma H, Qian W, Han F, Jiang H, Qi S, Qiu M (2018) Lung nodule classification using local kernel regression models with out-of-sample extension. Biomed Signal Process Control 40:1–9

    Article  Google Scholar 

  52. Wu S, Zhong S, Liu Y (2017) Deep residual learning for image steganalysis. Multimed Tools Appl 77(9):10437–10453. https://doi.org/10.1007/s11042-017-4440-4

    Article  Google Scholar 

  53. Zhu W, Liu C, Fan W, Xie X (2017) Deeplung: 3d deep convolutional nets for automated pulmonary nodule detection and classification. arXiv preprint arXiv:1709.05538

  54. Zoph B, Le QV (2016) Neural architecture search with reinforcement learning. CoRR arXiv:1611.01578

Download references

Acknowledgements

The authors acknowledge the financial support and encouragement from CNPq via Grant 304315/2017-6. The first author acknowledges the sponsorship from the Federal Institute of Education, Science and Technology of Ceará via grants PROINFRA/2017 and PROINFRA PPG/2017.

Author information

Authors and Affiliations

  1. Laboratório de Processamento Digital de Imagens, Sinais e Computação Aplicada, Instituto Federal de Federal de Educação, Ciência e Tecnologia do Ceará (IFCE), Fortaleza, Ceará, Brazil

    Raul Victor M. da Nóbrega, Pedro P. Rebouças Filho, Murillo B. Rodrigues, Suane P. P. da Silva & Carlos M. J. M. Dourado Júnior

  2. Programa de Pós-Graduação em Informática Aplicada, Laboratório de Bioinformática, Universidade de Fortaleza, Fortaleza, CE, Brazil

    Victor Hugo C. de Albuquerque

Authors
  1. Raul Victor M. da Nóbrega
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pedro P. Rebouças Filho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Murillo B. Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suane P. P. da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carlos M. J. M. Dourado Júnior
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Victor Hugo C. de Albuquerque
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro P. Rebouças Filho.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Nóbrega, R.V.M., Rebouças Filho, P.P., Rodrigues, M.B. et al. Lung nodule malignancy classification in chest computed tomography images using transfer learning and convolutional neural networks. Neural Comput & Applic 32, 11065–11082 (2020). https://doi.org/10.1007/s00521-018-3895-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3895-1

Keywords

Search

Navigation