Multi-scale Convolutional Neural Networks for Lung Nodule Classification

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9123)


We investigate the problem of diagnostic lung nodule classification using thoracic Computed Tomography (CT) screening. Unlike traditional studies primarily relying on nodule segmentation for regional analysis, we tackle a more challenging problem on directly modelling raw nodule patches without any prior definition of nodule morphology. We propose a hierarchical learning framework—Multi-scale Convolutional Neural Networks (MCNN)—to capture nodule heterogeneity by extracting discriminative features from alternatingly stacked layers. In particular, to sufficiently quantify nodule characteristics, our framework utilizes multi-scale nodule patches to learn a set of class-specific features simultaneously by concatenating response neuron activations obtained at the last layer from each input scale. We evaluate the proposed method on CT images from Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI), where both lung nodule screening and nodule annotations are provided. Experimental results demonstrate the effectiveness of our method on classifying malignant and benign nodules without nodule segmentation.


Lung nodule classification Computed Tomography (CT) Imaging Convolutional Neural Networks Computer-Aided Diagnoses (CAD) 



The authors acknowledge the National Cancer Institute and the Foundation for the National Institutes of Health, and their critical role in the creation of the free publicly available LIDC/IDRI Database used in this study. We also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Tesla K40 GPU used for this research. This paper is supported by the Chinese Academy of Sciences Key Deployment Program under Grant No. KGZD-EW-T03, the National Basic Research Program of China (973 Program) under Grant 2011CB707700, the National Natural Science Foundation of China under Grant No. 81227901, 61231004, 81370035, 81230030, 61301002, 61302025, major projects of Biomedicine Department of Shanghai Science and Technology Commission (13411950100), the Chinese Academy of Sciences Fellowship for Young International Scientists under Grant No. 2010Y2GA03, 2013Y1 GA0004, 2013Y1GB0005, the Chinese Academy of Sciences Visiting Professorship for Senior International Scientists under Grant No. 2012T1G0036, 2010T2G 36, 2012T1G0039, 2013T1G0013, the National High Technology Research and Development Program of China (863 Program) under 2012AA021105, the Guangdong Province-Chinese Academy of Sciences comprehensive strategic cooperation program under 2010A090100032 and 2012B090400039, the NSFC-NIH Biomedical collaborative research program under 81261120414, the National Science and Technology Supporting Plan under 2012BAI15B08, the Beijing Natural Science Foundation under Grant No. 4132080, the Fundamental Research Funds for the Central Universities under Grant No. 2013JBZ014.


  1. 1.
    Aberle, D.R., Adams, A.M., Berg, C.D., Black, W.C., Clapp, J.D., Fagerstrom, R.M., Gareen, I.F., Gatsonis, C., Marcus, P.M., Sicks, J.: Reduced lung-cancer mortality with low-dose computed tomographic screening. N. Engl. J. Med. 365(5), 395–409 (2011)CrossRefGoogle Scholar
  2. 2.
    Aerts, H.J., Velazquez, E.R., Leijenaar, R.T., Parmar, C., Grossmann, P., Cavalho, S., Bussink, J., Monshouwer, R., Haibe-Kains, B., Haibe-Kains, D., Rietveld, D., Hoebers, F., Rietbergen, M.M., Leemans, C.R., Dekker, A., Quackenbush, J., Gillies, R.J., Lambin, P.: Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat. Commun. 5, Article No. 4006 (2014). doi: 10.1038/ncomms5006
  3. 3.
    Armato III, S.G., McLennan, G., Bidaut, L., McNitt-Gray, M.F., Meyer, C.R., Reeves, A.P., Zhao, B., Aberle, D.R., Henschke, C.I., Hoffman, E.A., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. phys. 38(2), 915–931 (2011)CrossRefGoogle Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893 (2005)Google Scholar
  5. 5.
    El-Baz, A., Nitzken, M., Khalifa, F., Elnakib, A., Gimel’farb, G., Falk, R., El-Ghar, M.A.: 3D shape analysis for early diagnosis of malignant lung nodules. In: Székely, G., Hahn, H.K. (eds.) IPMI 2011. LNCS, vol. 6801, pp. 772–783. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  6. 6.
    Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)CrossRefGoogle Scholar
  7. 7.
    Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2, 1735–1742 (2006)Google Scholar
  8. 8.
    Han, F., Zhang, G., Wang, H., Song, B., Lu, H., Zhao, D., Zhao, H., Liang, Z.: A texture feature analysis for diagnosis of pulmonary nodules using LIDC-IDRI database. In: IEEE International Conference on Medical Imaging Physics and Engineering, pp. 14–18 (2013)Google Scholar
  9. 9.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding (2014). arXiv preprint arXiv:1408.5093
  10. 10.
    Jolliffe, I.: Principal Component Analysis. Wiley Online Library, Chichester (2005)CrossRefGoogle Scholar
  11. 11.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  12. 12.
    Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets (2014). arXiv preprint arXiv:1409.5185
  13. 13.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefGoogle Scholar
  14. 14.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)MATHMathSciNetGoogle Scholar
  15. 15.
    Prasanna, P., Tiwari, P., Madabhushi, A.: Co-occurrence of Local Anisotropic Gradient Orientations (CoLlAGe): distinguishing tumor confounders and molecular subtypes on MRI. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part III. LNCS, vol. 8675, pp. 73–80. Springer, Heidelberg (2014) Google Scholar
  16. 16.
    Roth, H.R., Lu, L., Seff, A., Cherry, K.M., Hoffman, J., Wang, S., Liu, J., Turkbey, E., Summers, R.M.: A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part I. LNCS, vol. 8673, pp. 520–527. Springer, Heidelberg (2014) Google Scholar
  17. 17.
    Suzuki, K., Li, F., Sone, S., Doi, K.: Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose ct by use of massive training artificial neural network. IEEE Trans. Med. Imaging 24(9), 1138–1150 (2005)CrossRefGoogle Scholar
  18. 18.
    van der Walt, S., Schönberger, J.L., Nunez-Iglesias, J., Boulogne, F., Warner, J.D., Yager, N., Gouillart, E., Yu, T.: Scikit-image: Image processing in python. Technical report, PeerJ PrePrints (2014)Google Scholar
  19. 19.
    Way, T.W., Hadjiiski, L.M., Sahiner, B., Chan, H.P., Cascade, P.N., Kazerooni, E.A., Bogot, N., Zhou, C.: Computer-aided diagnosis of pulmonary nodules on CT scans: segmentation and classification using 3D active contours. Med. Phys. 33(7), 2323–2337 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Key Laboratory of Molecular Imaging of Chinese Academy of SciencesInstitute of Automation, Chinese Academy of SciencesBeijingChina
  2. 2.School of Computer and Information TechnologyBeijing Jiaotong UniversityBeijingChina
  3. 3.Department of Computer Science and EngineeringUniversity of South FloridaTampaUSA

Personalised recommendations