Transferable Multi-model Ensemble for Benign-Malignant Lung Nodule Classification on Chest CT

  • Yutong Xie
  • Yong XiaEmail author
  • Jianpeng Zhang
  • David Dagan Feng
  • Michael Fulham
  • Weidong Cai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)


The classification of benign versus malignant lung nodules using chest CT plays a pivotal role in the early detection of lung cancer and this early detection has the best chance of cure. Although deep learning is now the most successful solution for image classification problems, it requires a myriad number of training data, which are not usually readily available for most routine medical imaging applications. In this paper, we propose the transferable multi-model ensemble (TMME) algorithm to separate malignant from benign lung nodules using limited chest CT data. This algorithm transfers the image representation abilities of three ResNet-50 models, which were pre-trained on the ImageNet database, to characterize the overall appearance, heterogeneity of voxel values and heterogeneity of shape of lung nodules, respectively, and jointly utilizes them to classify lung nodules with an adaptive weighting scheme learned during the error back propagation. Experimental results on the benchmark LIDC-IDRI dataset show that our proposed TMME algorithm achieves a lung nodule classification accuracy of 93.40%, which is markedly higher than the accuracy of seven state-of-the-art approaches.


Lung nodule classification Deep learning Ensemble learning Computed tomography (CT) 



This work was supported in part by the National Natural Science Foundation of China under Grants 61471297, in part by the Seed Foundation of Innovation and Creation for Graduate Students in Northwestern Polytechnical University under Grants Z2017041, and in part by the Australian Research Council (ARC) Grants. We acknowledged the National Cancer Institute and the Foundation for the National Institutes of Health, and their critical role in the creation of the free publicly available LIDC/IDRI Database used in this work.

Supplementary material

455908_1_En_75_MOESM1_ESM.docx (1.1 mb)
Supplementary material 1 (docx 1099 kb)


  1. 1.
    Siegel, R.L., Miller, K.D., Jemal, A.: Cancer statistics. CA Cancer J. Clin. 65(1), 5–29 (2015)CrossRefGoogle Scholar
  2. 2.
    Bach, P.B., Mirkin, J.N., Oliver, T.K., Azzoli, C.G., Berry, D.A., Brawley, O.W., Byers, T., Colditz, G.A., Gould, M.K., Jett, J.R.: Benefits and harms of CT screening for lung cancer: a systematic review. JAMA, J. Am. Med. Assoc. 307, 2418–2429 (2012)CrossRefGoogle Scholar
  3. 3.
    Abraham, J.: Reduced lung-cancer mortality with low-dose computed tomographic screening. New Engl. J. Med. 365, 395–409 (2011)CrossRefGoogle Scholar
  4. 4.
    American Thoracic Society: What is a lung nodule? Am. J. Respir. Crit. Care Med. 193, 11–12 (2016)CrossRefGoogle Scholar
  5. 5.
    Han, F., Wang, H., Zhang, G., Han, H., Song, B., Li, L., Moore, W., Lu, H., Zhao, H., Liang, Z.: Texture feature analysis for computer-aided diagnosis on pulmonary nodules. J. Digit. Imaging 28(1), 99–115 (2015)CrossRefGoogle Scholar
  6. 6.
    Dhara, A.K., Mukhopadhyay, S., Dutta, A., Garg, M., Khandelwal, N.: A combination of shape and texture features for classification of pulmonary nodules in lung CT images. J. Digit. Imaging 29(4), 466–475 (2016)CrossRefGoogle Scholar
  7. 7.
    Shen, W., Zhou, M., Yang, F., Yu, D., Dong, D., Yang, C., Zang, Y., Tian, J.: Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recogn. 61, 663–673 (2017)CrossRefGoogle Scholar
  8. 8.
    Hu, J., Lu, J., Tan, Y.P.: Deep transfer metric learning. In: CVPR 2015, pp. 325–333. IEEE Press, New York (2015)Google Scholar
  9. 9.
    Metz, S., Ganter, C., Lorenzen, S., Marwick, S.V., Holzapfel, K., Herrmann, K., Rummeny, E.J., Wester, H.J., Schwaiger, M., Nekolla, S.G.: Multiparametric MR and PET imaging of intratumoral biological heterogeneity in patients with metastatic lung cancer using voxel-by-voxel analysis. PLoS ONE 10(7), e0132386 (2014)CrossRefGoogle Scholar
  10. 10.
    Chen, S., Qin, J., Ji, X., Lei, B., Wang, T., Ni, D., Cheng, J.Z.: Automatic scoring of multiple semantic attributes with multi-task feature leverage: a study on pulmonary nodules in CT images. IEEE Trans. Med. Imaging 99, 1 (2016)Google Scholar
  11. 11.
    Xie, Y., Zhang, J., Liu, S., Cai, W., Xia, Y.: Lung nodule classification by jointly using visual descriptors and deep features. In: Müller, H., et al. (eds.) MCV 2016, BAMBI 2016. LNCS, vol. 10081. Springer, Cham (2017). doi: 10.1007/978-3-319-61188-4_11 CrossRefGoogle Scholar
  12. 12.
    Iii, S.G.A., Mclennan, G., Bidaut, L., Mcnittgray, M.F., Meyer, C.R., Reeves, A.P., Zhao, B., Aberle, D.R., Henschke, C.I., Hoffman, E.A.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38, 915–931 (2011)CrossRefGoogle Scholar
  13. 13.
    Hua, K.L., Hsu, C.H., Hidayati, S.C., Cheng, W.H., Chen, Y.J.: Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther. 8, 2015–2022 (2015)Google Scholar
  14. 14.
    Han, F., Zhang, G., Wang, H., Song, B.: A texture feature analysis for diagnosis of pulmonary nodules using LIDC-IDRI database. In: ICMIPE 2013, pp. 14–18. IEEE Press (2013)Google Scholar
  15. 15.
    Anand, S.K.V.: Segmentation coupled textural feature classification for lung tumor prediction. In: 2010 IEEE International Conference on Communication Control and Computing Technologies, pp. 518–524. IEEE Press, New York (2010)Google Scholar
  16. 16.
    Lampert, T.A., Stumpf, A., Gançarski, P.: An empirical study into annotator agreement, ground truth estimation, and algorithm evaluation. IEEE TIP 25(6), 2557–2572 (2016)MathSciNetzbMATHGoogle Scholar
  17. 17.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR 2016, pp. 770–778. IEEE Press, New York (2016)Google Scholar
  18. 18.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P.: Going deeper with convolutions. In: CVPR 2015, pp. 1–9. IEEE Press, New York (2016)Google Scholar
  19. 19.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409-1556 (2014)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yutong Xie
    • 1
  • Yong Xia
    • 1
    Email author
  • Jianpeng Zhang
    • 1
  • David Dagan Feng
    • 2
    • 5
  • Michael Fulham
    • 2
    • 3
    • 4
  • Weidong Cai
    • 2
  1. 1.Shaanxi Key Lab of Speech & Image Information Processing (SAIIP), School of Computer Science and EngineeringNorthwestern Polytechnical UniversityXi’anPeople’s Republic of China
  2. 2.Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information TechnologiesUniversity of SydneySydneyAustralia
  3. 3.Department of Molecular ImagingRoyal Prince Alfred HospitalCamperdownAustralia
  4. 4.Sydney Medical SchoolUniversity of SydneySydneyAustralia
  5. 5.Med-X Research InstituteShanghai Jiaotong UniversityShanghaiChina

Personalised recommendations