Skip to main content
SpringerLink
Log in

Combining multi-scale feature fusion with multi-attribute grading, a CNN model for benign and malignant classification of pulmonary nodules

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Lung cancer has the highest mortality rate of all cancers, and early detection can improve survival rates. In the recent years, low-dose CT has been widely used to detect lung cancer. However, the diagnosis is limited by the subjective experience of doctors. Therefore, the main purpose of this study is to use convolutional neural network to realize the benign and malignant classification of pulmonary nodules in CT images. We collected 1004 cases of pulmonary nodules from LIDC-IDRI dataset, among which 554 cases were benign and 450 cases were malignant. According to the doctors’ annotates on the center coordinates of the nodules, two 3D CT image patches of pulmonary nodules with different scales were extracted. In this study, our work focuses on two aspects. Firstly, we constructed a multi-stream multi-task network (MSMT), which combined multi-scale feature with multi-attribute classification for the first time, and applied it to the classification of benign and malignant pulmonary nodules. Secondly, we proposed a new loss function to balance the relationship between different attributes. The final experimental results showed that our model was effective compared with the same type of study. The area under ROC curve, accuracy, sensitivity, and specificity were 0.979, 93.92%, 92.60%, and 96.25%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A: Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians 68(6): 394-424, 2018

    Google Scholar 

  2. Siegel RL, Miller KD, Jemal A: Cancer statistics, 2018. CA: A Cancer Journal for Clinicians 68(1): 7-30, 2018

    Google Scholar 

  3. Oudkerk M, Devaraj A, Vliegenthart R et al.: European position statement on lung cancer screening. LANCET ONCOL 18(12): E754-E766, 2017

    Article  PubMed  Google Scholar 

  4. Gould MK, Maclean CC, Kuschner WG, Rydzak CE, Owens DK: Accuracy of positron emission tomography for diagnosis of pulmonary nodules and mass lesions: a meta-analysis. JAMA 285(7): 914-924, 2001

    Article  CAS  PubMed  Google Scholar 

  5. Liu Y, Balagurunathan Y, Atwater T: Radiological Image Traits Predictive of Cancer Status in Pulmonary Nodules. CLIN CANCER RES 23(6): 1442-1449, 2017

    Article  PubMed  Google Scholar 

  6. Zhang G, Yang Z, Gong L: An Appraisal of Nodule Diagnosis for Lung Cancer in CT Images.J MED SYST 43(7): 181, 2019

    Article  PubMed  Google Scholar 

  7. Han F, Wang H, Zhang G et al.: Texture Feature Analysis for Computer-Aided Diagnosis on Pulmonary Nodules. J DIGIT IMAGING 28(1): 99-115, 2015

    Article  PubMed  Google Scholar 

  8. Ye X, Lin X, Dehmeshki J, Slabaugh G, Beddoe G: Shape-based computer-aided detection of lung nodules in thoracic CT images. IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING 56(7): 1810-1820, 2009

  9. Dhara AK, Mukhopadhyay S, Dutta A, Garg M, Khandelwal N: A Combination of Shape and Texture Features for Classification of Pulmonary Nodules in Lung CT Images. J DIGIT IMAGING 29(4): 466-475, 2016

    Article  PubMed  PubMed Central  Google Scholar 

  10. Wang H, Zhao T, Li LC et al.: A hybrid CNN feature model for pulmonary nodule malignancy risk differentiation. J Xray Sci Technol 26(2): 171-187, 2018

    PubMed  Google Scholar 

  11. Farag AA, Ali A, Elshazly S, Farag AA: Feature fusion for lung nodule classification. INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY 12(10): 1809-1818, 2017

    Google Scholar 

  12. Nibali A, He Z, Wollersheim D: Pulmonary nodule classification with deep residual networks. INT J COMPUT ASS RAD 12(10): 1799-1808, 2017

    Google Scholar 

  13. Polat H, Mehr HD: Classification of Pulmonary CT Images by Using Hybrid 3D-Deep Convolutional Neural Network Architecture.Applied Sciences 9(5): 940, 2019

    Article  Google Scholar 

  14. Huang XJ, Shan JJ, Vaidya V: Lung nodule detection in CT using 3D convolutional neural networks. In: ISBI, 2017, pp 379-383

  15. Dou Q, Chen H, Yu L, Qin J, Heng PA: Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection. IEEE transactions on bio-medical engineering 64(7): 1558-1567, 2017

    Article  PubMed  Google Scholar 

  16. Liu K, Kang G: Multiview convolutional neural networks for lung nodule classification. INT J IMAG SYST TECH 27(1):12-22, 2017

    Article  Google Scholar 

  17. Ciompi F, Chung K, van Riel SJ: Towards automatic pulmonary nodule management in lung cancer screening with deep learning. SCI REP-UK 7: 46479, 2017

    Article  CAS  Google Scholar 

  18. Li X, Kao Y, Shen W, Li X, Xie G: Lung Nodule Malignancy Prediction Using Multi-task Convolutional Neural Network. In: Medical Imaging 2017: Computer-Aided Diagnosis, vol 10134, 2017, UNSP 1013424. International Society for Optics and Photonics

  19. Kang B, Zhu WP, Liang D: Robust multi-feature visual tracking via multi-task kernel-based sparse learning. IET IMAGE PROCESS 11(12): 1172-1178, 2017

    Article  Google Scholar 

  20. Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA et al.: The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A Completed Reference Database of Lung Nodules on CT Scans. MED PHYS 38(2): 915-931, 2011

    Article  PubMed  PubMed Central  Google Scholar 

  21. Kazerooni EA, Austin JHM, Black WC, Dyer DS, Hazelton TR, Leung AN, McNitt-Gray MF, Munden RF, Pipavath S: ACR-STR Practice Parameter for the Performance and Reporting of Lung Cancer Screening Thoracic Computed Tomography (CT). JOURNAL OF THORACIC IMAGING JOURNAL OF THORACIC IMAGING 29(5): 310-316, 2014

    Article  PubMed  Google Scholar 

  22. Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, Gareen IF, Gatsonis C, Marcus PM, Sicks JD: Reduced lung-cancer mortality with low-dose computed tomographic screening. NEW ENGL J MED 365(5): 395-409, 2011

    Article  PubMed  Google Scholar 

  23. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp 770–778

  24. He K, Zhang X, Ren S, Sun J: Identity Mappings in Deep Residual Networks. In: European Conference on Computer Vision. Springer, 2016, pp 630–645

  25. Kline DM, Berardi VL: Revisiting squared-error and cross-entropy functions for training neural network classifiers. Neural Computing and Applications 14(4): 310-318, 2015

    Article  Google Scholar 

  26. Facebook Inc. Pytorch. Available at https://pytorch.org. Accessed 16 August 2018

  27. Zhang Y, Yang Z, Lu H, Zhou X, Phillips P, Liu Q, Wang S: Facial Emotion Recognition Based on Biorthogonal Wavelet Entropy, Fuzzy Support Vector Machine, and Stratified Cross Validation. IEEE ACCESS 4:8375-8385, 2016

    Article  Google Scholar 

  28. Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. MED IMAGE ANAL 36: 61-78, 2017

    Article  PubMed  Google Scholar 

  29. Lyu J, Ling SH: Using multi-level convolutional neural network for classification of lung nodules on CT images. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 2018, pp 686-689

  30. Xie Y, Zhang J, Xia Y, Fulham M, Zhang Y: Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT. INFORM FUSION 42:102-110, 2018

    Article  Google Scholar 

  31. Xie Y, Xia Y, Zhang J: Knowledge-based Collaborative Deep Learning for Benign-Malignant Lung Nodule Classification on Chest CT. IEEE T MED IMAGING 38(4): 991-1004, 2019

    Article  Google Scholar 

  32. Causey JL, Zhang J, Ma S, Jiang B, Qualls JA, Politte DG, Prior F, Zhang S, Huang X: Highly accurate model for prediction of lung nodule malignancy with CT scans. SCI REP-UK 8:9286, 2018

    Article  CAS  Google Scholar 

Download references

Funding

The paper supported by the General Object of National Natural Science Foundation (61772358) Research on the key technology of BDS precision positioning in complex landform; Project supported by International Cooperation Project of Shanxi Province (Grant No.201603D421014) Three-Dimensional Reconstruction Research of Quantitative Multiple Sclerosis Demyelination; International Cooperation Project of Shanxi Province (Grant No. 201603D421012): Research on the key technology of GNSS area strengthen information extraction based on crowd sensing.

Author information

Authors and Affiliations

  1. College of Information and Computer, Taiyuan University of Technology, Jinzhong, China

    Jumin Zhao, Chen Zhang & Jing Niu

  2. Technology Research Center of Spatial Information Network Engineering of Shanxi, Jinzhong, China

    Jumin Zhao & Dengao Li

  3. College of Data Science, Taiyuan University of Technology, Jinzhong, China

    Dengao Li

Authors
  1. Jumin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dengao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dengao Li.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Zhang, C., Li, D. et al. Combining multi-scale feature fusion with multi-attribute grading, a CNN model for benign and malignant classification of pulmonary nodules. J Digit Imaging 33, 869–878 (2020). https://doi.org/10.1007/s10278-020-00333-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-020-00333-1

Keywords

Search

Navigation