A manifold learning regularization approach to enhance 3D CT image-based lung nodule classification

  • Ying Ren
  • Min-Yu Tsai
  • Liyuan Chen
  • Jing Wang
  • Shulong Li
  • Yufei Liu
  • Xun JiaEmail author
  • Chenyang ShenEmail author
Original Article



Diagnosis of lung cancer requires radiologists to review every lung nodule in CT images. Such a process can be very time-consuming, and the accuracy is affected by many factors, such as experience of radiologists and available diagnosis time. To address this problem, we proposed to develop a deep learning-based system to automatically classify benign and malignant lung nodules.


The proposed method automatically determines benignity or malignancy given the 3D CT image patch of a lung nodule to assist diagnosis process. Motivated by the fact that real structure among data is often embedded on a low-dimensional manifold, we developed a novel manifold regularized classification deep neural network (MRC-DNN) to perform classification directly based on the manifold representation of lung nodule images. The concise manifold representation revealing important data structure is expected to benefit the classification, while the manifold regularization enforces strong, but natural constraints on network training, preventing over-fitting.


The proposed method achieves accurate manifold learning with reconstruction error of ~ 30 HU on real lung nodule CT image data. In addition, the classification accuracy on testing data is 0.90 with sensitivity of 0.81 and specificity of 0.95, which outperforms state-of-the-art deep learning methods.


The proposed MRC-DNN facilitates an accurate manifold learning approach for lung nodule classification based on 3D CT images. More importantly, MRC-DNN suggests a new and effective idea of enforcing regularization for network training, possessing the potential impact to a board range of applications.


Diagnosis Lung nodule classification Deep learning Regularization Manifold learning 



This work was supported by the China National Key Research and Development Program of China (Grant Nos. 2016YFE0125200 and 2016YFC0101100) and the China Fundamental Research Funds for the Central Universities (Grant No. 2019CDQYGD020).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.


  1. 1.
    Armato SG III, Altman MB, Wilkie J, Sone S, Li F, Doi K, Roy AS (2003) Automated lung nodule classification following automated nodule detection on CT: a serial approach. Med Phys 30:1188–1197CrossRefGoogle Scholar
  2. 2.
    Lee SLA, Kouzani AZ, Hu EJ (2010) Random forest based lung nodule classification aided by clustering. Comput Med Imag Graph 34:535–542CrossRefGoogle Scholar
  3. 3.
    Zhang F, Song Y, Cai W, Lee M-Z, Zhou Y, Huang H, Shan S, Fulham MJ, Feng DD (2014) Lung nodule classification with multilevel patch-based context analysis. IEEE Trans Biomed Eng 61:1155–1166CrossRefGoogle Scholar
  4. 4.
    Shen W, Zhou M, Yang F, Yang C, Tian J (2015) Multi-scale convolutional neural networks for lung nodule classification. In: International conference on information processing in medical imaging. Springer, pp 588–599Google Scholar
  5. 5.
    Kumar D, Wong A, Clausi DA (2015) Lung nodule classification using deep features in CT images. In: 2015 12th conference on computer and robot vision. IEEE, pp 133–138Google Scholar
  6. 6.
    Hua K-L, Hsu C-H, Hidayati SC, Cheng W-H, Chen Y-J (2015) Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets and Ther 8:2015–2022Google Scholar
  7. 7.
    Ma J, Wang Q, Ren Y, Hu H, Zhao J (2016) Automatic lung nodule classification with radiomics approach. In: Medical imaging 2016: PACS and imaging informatics: next generation and innovations. International Society for Optics and Photonics, pp 978906Google Scholar
  8. 8.
    Kalpathy-Cramer J, Mamomov A, Zhao B, Lu L, Cherezov D, Napel S, Echegaray S, Rubin D, McNitt-Gray M, Lo P (2016) Radiomics of lung nodules: a multi-institutional study of robustness and agreement of quantitative imaging features. Tomography 2:430CrossRefGoogle Scholar
  9. 9.
    Shen W, Zhou M, Yang F, Yu D, Dong D, Yang C, Zang Y, Tian J (2017) Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recogn 61:663–673CrossRefGoogle Scholar
  10. 10.
    Song QZ, Zhao L, Luo XK, Dou XC (2017) Using deep learning for classification of lung nodules on computed tomography images. J Healthc Eng 2017:8314740. CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Chen C-H, Chang C-K, Tu C-Y, Liao W-C, Wu B-R, Chou K-T, Chiou Y-R, Yang S-N, Zhang G, Huang T-C (2018) Radiomic features analysis in computed tomography images of lung nodule classification. PLOS One 13:e0192002CrossRefGoogle Scholar
  12. 12.
    Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1:131–156CrossRefGoogle Scholar
  13. 13.
    Liu H, Yu L (2005) Toward integrating feature selection algorithms for classification and clustering. IEEE Trans Knowl Data Eng 17:491–502CrossRefGoogle Scholar
  14. 14.
    He X, Yan S, Hu Y, Niyogi P, Zhang H-J (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27:328–340CrossRefGoogle Scholar
  15. 15.
    He X, Niyogi P (2003) Locality preserving projections. In: Thrun S, Saul LK, Schölkopf B (eds) Proceedings of the 16th international conference on neural information processing systems (NIPS'03), MIT Press, Cambridge, MA, USA, pp 153–160Google Scholar
  16. 16.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323–2326CrossRefGoogle Scholar
  17. 17.
    Zhu J-Y, Krähenbühl P, Shechtman E, Efros AA (2016) Generative visual manipulation on the natural image manifold. In: European conference on computer vision, Springer, pp 597–613Google Scholar
  18. 18.
    Ma G, Shen C, Jia X (2018) Low dose ct reconstruction assisted by an image manifold prior. arXiv preprint arXiv:181012255
  19. 19.
    Brosch T, Tam R, Initiative AsDN (2013) Manifold learning of brain MRIS by deep learning. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N (eds) Medical image computing and computer-assisted intervention-MICCAI 2013. MICCAI 2013. Lecture notes in computer science. Springer, Berlin, pp 633–640Google Scholar
  20. 20.
    Dai Y, Yan S, Zheng B, Song C (2018) Incorporating automatically learned pulmonary nodule attributes into a convolutional neural network to improve accuracy of benign-malignant nodule classification. Phys Med Biol 63:245004CrossRefGoogle Scholar
  21. 21.
    Xie Y, Xia Y, Zhang J, Song Y, Feng D, Fulham M, Cai W (2019) Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest ct. IEEE Trans Med Imag 38:991–1004CrossRefGoogle Scholar
  22. 22.
    Al-Shabi M, Lan BL, Chan WY, Ng K-H, Tan M (2019) Lung nodule classification using deep local-global networks. Int J Comput Assist Radiol Surg 14:1815CrossRefGoogle Scholar
  23. 23.
    Li S, Xu P, Li B, Chen L, Zhou Z, Hao H, Duan Y, Folkert MR, Ma J, Huang S (2019) Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features. Phys Med Biol 64:175012CrossRefGoogle Scholar
  24. 24.
    Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of compstat’2010, Springer, pp 177–186Google Scholar
  25. 25.
    Armato SG, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP, Zhao B, Aberle DR, Henschke CI, Hoffman EA (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on ct scans. Med Phys 38:915–931CrossRefGoogle Scholar
  26. 26.
    Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M (2013) The cancer imaging archive (tcia): maintaining and operating a public information repository. J Digit Imag 26:1045–1057CrossRefGoogle Scholar
  27. 27.
    Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M (2016) Tensorflow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX symposium on operating systems design and implementation (OSDI ’16), pp 265–283Google Scholar
  28. 28.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105Google Scholar
  29. 29.
    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252CrossRefGoogle Scholar
  30. 30.
    Cui W, Zhou Q, Zheng Z (2018) Application of a hybrid model based on a convolutional auto-encoder and convolutional neural network in object-oriented remote sensing classification. Algorithms 11:9CrossRefGoogle Scholar
  31. 31.
    Chen M, Shi X, Zhang Y, Wu D, Guizani M (2017) Deep features learning for medical image analysis with convolutional autoencoder neural network. In: IEEE Transactions on Big DataGoogle Scholar

Copyright information

© CARS 2019

Authors and Affiliations

  1. 1.Department of NeurologyHeilongjiang Province Number III HospitalBeianChina
  2. 2.Innovative Technology Of Radiotherapy Computations and Hardware (iTORCH) LaboratoryUniversity of Texas Southwestern Medical CenterDallasUSA
  3. 3.Medical Artificial Intelligence and Automation (MAIA) LaboratoryUniversity of Texas Southwestern Medical CenterDallasUSA
  4. 4.Department of Radiation OncologyUniversity of Texas Southwestern Medical CenterDallasUSA
  5. 5.Key Laboratory of Optoelectronic Technology and Systems, Ministry of EducationChongqing UniversityChongqingChina
  6. 6.Centre for Intelligent Sensing Technology, College of Optoelectronic EngineeringChongqing UniversityChongqingChina

Personalised recommendations