Advertisement

Spatial-Frequency Non-local Convolutional LSTM Network for pRCC Classification

  • Yu Zhao
  • Yuan Liu
  • Yansheng Kan
  • Anjany Sekuboyina
  • Diana Waldmannstetter
  • Hongwei Li
  • Xiaobin HuEmail author
  • Xiaozhi Zhao
  • Kuangyu Shi
  • Bjoern Menze
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11769)

Abstract

The accurate classification of 3D medical images is a challenging task for current deep learning methods. Deep learning models struggle to extract features when the data size is small and the data dimension is large. To solve this problem, we develop a spatial-frequency non-local convolutional LSTM network for 3D image classification. Compared to traditional networks, the proposed model has the ability to extract features from both the spatial and frequency domains, which allows the frequency-domain features to contribute to the classification. Furthermore, the non-local blocks in our architecture enable it to capture the long-range dependencies directly in the feature space. Finally, to simplify the classification task and improve the performance, we utilize a two-stage framework that localizes lesions in the first step, and classifies them in the second. We evaluate our method on a challenging and important clinical task, i.e, the differentiation of papillary renal cell carcinoma (pRCC) into subtype 1 and subtype 2. To the best of our knowledge, this is the first time that the advantage of synthesizing spatial- and frequency-domain features by deep learning networks for medical image classification has been demonstrated. Experimental results demonstrate that the proposed method achieves competitive and often superior performance compared to state-of-the-art networks and three clinical experts.

Keywords

Deep neural network Convolutional LSTM Non-local network Spatial domain Frequency domain Papillary renal cell carcinoma 

References

  1. 1.
    Gupta, K., et al.: Epidemiologic and socioeconomic burden of metastatic renal cell carcinoma (mRCC): a literature review. Cancer Treat. Rev. 34(3), 193–205 (2008)CrossRefGoogle Scholar
  2. 2.
    Hazlett, H.C., et al.: Early brain development in infants at high risk for autism spectrum disorder. Nature 542(7641), 348 (2017)CrossRefGoogle Scholar
  3. 3.
    He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  4. 4.
    Huang, G., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)Google Scholar
  5. 5.
    Klatte, T., et al.: Cytogenetic and molecular tumor profiling for type 1 and type 2 papillary renal cell carcinoma. Clin. Cancer Res. 15(4), 1162–1169 (2009)CrossRefGoogle Scholar
  6. 6.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)CrossRefGoogle Scholar
  7. 7.
    Novikov, A.A., et al.: Deep sequential segmentation of organs in volumetric medical scans. IEEE Trans. Med. Imaging 36, 1359–1371 (2018)Google Scholar
  8. 8.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  9. 9.
    Roth, H.R., et al.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5), 1170–1181 (2016)CrossRefGoogle Scholar
  10. 10.
    Schlemper, J., et al.: Cardiac MR segmentation from undersampled k-space using deep latent representation learning. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 259–267. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00928-1_30CrossRefGoogle Scholar
  11. 11.
    Suk, H.I., et al.: Latent feature representation with stacked auto-encoder for AD/MCI diagnosis. Brain Struct. Func. 220(2), 841–859 (2015)CrossRefGoogle Scholar
  12. 12.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  13. 13.
    Jimenez-del Toro, O., et al.: Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms: visceral anatomy benchmarks. IEEE Trans. Med. Imaging 35(11), 2459–2475 (2016)CrossRefGoogle Scholar
  14. 14.
    Varghese, B.A., et al.: Differentiation of predominantly solid enhancing lipid-poor renal cell masses by use of contrast-enhanced CT: evaluating the role of texture in tumor subtyping. Am. J. Roentgenol. 211(6), W288–W296 (2018)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Wang, X., et al.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)Google Scholar
  16. 16.
    Zhu, B., et al.: Image reconstruction by domain-transform manifold learning. Nature 555(7697), 487 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yu Zhao
    • 1
  • Yuan Liu
    • 1
  • Yansheng Kan
    • 2
  • Anjany Sekuboyina
    • 1
  • Diana Waldmannstetter
    • 1
  • Hongwei Li
    • 1
  • Xiaobin Hu
    • 1
    Email author
  • Xiaozhi Zhao
    • 2
  • Kuangyu Shi
    • 3
  • Bjoern Menze
    • 1
  1. 1.Department of Computer ScienceTechnische Universität MünchenMunichGermany
  2. 2.Urology Department, The Affiliated Nanjing Drum Tower HospitalNanjing University Medical SchoolNanjingChina
  3. 3.Department of Nuclear MedicineUniversity of BernBernSwitzerland

Personalised recommendations