Advertisement

Spherical U-Net on Cortical Surfaces: Methods and Applications

  • Fenqiang Zhao
  • Shunren Xia
  • Zhengwang Wu
  • Dingna Duan
  • Li Wang
  • Weili Lin
  • John H. Gilmore
  • Dinggang Shen
  • Gang LiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11492)

Abstract

Convolutional Neural Networks (CNNs) have been providing the state-of-the-art performance for learning-related problems involving 2D/3D images in Euclidean space. However, unlike in the Euclidean space, the shapes of many structures in medical imaging have a spherical topology in a manifold space, e.g., brain cortical or subcortical surfaces represented by triangular meshes, with large inter-subject and intra-subject variations in vertex number and local connectivity. Hence, there is no consistent neighborhood definition and thus no straightforward convolution/transposed convolution operations for cortical/subcortical surface data. In this paper, by leveraging the regular and consistent geometric structure of the resampled cortical surface mapped onto the spherical space, we propose a novel convolution filter analogous to the standard convolution on the image grid. Accordingly, we develop corresponding operations for convolution, pooling, and transposed convolution for spherical surface data and thus construct spherical CNNs. Specifically, we propose the Spherical U-Net architecture by replacing all operations in the standard U-Net with their spherical operation counterparts. We then apply the Spherical U-Net to two challenging and neuroscientifically important tasks in infant brains: cortical surface parcellation and cortical attribute map development prediction. Both applications demonstrate the competitive performance in the accuracy, computational efficiency, and effectiveness of our proposed Spherical U-Net, in comparison with the state-of-the-art methods.

Keywords

Spherical U-Net Convolutional Neural Network Cortical surface Parcellation Prediction 

Notes

Acknowledgements

This work was partially supported by NIH grants (MH107815, MH108914, MH109773, MH116225, and MH117943) and China Scholarship Council.

References

  1. 1.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)CrossRefGoogle Scholar
  2. 2.
    Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond euclidean data. IEEE Sig. Process. Mag. 34(4), 18–42 (2017)CrossRefGoogle Scholar
  3. 3.
    Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203 (2013)
  4. 4.
    Cohen, T.S., Geiger, M., Köhler, J., Welling, M.: Spherical CNNs. arXiv preprint arXiv:1801.10130 (2018)
  5. 5.
    Coors, B., Paul Condurache, A., Geiger, A.: Spherenet: learning spherical representations for detection and classification in omnidirectional images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 518–533 (2018)CrossRefGoogle Scholar
  6. 6.
    Fischl, B.: Freesurfer. Neuroimage 62(2), 774–781 (2012)CrossRefGoogle Scholar
  7. 7.
    Gilmore, J.H., Knickmeyer, R.C., Gao, W.: Imaging structural and functional brain development in early childhood. Nat. Rev. Neurosci. 19(3), 123 (2018)CrossRefGoogle Scholar
  8. 8.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  9. 9.
    Li, G., Lin, W., Gilmore, J.H., Shen, D.: Spatial patterns, longitudinal development, and hemispheric asymmetries of cortical thickness in infants from birth to 2 years of age. J. Neurosci. 35(24), 9150–9162 (2015)CrossRefGoogle Scholar
  10. 10.
    Li, G., Wang, L., Shi, F., Gilmore, J.H., Lin, W., Shen, D.: Construction of 4D high-definition cortical surface atlases of infants: methods and applications. Med. Image Anal. 25(1), 22–36 (2015)CrossRefGoogle Scholar
  11. 11.
    Li, G., et al.: Computational neuroanatomy of baby brains: a review. Neuroimage 185, 906–925 (2018)CrossRefGoogle Scholar
  12. 12.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  13. 13.
    Meng, Y., et al.: Can we predict subject-specific dynamic cortical thickness maps during infancy from birth? Hum. Brain Mapp. 38(6), 2865–2874 (2017)CrossRefGoogle Scholar
  14. 14.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  15. 15.
    Seong, S.B., Pae, C., Park, H.J.: Geometric convolutional neural network for analyzing surface-based neuroimaging data. Front. Neuroinf. 12, 42 (2018)CrossRefGoogle Scholar
  16. 16.
    Wang, N., Li, S., Gupta, A., Yeung, D.Y.: Transferring rich feature hierarchies for robust visual tracking. arXiv preprint arXiv:1501.04587 (2015)
  17. 17.
    Wu, Z., et al.: Registration-free infant cortical surface parcellation using deep convolutional neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 672–680. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00931-1_77CrossRefGoogle Scholar
  18. 18.
    Zhang, Z., Xu, Y., Yu, J., Gao, S.: Saliency detection in 360\(^\circ \) videos. In: The European Conference on Computer Vision (ECCV) (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Fenqiang Zhao
    • 1
    • 2
  • Shunren Xia
    • 1
  • Zhengwang Wu
    • 2
  • Dingna Duan
    • 1
    • 2
  • Li Wang
    • 2
  • Weili Lin
    • 2
  • John H. Gilmore
    • 3
  • Dinggang Shen
    • 2
  • Gang Li
    • 2
    Email author
  1. 1.Key Laboratory of Biomedical Engineering of Ministry of EducationZhejiang UniversityHangzhouChina
  2. 2.Department of Radiology and BRICUniversity of North Carolina at Chapel HillChapel HillUSA
  3. 3.Department of PsychiatryUniversity of North Carolina at Chapel HillChapel HillUSA

Personalised recommendations