Advertisement

Autofocus Net: Auto-focused 3D CNN for Brain Tumour Segmentation

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1248)

Abstract

Several approaches based on convolutional neural networks (CNNs) are only able to process 2D images while most brain data consists of 3D volumes. Recent network architectures which have demonstrated promising results are able to process 3D images. In this work, we propose an adapted approach based on a CNN to process 3D contextual information in brain MRI scans for the challenging task of brain tumour segmentation. Our CNN is trained end-to-end on multi-modal MRI volumes and is able to predict segmentation for the binary case, which segments the whole tumour, and multi-class case, which segments the whole tumour (WT), tumour core (TC) and enhancing tumour (ET). Our network includes multiple layers of dilated convolutions and autofocus convolutions with residual connections to improve segmentation performance. Autofocus layers consist of multiple parallel convolutions each with a different dilation rate. We replaced standard convolutional layers with autofocus layers to adaptively change the size of the effective receptive field to generate more powerful features. Experiments with our autofocus settings on the BraTS 2018 glioma dataset show that the proposed method achieved average Dice scores of 83.92 for WT in the binary case and 66.88, 55.16, 64.13 for WT, TC and ET, respectively, in the multi-class case. We introduce the first publicly and freely available NiftyNet-based implementation of the autofocus convolutional layer for semantic image segmentation.

Keywords

Deep learning 3D CNN MRI Brain tumour segmentation Autofocus layer Glioma 

Notes

Acknowledgement

We would like to thank NVIDIA corporation, that donated a GPU to our group enabling this research. We would also like to thank the Engineering and Physical Sciences Research Council Impact Acceleration Impact Acceleration Account (EPSRC IAA) for supporting the initial stages of this work.

References

  1. 1.
    Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 170117 (2017).  https://doi.org/10.1038/sdata.2017.117CrossRefGoogle Scholar
  2. 2.
    Blionas, A., Giakoumettis, D., Klonou, A., Neromyliotis, E., Karydakis, P., Themistocleous, M.S.: Paediatric gliomas: diagnosis, molecular biology and management. Ann. Transl. Med. 6(12), 251 (2018).  https://doi.org/10.21037/atm.2018.05.11
  3. 3.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46723-8_49CrossRefGoogle Scholar
  4. 4.
    Davis, M.E.: Glioblastoma: overview of disease and treatment. Clin. J. Oncol. Nurs. (2016).  https://doi.org/10.1188/16.CJON.S1.2-8CrossRefGoogle Scholar
  5. 5.
    Havaei, M., et al.: Brain tumor segmentation with Deep Neural Networks. Med. Image Anal. 35, 18–31 (2017).  https://doi.org/10.1016/j.media.2016.05.004CrossRefGoogle Scholar
  6. 6.
    Isensee, F., et al.: Abstract: nnU-Net: self-adapting framework for U-net-based medical image segmentation. Bildverarbeitung für die Medizin 2019. I, p. 22. Springer, Wiesbaden (2019).  https://doi.org/10.1007/978-3-658-25326-4_7CrossRefGoogle Scholar
  7. 7.
    Kamnitsas, K., et al.: DeepMedic for brain tumor segmentation. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Winzeck, S., Handels, H. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2016. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10154, pp. 138–149. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-55524-9_14
  8. 8.
    Kamnitsas, K., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017).  https://doi.org/10.1016/j.media.2016.10.004CrossRefGoogle Scholar
  9. 9.
    Kingma, D.P., Ba, J.L.: Adam: a method for stochastic gradient descent. In: ICLR: International Conference on Learning Representations (2015)Google Scholar
  10. 10.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nat. Methods 13(1), 35 (2015).  https://doi.org/10.1038/nmeth.3707MathSciNetCrossRefGoogle Scholar
  11. 11.
    Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T.: On the compactness, efficiency, and representation of 3D convolutional networks: brain parcellation as a pretext task. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 348–360. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59050-9_28CrossRefGoogle Scholar
  12. 12.
    Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. (2017).  https://doi.org/10.1016/j.media.2017.07.005CrossRefGoogle Scholar
  13. 13.
    Liu, J., Li, M., Wang, J., Wu, F., Liu, T., Pan, Y.: A survey of MRI-based brain tumor segmentation methods (2014).  https://doi.org/10.1109/TST.2014.6961028
  14. 14.
    Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015).  https://doi.org/10.1109/TMI.2014.2377694CrossRefGoogle Scholar
  15. 15.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. J. Phys. E: Sci. Instrum. 6(9), 925–929 (2016).  https://doi.org/10.1088/0022-3735/6/9/035CrossRefGoogle Scholar
  16. 16.
    National Health Service. https://www.nhs.uk/conditions/brain-tumours/. Accessed 21 Mar 2020
  17. 17.
    Pereira, S., Pinto, A., Alves, V., Silva, C.A.: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med. Imaging 35(5), 1240–1251 (2016).  https://doi.org/10.1109/TMI.2016.2538465CrossRefGoogle Scholar
  18. 18.
    Qin, Y., et al.: Autofocus layer for semantic segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 603–611. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00931-1_69CrossRefGoogle Scholar
  19. 19.
    NiftyNet. http://www.niftynet.io/. Accessed 6 June 2019
  20. 20.
    MICCAI BraTS’18 Challenge. https://www.med.upenn.edu/sbia/brats2018.html. Accessed 6 June 2019
  21. 21.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  22. 22.
    Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017).  https://doi.org/10.1146/annurev-bioeng-071516-044442CrossRefGoogle Scholar
  23. 23.
    Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 178–190. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-75238-9_16CrossRefGoogle Scholar
  24. 24.
    Wang, G., Li, W., Ourselin, S., Vercauteren, T.: Automatic brain tumor segmentation using convolutional neural networks with test-time augmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11384, pp. 61–72. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-11726-9_6CrossRefGoogle Scholar
  25. 25.
    Wirsching, H.G., Galanis, E.: Glioblastoma. In: Handbook of Clinical Neurology (2016).  https://doi.org/10.1016/B978-0-12-802997-8.00023-2. Chapter 23
  26. 26.
    Yan, Y., Guo, Z.W., Zhang, H.B., Wang, N., Xu, Y.: Precision radiotherapy for brain tumors: a 10-year bibliometric analysis. Neural Regeneration Res. (2012).  https://doi.org/10.3969/j.issn.1673-5374CrossRefGoogle Scholar
  27. 27.
    Zikic, D., Ioannou, Y., Brown, M., Criminisi, A.: Segmentation of brain tumor tissues with convolutional neural networks. In: Proceedings MICCAI-BRATS (2014).  https://doi.org/10.1021/ol2032625

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Computer ScienceUniversity of St AndrewsSt AndrewsUK
  2. 2.The Institute of Cancer ResearchSuttonUK

Personalised recommendations