Skip to main content

Multi-modal PixelNet for Brain Tumor Segmentation

Part of the Lecture Notes in Computer Science book series (LNIP,volume 10670)

Abstract

Brain tumor segmentation using multi-modal MRI data sets is important for diagnosis, surgery and follow up evaluation. In this paper, a convolutional neural network (CNN) with hypercolumns features (e.g. PixelNet) utilizes for automatic brain tumor segmentation containing low and high-grade glioblastomas. Though pixel level convolutional predictors like CNNs, are computationally efficient, such approaches are not statistically efficient during learning precisely because spatial redundancy limits the information learned from neighboring pixels. PixelNet extracts features from multiple layers that correspond to the same pixel and samples a modest number of pixels across a small number of images for each SGD (Stochastic gradient descent) batch update. PixelNet has achieved whole tumor dice accuracy 87.6% and 85.8% for validation and testing data respectively in BraTS 2017 challenge.

Keywords

  • Brain tumor segmentation
  • Gliomas
  • BraTS
  • Deep learning
  • Convolutional neural network
  • Pixel level segmentation
  • Hypercolumn

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Nyul, L.G., Udupa, J.K., Zhang, X.: New variants of a method of MRI scale standardization. IEEE Trans. Med. Imaging 19(2), 143–150 (2000)

    CrossRef  Google Scholar 

  2. Ellwaa, A., et al.: Brain tumor segmantation using random forest trained on iterative selected patients. In: Proceedings of BRATS-MICCAI (2016)

    Google Scholar 

  3. Omuro, A., DeAngelis, L.M.: Glioblastoma and other malignant gliomas: a clinical review. JAMA 310, 1842–1850 (2013)

    CrossRef  Google Scholar 

  4. Bauer, S., et al.: A survey of MRI-based medical image analysis for brain tumor studies. Phys. Med. Biol. 58, R97 (2013)

    CrossRef  Google Scholar 

  5. Inda, M.M., Bonavia, R., Seoane, J.: Glioblastoma multiforme: a look inside its heterogeneous nature. Cancers 6(1), 226–239 (2014)

    CrossRef  Google Scholar 

  6. Baumgartner, C.F., Kamnitsas, K., Matthew, J., Smith, S., Kainz, B., Rueckert, D.: Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 203–211. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_24

    CrossRef  Google Scholar 

  7. Azizi, S., Imani, F., Ghavidel, S., Tahmasebi, A., Kwak, J.T., Xu, S., Turkbey, B., Choyke, P., Pinto, P., Wood, B., Mousavi, P., Abolmaesumi, P.: Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study. Int. J. Comput. Assist. Radiol. Surg. 11(6), 947–956 (2016)

    CrossRef  Google Scholar 

  8. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. arXiv:1505.04597v1 (2015)

  9. Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. arXiv:1606.04797v1 (2016)

  10. Havaei, M., Davy, A., Warde-Farley, D.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)

    CrossRef  Google Scholar 

  11. Kamnitsas, K., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)

    CrossRef  Google Scholar 

  12. Kamnitsas, K., et al: DeepMedic on brain tumor segmentation. In: Proceedings of BRATS-MICCAI (2016)

    Google Scholar 

  13. Pandian1, B., Boyle1, J., Orringer, D. A.: Multimodal tumor segmentation with 3D volumetric convolutional neural networks. In: Proceedings of BRATS-MICCAI (2016)

    Google Scholar 

  14. Casamitjana, A., et al.: 3D convolutional networks for brain tumor segmentation. In: Proceedings of BRATS-MICCAI (2016)

    Google Scholar 

  15. Alex, V., Krishnamurthi, G.: Brain tumor segmentation from multi modal MR images using stacked denoising autoencoders. In: Proceedings of BRATS-MICCAI (2016)

    Google Scholar 

  16. Randhawa, R., Modi, A., Jain, P., Warier, P.: Improving segment boundary classification for brain tumor segmentation and longitudinal disease progression. In: Proceedings of BRATS-MICCAI (2016)

    Google Scholar 

  17. Pereira, S., et al.: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med. Imaging 35, 1240–1251 (2016)

    CrossRef  Google Scholar 

  18. Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) Proceedings of COMPSTAT 2010, pp. 177–186. Springer, Heidelberg (2010)

    Google Scholar 

  19. Hyvärinen, A., Hurri, J., Hoyer, P. O.: Natural Image Statistics: A Probabilistic Approach to Early Computational Vision, vol. 39 (2009). https://doi.org/10.1007/978-1-84882-491-1. (1. Aufl. ed.)

  20. Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a Laplacian pyramid of adversarial networks. In: NIPS (2015)

    Google Scholar 

  21. Hariharan, B., Arbelaez, P., Girshick, R., Malik, J.: Hypercolumns for object segmentation and fine-grained localization. In: CVPR (2015)

    Google Scholar 

  22. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional models for semantic segmentation. In: CVPR (2015)

    Google Scholar 

  23. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A. L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. In: ICLR (2015)

    Google Scholar 

  24. Liu, W., Rabinovich, A., Berg, A. C.: Parsenet: looking wider to see better. arXiv preprint arXiv:1506.04579 (2015)

  25. Bansal, A., et al: PixelNet: Representation of the pixels, by the pixels, and for the pixels. arXiv:1702.06506v1 (2017)

  26. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  27. Menze, B., et al.: The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans. Med. Imaging 34, 1993–2024 (2015)

    CrossRef  Google Scholar 

  28. Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby J. S., Freymann, J. B., Farahani, K., Davatzikos, C.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Nat. Sci. Data (2017, in Press)

    Google Scholar 

  29. Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., Freymann, J., Farahani, K., Davatzikos, C.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. Cancer Imaging Arch. (2017). https://doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q

    Google Scholar 

  30. Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., Freymann, J., Fara-hani, K., Davatzikos, C.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection. Cancer Imaging Arch. (2017). https://doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF

    Google Scholar 

  31. Zhao, X., Wu, Y., Song, G., Li, Z., Fan, Y., Zhang, Y.: Brain tumor segmentation using a fully convolutional neural network with conditional random fields. In: Proceedings of BRATS-MICCAI (2016)

    Google Scholar 

  32. Shen, H., Wang, R., Zhang, J., McKenna, S.J.: Boundary-aware fully convolutional network for brain tumor segmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 433–441. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_49

    CrossRef  Google Scholar 

  33. Chen, H., Qi, X.J., Cheng, J.Z., Heng, P.A.: Deep contextual networks for neuronal structure segmentation. In: AAAI (2016)

    Google Scholar 

  34. Chen, H., Qi, X., Yu, L., Heng, P.A.: DCAN: deep contour-aware networks for accurate gland segmentation. In: CVPR, pp. 2487–2496 (2016)

    Google Scholar 

  35. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. CoRR, vol. abs/1505.04597 (2015)

    Google Scholar 

Download references

Acknowledgments

This work is supported by the Singapore Academic Research Fund under Grant R-397-000-227-112, NUSRI China Jiangsu Provincial Grant BK20150386 & BE2016077 and NMRC Bedside & Bench under grant R-397-000-245-511 awarded to Dr. Hongliang Ren. I would like to thank Aayush Bansal, the author of PixelNet [25], for the assistance to implement PixelNet for this project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mobarakol Islam .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Islam, M., Ren, H. (2018). Multi-modal PixelNet for Brain Tumor Segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2017. Lecture Notes in Computer Science(), vol 10670. Springer, Cham. https://doi.org/10.1007/978-3-319-75238-9_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-75238-9_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-75237-2

  • Online ISBN: 978-3-319-75238-9

  • eBook Packages: Computer ScienceComputer Science (R0)