Skip to main content
SpringerLink
Log in

Difficulty-Aware Brain Lesion Segmentation from MRI Scans

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The automatic segmentation of lesions from brain MR images is critical in diagnosing and treating diseases of the brain. Compared with laborious and time-consuming manual segmentation, computer-aided segmentation provides more efficient and reliable predictions. Recently, Deep Convolutional Neural Networks were proposed to show state-of-the-art performance. Training a Deep Convolutional Neural Network demands a large amount of labeled data from experts. However, limited by poor spatial resolution, low contrast, etc., identifying lesion boundaries is difficult with ambiguity, such difficult-to-segment regions pose difficulty and variation in training a segmentation model. In this paper, we present a novel teacher–student framework. A teacher model based on Bayesian Neural Network is used to identify these regions and quantify the degree of difficulty, and a dynamically weighted training loss is applied on the student model according to such difficulty. The experimental results on the BRATS 15 dataset and SPES 2015 dataset demonstrate the state-of-the-art performance of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Lo EH, Dalkara T, Moskowitz MA (2003) Mechanisms, challenges and opportunities in stroke. Nat Rev Neurosci 4(5):399–414

    Article  Google Scholar 

  2. Ferlay J, Soerjomataram I, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F (2015) Cancer incidence and mortality worldwide: sources, methods and major patterns in GLOBOCAN 2012. Int J Cancer 136(5):E359–E386

    Article  Google Scholar 

  3. Donnan GA, Fisher M, Macleod M, Davis SM (2008) Stroke. The Lancet 371(9624):1612–1623

    Article  Google Scholar 

  4. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  5. Wadhwa A, Bhardwaj A, Verma VS (2019) A review on brain tumor segmentation of MRI images. Magn Reson Imaging 61:247–259

    Article  Google Scholar 

  6. Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International conference on machine learning, PMLR, pp 1050–1059

  7. Ovadia Y, Fertig E, Ren J, Nado Z, Sculley D, Nowozin S, Dillon J, Lakshminarayanan B, Snoek J (2019) Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In: Advances in neural information processing systems, vol 32. Curran Associates Inc.

  8. Jungo A, McKinley R, Meier R, Knecht U, Vera L, Pérez-Beteta J, Molina-García D, Pérez-García VM, Wiest R, Reyes M (2017) Towards uncertainty-assisted brain tumor segmentation and survival prediction. In: International MICCAI Brainlesion workshop. Springer, pp 474–485

  9. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  10. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Comput Sci

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770–778

  12. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 4700–4708

  13. Ren S, He K, Girshick R, Sun J (2016) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  14. Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision. pp 2980–2988

  15. Tian Z, Shen C, Chen H, He T (2019) Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 9627–9636

  16. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  17. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFS. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  18. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision. pp 2961–2969

  19. Wang X, Zhang R, Kong T, Li L, Shen C (2020) Solov2: dynamic and fast instance segmentation. In: Advances in Neural information processing systems, vol 33. Curran Associates Inc, pp 17721–17732

  20. Yu J, Yao J, Zhang J, Yu Z, Tao D (2020) SPRNet: single-pixel reconstruction for one-stage instance segmentation. IEEE Trans Cybern 51(4):1731–1742

    Article  Google Scholar 

  21. Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35(5):1240–1251

    Article  Google Scholar 

  22. Kendall A, Gal Y (2017) What uncertainties do we need in Bayesian deep learning for computer vision? In: NIPS

  23. McKinley R, Meier R, Wiest R (2018) Ensembles of densely-connected CNNs with label-uncertainty for brain tumor segmentation. In: International MICCAI Brainlesion workshop. Springer, pp 456–465

  24. Zheng H, Chen Y, Yue X, Ma C, Liu X, Yang P, Lu J (2020) Deep pancreas segmentation with uncertain regions of shadowed sets. Magn Reson Imaging 68:45–52

    Article  Google Scholar 

  25. Jena R, Awate SP (2019) A Bayesian neural net to segment images with uncertainty estimates and good calibration. In: International conference on information processing in medical imaging. Springer, pp 3–15

  26. Jang E, Gu S, Poole B (2017) Categorical reparameterization with Gumbel-Softmax. In: ICLR

  27. Reiß S, Seibold C, Freytag A, Rodner E, Stiefelhagen R (2021) Every annotation counts: multi-label deep supervision for medical image segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9532–9542

  28. Yu J, Rui Y, Tao D (2014) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23(5):2019–2032

    Article  MathSciNet  Google Scholar 

  29. Yu J, Tan M, Zhang H, Tao D, Rui Y (2019) Hierarchical deep click feature prediction for fine-grained image recognition. IEEE Trans Pattern Anal Mach Intell

  30. Hinton G, Dean J, Vinyals O (2014) Distilling the knowledge in a neural network. In: NIPS

  31. Li K, Wang S, Yu L, Heng PA (2020) Dual-teacher: integrating intra-domain and inter-domain teachers for annotation-efficient cardiac segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 418–427

  32. Huo X, Xie L, He J, Yang Z, Zhou W, Li H, Tian Q (2021) ATSO: Asynchronous teacher–student optimization for semi-supervised image segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1235–1244

  33. Yu L, Wang S, Li X, Fu CW, Heng PA (2019) Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 605–613

  34. Ibrahim MS, Vahdat A, Ranjbar M, Macready WG (2020) Semi-supervised semantic image segmentation with self-correcting networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12715–12725

  35. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 424–432

  36. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R et al (2014) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024

    Article  Google Scholar 

  37. Maier O, Wilms M, von der Gablentz J, Krämer UM, Münte TF, Handels H (2015) Extra tree forests for sub-acute ischemic stroke lesion segmentation in MR sequences. J Neurosci Methods 240:89–100

    Article  Google Scholar 

  38. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  39. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vision 88(2):303–338

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Science and Engineering, Xiamen University, Xiamen, China

    Jianxiong Wu, Xiaoyu Liu & Yinghao Liao

Authors
  1. Jianxiong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yinghao Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinghao Liao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, J., Liu, X. & Liao, Y. Difficulty-Aware Brain Lesion Segmentation from MRI Scans. Neural Process Lett 54, 1961–1975 (2022). https://doi.org/10.1007/s11063-021-10714-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10714-4

Keywords

Search

Navigation