Skip to main content

Multiple sclerosis lesion segmentation from brain MRI using U-Net based on wavelet pooling

International Journal of Computer Assisted Radiology and Surgery volume 16, pages 1459–1467 (2021)Cite this article

Abstract

Purpose

The purpose of this work is to segment multiple sclerosis (MS) lesions in magnetic resonance imaging (MRI) images, in which lesions in different sizes are segmented with appropriate accuracy. Automated segmentation as a powerful tool can assist professionals to increase the accuracy of disease diagnosis and its level of progression.

Methods

We present a deep neural network based on the U-Net architecture in which wavelet transform-based pooling replaces max pooling. In the first part of the network, the wavelet transform is used, and in the second part, it’s inverse. In addition to decomposing the input image and reducing its size, the wavelet transform highlights sharp changes in the image and better describes local features. This transform has the multi-resolution characteristic, so its use provides improvement in the detection of lesions of different sizes and segmentation.

Results

The results of this study show that the proposed method has a better Dice similarity coefficient (DSC) value compared to the max pooling and average pooling methods.

Conclusion

The proposed method has better results for segmenting MS lesions of different sizes in MRI images than the max and average pooling methods and other methods studied.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Polman CH, Reingold SC, Banwell B, Clanet M, Cohen JA, Filippi M, Fujihara K, Havrdova E, Hutchinson M, Kappos L, Lublin FD, Montalban X, O’Connor P, Sandberg-Wollheim M, Thompson AJ, Waubant E, Weinshenker B, Wolinsky JS (2011) Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol 69(2):292–302. https://doi.org/10.1002/ana.22366

    Article  PubMed  PubMed Central  Google Scholar 

  2. Reich DS, Lucchinetti CF, Calabresi PA (2018) Multiple Sclerosis. N Engl J Med 378(2):169–180. https://doi.org/10.1056/NEJMra1401483

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  3. Sommer NN, Saam T, Coppenrath E, Kooijman H, Kumpfel T, Patzig M, Beyer SE, Sommer WH, Reiser MF, Ertl-Wagner B, Treitl KM (2018) Multiple Sclerosis: improved detection of active cerebral lesions With 3-dimensional T1 black-blood Magnetic Resonance Imaging compared with Conventional 3-Dimensional T1 GRE imaging. Invest Radiol 53(1):13–19. https://doi.org/10.1097/rli.0000000000000410

    Article  PubMed  Google Scholar 

  4. Thompson AJ, Banwell BL, Barkhof F, Carroll WM, Coetzee T, Comi G, Correale J, Fazekas F, Filippi M, Freedman MS, Fujihara K, Galetta SL, Hartung HP, Kappos L, Lublin FD, Marrie RA, Miller AE, Miller DH, Montalban X, Mowry EM, Sorensen PS, Tintore M, Traboulsee AL, Trojano M, Uitdehaag BMJ, Vukusic S, Waubant E, Weinshenker BG, Reingold SC, Cohen JA (2018) Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol 17(2):162–173. https://doi.org/10.1016/s1474-4422(17)30470-2

    Article  PubMed  Google Scholar 

  5. Filippi M, Rocca MA, Ciccarelli O, De Stefano N, Evangelou N, Kappos L, Rovira A, Sastre-Garriga J, Tintore M, Frederiksen JL, Gasperini C, Palace J, Reich DS, Banwell B, Montalban X, Barkhof F (2016) MRI criteria for the diagnosis of multiple sclerosis: MAGNIMS consensus guidelines. Lancet Neurol 15(3):292–303. https://doi.org/10.1016/s1474-4422(15)00393-2

    Article  PubMed  PubMed Central  Google Scholar 

  6. Zivadinov R, Zorzon M, De Masi R, Nasuelli D, Cazzato G (2008) Effect of intravenous methylprednisolone on the number, size and confluence of plaques in relapsing–remitting multiple sclerosis. J Neurol Sci 267(1):28–35. https://doi.org/10.1016/j.jns.2007.09.025

    CAS  Article  PubMed  Google Scholar 

  7. Jain S, Sima DM, Ribbens A, Cambron M, Maertens A, Van Hecke W, De Mey J, Barkhof F, Steenwijk MD, Daams M, Maes F, Van Huffel S, Vrenken H, Smeets D (2015) Automatic segmentation and volumetry of multiple sclerosis brain lesions from MR images. NeuroImage Clin 8:367–375. https://doi.org/10.1016/j.nicl.2015.05.003

    Article  PubMed  PubMed Central  Google Scholar 

  8. Bullmore E, Fadili J, Maxim V, Sendur L, Whitcher B, Suckling J, Brammer M, Breakspear M (2004) Wavelets and functional Magnetic Resonance Imaging of the human brain. Neuroimage 23(Suppl 1):S234-249. https://doi.org/10.1016/j.neuroimage.2004.07.012

    Article  PubMed  Google Scholar 

  9. Selesnick IW, Baraniuk RG, Kingsbury NC (2005) The dual-tree complex wavelet transform. IEEE Signal Process Mag 22(6):123–151. https://doi.org/10.1109/MSP.2005.1550194

    Article  Google Scholar 

  10. Brosch T, Tang LYW, Yoo Y, Li DKB, Traboulsee A, Tam R (2016) Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to Multiple Sclerosis lesion segmentation. IEEE Trans Med Imaging 35(5):1229–1239. https://doi.org/10.1109/TMI.2016.2528821

    Article  PubMed  Google Scholar 

  11. Birenbaum A, Greenspan H (2017) Multi-view longitudinal CNN for Multiple Sclerosis lesion segmentation. Eng Appl Artif Intell 65:111–118. https://doi.org/10.1016/j.engappai.2017.06.006

    Article  Google Scholar 

  12. Valverde S, Cabezas M, Roura E, González-Villà S, Pareto D, Vilanova J-C, Ramió-Torrentà L, Rovira À, Oliver A, Lladó X (2017) Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach. arXiv e-prints: arXiv:1702.04869

  13. Kazancli E, Prčkovska V, Rodrigues P, Villoslada P, Igual L (2018) Multiple Sclerosis lesion segmentation using improved convolutional neural networks. In: Proceedings of 13th International Joint Conference on Computer Vision Imaging and Computer Graphics Theory and Applications (VISIGRAPP:4 ), pp 260–269. doi: https://doi.org/https://doi.org/10.5220/0006540902600269

  14. Raein Hashemi S, Sadegh Mohseni Salehi S, Erdogmus D, Prabhu SP, Warfield SK, Gholipour A (2018) Asymmetric loss functions and deep densely connected networks for highly imbalanced medical image segmentation: application to Multiple Sclerosis lesion detection. arXiv e-prints: arXiv:1803.11078

  15. Aslani S, Dayan M, Storelli L, Filippi M, Murino V, Rocca MA, Sona D (2019) Multi-branch convolutional neural network for Multiple Sclerosis lesion segmentation. Neuroimage 196:1–15. https://doi.org/10.1016/j.neuroimage.2019.03.068

    Article  PubMed  Google Scholar 

  16. Kumar A, Murthy ON, Shrish, Ghosal P, Mukherjee A, Nandi D (2019) A dense U-Net architecture for Multiple Sclerosis lesion segmentation. In: TENCON 2019—2019 IEEE Region 10 Conference (TENCON), 17–20 2019. pp 662–667. doi:https://doi.org/https://doi.org/10.1109/TENCON.2019.8929615

  17. Ghosal P, Prasad PKC, Nandi D (2019) A light weighted deep learning framework for Multiple Sclerosis lesion segmentation. In: 2019 Fifth International Conference on Image Information Processing (ICIIP), 15–17 2019. pp 526–531. doi: https://doi.org/https://doi.org/10.1109/ICIIP47207.2019.8985674

  18. Williams T, Li R Advanced image classification using wavelets and convolutional neural networks. In: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), 18–20 Dec. 2016. pp 233–239. doi: https://doi.org/10.1109/ICMLA.2016.0046

  19. Rossetto A, Zhou W (2019) Improving classification with CNNs using wavelet pooling with nesterov-accelerated adam. In: Eulenstein O (ed) Proceedings of 11th International Conference on Bioinformatics and Computational Biology. EasyChair, pp 84–93. doi: https://doi.org/https://doi.org/10.29007/9c5j

  20. McKinley R, Wepfer R, Gundersen T, Wagner F, Chan A, Wiest R, Reyes M (2016) Nabla-net: a deep dag-like convolutional architecture for biomedical image segmentation. Image segmentation. 119–128. doi: https://doi.org/https://doi.org/10.1007/978-3-319-55524-9_12

  21. Beaumont J, Commowick O, Barillot C (2016) Multiple Sclerosis lesion segmentation using an automated multimodal Graph Cut. Med Image Comput Comput Assist Interv 12(2):584–591. https://doi.org/10.1007/978-3-642-04271-3_7

    Article  Google Scholar 

  22. Knight J, Khademi A (2016) MS Lesion Segmentation Using FLAIR MRI Only. Paper presented at the 19th International Conference on medical Image Computing & Computer Assisted Intervention, Greece, 21–28 (2016)

  23. F Vera-Olmos, H Melero, Malpica N (2016) Random forest for multiple sclerosis lesion segmentation. Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure-MICCAI-MSSEG: 81–86

  24. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical image computing and computer-assisted intervention—MICCAI 2015, Springer, Cham, pp 234–241. doi: doi.org/https://doi.org/10.1007/978-3-319-24574-4_28

  25. , Zeiler MD, Ranzato MA, Monga R, Mao MZ, Yang K, Le QV, Nguyen P, Senior AW, Vanhoucke V, Hinton GE (2013) On rectified linear units for speech processing. IEEE International Conference on Acoustics, Speech and Signal Processing: 3517–3521

  26. Zeiler MD, Fergus R (2013) Stochastic pooling for regularization of deep convolutional neural networks. arXiv e-prints: arXiv:1301.3557

  27. Boussion N, Hatt M, Lamare F, Bizais Y, Turzo A, Rest CC-L, Visvikis D (2006) A multiresolution image based approach for correction of partial volume effects in emission tomography. Phys Med Biol 51(7):1857–1876. https://doi.org/10.1088/0031-9155/51/7/016

    CAS  Article  PubMed  Google Scholar 

  28. Jingjing S, Ming Y, Bugao X, Bel P (2011) Fabric wrinkle characterization and classification using modified wavelet coefficients and support-vector-machine classifiers. Text Res J 81(9):902–913. https://doi.org/10.1177/0040517510391702

    CAS  Article  Google Scholar 

  29. Mallat S (2008) A wavelet tour of signal processing, 3rd edn. Academic Press Inc, The Sparse Way

    Google Scholar 

  30. Commowick O, Cervenansky F, Ameli R (2016) MSSEG challenge proceedings: Multiple Sclerosis lesions segmentation challenge using a data management and processing infrastructure. In: MICCAI 2016

  31. MikpBajczyk A, Grochowski M (2018) Data augmentation for improving deep learning in image classification problem. 2018 International Interdisciplinary Ph.D Workshop (IIPhDW):117–122

  32. Abadi AA M, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow IJ, Harp A, Irving G, Isard M, Jia Y, Józefowicz R, Kaiser L, Kudlur M, Levenberg J, Mane D, Monga R, Moore S, Murray DG, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker PA, Vanhoucke V, Vasudevan V, Viégas FB, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2016) TensorFlow: a system for large-scale machine learning. Paper presented at the 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), Savannah, GA

Download references

Author information

Affiliations

  1. Department of Computer Engineering, Rasht Branch, Islamic Azad University, Rasht, Iran

    Ali Alijamaat & Peyman Bayat

  2. Department of Computer Engineering, Karaj Branch, Islamic Azad University, Karaj, Iran

    Alireza NikravanShalmani

Authors
  1. Ali Alijamaat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alireza NikravanShalmani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peyman Bayat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alireza NikravanShalmani.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Alijamaat, A., NikravanShalmani, A. & Bayat, P. Multiple sclerosis lesion segmentation from brain MRI using U-Net based on wavelet pooling. Int J CARS 16, 1459–1467 (2021). https://doi.org/10.1007/s11548-021-02327-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02327-y

Keywords

  • Deep learning
  • Multiple sclerosis
  • Magnetic resonance imaging
  • U-Net neural network
  • Wavelet