Skip to main content

Multiple Sclerosis Lesion Filling Using a Non-lesion Attention Based Convolutional Network

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12532))

Included in the following conference series:

Abstract

Multiple sclerosis (MS) is an inflammatory demyelinating disease of the central nervous system (CNS) that results in focal injury to the grey and white matter. The presence of white matter lesions biases morphometric analyses such as registration, individual longitudinal measurements and tissue segmentation for brain volume measurements. Lesion-inpainting with intensities derived from surrounding healthy tissue represents one approach to alleviate such problems. However, existing methods fill lesions based on texture information derived from local surrounding tissue, often leading to inconsistent inpainting and the generation of artifacts such as intensity discrepancy and blurriness. Based on these observations, we propose a non-lesion attention network (NLAN) that integrates an elaborately designed network with non-lesion attention modules and a designed loss function. The non-lesion attention module is exploited to capture long range dependencies between the lesion area and remaining normal-appearing brain regions, and also eliminates the impact of other lesions on local lesion filling. Meanwhile, the designed loss function ensures that high-quality output can be generated. As a result, this method generates inpainted regions that appear more realistic; more importantly, quantitative morphometric analyses incorporating our NLAN demonstrate superiority of this technique of existing state-of-the-art lesion filling methods.

H. Xiong and C. Wang—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://brain-development.org/ixi-dataset/.

References

  1. Battaglini, M., Jenkinson, M., Stefano, N.D.: Evaluating and reducing the impact of white matter lesions on brain volume measurements. Hum. Brain Mapp. 33, 2062–2071 (2014)

    Article  Google Scholar 

  2. Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Computer Vision and Pattern Recognition, pp. 60–65 (2005)

    Google Scholar 

  3. Chard, D., Jackson, J., Miller, D., Wheeler-Kingshott, C.: Reducing the impact of white matter lesions on automated measures of brain gray and white matter volumes. J. Magn. Reson. Imaging. 32, 223–228 (2010). https://doi.org/10.1002/jmri.22214

    Article  Google Scholar 

  4. Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process 13, 1200–1212 (2004)

    Article  Google Scholar 

  5. Diez, Y., et al.: Intensity based methods for brain MRI longitudinal registration. A study on multiple sclerosis patients. Neuroinformatics 12, 365–379 (2014)

    Article  Google Scholar 

  6. Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: International Conference on Computer Vision (1999)

    Google Scholar 

  7. Ganiler, O., et al.: A subtraction pipeline for automatic detection of new appearing multiple sclerosis lesions in longitudinal studies. Neuroradiology 56(5), 363–374 (2014). https://doi.org/10.1007/s00234-014-1343-1

    Article  Google Scholar 

  8. Gelineau-Morel, R., Tomassini, V., Jenkinson, M., Johansen-Berg, H., Matthews, P., Palace, J.: The effect of hypointense white matter lesions on automated gray matter segmentation in multiple sclerosis. Hum. Brain Mapp. 33, 2802–2814 (2012)

    Article  Google Scholar 

  9. Guizard, N., Nakamura, K., Coupe, P., Arnold, D., Collins, D.: Non-local MS MRI lesion inpainting method for image processing. In: The endMS Conference (2013)

    Google Scholar 

  10. Guizard, N., et al.: Non-local means inpainting of MS lesions in longitudinal image processing. Front. Neurosci. 456 (2015)

    Google Scholar 

  11. Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. (TOG) 36(4), 107 (2017)

    Article  Google Scholar 

  12. Jenkinson, M., Beckmann, C., Behrens, T., Woolrich, M., Smith, S.: FSL. NeuroImage 62, 782–790 (2012)

    Article  Google Scholar 

  13. Li, Y., Liu, S., Yang, J., Yang, M.H.: Generative face completion. arXiv preprint arXiv:1704.05838 (2017)

  14. Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. arXiv preprint arXiv:1804.07723 (2018)

  15. Magon, S., et al.: White matter lesion filling improves the accuracy of cortical thickness measurements in multiple sclerosis patients: a longitudinal study. BMC Neurosci. 106 (2014)

    Google Scholar 

  16. Nakamura, K., Fisher, E.: Segmentation of brain magnetic resonance images for measurement of gray matter atrophy in multiple sclerosis patients. Neuroimage 44, 769–776 (2009)

    Article  Google Scholar 

  17. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536–2544 (2016)

    Google Scholar 

  18. Prados, F., et al.: A multi-time-point modality-agnostic patch-based method for lesion filling in multiple sclerosis. Neuroimage 139, 376–384 (2016)

    Article  Google Scholar 

  19. Sdika, M., Pelletier, D.: Nonrigid registration of multiple sclerosis brain images using lesion inpainting for morphometry or lesion mapping. Hum. Brain Mapp. 30, 1060–1067 (2008)

    Article  Google Scholar 

  20. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  21. Smith, S., et al.: Accurate, robust and automated longitudinal and cross-sectional brain change analysis. NeuroImage 17(1), 479–489 (2002)

    Article  Google Scholar 

  22. Valverde, S., Oliver, A., Lladó, X.: A white matter lesion-filling approach to improve brain tissue volume measurements. Neuroimage Clin. 6, 86–92 (2014)

    Article  Google Scholar 

  23. Wong, A., Orchard, J.: A nonlocal-means approach to exemplar-based inpainting. In: International Conference on Image Processing (2008)

    Google Scholar 

  24. Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., Li, H.: High-resolution image inpainting using multi-scale neural patch synthesis. arXiv preprint arXiv:1611.09969 (2016)

  25. Yeh, R., Chen, C., Lim, T.Y., Hasegawa-Johnson, M., Do, M.N.: Semantic image inpainting with perceptual and contextual losses. arXiv preprint arXiv:1607.07539 (2016)

  26. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)

  27. Zijdenbos, A., Forghani, R., Evans, A.: Automatic quantification of MS lesions in 3D MRI brain data sets: validation of INSECT. In: Wells, W.M., Colchester, A., Delp, S. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 439–448. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056229

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hao Xiong .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xiong, H., Wang, C., Barnett, M., Wang, C. (2020). Multiple Sclerosis Lesion Filling Using a Non-lesion Attention Based Convolutional Network. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63830-6_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63829-0

  • Online ISBN: 978-3-030-63830-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics