Abstract
Multiple sclerosis (MS) is an inflammatory demyelinating disease of the central nervous system (CNS) that results in focal injury to the grey and white matter. The presence of white matter lesions biases morphometric analyses such as registration, individual longitudinal measurements and tissue segmentation for brain volume measurements. Lesion-inpainting with intensities derived from surrounding healthy tissue represents one approach to alleviate such problems. However, existing methods fill lesions based on texture information derived from local surrounding tissue, often leading to inconsistent inpainting and the generation of artifacts such as intensity discrepancy and blurriness. Based on these observations, we propose a non-lesion attention network (NLAN) that integrates an elaborately designed network with non-lesion attention modules and a designed loss function. The non-lesion attention module is exploited to capture long range dependencies between the lesion area and remaining normal-appearing brain regions, and also eliminates the impact of other lesions on local lesion filling. Meanwhile, the designed loss function ensures that high-quality output can be generated. As a result, this method generates inpainted regions that appear more realistic; more importantly, quantitative morphometric analyses incorporating our NLAN demonstrate superiority of this technique of existing state-of-the-art lesion filling methods.
H. Xiong and C. Wang—Equal contribution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Battaglini, M., Jenkinson, M., Stefano, N.D.: Evaluating and reducing the impact of white matter lesions on brain volume measurements. Hum. Brain Mapp. 33, 2062–2071 (2014)
Buades, A., Coll, B., Morel, J.M.: A non-local algorithm for image denoising. In: Computer Vision and Pattern Recognition, pp. 60–65 (2005)
Chard, D., Jackson, J., Miller, D., Wheeler-Kingshott, C.: Reducing the impact of white matter lesions on automated measures of brain gray and white matter volumes. J. Magn. Reson. Imaging. 32, 223–228 (2010). https://doi.org/10.1002/jmri.22214
Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process 13, 1200–1212 (2004)
Diez, Y., et al.: Intensity based methods for brain MRI longitudinal registration. A study on multiple sclerosis patients. Neuroinformatics 12, 365–379 (2014)
Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: International Conference on Computer Vision (1999)
Ganiler, O., et al.: A subtraction pipeline for automatic detection of new appearing multiple sclerosis lesions in longitudinal studies. Neuroradiology 56(5), 363–374 (2014). https://doi.org/10.1007/s00234-014-1343-1
Gelineau-Morel, R., Tomassini, V., Jenkinson, M., Johansen-Berg, H., Matthews, P., Palace, J.: The effect of hypointense white matter lesions on automated gray matter segmentation in multiple sclerosis. Hum. Brain Mapp. 33, 2802–2814 (2012)
Guizard, N., Nakamura, K., Coupe, P., Arnold, D., Collins, D.: Non-local MS MRI lesion inpainting method for image processing. In: The endMS Conference (2013)
Guizard, N., et al.: Non-local means inpainting of MS lesions in longitudinal image processing. Front. Neurosci. 456 (2015)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. (TOG) 36(4), 107 (2017)
Jenkinson, M., Beckmann, C., Behrens, T., Woolrich, M., Smith, S.: FSL. NeuroImage 62, 782–790 (2012)
Li, Y., Liu, S., Yang, J., Yang, M.H.: Generative face completion. arXiv preprint arXiv:1704.05838 (2017)
Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. arXiv preprint arXiv:1804.07723 (2018)
Magon, S., et al.: White matter lesion filling improves the accuracy of cortical thickness measurements in multiple sclerosis patients: a longitudinal study. BMC Neurosci. 106 (2014)
Nakamura, K., Fisher, E.: Segmentation of brain magnetic resonance images for measurement of gray matter atrophy in multiple sclerosis patients. Neuroimage 44, 769–776 (2009)
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536–2544 (2016)
Prados, F., et al.: A multi-time-point modality-agnostic patch-based method for lesion filling in multiple sclerosis. Neuroimage 139, 376–384 (2016)
Sdika, M., Pelletier, D.: Nonrigid registration of multiple sclerosis brain images using lesion inpainting for morphometry or lesion mapping. Hum. Brain Mapp. 30, 1060–1067 (2008)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Smith, S., et al.: Accurate, robust and automated longitudinal and cross-sectional brain change analysis. NeuroImage 17(1), 479–489 (2002)
Valverde, S., Oliver, A., Lladó, X.: A white matter lesion-filling approach to improve brain tissue volume measurements. Neuroimage Clin. 6, 86–92 (2014)
Wong, A., Orchard, J.: A nonlocal-means approach to exemplar-based inpainting. In: International Conference on Image Processing (2008)
Yang, C., Lu, X., Lin, Z., Shechtman, E., Wang, O., Li, H.: High-resolution image inpainting using multi-scale neural patch synthesis. arXiv preprint arXiv:1611.09969 (2016)
Yeh, R., Chen, C., Lim, T.Y., Hasegawa-Johnson, M., Do, M.N.: Semantic image inpainting with perceptual and contextual losses. arXiv preprint arXiv:1607.07539 (2016)
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)
Zijdenbos, A., Forghani, R., Evans, A.: Automatic quantification of MS lesions in 3D MRI brain data sets: validation of INSECT. In: Wells, W.M., Colchester, A., Delp, S. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 439–448. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0056229
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Xiong, H., Wang, C., Barnett, M., Wang, C. (2020). Multiple Sclerosis Lesion Filling Using a Non-lesion Attention Based Convolutional Network. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-63830-6_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63829-0
Online ISBN: 978-3-030-63830-6
eBook Packages: Computer ScienceComputer Science (R0)