Skip to main content
SpringerLink
Log in

Subject-Specific Lesion Generation and Pseudo-Healthy Synthesis for Multiple Sclerosis Brain Images

  • Conference paper
  • First Online:
Simulation and Synthesis in Medical Imaging (SASHIMI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13570))

Included in the following conference series:

Abstract

Understanding the intensity characteristics of brain lesions is key for defining image-based biomarkers in neurological studies and for predicting disease burden and outcome. In this work, we present a novel foreground-based generative method for modelling the local lesion characteristics that can both generate synthetic lesions on healthy images and synthesize subject-specific pseudo-healthy images from pathological images. Furthermore, the proposed method can be used as a data augmentation module to generate synthetic images for training brain image segmentation networks. Experiments on multiple sclerosis (MS) brain images acquired on magnetic resonance imaging (MRI) demonstrate that the proposed method can generate highly realistic pseudo-healthy and pseudo-pathological brain images. Data augmentation using the synthetic images improves the brain image segmentation performance compared to traditional data augmentation methods as well as a recent lesion-aware data augmentation technique, CarveMix. The code will be released at https://github.com/dogabasaran/lesion-synthesis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bakshi, R., Ariyaratana, S., Benedict, R.H.B., Jacobs, L.: Fluid-attenuated inversion recovery magnetic resonance imaging detects cortical and juxtacortical multiple sclerosis lesions. Arch. Neurol. 58(5), 742–748 (2001). https://doi.org/10.1001/archneur.58.5.742

    Article  Google Scholar 

  2. Bissoto, A., Perez, F., Valle, E., Avila, S.: Skin lesion synthesis with generative adversarial networks. In: Stoyanov, D., et al. (eds.) CARE/CLIP/OR 2.0/ISIC -2018. LNCS, vol. 11041, pp. 294–302. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01201-4_32

    Chapter  Google Scholar 

  3. Bowles, C., Qin, C., Guerrero, R., et al.: Brain lesion segmentation through image synthesis and outlier detection. NeuroImage Clin. 16, 643–658 (2017). https://doi.org/10.1016/j.nicl.2017.09.003

    Article  Google Scholar 

  4. Carass, A., Roy, S., Jog, A., et al.: Longitudinal multiple sclerosis lesion segmentation: resource and challenge. Neuroimage 148, 77–102 (2017). https://doi.org/10.1016/j.neuroimage.2016.12.064

    Article  Google Scholar 

  5. Chartsias, A., Joyce, T., Papanastasiou, G., et al.: Disentangled representation learning in cardiac image analysis. Med. Image Anal. 58, 101535 (2019). https://doi.org/10.1016/j.media.2019.101535

    Article  Google Scholar 

  6. Commowick, O., Istace, A., Kain, M., et al.: Objective evaluation of multiple sclerosis lesion segmentation using a data management and processing infrastructure. Sci. Rep. 8(1), 13650 (2018). https://doi.org/10.1038/s41598-018-31911-7

    Article  Google Scholar 

  7. Dalton, C.M., Brex, P.A., Jenkins, R., et al.: Progressive ventricular enlargement in patients with clinically isolated syndromes is associated with the early development of multiple sclerosis. J. Neurol. Neurosurg. Psychiatry 73(2), 141–147 (2002). https://doi.org/10.1136/jnnp.73.2.141

    Article  Google Scholar 

  8. Dalton, C.M., Miszkiel, K.A., O’Connor, P.W., et al.: Ventricular enlargement in MS. Neurology 66(5), 693–698 (2006). https://doi.org/10.1212/01.wnl.0000201183.87175.9f

    Article  Google Scholar 

  9. Ghasemi, N., Razavi, S., Nikzad, E.: Multiple sclerosis: pathogenesis, symptoms, diagnoses and cell-based therapy. Cell J. 19(191), 1–10 (2017). https://doi.org/10.22074/cellj.2016.4867

  10. Guptha, S.H., Holroyd, E., Campbell, G.: Progressive lateral ventricular enlargement as a clue to Alzheimer’s disease. The Lancet 359(9322), 2040 (2002). https://doi.org/10.1016/S0140-6736(02)08806-2

    Article  Google Scholar 

  11. Huang, H., Yu, P.S., Wang, C.: An introduction to image synthesis with generative adversarial nets (2018). https://doi.org/10.48550/ARXIV.1803.04469

  12. Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021). https://doi.org/10.1038/s41592-020-01008-z

    Article  Google Scholar 

  13. Jin, Q., Cui, H., Sun, C., Meng, Z., Su, R.: Free-form tumor synthesis in computed tomography images via richer generative adversarial network. Knowl.-Based Syst. 218, 106753 (2021). https://doi.org/10.1016/j.knosys.2021.106753

    Article  Google Scholar 

  14. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396–4405 (2019). https://doi.org/10.1109/CVPR.2019.00453

  15. Li, Q., Yu, Z., Wang, Y., Zheng, H.: TumorGAN: a multi-modal data augmentation framework for brain tumor segmentation. Sensors 20(15), 4203 (2020). https://doi.org/10.3390/s20154203

    Article  Google Scholar 

  16. Luxenberg, J.S., Haxby, J.V., Creasey, H., Sundaram, M., Rapoport, S.I.: Rate of ventricular enlargement in dementia of the Alzheimer type correlates with rate of neuropsychological deterioration. Neurology 37(7), 1135 (1987). https://doi.org/10.1212/WNL.37.7.1135

    Article  Google Scholar 

  17. Mao, X., Li, Q., Xie, H., et al.: Least squares generative adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2813–2821 (2017). https://doi.org/10.1109/ICCV.2017.304

  18. Reed, S., Akata, Z., Yan, X., et al.: Generative adversarial text to image synthesis. In: Proceedings of the 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research, New York, USA, 20–22 June 2016, vol. 48, pp. 1060–1069. PMLR, New York (2016)

    Google Scholar 

  19. Reinhold, J.C., Carass, A., Prince, J.L.: A structural causal model for MR images of multiple sclerosis. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 782–792. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_75

    Chapter  Google Scholar 

  20. Salem, M., Valverde, S., Cabezas, M., et al.: Multiple sclerosis lesion synthesis in MRI using an encoder-decoder U-NET. IEEE Access 7, 25171–25184 (2019). https://doi.org/10.1109/ACCESS.2019.2900198

    Article  Google Scholar 

  21. Styner, M., Lee, J., Chin, B., et al.: 3D segmentation in the clinic: a grand challenge II: MS lesion segmentation (2008). https://doi.org/10.54294/lmkqvm

  22. Sun, L., Wang, J., Huang, Y., et al.: An adversarial learning approach to medical image synthesis for lesion detection. IEEE J. Biomed. Health Inform. 24(8), 2303–2314 (2020). https://doi.org/10.1109/JBHI.2020.2964016

    Article  Google Scholar 

  23. Tang, H., Xu, D., Sebe, N., Yan, Y.: Attention-guided generative adversarial networks for unsupervised image-to-image translation. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019). https://doi.org/10.1109/IJCNN.2019.8851881

  24. Xia, T., Chartsias, A., Tsaftaris, S.A.: Pseudo-healthy synthesis with pathology disentanglement and adversarial learning. Med. Image Anal. 64, 101719 (2020). https://doi.org/10.1016/j.media.2020.101719

    Article  Google Scholar 

  25. Yun, S., Han, D., Chun, S., et al.: Cutmix: regularization strategy to train strong classifiers with localizable features. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6022–6031 (2019). https://doi.org/10.1109/ICCV.2019.00612

  26. Zeng, C., Gu, L., Liu, Z., Zhao, S.: Review of deep learning approaches for the segmentation of multiple sclerosis lesions on brain MRI. Front. Neuroinform. 14, 610967 (2020). https://doi.org/10.3389/fninf.2020.610967

  27. Zhang, X., et al.: CarveMix: a simple data augmentation method for brain lesion segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 196–205. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_19

    Chapter  Google Scholar 

  28. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017). https://doi.org/10.1109/ICCV.2017.244

Download references

Acknowledgements

This work is supported by the UKRI CDT in AI for Healthcare http://ai4health.io (Grant No. EP/S023283/1). For the purpose of open access, the author has applied a ‘Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.

Author information

Authors and Affiliations

  1. Department of Computing, Imperial College London, London, UK

    Berke Doga Basaran & Wenjia Bai

  2. Data Science Institute, Imperial College London, London, UK

    Berke Doga Basaran, Mengyun Qiao & Wenjia Bai

  3. Department of Brain Sciences, Imperial College London, London, UK

    Paul M. Matthews & Wenjia Bai

  4. UK Dementia Research Institute, Imperial College London, London, UK

    Paul M. Matthews

Authors
  1. Berke Doga Basaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mengyun Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul M. Matthews
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenjia Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Berke Doga Basaran .

Editor information

Editors and Affiliations

  1. NVIDIA, Santa Clara, CA, USA

    Can Zhao

  2. Masaryk University, Brno, Czech Republic

    David Svoboda

  3. University of Twente, Enschede, The Netherlands

    Jelmer M. Wolterink

  4. Universidad de Los Andes, Bogotá, Colombia

    Maria Escobar

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1694 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Basaran, B.D., Qiao, M., Matthews, P.M., Bai, W. (2022). Subject-Specific Lesion Generation and Pseudo-Healthy Synthesis for Multiple Sclerosis Brain Images. In: Zhao, C., Svoboda, D., Wolterink, J.M., Escobar, M. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2022. Lecture Notes in Computer Science, vol 13570. Springer, Cham. https://doi.org/10.1007/978-3-031-16980-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-16980-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16979-3

  • Online ISBN: 978-3-031-16980-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation