Skip to main content

Contrast Adaptive Tissue Classification by Alternating Segmentation and Synthesis

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12417)


Deep learning approaches to the segmentation of magnetic resonance images have shown significant promise in automating the quantitative analysis of brain images. However, a continuing challenge has been its sensitivity to the variability of acquisition protocols. Attempting to segment images that have different contrast properties from those within the training data generally leads to significantly reduced performance. Furthermore, heterogeneous data sets cannot be easily evaluated because the quantitative variation due to acquisition differences often dwarfs the variation due to the biological differences that one seeks to measure. In this work, we describe an approach using alternating segmentation and synthesis steps that adapts the contrast properties of the training data to the input image. This allows input images that do not resemble the training data to be more consistently segmented. A notable advantage of this approach is that only a single example of the acquisition protocol is required to adapt to its contrast properties. We demonstrate the efficacy of our approaching using brain images from a set of human subjects scanned with two different T1-weighted volumetric protocols.


  • Segmentation
  • Synthesis
  • Magnetic resonance imaging
  • Harmonization
  • Domain adaptation

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Avants, B.B., Tustison, N.J., Stauffer, M., Song, G., Wu, B., Gee, J.C.: The Insight ToolKit image registration framework. Front. Neuroinform. 8, 44 (2014)

    CrossRef  Google Scholar 

  2. Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.A.: Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Trans. Med. Imaging 39, 2494–2505 (2020)

    CrossRef  Google Scholar 

  3. Chua, A.S., et al.: Handling changes in MRI acquisition parameters in modeling whole brain lesion volume and atrophy data in multiple sclerosis subjects: comparison of linear mixed-effect models. NeuroImage Clin. 8, 606–610 (2015)

    CrossRef  Google Scholar 

  4. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016).

    CrossRef  Google Scholar 

  5. Collins, D.L., et al.: Design and construction of a realistic digital brain phantom. IEEE Trans. Med. Imaging 17(3), 463–468 (1998)

    CrossRef  Google Scholar 

  6. Dewey, B., et al.: DeepHarmony: a deep learning approach to contrast harmonization across scanner changes. Magn. Reson. Imaging 64, 160–170 (2019)

    CrossRef  Google Scholar 

  7. Dou, Q., Castro, D.C., Kamnitsas, K., Glocker, B.: Domain generalization via model-agnostic learning of semantic features (NeurIPS) (2019)

    Google Scholar 

  8. Erus, G., Doshi, J., An, Y., Verganelakis, D., Resnick, S.M., Davatzikos, C.: Longitudinally and inter-site consistent multi-atlas based parcellation of brain anatomy using harmonized atlases. Neuroimage 166, 71–78 (2018)

    CrossRef  Google Scholar 

  9. Fischl, B.: FreeSurfer. Neuroimage 62(2), 774–781 (2012)

    CrossRef  Google Scholar 

  10. Fortin, J.P., Sweeney, E.M., Muschelli, J., Crainiceanu, C.M., Shinohara, R.T.: Removing inter-subject technical variability in magnetic resonance imaging studies. Neuroimage 132, 198–212 (2016)

    CrossRef  Google Scholar 

  11. George, A., Kuzniecky, R., Rusinek, H., Pardoe, H.R.: Standardized brain MRI acquisition protocols improve statistical power in multicenter quantitative morphometry studies. J. Neuroimaging 30(1), 126–133 (2020)

    CrossRef  Google Scholar 

  12. Glocker, B., Robinson, R., Castro, D.C., Dou, Q., Konukoglu, E.: Machine learning with multi-site imaging data: an empirical study on the impact of scanner effects, pp. 1–5 (2019)

    Google Scholar 

  13. Iglesias, J.E., Liu, C.Y., Thompson, P.M., Tu, Z.: Robust brain extraction across datasets and comparison with publicly available methods. IEEE Trans. Med. Imaging 30(9), 1617–1634 (2011)

    CrossRef  Google Scholar 

  14. Jog, A., Carass, A., Roy, S., Pham, D.L., Prince, J.L.: MR image synthesis by contrast learning on neighborhood ensembles. Med. Image Anal. 24(1), 63–76 (2015)

    CrossRef  Google Scholar 

  15. Jog, A., Hoopes, A., Greve, D.N., Van Leemput, K., Fischl, B.: PSACNN: pulse sequence adaptive fast whole brain segmentation. Neuroimage 199, 553–569 (2019)

    CrossRef  Google Scholar 

  16. Kamnitsas, K., et al.: Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. In: Niethammer, M., et al. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 597–609. Springer, Cham (2017).

    CrossRef  Google Scholar 

  17. Nyul, L., Udupa, J., Zhang, X.: New variants of a method of MRI scale standardization. IEEE Trans. Med. Imaging 19(2), 143–150 (2000)

    CrossRef  Google Scholar 

  18. Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S.: Continual lifelong learning with neural networks: a review. Neural Netw. 113, 54–71 (2019)

    CrossRef  Google Scholar 

  19. Pham, D., Roy, S.: Alternating segmentation and simulation for contrast adaptive tissue classification. In: SPIE Medical Imaging, Houston, TX, February 10–15 2018

    Google Scholar 

  20. Puonti, O., Iglesias, J.E., Van Leemput, K.: Fast and sequence-adaptive whole-brain segmentation using parametric Bayesian modeling. Neuroimage 143, 235–249 (2016)

    CrossRef  Google Scholar 

  21. Shah, M., et al.: Evaluating intensity normalization on MRIs of human brain with multiple sclerosis. Med. Image Anal. 15(2), 267–282 (2011)

    CrossRef  Google Scholar 

  22. Shinohara, R., et al.: Volumetric analysis from a harmonized multi-site brain MRI study of a single-subject with multiple sclerosis. Am. J. Neuroradiol. 38, 1501–1509 (2017)

    CrossRef  Google Scholar 

  23. Tustison, N.J., Avants, B.B., Cook, P.A., Zheng, Y., Egan, A., Yushkevich, P.A., Gee, J.C.: N4ITK: improved N3 bias correction. IEEE Trans. Med. Imaging 29(6), 1310–1320 (2010)

    CrossRef  Google Scholar 

Download references


This work was supported by a research grant from the National Multiple Sclerosis Society (RG-1907-34570), by the Department of Defense in the Center for Neuroscience and Regenerative Medicine, the intramural research program of the National Institute of Neurological Disorders and Stroke, and the intramural research program of the Clinical Center in the National Institutes of Health.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Dzung L. Pham .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pham, D.L., Chou, YY., Dewey, B.E., Reich, D.S., Butman, J.A., Roy, S. (2020). Contrast Adaptive Tissue Classification by Alternating Segmentation and Synthesis. In: Burgos, N., Svoboda, D., Wolterink, J.M., Zhao, C. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2020. Lecture Notes in Computer Science(), vol 12417. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59519-7

  • Online ISBN: 978-3-030-59520-3

  • eBook Packages: Computer ScienceComputer Science (R0)