Transfer Learning for Brain Segmentation: Pre-task Selection and Data Limitations

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1248)


Manual segmentations of anatomical regions in the brain are time consuming and costly to acquire. In a clinical trial setting, this is prohibitive and automated methods are needed for routine application. We propose a deep-learning architecture that automatically delineates sub-cortical regions in the brain (example biomarkers for monitoring the development of Huntington’s disease). Neural networks, despite typically reaching state-of-the-art performance, are sensitive to differing scanner protocols and pre-processing methods. To address this challenge, one can pre-train a model on an existing data set and then fine-tune this model using a small amount of labelled data from the target domain. This work investigates the impact of the pre-training task and the amount of data required via a systematic study. We show that use of just a few samples from the same task (but a different domain) can achieve state-of-the-art performance. Further, this pre-training task utilises automated labels, meaning the pipeline requires very few manually segmented data points. On the other hand, using a different task for pre-training is shown to be less successful. We then conclude, by showing that, whilst fine-tuning is very powerful for a specific data distribution, models developed in this fashion are considerably more fragile when used on completely unseen data.


Brain segmentation Deep learning Transfer learning 



Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database ( As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012).


  1. 1.
    Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). software available from
  2. 2.
    Alex, V., Vaidhya, K., Thirunavukkarasu, S., Kesavadas, C., Krishnamurthi, G.: Semisupervised learning using denoising autoencoders for brain lesion detection and segmentation. J. Med. Imaging 4(4), 041311 (2017)Google Scholar
  3. 3.
    Balafar, M.A., Ramli, A.R., Saripan, M.I., Mashohor, S.: Review of brain MRI image segmentation methods. Artif. Intell. Rev. 33(3), 261–274 (2010). Scholar
  4. 4.
    Bowles, C., et al.: GAN augmentation: augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863 (2018)
  5. 5.
    Brusini, I., Lindberg, O., Muehlboeck, J.S., Smedby, Ö., Westman, E., Wang, C.: Shape information improves the cross-cohort performance of deep learning-based segmentation of the hippocampus. Front. Neurosci. 14, 15 (2020)Google Scholar
  6. 6.
    Cheplygina, V., de Bruijne, M., Pluim, J.P.: Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med. Image Anal. 54, 280–296 (2019)Google Scholar
  7. 7.
    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). Scholar
  8. 8.
    Freeborough, P.A., Fox, N.C.: The boundary shift integral: an accurate and robust measure of cerebral volume changes from registered repeat MRI. IEEE Trans. Med. Imaging 16(5), 623–629 (1997)Google Scholar
  9. 9.
    Georgiou-Karistianis, N., Hannan, A.J., Egan, G.F.: Magnetic resonance imaging as an approach towards identifying neuropathological biomarkers for Huntington’s disease. Brain Res. Rev. 58(1), 209–225 (2008)Google Scholar
  10. 10.
    Ghafoorian, M., et al.: Transfer learning for domain adaptation in MRI: application in brain lesion segmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 516–524. Springer, Cham (2017). Scholar
  11. 11.
    Giorgio, A., De Stefano, N.: Clinical use of brain volumetry. J. Magn. Reson. Imaging 37(1), 1–14 (2013)Google Scholar
  12. 12.
    Henley, S.M., Bates, G.P., Tabrizi, S.J.: Biomarkers for neurodegenerative diseases. Curr. Opin. Neurol. 18(6), 698–705 (2005)Google Scholar
  13. 13.
    Jack Jr., C.R., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging Off. J. Int. Soc. Magn. Reson. Med. 27(4), 685–691 (2008)Google Scholar
  14. 14.
    Johnson, E.B., et al.: Recommendations for the use of automated gray matter segmentation tools: evidence from Huntington’s disease. Front. Neurol. 8, 519 (2017)Google Scholar
  15. 15.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  16. 16.
    Krivov, E., Pisov, M., Belyaev, M.: MRI augmentation via elastic registration for brain lesions segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Menze, B., Reyes, M. (eds.) BrainLes 2017. LNCS, vol. 10670, pp. 369–380. Springer, Cham (2018). Scholar
  17. 17.
    Ledig, C., Schuh, A., Guerrero, R., Heckemann, R.A., Rueckert, D.: Structural brain imaging in Alzheimer’s disease and mild cognitive impairment: biomarker analysis and shared morphometry database. Sci. Rep. 8(1), 1–16 (2018)Google Scholar
  18. 18.
    Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)Google Scholar
  19. 19.
    Milletari, F., et al.: Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound. Comput. Vis. Image Underst. 164, 92–102 (2017)Google Scholar
  20. 20.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  21. 21.
    Roy, A.G., Conjeti, S., Navab, N., Wachinger, C., Initiative, A.D.N., et al.: QuickNAT: a fully convolutional network for quick and accurate segmentation of neuroanatomy. NeuroImage 186, 713–727 (2019)Google Scholar
  22. 22.
    Shin, H.C., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35(5), 1285–1298 (2016)Google Scholar
  23. 23.
    Sled, J.G., Zijdenbos, A.P., Evans, A.C.: A nonparametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Trans. Med. Imaging 17(1), 87–97 (1998)Google Scholar
  24. 24.
    Tajbakhsh, N., et al.: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans. Med. Imaging 35(5), 1299–1312 (2016)Google Scholar
  25. 25.
    Weese, J., Lorenz, C.: Four challenges in medical image analysis from an industrial perspective. Med. Image Anal. 33, 44–49 (2016)Google Scholar
  26. 26.
    Wolz, R., Aljabar, P., Hajnal, J.V., Hammers, A., Rueckert, D., Initiative, A.D.N., et al.: LEAP: learning embeddings for atlas propagation. NeuroImage 49(2), 1316–1325 (2010)Google Scholar
  27. 27.
    Zavala-Romero, O., et al.: Segmentation of prostate and prostate zones using deep learning. Strahlentherapie und Onkologie (2020).
  28. 28.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.IXICO plcLondonUK
  2. 2.Imperial CollegeLondonUK

Personalised recommendations