Whole Brain Segmentation and Labeling from CT Using Synthetic MR Images

  • Can ZhaoEmail author
  • Aaron Carass
  • Junghoon Lee
  • Yufan He
  • Jerry L. Prince
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10541)


To achieve whole-brain segmentation—i.e., classifying tissues within and immediately around the brain as gray matter (GM), white matter (WM), and cerebrospinal fluid—magnetic resonance (MR) imaging is nearly always used. However, there are many clinical scenarios where computed tomography (CT) is the only modality that is acquired and yet whole brain segmentation (and labeling) is desired. This is a very challenging task, primarily because CT has poor soft tissue contrast; very few segmentation methods have been reported to date and there are no reports on automatic labeling. This paper presents a whole brain segmentation and labeling method for non-contrast CT images that first uses a fully convolutional network (FCN) to synthesize an MR image from a CT image and then uses the synthetic MR image in a standard pipeline for whole brain segmentation and labeling. The FCN was trained on image patches derived from ten co-registered MR and CT images and the segmentation and labeling method was tested on sixteen CT scans in which co-registered MR images are available for performance evaluation. Results show excellent MR image synthesis from CT images and improved soft tissue segmentation and labeling over a multi-atlas segmentation approach.


Synthesis MR CT Deep learning CNN FCN U-net Segmentation 



This work was supported by NIH/NIBIB under grant R01 EB017743.


  1. 1.
    Burgos, N., Cardoso, M.J., Thielemans, K., Modat, M., Pedemonte, S., Dickson, J., Barnes, A., Ahmed, R., Mahoney, C.J., Schott, J.M., et al.: Attenuation correction synthesis for hybrid PET-MR scanners: application to brain studies. IEEE Trans. Med. Imag. 33(12), 2332–2341 (2014)CrossRefGoogle Scholar
  2. 2.
    Cao, X., Yang, J., Gao, Y., Guo, Y., Wu, G., Shen, D.: Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis. Med. Image Anal. (2017, in press)Google Scholar
  3. 3.
    Chen, M., Carass, A., Jog, A., Lee, J., Roy, S., Prince, J.L.: Cross contrast multi-channel image registration using image synthesis for MR brain images. Med. Image Anal. 36, 2–14 (2017)CrossRefGoogle Scholar
  4. 4.
    Dodge, S., Karam, L.: Understanding how image quality affects deep neural networks. In: 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–6. IEEE (2016)Google Scholar
  5. 5.
    Fischl, B.: Freesurfer. Neuroimage 62(2), 774–781 (2012)CrossRefGoogle Scholar
  6. 6.
    Gupta, V., Ambrosius, W., Qian, G., Blazejewska, A., Kazmierski, R., Urbanik, A., Nowinski, W.L.: Automatic segmentation of cerebrospinal fluid, white and gray matter in unenhanced computed tomography images. Acad. Radiol. 17(11), 1350–1358 (2010)CrossRefGoogle Scholar
  7. 7.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  8. 8.
    Hu, Q., Qian, G., Aziz, A., Nowinski, W.L.: Segmentation of brain from computed tomography head images. In: 27th Annual International Conference of the Engineering in Medicine and Biology Society, IEEE-EMBS 2005, pp. 3375–3378. IEEE (2006)Google Scholar
  9. 9.
    Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017)CrossRefGoogle Scholar
  10. 10.
    Kemmling, A., Wersching, H., Berger, K., Knecht, S., Groden, C., Nölte, I.: Decomposing the hounsfield unit. Clin. Neuroradiol. 22(1), 79–91 (2012)CrossRefGoogle Scholar
  11. 11.
    Li, R., Zhang, W., Suk, H.-I., Wang, L., Li, J., Shen, D., Ji, S.: Deep learning based imaging data completion for improved brain disease diagnosis. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8675, pp. 305–312. Springer, Cham (2014). doi: 10.1007/978-3-319-10443-0_39 Google Scholar
  12. 12.
    Manniesing, R., Oei, M.T., Oostveen, L.J., Melendez, J., Smit, E.J., Platel, B., Sánchez, C.I., Meijer, F.J., Prokop, M., van Ginneken, B.: White matter and gray matter segmentation in 4D computed tomography. Sci. Rep. 7 (2017)Google Scholar
  13. 13.
    Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19(9), 1498–1507 (2007)CrossRefGoogle Scholar
  14. 14.
    Moeskops, P., Viergever, M.A., Mendrik, A.M., de Vries, L.S., Benders, M.J., Išgum, I.: Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans. Med. Imag. 35(5), 1252–1261 (2016)CrossRefGoogle Scholar
  15. 15.
    Ng, C.R., Than, J.C.M., Noor, N.M., Rijal, O.M.: Preliminary brain region segmentation using FCM and graph cut for CT scan images. In: 2015 International Conference on BioSignal Analysis, Processing and Systems (ICBAPS), pp. 52–56. IEEE (2015)Google Scholar
  16. 16.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  17. 17.
    Roy, S., Wang, W.T., Carass, A., Prince, J.L., Butman, J.A., Pham, D.L.: PET attenuation correction using synthetic CT from ultrashort echo-time MR imaging. J. Nuclear Med. 55(12), 2071–2077 (2014)CrossRefGoogle Scholar
  18. 18.
    Wang, H., Suh, J.W., Das, S.R., Pluta, J.B., Craige, C., Yushkevich, P.A.: Multi-atlas segmentation with joint label fusion. IEEE Trans. Patt. Anal. Mach. Intell. 35(3), 611–623 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Can Zhao
    • 1
    Email author
  • Aaron Carass
    • 1
  • Junghoon Lee
    • 2
  • Yufan He
    • 1
  • Jerry L. Prince
    • 1
  1. 1.Department of Electrical and Computer EngineeringThe Johns Hopkins UniversityBaltimoreUSA
  2. 2.Department of Radiation OncologyThe Johns Hopkins School of MedicineBaltimoreUSA

Personalised recommendations