A Supervoxel Based Random Forest Synthesis Framework for Bidirectional MR/CT Synthesis

  • Can Zhao
  • Aaron Carass
  • Junghoon Lee
  • Amod Jog
  • Jerry L. Prince
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10557)


Synthesizing magnetic resonance (MR) and computed tomography (CT) images (from each other) has important implications for clinical neuroimaging. The MR to CT direction is critical for MRI-based radiotherapy planning and dose computation, whereas the CT to MR direction can provide an economic alternative to real MRI for image processing tasks. Additionally, synthesis in both directions can enhance MR/CT multi-modal image registration. Existing approaches have focused on synthesizing CT from MR. In this paper, we propose a multi-atlas based hybrid method to synthesize T1-weighted MR images from CT and CT images from T1-weighted MR images using a common framework. The task is carried out by: (a) computing a label field based on supervoxels for the subject image using joint label fusion; (b) correcting this result using a random forest classifier (RF-C); (c) spatial smoothing using a Markov random field; (d) synthesizing intensities using a set of RF regressors, one trained for each label. The algorithm is evaluated using a set of six registered CT and MR image pairs of the whole head.


Synthesis MR CT JLF Segmentation Random forest MRF 



This work was supported by NIH/NIBIB grant R01-EB017743.


  1. 1.
    Achanta, R., et al.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  2. 2.
    Bai, W., et al.: Multi-atlas segmentation with augmented features for cardiac MR images. Med. Image Anal. 19(1), 98–109 (2015)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Boykov, Y., et al.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  4. 4.
    Burgos, N., et al.: Attenuation correction synthesis for hybrid PET-MR scanners: application to brain studies. IEEE Trans. Med. Imaging 33(12), 2332–2341 (2014)CrossRefGoogle Scholar
  5. 5.
    Burgos, N., et al.: Robust CT synthesis for radiotherapy planning: application to the head and neck region. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9350, pp. 476–484. Springer, Cham (2015). doi: 10.1007/978-3-319-24571-3_57 CrossRefGoogle Scholar
  6. 6.
    Cao, X., et al.: Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis. Med. Image Anal. 41, 18–31 (2017)CrossRefGoogle Scholar
  7. 7.
    Chen, M., et al.: Cross contrast multi-channel image registration using image synthesis for MR brain images. Med. Image Anal. 36, 2–14 (2017)CrossRefGoogle Scholar
  8. 8.
    Huynh, T., et al.: Estimating CT image from MRI data using structured random forest and auto-context model. IEEE Trans. Med. Imaging 35(1), 174–183 (2016)CrossRefGoogle Scholar
  9. 9.
    Iglesias, J.E., Konukoglu, E., Zikic, D., Glocker, B., Van Leemput, K., Fischl, B.: Is synthesizing MRI contrast useful for inter-modality analysis? In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 631–638. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40811-3_79 CrossRefGoogle Scholar
  10. 10.
    Jog, A., et al.: Random forest regression for magnetic resonance image synthesis. Med. Image Anal. 35, 475–488 (2017)CrossRefGoogle Scholar
  11. 11.
    Komodakis, N.: Image completion using global optimization. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 442–452. IEEE (2006)Google Scholar
  12. 12.
    Lee, J., et al.: Multi-atlas-based CT synthesis from conventional MRI with patch-based refinement for MRI-based radiotherapy planning. In: SPIE Medical Imaging, pp. 101331I–101331I-6. International Society for Optics and Photonics (2017)Google Scholar
  13. 13.
    Pham, D.L., et al.: Current methods in medical image segmentation. Annu. Rev. Biomed. Eng. 2(1), 315–337 (2000)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Riederer, S.J., et al.: Automated MR image synthesis: feasibility studies. Radiology 153(1), 203–206 (1984)CrossRefGoogle Scholar
  15. 15.
    Roy, S., et al.: Magnetic resonance image example-based contrast synthesis. IEEE Trans. Med. Imaging 32(12), 2348–2363 (2013)CrossRefGoogle Scholar
  16. 16.
    Roy, S., et al.: PET attenuation correction using synthetic CT from ultrashort echo-time MR imaging. J. Nucl. Med. 55(12), 2071–2077 (2014)CrossRefGoogle Scholar
  17. 17.
    Wang, H., et al.: Multi-atlas segmentation with joint label fusion. IEEE Trans. Pattern Anal. Mach. Intell. 35(3), 611–623 (2013)CrossRefGoogle Scholar
  18. 18.
    Zhao, C., et al.: Whole brain segmentation and labeling from CT using synthetic MR images. In: 8th International Workshop on Machine Learning in Medical Imaging. LNCS. Springer, Heidelberg (2017)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Can Zhao
    • 1
  • Aaron Carass
    • 1
  • Junghoon Lee
    • 2
  • Amod Jog
    • 1
  • Jerry L. Prince
    • 1
  1. 1.Department of Electrical and Computer EngineeringThe Johns Hopkins UniversityBaltimoreUSA
  2. 2.Department of Radiation OncologyThe Johns Hopkins School of MedicineBaltimoreUSA

Personalised recommendations