Why Does Synthesized Data Improve Multi-sequence Classification?

  • Gijs van Tulder
  • Marleen de Bruijne
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9349)


The classification and registration of incomplete multi-modal medical images, such as multi-sequence MRI with missing sequences, can sometimes be improved by replacing the missing modalities with synthetic data. This may seem counter-intuitive: synthetic data is derived from data that is already available, so it does not add new information. Why can it still improve performance? In this paper we discuss possible explanations. If the synthesis model is more flexible than the classifier, the synthesis model can provide features that the classifier could not have extracted from the original data. In addition, using synthetic information to complete incomplete samples increases the size of the training set.

We present experiments with two classifiers, linear support vector machines (SVMs) and random forests, together with two synthesis methods that can replace missing data in an image classification problem: neural networks and restricted Boltzmann machines (RBMs). We used data from the BRATS 2013 brain tumor segmentation challenge, which includes multi-modal MRI scans with T1, T1 post-contrast, T2 and FLAIR sequences. The linear SVMs appear to benefit from the complex transformations offered by the synthesis models, whereas the random forests mostly benefit from having more training data. Training on the hidden representation from the RBM brought the accuracy of the linear SVMs close to that of random forests.


Hide Layer Random Forest Synthetic Data Hide Node Unlabeled Data 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Little, R.J.A., Rubin, D.B.: Statistical analysis with missing data, 2nd edn. Wiley, New York (2002)zbMATHGoogle Scholar
  2. 2.
    Fischl, B., Salat, D.H., van der Kouwe, A.J.W., Makris, N., Ségonne, F., Quinn, B.T., Dale, A.M.: Sequence-independent segmentation of magnetic resonance images. NeuroImage 23, S69–S84 (2004)Google Scholar
  3. 3.
    Johansson, A., Karlsson, M., Nyholm, T.: CT substitute derived from MRI sequences with ultrashort echo time. Medical Physics 38(5) (2011)Google Scholar
  4. 4.
    Johansson, A., Garpebring, A., Asklund, T., Nyholm, T.: CT substituties derived from MR images reconstructed with parallel imaging. Medical Physics 41 (2014)Google Scholar
  5. 5.
    Eilertsen, K., Vestad, L.N.T.A., Geier, O., Skretting, A.: A simulation of MRI based dose calculations on the basis of radiotherapy planning CT images. Acta Oncologica 47(7), 1294–1302 (2008)CrossRefGoogle Scholar
  6. 6.
    Kapanen, M., Tenhunen, M.: T1/T2*-weighted MRI provides clinically relevant pseudo-CT density data for the pelvic bones in MRI-only based radiotherapy treatment planning. Acta Oncologica (Stockholm, Sweden) 52(3), 612–618 (2013)CrossRefGoogle Scholar
  7. 7.
    Larsson, A., Johansson, A., Axelsson, J., Nyholm, T., Asklund, T., Riklund, K., Karlsson, M.: Evaluation of an attenuation correction method for PET/MR imaging of the head based on substitute CT images. Magnetic Resonance Materials in Physics, Biology and Medicine 26(1), 127–136 (2013)CrossRefGoogle Scholar
  8. 8.
    Hofmann, M., Steinke, F., Scheel, V., Charpiat, G., Farquhar, J., Aschoff, P., Brady, M., Schölkopf, B., Pichler, B.J.: MRI-based attenuation correction for PET/MRI: a novel approach combining pattern recognition and atlas registration. Journal of Nuclear Medicine 49(11), 1875–1883 (2008)CrossRefGoogle Scholar
  9. 9.
    Hofmann, M., Pichler, B., Schölkopf, B., Beyer, T.: Towards quantitative PET/MRI: a review of MR-based attenuation correction techniques. European Journal of Nuclear Medicine and Molecular Imaging 36(suppl. 1), March 2009Google Scholar
  10. 10.
    Iglesias, J.E., Konukoglu, E., Zikic, D., Glocker, B., Van Leemput, K., Fischl, B.: Is synthesizing MRI contrast useful for inter-modality analysis? In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 631–638. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  11. 11.
    Roy, S., Carass, A., Prince, J.: A compressed sensing approach for MR tissue contrast synthesis. In: Székely, G., Hahn, H.K. (eds.) IPMI 2011. LNCS, vol. 6801, pp. 371–383. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  12. 12.
    Li, R., Zhang, W., Suk, H.-I., Wang, L., Li, J., Shen, D., Ji, S.: Deep learning based imaging data completion for improved brain disease diagnosis. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part III. LNCS, vol. 8675, pp. 305–312. Springer, Heidelberg (2014)Google Scholar
  13. 13.
    Menze, B.H., Jakab, A., Bauer, S., et al.: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Transactions on Medical Imaging (2014)Google Scholar
  14. 14.
    Bengio, Y., Courville, A., Vincent, P.: Representation Learning: A Review and New Perspectives. Technical report, Université de Montréal (2012)Google Scholar
  15. 15.
    Hinton, G.E.: A Practical Guide to Training Restricted Boltzmann Machines. Technical report, University of Toronto (2010)Google Scholar
  16. 16.
    Tieleman, T.: Training restricted Boltzmann machines using approximations to the likelihood gradient. In: ICML (2008)Google Scholar
  17. 17.
    Bergstra, J., et al.: Theano: A CPU and GPU Math Compiler in Python. In: Proceedings of the Python for Scientific Computing Conference, SciPy (2010)Google Scholar
  18. 18.
    Pedregosa, F., et al.: Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12, 2825–2830 (2011)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Gijs van Tulder
    • 1
  • Marleen de Bruijne
    • 1
    • 2
  1. 1.Biomedical Imaging Group RotterdamErasmus MC University Medical CenterRotterdamThe Netherlands
  2. 2.Department of Computer ScienceUniversity of CopenhagenCopenhagenDenmark

Personalised recommendations