Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Optimal Symmetric Multimodal Templates and Concatenated Random Forests for Supervised Brain Tumor Segmentation (Simplified) with ANTsR


Segmenting and quantifying gliomas from MRI is an important task for diagnosis, planning intervention, and for tracking tumor changes over time. However, this task is complicated by the lack of prior knowledge concerning tumor location, spatial extent, shape, possible displacement of normal tissue, and intensity signature. To accommodate such complications, we introduce a framework for supervised segmentation based on multiple modality intensity, geometry, and asymmetry feature sets. These features drive a supervised whole-brain and tumor segmentation approach based on random forest-derived probabilities. The asymmetry-related features (based on optimal symmetric multimodal templates) demonstrate excellent discriminative properties within this framework. We also gain performance by generating probability maps from random forest models and using these maps for a refining Markov random field regularized probabilistic segmentation. This strategy allows us to interface the supervised learning capabilities of the random forest model with regularized probabilistic segmentation using the recently developed ANTsR package—a comprehensive statistical and visualization interface between the popular Advanced Normalization Tools (ANTs) and the R statistical project. The reported algorithmic framework was the top-performing entry in the MICCAI 2013 Multimodal Brain Tumor Segmentation challenge. The challenge data were widely varying consisting of both high-grade and low-grade glioma tumor four-modality MRI from five different institutions. Average Dice overlap measures for the final algorithmic assessment were 0.87, 0.78, and 0.74 for “complete”, “core”, and “enhanced” tumor components, respectively.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. 1.


  2. 2.



  1. Amit, Y., & Geman, D. (1997). Shape quantization and recognition with randomized trees. Neural Computation, 9, 1545–1588.

  2. Angelini, E. D., Clatz, O., Mandonnet, E., Konukoglu, E., Capelle, L., Duffau, H. (2007). Glioma dynamics and computational models: A review of segmentation, registration, and in silico growth algorithms and their clinical applications. Current Medical Imaging Reviews, 3, 262–276.

  3. Ashburner, J., & Friston, K. (1997). Multimodal image coregistration and partitioning–a unified framework. Neuroimage, 6 (3), 209–217. doi:doi:10.1006/nimg.1997.0290.

  4. Avants, B., Duda, J.T., Kim, J., Zhang, H., Pluta, J., Gee, J.C., Whyte, J. (2008a). Multivariate analysis of structural and diffusion imaging in traumatic brain injury. Academic Radiology, 15 (11), 1360–1375. doi:10.1016/j.acra.2008.07.007.

  5. Avants, B.B., Epstein, C.L., Grossman, M., Gee, J.C. (2008b). Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis, 12 (1), 26–41. doi:10.1016/j.media.2007.06.004.

  6. Avants, B.B., Yushkevich, P., Pluta, J., Minkoff, D., Korczykowski, M., Detre, J., Gee, J.C. (2010). The optimal template effect in hippocampus studies of diseased populations. Neuroimage, 49 (3), 2457–2466. doi:10.1016/j.neuroimage.2009.09.062.

  7. Avants, B.B., Tustison, N.J., Song, G., Cook, P.A., Klein, A., Gee, J.C. (2011a). A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage, 54 (3), 2033–2044. doi:10.1016/j.neuroimage.2010.09.025.

  8. Avants, B.B., Tustison, N.J., Wu, J., Cook, P.A., Gee, J.C. (2011b). An open source multivariate framework for n-tissue segmentation with evaluation on public data. Neuroinformatics, 9 (4), 381–400. doi:10.1007/s12021-011-9109-y.

  9. Bauer, S., Nolte, L.P., Reyes, M. (2011). Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. Medical Image Computing and Computer Assisted Intervention, 14 (Pt 3), 354–361.

  10. Bauer, S., Fejes, T., Slotboom, J., Wiest, R., Nolte, L.P., Reyes, M. (2012). Segmentation of brain tumor images based on integrated hierarchical classification and regularization. In Proceedings of MICCAI-BRATS 2012, (pp. 10–13).

  11. Bauer, S., Wiest, R., Nolte, L.P., Reyes, M. (2013). A survey of MRI-based medical image analysis for brain tumor studies. Physics in Medicine and Biology, 58 (13), R97–R129. doi:10.1088/0031-9155/58/13/R97.

  12. Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123–140.

  13. Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32.

  14. Cha, S. (2005). Update on brain tumor imaging. Current Neurological Neuroscience Report, 5 (3), 169–177.

  15. Childs, H., Geveci, B., Schroeder, W.J., Meredith, J.S., Moreland, K., Sewell, C., Kuhlen, T., Bethel, E.W. (2013). Research challenges for visualization software. IEEE Computer, 46 (5), 34–42.

  16. Criminisi, A., Shotton, J., Konukoglu, E. (2011). Decision forests for classification, regression, density estimation, manifold learning and semi-supervised learning. Tech. rep., Microsoft Resaerch.

  17. Criminisi, A., Robertson, D., Konukoglu, E., Shotton, J., Pathak, S., White, S., Siddiqui, K. (2013). Regression forests for efficient anatomy detection and localization in computed tomography scans. Medical Image Analysis. doi:10.1016/j.media.2013.01.001.

  18. Das, S.R., Avants, B.B., Grossman, M., Gee, J.C. (2009). Registration based cortical thickness measurement. Neuroimage, 45 (3), 867–879. doi:10.1016/j.neuroimage.2008.12.016.

  19. Durst, C.R., Raghavan, P., Shaffrey, M.E., Schiff, D., Lopes, M.B., Sheehan, J.P., Tustison, N.J., Patrie, J.T., Xin, W., Elias, W.J., Liu, K.C., Helm, G.A., Cupino, A., Wintermark, M. (2014). Multimodal MR imaging model to predict tumor infiltration in patients with gliomas. Neuroradiology, 56 (2), 107–115. doi:10.1007/s00234-013-1308-9.

  20. Freund, Y., & Schapire, R. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55, 119–139.

  21. Geremia, E., Clatz, O., Menze, B.H., Konukoglu, E., Criminisi, A., Ayache, N. (2011). Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images. Neuroimage, 57 (2), 378–90. doi:10.1016/j.neuroimage.2011.03.080.

  22. Geremia, E., Menze, B. H., Ayache, N. (2012). Spatial decision forests for glioma segmentation in multi-channel MR images. In: Proceedings of MICCAI-BRATS 2012.

  23. Gray, K.R., Aljabar, P., Heckemann, R.A., Hammers, A., Rueckert, D. (2013). Alzheimer’s Disease Neuroimaging Initiative Random forest-based similarity measures for multi-modal classification of alzheimer’s disease. Neuroimage, 65, 167–175. doi:10.1016/j.neuroimage.2012.09.065.

  24. Greenspan, H.P. (1972). Models for the growth of a solid tumor by diffusion. Studies in Applied Mathematics, 52 (4), 317–340.

  25. Ho, T.K. (1995). Random decision forests. In Proceedings of the 3rd international conference on document analysis and recognition, 1995 (Vol. 1, pp. 278–282). doi:10.1109/ICDAR.1995.598994.

  26. Iglesias, J.E., Liu, C.Y., Thompson, P., Tu, Z. (2010). Agreement-based semi-supervised learning for skull stripping. Medical Image Computing and Computer Assisted Intervention, 13(Pt3), 147– 54.

  27. Ince, D.C., Hatton, L., Graham-Cumming, J. (2012). The case for open computer programs. Nature, 482 (7386), 485–488. doi:10.1038/nature10836.

  28. Kehrer, J., & Hauser, H. (2013). Visualization and visual analysis of multifaceted scientific data: a survey. IEEE Transaction Vis Computer Graph, 19 (3), 495–513. doi:10.1109/TVCG.2012.110.

  29. Landman, B.A., Huang, A.J., Gifford, A., Vikram, D.S., Lim, I.A.L., Farrell, J.A.D., Bogovic, J.A., Hua, J., Chen, M., Jarso, S., Smith, S.A., Joel, S., Mori, S., Pekar, J.J., Barker, P.B., Prince, J.L., van Z.ijl, P.C.M. (2011). Multi-parametric neuroimaging reproducibility: a 3-T resource study. Neuroimage, 54 (4), 2854–2866. doi:10.1016/j.neuroimage.2010.11.047.

  30. Liaw, A., & Wiener, M. (2002). Classification and regression by random forest. R News, 2 (3), 18–22.

  31. Louis, D.N., Ohgaki, H., Wiestler, O.D., Cavenee, W.K., Burger, P.C., Jouvet, A., Scheithauer, B.W., Kleihues, P. (2007). The 2007 who classification of tumours of the central nervous system. Acta Neuropathologica, 114 (2), 97–109. doi:10.1007/s00401-007-0243-4.

  32. Maurer, C.R., Rensheng, Q., Raghavan, V. (2003). A linear time algorithm for computing exact Euclidean distance transforms of binary images in arbitrary dimensions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25 (2), 265–270. doi:10.1109/TPAMI.2003.1177156.

  33. Menze, B., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.A., Arbel, T., Avants, B., Ayache, N., Buendia, P., Collins, L., Cordier, N., Corso, J., Criminisi, A., Das, T., Delingette, H., Demiralp, C., Durst, C., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K., Jena, R., John, N., Konukoglu, E., Lashkari, D., Antonio Mariz, J., Meier, R., Pereira, S., Precup, D., Price, S., Reza, S., Ryan, M., Schwartz, L., Shin, H.C., Shotton, J., Silva, C., Sousa, N., Subbanna, N., Szekely, G., Taylor, T., Thomas, O., Tustison, N., Unal, G., Vasseur, F., Wintermark, M., Hye Ye, D., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Van Leemput, K. (2014). The multimodal brain tumor image segmentation benchmark (brats). http://hal.inria.fr/hal-00935640.

  34. Menze, B.H., Van Leemput, K., Lashkari, D., Weber, M.A., Ayache, N., Golland, P. (2010). A generative model for brain tumor segmentation in multi-modal images. Medical Image Computing Computer Assisted Intervention, 13 (Pt 2), 151–159.

  35. Nyúl, L.G., Udupa, J.K., Zhang, X. (2000). New variants of a method of MRI scale standardization. IEEE Transactions on Medical Imaging, 19 (2), 143–150. doi:10.1109/42.836373.

  36. Padfield, D., & Miller, J. (2008). A label geometry image filter for multiple object measurement. Insight Journal, 1–13. http://hdl.handle.net/1926/1493.

  37. Prastawa, M., Bullitt, E., Moon, N., Van Leemput, K., Gerig, G. (2003). Automatic brain tumor segmentation by subject specific modification of atlas priors. Academic Radiology, 10 (12), 1341–1348.

  38. Price, S.J., Burnet, N.G., Donovan, T., Green, H.A.L., Pena, A., Antoun, N.M., Pickard, J.D., Carpenter, T.A., Gillard, J.H. (2003). Diffusion tensor imaging of brain tumours at 3T: a potential tool for assessing white matter tract invasion? Clinical Radiology, 58 (6), 455–462.

  39. Prima, S., Ourselin, S., Ayache, N. (2002). Computation of the mid-sagittal plane in 3-D brain images. IEEE Transactions on Medical Imaging, 21 (2), 122–138. doi:10.1109/42.993131.

  40. Reynolds, D.A. (2009). Gaussian mixture modeling. In S.Z. Li & A.K. Jain (Eds), Encyclopedia of biometrics (pp. 659-663). USA: Springer.

  41. Schapire, R. (1990). The strength of weak learnability. Machine Learning, 5, 197–227.

  42. Tustison, N.J., & Avants, B.B. (2013). Explicit B-spline regularization in diffeomorphic image registration. Frontiers in Neuroinformatics, 7, 39. doi:10.3389/fninf.2013.00039.

  43. Tustison, N.J., & Gee, J.C. (2009). Introducing Dice, Jaccard, and other label overlap measures to ITK, Insight Journal, 1–4. http://hdl.handle.net/10380/3141.

  44. Tustison, N.J., Avants, B.B., Cook, P.A., Zheng, Y., Egan, A., Yushkevich, P.A., Gee, J.C. (2010). N4ITK: improved N3 bias correction. IEEE Transactions on Medical Imaging, 29 (6), 1310–1320. doi:10.1109/TMI.2010.2046908.

  45. Tustison, N.J., Johnson, H.J., Rohlfing, T., Klein, A., Ghosh, S.S., Ibanez, L., Avants, B.B. (2013). Instrumentation bias in the use and evaluation of scientific software: recommendations for reproducible practices in the computational sciences. Frontiers in Neuroscience, 7, 162. doi:10.3389/fnins.2013.00162.

  46. VanHorn, J.D., & Toga, A.W. (2013). Human neuroimaging as a “big data” science. Brain Imaging Behav. doi:10.1007/s11682-013-9255-y.

  47. Verhoek, M., Yaqub, M., McManigle, J., Noble, J.A. (2011). Learning optical flow propagation strategies using random forests for fast segmentation, in dynamic 2D and 3D echocardiographys. In K. Suzuki, F. Wang, D. Shen, P. Yan (Eds.), Machine learning in medical imaging, lecture notes in computer science (Vol. 7009, pp. 75-82). Berlin Heidelberg: Springer.

  48. Viola, P., Jones, M., Snow, D. (2005). Detecting pedestrians using patterns of motion and appearance. International Journal of Computer Vision, 63, 153–161.

  49. Wilkinson, G.N., & Rogers, C.E. (1973). Symbolic description of factorial models for analysis of variance. Journal of the Royal Statistical Society Series C (Applied Statistics), 22 (3), 392– 399.

  50. Yi, Z., Criminisi, A., Shotton, J., Blake, A. (2009). Discriminative, semantic segmentation of brain tissue in MR images. Medical Image Computing Computed Assisted Intervention, 12 (Pt 2), 558–565.

  51. Zikic, D., Glocker, B., Konukoglu, E., Shotton, J., Criminisi, A., Ye, D.H., Demiralp, C., Thomas, O. M., Das, T., Jena, R., Price, S.J. (2012). Context-sensitive classification forests for segmentation of brain tumor tissues. In: Proceedings of MICCAI-BRATS 2012, (pp. 1–9).

Download references

Author information

Correspondence to Nicholas J. Tustison.

Additional information


The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tustison, N.J., Shrinidhi, K.L., Wintermark, M. et al. Optimal Symmetric Multimodal Templates and Concatenated Random Forests for Supervised Brain Tumor Segmentation (Simplified) with ANTsR . Neuroinform 13, 209–225 (2015). https://doi.org/10.1007/s12021-014-9245-2

Download citation


  • Advanced normalization tools
  • Glioma
  • Brain tumor segmentation
  • R project for statistical computing