Lesion Detection, Segmentation and Prediction in Multiple Sclerosis Clinical Trials

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10670)


A variety of automatic segmentation techniques have been successfully applied to the delineation of larger T2 lesions in patient MRI in the context of Multiple Sclerosis (MS), assisting in the estimation of lesion volume, a common clinical measure of disease activity and stage. In the context of clinical trials, however, a wider number of metrics are required to determine the “burden of disease” and activity in order to measure treatment efficacy. These include: (1) the number and volume of T2 lesions in MRI, (2) the number of new and enlarging T2 volumes in longitudinal MRI, and (3) the number of gadolinium enhancing lesions in T1 MRI, the portion of lesions that enhance in T1w MRI after injection with a contrast agent, often associated with active inflammations. In this context, accurate lesion detection must ensure that even small lesions (e.g. 3 to 10 voxels) are detected as they are prevalent in trials. Manual or semi-manual approaches are too time-consuming, inconsistent and expensive to be practical in large clinical trials. To this end, we present a series of fully-automatic, probabilistic machine learning frameworks to detect and segment all lesions in patient MRI, and show their accuracy and robustness in large multi-center, multi-scanner, clinical trial datasets. Several of these algorithms have been placed into a commercial software analysis pipeline, where they have assisted in improving the efficiency and precision of the development of most new MS treatments worldwide. Recent work has shown how a new Bag-of-Lesions brain representation can be used in the context of clinical trials to automatically predict the probability of future disease activity and potential treatment responders, leading to the possibility of personalized medicine.


Multiple Sclerosis Clinical Trials Clinical Trial Dataset Measure Treatment Efficacy Future Disease Activity Gadolinium-enhancing Lesions 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by a Canadian Natural Science and Engineering Research Council collaborative Research and Development Grant (CRDPJ 411455-10), and an International Progressive MS Alliance Collaborative Network Award (PA-1603-08175).


  1. 1.
    Bakshi, R., et al.: MRI in Multiple Sclerosis: current status and future prospects. Lancet Neurol. 7(7), 615–625 (2008)CrossRefGoogle Scholar
  2. 2.
    Barkhof, F., et al.: Comparison of MRI criteria at first presentation to predict conversion to clinically definite Multiple Sclerosis. Brain 120(11), 2059–2069 (1997)CrossRefGoogle Scholar
  3. 3.
    Brosch, T., Yoo, Y., Li, D.K.B., Traboulsee, A., Tam, R.: Modeling the variability in brain morphology and lesion distribution in multiple sclerosis by deep learning. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 462–469. Springer, Cham (2014). Google Scholar
  4. 4.
    Cabezas, M., et al.: A review of atlas-based segmentation for magnetic resonance brain images. Comput. Methods Programs Biomed. 104(3), e158–177 (2011)CrossRefGoogle Scholar
  5. 5.
    Díaz-Uriarte, R., De Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC Bioinform. 7(1), 3 (2006)CrossRefGoogle Scholar
  6. 6.
    Doyle, A., Precup, D., Arnold, D.L., Arbel, T.: Predicting future disease activity and treatment responders for multiple sclerosis patients using a bag-of-lesions brain representation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 186–194. Springer, Cham (2017). CrossRefGoogle Scholar
  7. 7.
    Elliott, C.: A Bayesian framework for 4-D segmentation of Multiple Sclerosis lesions in serial MRI in the brain. Ph.D. thesis, McGill University Libraries (2016)Google Scholar
  8. 8.
    Elliott, C., et al.: Temporally consistent probabilistic detection of new Multiple Sclerosis lesions in brain MRI. IEEE TMI 32(8), 1490–1503 (2013)Google Scholar
  9. 9.
    Elliott, C., et al.: A generative model for automatic detection of resolving Multiple Sclerosis lesions. In: BAMBI (2014)Google Scholar
  10. 10.
    Filippi, M., et al.: Association between pathological and MRI findings in Multiple Sclerosis. Lancet Neurol. 11(4), 349–360 (2012)CrossRefGoogle Scholar
  11. 11.
    Fischl, B., et al.: Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 33(3), 341–355 (2002)CrossRefGoogle Scholar
  12. 12.
    Karimaghaloo, Z., et al.: Adaptive multi-level conditional random fields for detection and segmentation of small enhanced pathology in medical images. MIA 27, 17–30 (2016)Google Scholar
  13. 13.
    Karimaghaloo, Z., et al.: Temporal hierarchical adaptive texture CRF for automatic detection of gadolinium-enhancing Multiple Sclerosis lesions in brain MRI. IEEE TMI 34(6), 1227–1241 (2015)Google Scholar
  14. 14.
    Karimaghaloo, Z., Shah, M., Francis, S.J., Arnold, D.L., Collins, D.L., Arbel, T.: Detection of gad-enhancing lesions in multiple sclerosis using conditional random fields. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6363, pp. 41–48. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  15. 15.
    Karimaghaloo, Z., et al.: Automatic detection of gadolinium-enhancing Multiple Sclerosis lesions in brain MRI using conditional random fields. IEEE TMI 31(6), 1181–1194 (2012)Google Scholar
  16. 16.
    Lee, M., et al.: Defining Multiple Sclerosis disease activity using MRI T2-weighted difference imaging. Brain 121(11), 2095–2102 (1998)CrossRefGoogle Scholar
  17. 17.
    Meier, D., et al.: MR imaging intensity modeling of damage and repair in Multiple Sclerosis: relationship of short-term lesion recovery to progression and disability. Am. J. Neuroradiol. 28(10), 1956–1963 (2007)CrossRefGoogle Scholar
  18. 18.
    Milletari, F., et al.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 3D Vision, pp. 565–571. IEEE (2016)Google Scholar
  19. 19.
    Sormani, M.P., et al.: Magnetic resonance active lesions as individual-level surrogate for relapses in Multiple Sclerosis. Mult. Scler. J. 17(5), 541–549 (2011)CrossRefGoogle Scholar
  20. 20.
    Subbanna, N., Precup, D., Arnold, D., Arbel, T.: IMaGe: iterative multilevel probabilistic graphical model for detection and segmentation of Multiple Sclerosis lesions in Brain MRI. In: Ourselin, S., Alexander, D.C., Westin, C.-F., Cardoso, M.J. (eds.) IPMI 2015. LNCS, vol. 9123, pp. 514–526. Springer, Cham (2015). CrossRefGoogle Scholar
  21. 21.
    Subbanna, N.: Iterative Multilevel Probabilistic Graphical Model for Detection and Segmentation of Tumours and Lesions in Brain MRI. Ph.D. thesis, McGill University (2016)Google Scholar
  22. 22.
    Wang, H., et al.: Multi-atlas segmentation with joint label fusion. IEEE TPAMI 35(3), 611–623 (2013)CrossRefGoogle Scholar
  23. 23.
    Warfield, S.K., et al.: Adaptive, template moderated, spatially varying statistical classification. MIA 4(1), 43–55 (2000)Google Scholar
  24. 24.
    Yoo, Y., Tang, L.W., Brosch, T., Li, D.K.B., Metz, L., Traboulsee, A., Tam, R.: Deep learning of brain lesion patterns for predicting future disease activity in patients with early symptoms of multiple sclerosis. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 86–94. Springer, Cham (2016). Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Centre for Intelligent MachinesMcGill UniversityMontréalCanada
  2. 2.NeuroRx ResearchMontréalCanada

Personalised recommendations