Predicting Future Disease Activity and Treatment Responders for Multiple Sclerosis Patients Using a Bag-of-Lesions Brain Representation

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10435)


The growth of lesions and the development of new lesions in MRI are markers of new disease activity in Multiple Sclerosis (MS) patients. Successfully predicting future lesion activity could lead to a better understanding of disease worsening, as well as prediction of treatment efficacy. We introduce the first, fully automatic, probabilistic framework for the prediction of future lesion activity in relapsing-remitting MS patients, based only on baseline multi-modal MRI, and use it to successfully identify responders to two different treatments. We develop a new Bag-of-Lesions (BoL) representation for patient images based on a variety of features extracted from lesions. A probabilistic codebook of lesion types is created by clustering features using Gaussian mixture models. Patients are represented as a probabilistic histogram of lesion-types. A Random Forest classifier is trained to automatically predict future MS activity up to two years ahead based on the patient’s baseline BoL representation. The framework is trained and tested on a large, proprietary, multi-centre, multi-modal clinical trial dataset consisting of 1048 patients. Testing based on 50-fold cross validation shows that our framework compares favourably to several other classifiers. Automated identification of responders in two different treated groups of patients leads to sensitivity of 82% and 84% and specificity of 92% and 94% respectively, showing that this is a very promising approach towards personalized treatment for MS patients.



This work was supported by the Canadian NSERC Discovery and CREATE grants. We would like to thank Drs. Narayanan and Maranzano for their clinical advice, and Mr. A. Zografos Caramanos for data preparation. All patient MRI are courtesy of NeuroRx Research.


  1. 1.
    Gold, R., et al.: Placebo-controlled phase 3 study of oral BG-12 for relapsing multiple sclerosis. New Engl. J. Med. 367(12), 1098–1107Google Scholar
  2. 2.
    Brown, J.W.L., Chard, D.T.: The role of MRI in the evaluation of secondary progressive multiple sclerosis. Expert Rev. Neurother. 16(2), 157–171 (2016)CrossRefGoogle Scholar
  3. 3.
    Barkhof, F., et al.: Comparison of MRI criteria at first presentation to predict conversion to clinically definite multiple sclerosis. Brain 120(11), 2059–2069 (1997)CrossRefGoogle Scholar
  4. 4.
    Brosch, T., Yoo, Y., Li, D.K.B., Traboulsee, A., Tam, R.: Modeling the variability in brain morphology and lesion distribution in multiple sclerosis by deep learning. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 462–469. Springer, Cham (2014). doi: 10.1007/978-3-319-10470-6_58 CrossRefGoogle Scholar
  5. 5.
    Yoo, Y., et al.: Deep learning of brain lesion patterns for predicting future disease activity in patients with early symptoms of multiple sclerosis. In: International Workshop Large-Scale Annotation of Biomedical Data, pp. 86–94 (2016)Google Scholar
  6. 6.
    Popescu, V., et al.: Brain atrophy and lesion load predict long term disability in multiple sclerosis. J. Neurol. Neurosurg. Psych. 84(10), 1082–1091 (2013)CrossRefGoogle Scholar
  7. 7.
    Bosch, A., Zisserman, A., Muñoz, X.: Scene classification via pLSA. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 517–530. Springer, Heidelberg (2006). doi: 10.1007/11744085_40 CrossRefGoogle Scholar
  8. 8.
    Shiee, N., et al.: A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions. NeuroImage 49(2), 1524–1535 (2010)CrossRefGoogle Scholar
  9. 9.
    Díaz-Uriarte, R., De Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC Bioinform. 7(1), 3 (2006)CrossRefGoogle Scholar
  10. 10.
    Lazebnik, S., et al.: A sparse texture representation using local affine regions. PAMI 27, 1265–1278 (2005)CrossRefGoogle Scholar
  11. 11.
    Ahonen, T., et al.: Face description with local binary patterns: application to face recognition. PAMI 28(12), 2037–2041 (2006)CrossRefGoogle Scholar
  12. 12.
    Smith, S.M.: Fast robust automated brain extraction. Hum. Brain Mapp. 17(3), 143–155 (2002)CrossRefGoogle Scholar
  13. 13.
    Sled, J.G., et al.: A nonparametric method for automatic correction of intensity nonuniformity in MRI data. TMI 17(1), 87–97 (1998)Google Scholar
  14. 14.
    Elliott, C., et al.: Temporally consistent probabilistic detection of new multiple sclerosis lesions in brain MRI. TMI 32(8), 1490–1503 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Centre for Intelligent MachinesMcGill UniversityMontréalCanada
  2. 2.School of Computer ScienceMcGill UniversityMontréalCanada
  3. 3.NeuroRx ResearchMontréalCanada

Personalised recommendations