Multi-Modal Multi-Task Learning for Joint Prediction of Clinical Scores in Alzheimer’s Disease

  • Daoqiang Zhang
  • Dinggang Shen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7012)


One recent interest in computer-aided diagnosis of neurological diseases is to predict the clinical scores from brain images. Most existing methods usually estimate multiple clinical variables separately, without considering the useful correlation information among them. On the other hand, nearly all methods use only one modality of data (mostly structural MRI) for regression, and thus ignore the complementary information among different modalities. To address these issues, in this paper, we present a general methodology, namely Multi-Modal Multi-Task (M3T) learning, to jointly predict multiple variables from multi-modal data. Our method contains three major subsequent steps: (1) a multi-task feature selection which selects the common subset of relevant features for the related multiple clinical variables from each modality; (2) a kernel-based multimodal data fusion which fuses the above-selected features from all modalities; (3) a support vector regression which predicts multiple clinical variables based on the previously learnt mixed kernel. Experimental results on ADNI dataset with both imaging modalities (MRI and PET) and biological modality (CSF) validate the efficacy of the proposed M3T learning method.


Feature Selection Mild Cognitive Impairment Mini Mental State Examination Support Vector Regression Clinical Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kloppel, S., Stonnington, C.M., Chu, C., Draganski, B., Scahill, R.I., Rohrer, J.D., Fox, N.C., Jack Jr., C.R., Ashburner, J., Frackowiak, R.S.: Automatic classification of MR scans in Alzheimer’s disease. Brain 131, 681–689 (2008)CrossRefGoogle Scholar
  2. 2.
    Vemuri, P., Wiste, H.J., Weigand, S.D., Shaw, L.M., Trojanowski, J.Q., Weiner, M.W., Knopman, D.S., Petersen, R.C., Jack Jr., C.R.: MRI and CSF biomarkers in normal, MCI, and AD subjects: predicting future clinical change. Neurology 73, 294–301 (2009)CrossRefGoogle Scholar
  3. 3.
    Walhovd, K.B., Fjell, A.M., Dale, A.M., McEvoy, L.K., Brewer, J., Karow, D.S., Salmon, D.P., Fennema-Notestine, C.: Multi-modal imaging predicts memory performance in normal aging and cognitive decline. Neurobiol. Aging 31, 1107–1121 (2010)CrossRefGoogle Scholar
  4. 4.
    Hinrichs, C., Singh, V., Xu, G., Johnson, S.: MKL for robust multi-modality AD classification. Med. Image Comput. Comput. Assist. Interv. 12, 786–794 (2009)Google Scholar
  5. 5.
    Ye, J., Chen, K., Wu, T., Li, J., Zhao, Z., Patel, R., Bae, M., Janardan, R., Liu, H., Alexander, G., Reiman, E.M.: Heterogeneous data fusion for Alzheimer’s disease study. In: ACM International Conference on Knowledge Discovery and Data Mining (2008)Google Scholar
  6. 6.
    Duchesne, S., Caroli, A., Geroldi, C., Frisoni, G.B., Collins, D.L.: Predicting clinical variable from MRI features: Application to MMSE in MCI. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3749, pp. 392–399. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Wang, Y., Fan, Y., Bhatt, P., Davatzikos, C.: High-dimensional pattern regression using machine learning: from medical images to continuous variables. NeuroImage 50, 1519–1535 (2010)CrossRefGoogle Scholar
  8. 8.
    Stonnington, C.M., Chu, C., Kloppel, S., Jack Jr., C.R., Ashburner, J., Frackowiak, R.S.J.: Predicting clinical scores from magnetic resonance scans in Alzheimer’s disease. NeuroImage 51, 1405–1413 (2010)CrossRefGoogle Scholar
  9. 9.
    Fan, Y., Kaufer, D., Shen, D.: Joint estimation of multiple clinical variables of neurological diseases from imaging patterns. In: The 2010 IEEE International Conference on Biomedical Imaging: from Nano to Macro, pp. 852–855 (2010)Google Scholar
  10. 10.
    Obozinski, G., Taskar, B., Jordan, M.I.: Multi-task feature selection. Technical report, Statistics Department, UC Berkeley (2006)Google Scholar
  11. 11.
    Liu, J., Ji, S., Ye, J.: Multi-task feature learning via efficient l2,1-norm minimization. In: Uncertainty in Artificial Intelligence (2009)Google Scholar
  12. 12.
    Yang, X., Kim, S., Xing, E.P.: Heterogeneous multitask learning with joint sparsity constraints. In: Advances in Neural Information Processing Systems (2009) Google Scholar
  13. 13.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)zbMATHGoogle Scholar
  14. 14.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B 58, 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  15. 15.
    Liu, J., Ji, S., Ye, J.: SLEP: Sparse Learning with Efficient Projections. Technical report, Arizona State University (2009)Google Scholar
  16. 16.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Daoqiang Zhang
    • 1
    • 2
  • Dinggang Shen
    • 1
  1. 1.Dept. of Radiology and BRICUniversity of North Carolina at Chapel Hill
  2. 2.Dept. of Computer Science and EngineeringNanjing University of Aeronautics and AstronauticsNanjingChina

Personalised recommendations