, Volume 11, Issue 3, pp 339–353 | Cite as

Semi-Supervised Multimodal Relevance Vector Regression Improves Cognitive Performance Estimation from Imaging and Biological Biomarkers

  • Bo Cheng
  • Daoqiang ZhangEmail author
  • Songcan Chen
  • Daniel I. Kaufer
  • Dinggang ShenEmail author
  • the Alzheimer’s Disease Neuroimaging Initiative
Original Article


Accurate estimation of cognitive scores for patients can help track the progress of neurological diseases. In this paper, we present a novel semi-supervised multimodal relevance vector regression (SM-RVR) method for predicting clinical scores of neurological diseases from multimodal imaging and biological biomarker, to help evaluate pathological stage and predict progression of diseases, e.g., Alzheimer’s diseases (AD). Unlike most existing methods, we predict clinical scores from multimodal (imaging and biological) biomarkers, including MRI, FDG-PET, and CSF. Considering that the clinical scores of mild cognitive impairment (MCI) subjects are often less stable compared to those of AD and normal control (NC) subjects due to the heterogeneity of MCI, we use only the multimodal data of MCI subjects, but no corresponding clinical scores, to train a semi-supervised model for enhancing the estimation of clinical scores for AD and NC subjects. We also develop a new strategy for selecting the most informative MCI subjects. We evaluate the performance of our approach on 202 subjects with all three modalities of data (MRI, FDG-PET and CSF) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. The experimental results show that our SM-RVR method achieves a root-mean-square error (RMSE) of 1.91 and a correlation coefficient (CORR) of 0.80 for estimating the MMSE scores, and also a RMSE of 4.45 and a CORR of 0.78 for estimating the ADAS-Cog scores, demonstrating very promising performances in AD studies.


Alzheimer’s disease (AD) Mild cognitive impairment (MCI) Semi-supervised learning Relevance vector regression (RVR) Multimodality 



Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: Abbott, AstraZeneca AB, Bayer Schering Pharma AG, Bristol-Myers Squibb, Eisai Global Clinical Development, Elan Corporation, Genentech, GE Healthcare, GlaxoSmithKline, Innogenetics, Johnson and Johnson, Eli Lilly and Co., Medpace, Inc., Merck and Co., Inc., Novartis AG, Pfizer Inc, F. Hoffman-La Roche, Schering-Plough, Synarc, Inc., as well as non-profit partners the Alzheimer’s Association and Alzheimer’s Drug Discovery Foundation, with participation from the U.S. Food and Drug Administration. Private sector contributions to ADNI are facilitated by the Foundation for the National Institutes of Health ( The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Disease Cooperative Study at the University of California, San Diego. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of California, Los Angeles.

This work was supported in part by NIH grants EB006733, EB008374, EB009634, and AG041721, by NSFC grants 61075010 and 61170151, by SRFDP grant 20123218110009, by Qing Lan Project, and also by The National Basic Research Program of China (973 Program) grant No. 2010CB732505.


  1. Adams, N. (2009). Semi-supervised learning. Journal of the Royal Statistical Society Series a-Statistics in Society, 172, 530–530.CrossRefGoogle Scholar
  2. Belkin, M., & Niyogi, P. (2004). Semi-supervised learning on Riemannian manifolds. Machine Learning, 56, 209–239.CrossRefGoogle Scholar
  3. Belkin, M., Matveeva, I., & Niyogi, P. (2004). Regularization and semi-supervised learning on large graphs. Learning Theory, Proceedings, 3120, 624–638.CrossRefGoogle Scholar
  4. Belkin, M., Niyogi, P., & Sindhwani, V. (2006). Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7, 2399–2434.Google Scholar
  5. Bouwman, F. H., Schoonenboom, S. N., van der Flier, W. M., van Elk, E. J., Kok, A., Barkhof, F., et al. (2007). CSF biomarkers and medial temporal lobe atrophy predict dementia in mild cognitive impairment. Neurobiology of Aging, 28, 1070–1074.PubMedCrossRefGoogle Scholar
  6. Bouwman, F. H., van der Flier, W. M., Schoonenboom, N. S. M., van Elk, E. J., Kok, A., Rijmen, F., et al. (2007). Longitudinal changes of CSF biomarkers in memory clinic patients. Neurology, 69, 1006–1011.PubMedCrossRefGoogle Scholar
  7. Bruzzone, L., Chi, M. M., & Marconcini, M. (2006). A novel transductive SVM for semisupervised classification of remote-sensing images. IEEE Transactions on Geoscience and Remote Sensing, 44, 3363–3373.CrossRefGoogle Scholar
  8. Caselli, R. J., Reiman, E. M., Locke, D. E., Hutton, M. L., Hentz, J. G., Hoffman-Snyder, C., et al. (2007). Cognitive domain decline in healthy apolipoprotein E epsilon4 homozygotes before the diagnosis of mild cognitive impairment. Archives of Neurology, 64, 1306–1311.PubMedCrossRefGoogle Scholar
  9. Caselli, R. J., Dueck, A. C., Osborne, D., Sabbagh, M. N., Connor, D. J., Ahern, G. L., et al. (2009). Longitudinal modeling of age-related memory decline and the APOE epsilon 4 effect. The New England Journal of Medicine, 361, 255–263.PubMedCrossRefGoogle Scholar
  10. Chetelat, G., Eustache, F., Viader, F., De la Sayette, V., Pelerin, A., Mezenge, F., et al. (2005). FDG-PET measurement is more accurate than neuropsychological assessments to predict global cognitive deterioration in patients with mild cognitive impairment. Neurocase, 11, 14–25.PubMedCrossRefGoogle Scholar
  11. de Leon, M. J., Mosconi, L., Li, J., De Santi, S., Yao, Y., Tsui, W. H., et al. (2007). Longitudinal CSF isoprostane and MRI atrophy in the progression to AD. Journal of Neurology, 254, 1666–1675.PubMedCrossRefGoogle Scholar
  12. De Santi, S., de Leon, M. J., Rusinek, H., Convit, A., Tarshish, C. Y., Roche, A., et al. (2001). Hippocampal formation glucose metabolism and volume losses in MCI and AD. Neurobiology of Aging, 22, 529–539.PubMedCrossRefGoogle Scholar
  13. Diehl, J., Grimmer, T., Drzezga, A., Riemenschneider, M., Forstl, H., & Kurz, A. (2004). Cerebral metabolic patterns at early stages of frontotemporal dementia and semantic dementia. A PET study. Neurobiology of Aging, 25, 1051–1056.PubMedCrossRefGoogle Scholar
  14. Ding, L., & Zhao, P. B. (2010). Semi-supervised learning with varifold Laplacians. Neurocomputing, 73, 1580–1586.CrossRefGoogle Scholar
  15. Drzezga, A., Lautenschlager, N., Siebner, H., Riemenschneider, M., Willoch, F., Minoshima, S., et al. (2003). Cerebral metabolic changes accompanying conversion of mild cognitive impairment into Alzheimer’s disease: a PET follow-up study. European Journal of Nuclear Medicine and Molecular Imaging, 30, 1104–1113.PubMedCrossRefGoogle Scholar
  16. Duchesne, S., Caroli, A., Geroldi, C., Collins, D. L., & Frisoni, G. B. (2009). Relating one-year cognitive change in mild cognitive impairment to baseline MRI features. NeuroImage, 47, 1363–1370.PubMedCrossRefGoogle Scholar
  17. Fan, Y., Batmanghelich, N., Clark, C. M., Davatzikos, C., & Initia, A. D. N. (2008). Spatial patterns of brain atrophy in MCI patients, identified via high-dimensional pattern classification, predict subsequent cognitive decline. NeuroImage, 39, 1731–1743.PubMedCrossRefGoogle Scholar
  18. Fan, Y., Resnick, S. M., Wu, X., & Davatzikos, C. (2008). Structural and functional biomarkers of prodromal Alzheimer’s disease: a high-dimensional pattern classification study. NeuroImage, 41, 277–285.PubMedCrossRefGoogle Scholar
  19. Fan, Y., Kaufer, D., Shen, D., (2010). Joint estimation of multiple clinical variables of neurological diseases from imaging patterns. In Proceedings of 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI 2010), 852–855.Google Scholar
  20. Fellgiebel, A., Scheurich, A., Bartenstein, P., & Muller, M. J. (2007). FDG-PET and CSF phospho-tau for prediction of cognitive decline in mild cognitive impairment. Psychiatry Research-Neuroimaging, 155, 167–171.CrossRefGoogle Scholar
  21. Filipovych, R., Davatzikos, C., & Initia, A. D. N. (2011). Semi-supervised pattern classification of medical images: application to mild cognitive impairment (MCI). NeuroImage, 55, 1109–1119.PubMedCrossRefGoogle Scholar
  22. Fjell, A. M., Walhovd, K. B., Fennema-Notestine, C., McEvoy, L. K., Hagler, D. J., Holland, D., et al. (2010). CSF biomarkers in prediction of cerebral and clinical change in mild cognitive impairment and Alzheimer’s disease. Journal of Neuroscience, 30, 2088–2101.PubMedCrossRefGoogle Scholar
  23. Geroldi, C., Rossi, R., Calvagna, C., Testa, C., Bresciani, L., Binetti, G., et al. (2006). Medial temporal atrophy but not memory deficit predicts progression to dementia in patients with mild cognitive impairment. Journal of Neurology, Neurosurgery, and Psychiatry, 77, 1219–1222.PubMedCrossRefGoogle Scholar
  24. Hinrichs, C., Singh, V., Mukherjee, L., Xu, G. F., Chung, M. K., & Johnson, S. C. (2009a). Spatially augmented LPboosting for AD classification with evaluations on the ADNI dataset. NeuroImage, 48, 138–149.CrossRefGoogle Scholar
  25. Hinrichs, C., Singh, V., Xu, G., & Johnson, S. (2009b). MKL for robust multi-modality AD classification. Medical Image Computing and Computer Assisted Intervention, 12, 786–794.Google Scholar
  26. Hinrichs, C., Singh, V., Xu, G., & Johnson, S. C. (2011). Predictive markers for AD in a multi-modality framework: an analysis of MCI progression in the ADNI population. NeuroImage, 55, 574–589.PubMedCrossRefGoogle Scholar
  27. Inoue, M., Jimbo, D., Taniguchi, M., & Urakami, K. (2011). Touch Panel-type Dementia Assessment Scale: a new computer-based rating scale for Alzheimer’s disease. Psychogeriatrics, 11, 28–33.PubMedCrossRefGoogle Scholar
  28. Ni, B. B., Yan, S. C., & Kassim, A. A. (2012). Learning a propagable graph for semisupervised learning: classification and regression. Ieee Transactions on Knowledge and Data Engineering, 24, 114–126.CrossRefGoogle Scholar
  29. Rakotomamonjy, A., Bach, F. R., Canu, S., & Grandvalet, Y. (2008). SimpleMKL. Journal of Machine Learning Research, 9, 2491–2521.Google Scholar
  30. Reiman, E. M., Caselli, R. J., Yun, L. S., Chen, K. W., Bandy, D., Minoshima, S., et al. (1996). Preclinical evidence of Alzheimer’s disease in persons homozygous for the epsilon 4 allele for apolipoprotein E. The New England Journal of Medicine, 334, 752–758.PubMedCrossRefGoogle Scholar
  31. Reiman, E. M., Chen, K. W., Liu, X. F., Bandy, D., Yu, M. X., Lee, W. D., et al. (2009). Fibrillar amyloid-beta burden in cognitively normal people at 3 levels of genetic risk for Alzheimer’s disease. Proceedings of the National Academy of Sciences of the United States of America, 106, 6820–6825.PubMedCrossRefGoogle Scholar
  32. Rosen, W. G., Mohs, R. C., & Davis, K. L. (1984). A new rating scale for Alzheimer’s disease. The American Journal of Psychiatry, 141, 1356–1364.PubMedGoogle Scholar
  33. Shattuck, D. W., Sandor-Leahy, S. R., Schaper, K. A., Rottenberg, D. A., & Leahy, R. M. (2001). Magnetic resonance image tissue classification using a partial volume model. NeuroImage, 13, 856–876.PubMedCrossRefGoogle Scholar
  34. Sled, J. G., Zijdenbos, A. P., & Evans, A. C. (1998). A nonparametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Transactions on Medical Imaging, 17, 87–97.PubMedCrossRefGoogle Scholar
  35. Smith, S. M. (2002). Fast robust automated brain extraction. Human Brain Mapping, 17, 143–155.PubMedCrossRefGoogle Scholar
  36. Stonnington, C. M., Chu, C., Kloppel, S., Jack, C. R., Ashburner, J., Frackowiak, R. S. J., et al. (2010). Predicting clinical scores from magnetic resonance scans in Alzheimer’s disease. NeuroImage, 51, 1405–1413.PubMedCrossRefGoogle Scholar
  37. Tipping, M. E. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1, 211–244.Google Scholar
  38. Twamley, E. W., Ropacki, S. A. L., & Bondi, M. W. (2006). Neuropsychological and neuroimaging changes in preclinical Alzheimer’s disease. Journal of the International Neuropsychological Society, 12, 707–735.PubMedCrossRefGoogle Scholar
  39. Vemuri, P., Wiste, H. J., Weigand, S. D., Shaw, L. M., Trojanowski, J. Q., Weiner, M. W., et al. (2009). MRI and CSF biomarkers in normal, MCI, and AD subjects predicting future clinical change. Neurology, 73, 294–301.PubMedCrossRefGoogle Scholar
  40. Visser, P. J., Verhey, F. R. J., Hofman, P. A., Scheltens, P., & Jolles, J. (2002). Medial temporal lobe atrophy predicts Alzheimer’s disease in patients with minor cognitive impairment. Journal of Neurology, Neurosurgery, and Psychiatry, 72, 491–497.PubMedGoogle Scholar
  41. Walhovd, K. B., Fjell, A. M., Brewer, J., McEvoy, L. K., Fennema-Notestine, C., Hagler, D. J., Jr., et al. (2010). Combining MR imaging, positron-emission tomography, and CSF biomarkers in the diagnosis and prognosis of Alzheimer disease. AJNR. American Journal of Neuroradiology, 31, 347–354.PubMedCrossRefGoogle Scholar
  42. Walhovd, K. B., Fjell, A. M., Dale, A. M., McEvoy, L. K., Brewer, J., Karow, D. S., et al. (2010b). Multi-modal imaging predicts memory performance in normal aging and cognitive decline. Neurobiology of Aging, 31, 1107–1121.CrossRefGoogle Scholar
  43. Wang, Z., Chen, S. C., & Sun, T. K. (2008). MultiK-MHKS: a novel multiple kernel learning algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 348–353.PubMedCrossRefGoogle Scholar
  44. Wang, Y., Fan, Y., Bhatt, P., & Davatzikos, C. (2010). High-dimensional pattern regression using machine learning: from medical images to continuous clinical variables. NeuroImage, 50, 1519–1535.PubMedCrossRefGoogle Scholar
  45. Xu, Z., Jin, R., Yang, H., King, I., Lyu, M. R. (2010). Simple and Efficient Multiple Kernel Learning by Group Lasso. In Proceedings of the 27th Conference on Machine Learning (ICML 2010).Google Scholar
  46. Zhang, D., & Shen, D. (2011). Semi-supervised multimodal classification of Alzheimer’s disease. In Proceedings of 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI 2011), 1628–1631.Google Scholar
  47. Zhang, D., & Shen, D. (2012). Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. NeuroImage, 59, 895–907.PubMedCrossRefGoogle Scholar
  48. Zhang, Y. Y., Brady, M., & Smith, S. (2001). Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Transactions on Medical Imaging, 20, 45–57.PubMedCrossRefGoogle Scholar
  49. Zhang, D., Wang, Y., Zhou, L., Yuan, H., Shen, D., & Initia, A. D. N. (2011). Multimodal classification of Alzheimer’s disease and mild cognitive impairment. NeuroImage, 55, 856–867.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Bo Cheng
    • 1
    • 2
  • Daoqiang Zhang
    • 1
    • 2
    Email author
  • Songcan Chen
    • 1
  • Daniel I. Kaufer
    • 3
  • Dinggang Shen
    • 2
    Email author
  • the Alzheimer’s Disease Neuroimaging Initiative
    • 2
  1. 1.Department of Computer Science and EngineeringNanjing University of Aeronautics and AstronauticsNanjingChina
  2. 2.Department of Radiology and BRICUniversity of North CarolinaChapel HillUSA
  3. 3.Department of NeurologyUniversity of North CarolinaChapel HillUSA

Personalised recommendations