Anatomical-Landmark-Based Deep Learning for Alzheimer’s Disease Diagnosis with Structural Magnetic Resonance Imaging

  • Mingxia LiuEmail author
  • Chunfeng Lian
  • Dinggang Shen
Part of the Intelligent Systems Reference Library book series (ISRL, volume 171)


Structural magnetic resonance imaging (sMRI) has been widely used in computer-aided diagnosis of brain diseases, such as Alzheimer’s disease (AD) and its prodromal stage, i.e., mild cognitive impairment (MCI). Based on sMRI data, anatomical-landmark-based deep learning has been recently proposed for AD and MCI diagnosis. These methods usually first locate informative anatomical landmarks in brain sMR images, and then integrate both feature learning and classification training into a unified framework. This chapter presents the latest anatomical-landmark-based deep learning approaches for automatic diagnosis of AD and MCI. Specifically, an automatic landmark discovery method is first introduced to identify discriminative regions in brain sMR images. Then, a landmark-based deep learning framework is presented for AD/MCI classification, by jointly performing feature extraction and classifier training. Experimental results on three public databases demonstrate that the proposed framework boosts the disease diagnosis performance, compared with several state-of-the-art sMRI-based methods.



This study was partly supported by NIH grants (EB006733, EB008374, EB009634, MH100217, AG041721, AG042599, AG010129, and AG030514). Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. The investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report, with details shown online.


  1. 1.
    Wolz, R., Aljabar, P., Hajnal, J.V., Hammers, A., Rueckert, D.: LEAP: learning embeddings for atlas propagation. NeuroImage 49(2), 1316–1325 (2010)CrossRefGoogle Scholar
  2. 2.
    Zhang, J., Gao, Y., Gao, Y., Munsell, B., Shen, D.: Detecting anatomical landmarks for fast Alzheimer’s disease diagnosis. IEEE Trans. Med. Imaging 35(12), 2524–2533 (2016)CrossRefGoogle Scholar
  3. 3.
    Ashburner, J., Friston, K.J.: Voxel-based morphometry: the methods. NeuroImage 11(6), 805–821 (2000)CrossRefGoogle Scholar
  4. 4.
    Jack, C., Petersen, R.C., Xu, Y.C., O’Brien, P.C., Smith, G.E., Ivnik, R.J., Boeve, B.F., Waring, S.C., Tangalos, E.G., Kokmen, E.: Prediction of AD with MRI-based hippocampal volume in mild cognitive impairment. Neurology 52(7), 1397 (1999)CrossRefGoogle Scholar
  5. 5.
    Atiya, M., Hyman, B.T., Albert, M.S., Killiany, R.: Structural magnetic resonance imaging in established and prodromal Alzheimer disease: a review. Alzheimer Dis. Assoc. Disord. 17(3), 177–195 (2003)CrossRefGoogle Scholar
  6. 6.
    Dubois, B., Chupin, M., Hampel, H., Lista, S., Cavedo, E., Croisile, B., Tisserand, G.L., Touchon, J., Bonafe, A., Ousset, P.J., et al.: Donepezil decreases annual rate of hippocampal atrophy in suspected prodromal Alzheimer’s disease. Alzheimer’s Dement. 11(9), 1041–1049 (2015)CrossRefGoogle Scholar
  7. 7.
    Cuingnet, R., Gerardin, E., Tessieras, J., Auzias, G., Lehéricy, S., Habert, M.O., Chupin, M., Benali, H., Colliot, O.: Automatic classification of patients with Alzheimer’s disease from structural MRI: a comparison of ten methods using the ADNI database. NeuroImage 56(2), 766–781 (2011)CrossRefGoogle Scholar
  8. 8.
    Lötjönen, J., Wolz, R., Koikkalainen, J., Julkunen, V., Thurfjell, L., Lundqvist, R., Waldemar, G., Soininen, H., Rueckert, D.: Fast and robust extraction of hippocampus from MR images for diagnostics of Alzheimer’s disease. NeuroImage 56(1), 185–196 (2011)CrossRefGoogle Scholar
  9. 9.
    Liu, M., Zhang, D., Shen, D.: View-centralized multi-atlas classification for Alzheimer’s disease diagnosis. Hum. Brain Mapp. 36(5), 1847–1865 (2015)CrossRefGoogle Scholar
  10. 10.
    Montagne, A., Barnes, S.R., Sweeney, M.D., Halliday, M.R., Sagare, A.P., Zhao, Z., Toga, A.W., Jacobs, R.E., Liu, C.Y., Amezcua, L., et al.: Blood-brain barrier breakdown in the aging human hippocampus. Neuron 85(2), 296–302 (2015)CrossRefGoogle Scholar
  11. 11.
    Liu, M., Zhang, J., Yap, P.T., Shen, D.: View-aligned hypergraph learning for Alzheimer’s disease diagnosis with incomplete multi-modality data. Med. Image Anal. 36, 123–134 (2017)CrossRefGoogle Scholar
  12. 12.
    Liu, M., Zhang, D., Shen, D.: Relationship induced multi-template learning for diagnosis of Alzheimer’s disease and mild cognitive impairment. IEEE Trans. Med. Imaging 35(6), 1463–1474 (2016)CrossRefGoogle Scholar
  13. 13.
    Lian, C., Zhang, J., Liu, M., Zong, X., Hung, S.C., Lin, W., Shen, D.: Multi-channel multi-scale fully convolutional network for 3D perivascular spaces segmentation in 7T MR images. Med. Image Anal. 46, 106–117 (2018)CrossRefGoogle Scholar
  14. 14.
    Liu, M., Zhang, J., Adeli, E., Shen, D.: Joint classification and regression via deep multi-task multi-channel learning for Alzheimer’s disease diagnosis. IEEE Trans. Biomed. Eng. (2018)Google Scholar
  15. 15.
    Liu, M., Zhang, D., Adeli, E., Shen, D.: Inherent structure-based multiview learning with multitemplate feature representation for Alzheimer’s disease diagnosis. IEEE Trans. Biomed. Eng. 63(7), 1473–1482 (2016)CrossRefGoogle Scholar
  16. 16.
    Liu, M., Zhang, J., Nie, D., Yap, P.T., Shen, D.: Anatomical landmark based deep feature representation for MR images in brain disease diagnosis. IEEE J. Biomed. Health Inform. 22(5), 1476–1485 (2018)CrossRefGoogle Scholar
  17. 17.
    Friedman, J., Hastie, T., Tibshirani, R.: The Elements of Statistical Learning. Springer Series in Statistics, vol. 1. Springer, Berlin (2001)zbMATHGoogle Scholar
  18. 18.
    Small, G.W., Ercoli, L.M., Silverman, D.H., Huang, S.C., Komo, S., Bookheimer, S.Y., Lavretsky, H., Miller, K., Siddarth, P., Rasgon, N.L., et al.: Cerebral metabolic and cognitive decline in persons at genetic risk for Alzheimer’s disease. Proc. Natl. Acad. Sci. 97(11), 6037–6042 (2000)CrossRefGoogle Scholar
  19. 19.
    Lian, C., Ruan, S., Denœux, T., Jardin, F., Vera, P.: Selecting radiomic features from FDG-PET images for cancer treatment outcome prediction. Med. Image Anal. 32, 257–268 (2016)CrossRefGoogle Scholar
  20. 20.
    Wolz, R., Aljabar, P., Hajnal, J.V., Lötjönen, J., Rueckert, D.: Nonlinear dimensionality reduction combining MR imaging with non-imaging information. Med. Image Anal. 16(4), 819–830 (2012)CrossRefGoogle Scholar
  21. 21.
    Tong, T., Wolz, R., Gao, Q., Guerrero, R., Hajnal, J.V., Rueckert, D.: Multiple instance learning for classification of dementia in brain MRI. Med. Image Anal. 18(5), 808–818 (2014)CrossRefGoogle Scholar
  22. 22.
    Coupé, P., Eskildsen, S.F., Manjón, J.V., Fonov, V.S., Pruessner, J.C., Allard, M., Collins, D.L.: Scoring by nonlocal image patch estimator for early detection of Alzheimer’s disease. NeuroImage: Clin. 1(1) (2012) 141–152Google Scholar
  23. 23.
    Lian, C., Ruan, S., Denœux, T., Li, H., Vera, P.: Spatial evidential clustering with adaptive distance metric for tumor segmentation in FDG-PET images. IEEE Trans. Biomed. Eng. 65(1), 21–30 (2017)CrossRefGoogle Scholar
  24. 24.
    Lian, C., Ruan, S., Denœux, T., Li, H., Vera, P.: Joint tumor segmentation in PET-CT images using co-clustering and fusion based on belief functions. IEEE Trans. Image Process. 28(2), 755–766 (2019)CrossRefMathSciNetzbMATHGoogle Scholar
  25. 25.
    Liu, M., Zhang, J., Adeli, E., Shen, D.: Landmark-based deep multi-instance learning for brain disease diagnosis. Med. Image Anal. 43, 157–168 (2018)CrossRefGoogle Scholar
  26. 26.
    Jack, C.R., Bernstein, M.A., Fox, N.C., Thompson, P., Alexander, G., Harvey, D., Borowski, B., Britson, P.J., L Whitwell, J., Ward, C.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging 27(4), 685–691 (2008)Google Scholar
  27. 27.
    Malone, I.B., Cash, D., Ridgway, G.R., MacManus, D.G., Ourselin, S., Fox, N.C., Schott, J.M.: MIRIAD-Public release of a multiple time point Alzheimer’s MR imaging dataset. NeuroImage 70, 33–36 (2013)CrossRefGoogle Scholar
  28. 28.
    Cheng, B., Liu, M., Suk, H.I., Shen, D., Zhang, D.: Multimodal manifold-regularized transfer learning for MCI conversion prediction. Brain Imaging Behav. 1–14 (2015)Google Scholar
  29. 29.
    Sled, J.G., Zijdenbos, A.P., Evans, A.C.: A nonparametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Trans. Med. Imaging 17(1), 87–97 (1998)CrossRefGoogle Scholar
  30. 30.
    Mardia, K.: Assessment of multinormality and the robustness of Hotelling’s T\(^2\) test. Appl. Stat. 163–171 (1975)Google Scholar
  31. 31.
    Holmes, C.J., Hoge, R., Collins, L., Woods, R., Toga, A.W., Evans, A.C.: Enhancement of MR images using registration for signal averaging. J. Comput. Assist. Tomogr. 22(2), 324–333 (1998)CrossRefGoogle Scholar
  32. 32.
    Ashburner, J., Friston, K.J.: Why voxel-based morphometry should be used. NeuroImage 14(6), 1238–1243 (2001)CrossRefGoogle Scholar
  33. 33.
    Zhang, J., Liang, J., Zhao, H.: Local energy pattern for texture classification using self-adaptive quantization thresholds. IEEE Trans. Image Process. 22(1), 31–42 (2013)CrossRefMathSciNetzbMATHGoogle Scholar
  34. 34.
    Leung, T., Malik, J.: Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vis. 43(1), 29–44 (2001)CrossRefzbMATHGoogle Scholar
  35. 35.
    De Jong, L., Van der Hiele, K., Veer, I., Houwing, J., Westendorp, R., Bollen, E., De Bruin, P., Middelkoop, H., Van Buchem, M., Van Der Grond, J.: Strongly reduced volumes of putamen and thalamus in Alzheimer’s disease: an MRI study. Brain 131(12), 3277–3285 (2008)CrossRefGoogle Scholar
  36. 36.
    Liu, M., Zhang, D., Shen, D.: Ensemble sparse classification of Alzheimer’s disease. NeuroImage 60(2), 1106–1116 (2012)CrossRefGoogle Scholar
  37. 37.
    Yan, Z., Zhan, Y., Peng, Z., Liao, S., Shinagawa, Y., Zhang, S., Metaxas, D.N., Zhou, X.S.: Multi-instance deep learning: discover discriminative local anatomies for bodypart recognition. IEEE Trans. Med. Imaging 35(5), 1332–1343 (2016)CrossRefGoogle Scholar
  38. 38.
    Amores, J.: Multiple instance classification: review, taxonomy and comparative study. Artif. Intell. 201, 81–105 (2013)CrossRefMathSciNetzbMATHGoogle Scholar
  39. 39.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)Google Scholar
  40. 40.
    Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al.: Tensorflow: A system for large-scale machine learning. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, vol. 16, pp. 265–283 (2016)Google Scholar
  41. 41.
    Zhang, D., Shen, D.: Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. NeuroImage 59(2), 895–907 (2012)CrossRefMathSciNetGoogle Scholar
  42. 42.
    Cheng, B., Liu, M., Zhang, D., Munsell, B.C., Shen, D.: Domain transfer learning for MCI conversion prediction. IEEE Trans. Biomed. Eng. 62(7), 1805–1817 (2015)CrossRefGoogle Scholar
  43. 43.
    Liu, M., Zhang, D., Chen, S., Xue, H.: Joint binary classifier learning for ECOC-based multi-class classification. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2335–2341 (2016)CrossRefGoogle Scholar
  44. 44.
    Zhang, Y., Brady, M., Smith, S.: Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging 20(1), 45–57 (2001)CrossRefGoogle Scholar
  45. 45.
    Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., Joliot, M.: Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage 15(1), 273–289 (2002)CrossRefGoogle Scholar
  46. 46.
    Shen, D., Davatzikos, C.: HAMMER: hierarchical attribute matching mechanism for elastic registration. IEEE Trans. Med. Imaging 21(11), 1421–1439 (2002)CrossRefGoogle Scholar
  47. 47.
    Matthews, B.W.: Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405(2) (1975) 442–451Google Scholar
  48. 48.
    Wang, M., Zhang, D., Shen, D., Liu, M.: Multi-task exclusive relationship learning for Alzheimer’s disease progression prediction with longitudinal data. Med. Image Anal. 53, 111–122 (2019)CrossRefGoogle Scholar
  49. 49.
    Jie, B., Liu, M., Liu, J., Zhang, D., Shen, D.: Temporally constrained group sparse learning for longitudinal data analysis in Alzheimer’s disease. IEEE Trans. Biomed. Eng. 64(1), 238–249 (2017)CrossRefGoogle Scholar
  50. 50.
    Lian, C., Liu, M., Zhang, J., Shen, D.: Hierarchical fully convolutional network for joint atrophy localization and Alzheimer’s disease diagnosis using structural MRI. IEEE Trans. Pattern Anal. Mach. Intell. (2019)Google Scholar
  51. 51.
    Wang, M., Zhang, D., Huang, J., Shen, D., Liu, M.: Low-rank representation for multi-center autism spectrum disorder identification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 647–654. Springer (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Radiology and BRICUniversity of North Carolina at Chapel HillChapel HillUSA

Personalised recommendations