Abstract
Using multimodal neuroimaging data to characterize brain network is currently an advanced technique for Alzheimer’s disease(AD) Analysis. Over recent years the neuroimaging community has made tremendous progress in the study of resting-state functional magnetic resonance imaging (rs-fMRI) derived from blood-oxygen-level-dependent (BOLD) signals and Diffusion Tensor Imaging (DTI) derived from white matter fiber tractography. However, Due to the heterogeneity and complexity between BOLD signals and fiber tractography, Most existing multimodal data fusion algorithms can not sufficiently take advantage of the complementary information between rs-fMRI and DTI. To overcome this problem, a novel Hypergraph Generative Adversarial Networks (HGGAN) is proposed in this paper, which utilizes Interactive Hyperedge Neurons module (IHEN) and Optimal Hypergraph Homomorphism algorithm (OHGH) to generate multimodal connectivity of Brain Network from rs-fMRI combination with DTI. To evaluate the performance of this model, We use publicly available data from the ADNI database to demonstrate that the proposed model not only can identify discriminative brain regions of AD but also can effectively improve classification performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dadar, M., et al.: Validation of a regression technique for segmentation of white matter hyperintensities in Alzheimers disease. IEEE Trans. Med. Imaging 36(8), 1758–1768 (2017)
Association, A., et al.: 2018 Alzheimer’s disease facts and figures. Alzheimer’s Dementia 14(3), 367–429 (2018)
Wang, S., Wang, H., Shen, Y., Wang, X.: Automatic recognition of mild cognitive impairment and Alzheimers disease using ensemble based 3D densely connected convolutional networks. In: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 517–523. IEEE (2018)
Huettel, S., Song, A., McCarthy, G.: Functional magnetic resonance imaging. Sinauer associates. Inc, Sunderland, MA, pp. 162–170 (2004)
Westlye, L.T., et al.: Life-span changes of the human brain white matter: diffusion tensor imaging (DTI) and volumetry. Cereb. Cortex 20(9), 2055–2068 (2010)
Wang, S., Hu, Y., Shen, Y., Li, H.: Classification of diffusion tensor metrics for the diagnosis of a myelopathic cord using machine learning. Int. J. Neural Syst. 28(02), 1750036 (2018)
Wang, S., et al.: Skeletal maturity recognition using a fully automated system with convolutional neural networks. IEEE Access 6, 29979–29993 (2018)
Wang, S., Shen, Y., Zeng, D., Hu, Y.: Bone age assessment using convolutional neural networks. In: 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), pp. 175–178. IEEE (2018)
Zeng, D., Wang, S., Shen, Y., Shi, C.: A GA-based feature selection and parameter optimization for support tucker machine. Procedia Comput. Sci. 111, 17–23 (2017)
Jeon, E., Kang, E., Lee, J., Lee, J., Kam, T.-E., Suk, H.-I.: Enriched representation learning in resting-state fMRI for early MCI diagnosis. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 397–406. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_39
Wang, S.Q., Li, X., Cui, J.L., Li, H.X., Luk, K.D., Hu, Y.: Prediction of myelopathic level in cervical spondylotic myelopathy using diffusion tensor imaging. J. Magn. Reson. Imaging 41(6), 1682–1688 (2015)
Mo, L.F., Wang, S.Q.: A variational approach to nonlinear two-point boundary value problems. Nonlinear Anal. Theory Meth. Appl. 71(12), e834–e838 (2009)
Wang, S.Q.: A variational approach to nonlinear two-point boundary value problems. Comput. Math. Appl. 58(11–12), 2452–2455 (2009)
Wang, S.Q., He, J.H.: Variational iteration method for a nonlinear reaction-diffusion process. Int. J. Chemi. Reactor Eng. 6(1) (2008)
Wang, S., et al.: An ensemble-based densely-connected deep learning system for assessment of skeletal maturity. IEEE Trans. Syst. Man Cybern. Syst. (2020)
Wu, K., Shen, Y., Wang, S.: 3D convolutional neural network for regional precipitation nowcasting. J. Image Signal Process. 7(4), 200–212 (2018)
Zhang, D., Shen, D., Initiative, A.D.N., et al.: Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. Neuroimage 59(2), 895–907 (2012)
Wang, S., Shen, Y., Chen, W., Xiao, T., Hu, J.: Automatic recognition of mild cognitive impairment from MRI images using expedited convolutional neural networks. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10613, pp. 373–380. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_43
Lei, B., et al.: Self-calibrated brain network estimation and joint non-convex multi-task learning for identification of early Alzheimer’s disease. Med. Image Anal. 61, 101652 (2020)
Lei, B., et al.: Deep and joint learning of longitudinal data for Alzheimer’s disease prediction. Patt. Recogn. 102, 107247 (2020)
Wang, S., Wang, H., Cheung, A.C., Shen, Y., Gan, M.: Ensemble of 3D densely connected convolutional network for diagnosis of mild cognitive impairment and. Deep Learn. Appl. 1098, 53 (2020)
Hu, S., Yuan, J., Wang, S.: Cross-modality synthesis from MRI to pet using adversarial U-Net with different normalization. In: 2019 International Conference on Medical Imaging Physics and Engineering (ICMIPE), pp. 1–5. IEEE (2019)
Yu, S., et al.: Multi-scale enhanced graph convolutional network for early mild cognitive impairment detection. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 228–237. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_23
Lee, M.H., Smyser, C.D., Shimony, J.S.: Resting-state fMRI: a review of methods and clinical applications. Am. J. Neuroradiol. 34(10), 1866–1872 (2013)
Cao, P., et al.: Generalized fused group lasso regularized multi-task feature learning for predicting cognitive outcomes in Alzheimers disease. Comput. Methods Programs Biomed. 162, 19–45 (2018)
Munsell, B.C., Wu, G., Gao, Y., Desisto, N., Styner, M.: Identifying relationships in functional and structural connectome data using a hypergraph learning method. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 9–17. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_2
Goodfellow, I., et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014)
Wang, S., et al.: Diabetic retinopathy diagnosis using multichannel generative adversarial network with semisupervision. IEEE Trans. Autom. Sci. Eng. 18(2), 574–585 (2020)
Yu, W., Lei, B., Ng, M.K., Cheung, A.C., Shen, Y., Wang, S.: Tensorizing GAN with high-order pooling for Alzheimer’s disease assessment. IEEE Trans. Neural Networks Learn. Syst. (2021)
Hu, S., Shen, Y., Wang, S., Lei, B.: Brain MR to PET synthesis via bidirectional generative adversarial network. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 698–707. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_67
Hu, S., Yu, W., Chen, Z., Wang, S.: Medical image reconstruction using generative adversarial network for Alzheimer disease assessment with class-imbalance problem. In: 2020 IEEE 6th International Conference on Computer and Communications (ICCC), pp. 1323–1327. IEEE (2020)
Lei, B., et al.: Skin lesion segmentation via generative adversarial networks with dual discriminators. Med. Image Anal. 64, 101716 (2020)
Dong, Y., Sawin, W., Bengio, Y.: HNHN: hypergraph networks with hyperedge neurons. arXiv preprint arXiv:2006.12278 (2020)
Tzourio-Mazoyer, N., et al.: Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15(1), 273–289 (2002)
Cui, Z., Zhong, S., Xu, P., Gong, G., He, Y.: Panda: a pipeline toolbox for analyzing brain diffusion images. Front. Hum. Neurosci. 7, 42 (2013)
Wang, J., Wang, X., Xia, M., Liao, X., Evans, A., He, Y.: Gretna: a graph theoretical network analysis toolbox for imaging connectomics. Front. Hum. Neurosci. 9, 386 (2015)
Yan, C., Zang, Y.: DPARSF: a MATLAB toolbox for “pipeline’’ data analysis of resting-state fMRI. Front. Syst. Neurosci. 4, 13 (2010)
Ashburner, J., et al.: Spm12 manual. wellcome trust centre for neuroimaging. London, UK (2014)
Jiang, J., Wei, Y., Feng, Y., Cao, J., Gao, Y.: Dynamic hypergraph neural networks. In: IJCAI, pp. 2635–2641 (2019)
Golkov, V., et al.: Q-space deep learning: twelve-fold shorter and model-free diffusion MRI scans. IEEE Trans. Med. Imaging 35(5), 1344–1351 (2016)
Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)
Gray, K.R., Aljabar, P., Heckemann, R.A., Hammers, A., Rueckert, D., Initiative, A.D.N., et al.: Random forest-based similarity measures for multi-modal classification of Alzheimer’s disease. Neuroimage 65, 167–175 (2013)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Veitch, D.P., et al.: Understanding disease progression and improving Alzheimer’s disease clinical trials: Recent highlights from the Alzheimer’s disease neuroimaging initiative. Alzheimer’s Dementia 15(1), 106–152 (2019)
Acknowledgement
This work was supported by the National Natural Science Foundations of China under Grant 61872351, the International Science and Technology Cooperation Projects of Guangdong under Grant 2019A050510030, the Distinguished Young Scholars Fund of Guangdong under Grant 2021B1515020019, the Excellent Young Scholars of Shenzhen under Grant RCYX20200714114641211 and Shenzhen Key Basic Research Project under Grant JCYJ20200109115641762.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Pan, J., Lei, B., Shen, Y., Liu, Y., Feng, Z., Wang, S. (2021). Characterization Multimodal Connectivity of Brain Network by Hypergraph GAN for Alzheimer’s Disease Analysis. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13021. Springer, Cham. https://doi.org/10.1007/978-3-030-88010-1_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-88010-1_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88009-5
Online ISBN: 978-3-030-88010-1
eBook Packages: Computer ScienceComputer Science (R0)