Interpretable Multimodality Embedding of Cerebral Cortex Using Attention Graph Network for Identifying Bipolar Disorder

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11766)


Bipolar Disorder (BP) is a mental disorder that affects 1–2% of the population. Early diagnosis and targeted treatment can benefit from associated biological markers (biomarkers). The existing methods typically utilize biomarkers from anatomical MRI or functional BOLD imaging but lack the ability of revealing the relationship between integrated modalities and disease. In this paper, we developed an Edge-weighted Graph Attention Network (EGAT) with Dense Hierarchical Pooling (DHP), to better understand the underlying roots of the disorder from the view of structure-function integration. EGAT is an interpretable framework for integrating multi-modality features without loss of prediction accuracy. For the input, the underlying graph is constructed from functional connectivity matrices and the nodal features consist of both the anatomical features and the statistics of the connectivity. We investigated the potential benefits of using EGAT to classify BP vs. Healthy Control (HC), by examining the attention map and gradient sensitivity of nodal features. We indicated that associated with the abnormality of anatomical geometric properties, multiple interactive patterns among Default Mode, Fronto-parietal and Cingulo-opercular networks contribute to identifying BP.


  1. 1.
    Baker, J.T., et al.: Disruption of cortical association networks in schizophrenia and psychotic bipolar disorder. JAMA Psychiatry 71(2), 109–118 (2014)CrossRefGoogle Scholar
  2. 2.
    Calhoun, V.D., Sui, J.: Multimodal fusion of brain imaging data: a key to finding the missing link(s) in complex mental illness. Biol. Psychiatry: Cogn. Neurosci. neuroimaging 1(3), 230–244 (2016)Google Scholar
  3. 3.
    Cao, B., Zhan, L., Kong, X., Yu, P.S., Vizueta, N., Altshuler, L.L., Leow, A.D.: Identification of discriminative subgraph patterns in fmri brain networks in bipolar affective disorder. In: Guo, Y., Friston, K., Aldo, F., Hill, S., Peng, H. (eds.) BIH 2015. LNCS (LNAI), vol. 9250, pp. 105–114. Springer, Cham (2015). Scholar
  4. 4.
    Daducci, A., et al.: The connectome mapper: an open-source processing pipeline to map connectomes with MRI. PLoS ONE 7(12), 1–9 (2012)CrossRefGoogle Scholar
  5. 5.
    Deppe, M., et al.: Increased cortical curvature reflects white matter atrophy in individual patients with early multiple sclerosis. NeuroImage: Clin. 6, 475–487 (2014)CrossRefGoogle Scholar
  6. 6.
    Fischl, B.: Freesurfer. Neuroimage 62(2), 774–781 (2012)CrossRefGoogle Scholar
  7. 7.
    Hibar, D., et al.: Cortical abnormalities in bipolar disorder: an MRI analysis of 6503 individuals from the enigma bipolar disorder working group. Mol. Psychiatry 23(4), 932 (2018)CrossRefGoogle Scholar
  8. 8.
    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
  9. 9.
    Li, X., Dvornek, N.C., Zhou, Y., Zhuang, J., Ventola, P., Duncan, J.S.: Efficient interpretation of deep learning models using graph structure and cooperative game theory: application to ASD biomarker discovery. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 718–730. Springer, Cham (2019). Scholar
  10. 10.
    Lim, K., et al.: Cortical gray matter deficit in patients with bipolar disorder. Schizophrenia Res. 40(3), 219–227 (1999)CrossRefGoogle Scholar
  11. 11.
    Liu, C.H., et al.: Regional homogeneity within the default mode network in bipolar depression: a resting-state functional magnetic resonance imaging study. PLoS ONE 7(11), e48181 (2012)CrossRefGoogle Scholar
  12. 12.
    Meunier, D., et al.: Modular and hierarchically modular organization of brain networks. Front. Neurosci. 4, 200 (2010)CrossRefGoogle Scholar
  13. 13.
    Öngür, D., et al.: Default mode network abnormalities in bipolar disorder and schizophrenia. Psychiatry Res. Neuroimaging 183(1), 59–68 (2010)CrossRefGoogle Scholar
  14. 14.
    Power, J.D., et al.: Functional network organization of the human brain. Neuron 72(4), 665–678 (2011)CrossRefGoogle Scholar
  15. 15.
    Sheffield, J.M., et al.: Fronto-parietal and cingulo-opercular network integrity and cognition in health and schizophrenia. Neuropsychologia 73, 82–93 (2015)CrossRefGoogle Scholar
  16. 16.
    Sporns, O.: Contributions and challenges for network models in cognitive neuroscience. Nat. Neurosci. 17(5), 652 (2014)CrossRefGoogle Scholar
  17. 17.
    Sui, J., et al.: Function-structure associations of the brain: evidence from multimodal connectivity and covariance studies. Neuroimage 102, 11–23 (2014)CrossRefGoogle Scholar
  18. 18.
    Tost, H., et al.: Prefrontal-temporal gray matter deficits in bipolar disorder patients with persecutory delusions. J. Affect. Disord. 120(1–3), 54–61 (2010)CrossRefGoogle Scholar
  19. 19.
    Veličković, P., et al.: Graph attention networks. In: ICLR (2018)Google Scholar
  20. 20.
    Wang, F., et al.: Functional and structural connectivity between the perigenual anterior cingulate and amygdala in bipolar disorder. Biol. Psychiatry 66(5), 516–521 (2009)CrossRefGoogle Scholar
  21. 21.
    Ying, Z., et al.: Hierarchical graph representation learning with differentiable pooling. In: NeurIPS (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina
  2. 2.Department of Biomedical EngineeringYale UniversityNew HavenUSA
  3. 3.Department of BioengineeringUniversity of PennsylvaniaPhiladelphiaUSA
  4. 4.Department of Radiology, West China HospitalSichuan UniversityChengduChina

Personalised recommendations