Multimodal Fusion of Structural and Functional Brain Imaging Data

  • Jing Sui
  • Vince D. CalhounEmail author
Part of the Neuromethods book series (NM, volume 119)


Recent years have witnessed a rapid growth of interest in moving functional magnetic resonance imaging (fMRI) beyond simple scan-length averages and into approaches that can integrate structural MRI measures and capture rich multimodal interactions. It is becoming increasingly clear that multimodal fusion is able to provide more information for individual subjects by exploiting covariation between modalities, rather an analysis of each modality alone. Multimodal fusion is a more complicated endeavor that must be approached carefully and efficient methods should be developed to draw generalized and valid conclusions out of high dimensional data with a limited number of subjects, such as patients with brain disorders. Numerous research efforts have been reported in the field based on various statistical models, including independent component analysis (ICA), canonical correlation analysis (CCA), and partial least squares (PLS). In this chapter, we survey a number of methods previously shown in multimodal fusion reports, performed with or without prior information, and with their possible strengths and limitations addressed. To examine the function–structure associations of the brain in a more comprehensive and integrated manner, we also reviewed a number of multimodal studies that combined fMRI and structural (sMRI and/or diffusion tensor MRI) measures, which could reveal important brain alterations that may not be fully detected by employing separate analysis of individual modalities, and also enable us to identify potential brain illness biomarkers.

Key words

Multimodal fusion methods Data driven Functional magnetic resonance imaging Structural MRI Diffusion MRI Independent component analysis Canonical correlation analysis 


  1. 1.
    Skudlarski P, Jagannathan K, Calhoun VD et al (2008) Measuring brain connectivity: diffusion tensor imaging validates resting state temporal correlations. Neuroimage 43(3):554–561PubMedPubMedCentralCrossRefGoogle Scholar
  2. 2.
    Savopol F, Armenakis C (2002) Mergine of heterogeneous data for emergency mapping: data integration or data fusion? Proc. ISPRSGoogle Scholar
  3. 3.
    Ardnt C, Loffeld O (1996) Information gained by data fusion, Proc. SPIE, vol. 2784 Google Scholar
  4. 4.
    Sui J, Adali T, Yu Q et al (2012) A review of multivariate methods for multimodal fusion of brain imaging data. J Neurosci Methods 204(1):68–81PubMedCrossRefGoogle Scholar
  5. 5.
    Sui J, Huster R, Yu Q et al (2014) Function-structure associations of the brain: evidence from multimodal connectivity and covariance studies. Neuroimage 102P1:11–23CrossRefGoogle Scholar
  6. 6.
    Schultz CC, Fusar-Poli P, Wagner G et al (2012) Multimodal functional and structural imaging investigations in psychosis research. Eur Arch Psychiatry Clin Neurosci 262(Suppl 2):S97–S106PubMedCrossRefGoogle Scholar
  7. 7.
    Horwitz B, Poeppel D (2002) How can EEG/MEG and fMRI/PET data be combined? Hum Brain Mapp 17(1):1–3PubMedCrossRefGoogle Scholar
  8. 8.
    Calhoun VD, Adali T (2009) Feature-based fusion of medical imaging data. IEEE Trans Inf Technol Biomed 13(5):711–720PubMedCrossRefGoogle Scholar
  9. 9.
    Blum AL, Langley P (1997) Selection of relevant features and examples in machine learning. Artif Intell 97(1–2):245–271CrossRefGoogle Scholar
  10. 10.
    Smith SM, Fox PT, Miller KL et al (2009) Correspondence of the brain’s functional architecture during activation and rest. Proc Natl Acad Sci U S A 106(31):13040–13045PubMedPubMedCentralCrossRefGoogle Scholar
  11. 11.
    Goldberg-Zimring D, Mewes AU, Maddah M et al (2005) Diffusion tensor magnetic resonance imaging in multiple sclerosis. J Neuroimaging 15(4 Suppl):68S–81SPubMedCrossRefGoogle Scholar
  12. 12.
    Ramnani N, Lee L, Mechelli A et al (2002) Exploring brain connectivity: a new frontier in systems neuroscience. Functional Brain Connectivity, 4–6 April 2002, Dusseldorf, Germany. Trends Neurosci 25:496–497PubMedCrossRefGoogle Scholar
  13. 13.
    Henson RN, Flandin G, Friston KJ et al (2010) A parametric empirical Bayesian framework for fMRI-constrained MEG/EEG source reconstruction. Hum Brain Mapp 31(10):1512–1531PubMedPubMedCentralCrossRefGoogle Scholar
  14. 14.
    Lemieux L (2004) Electroencephalography-correlated functional MR imaging studies of epileptic activity. Neuroimaging Clin N Am 14(3):487–506PubMedCrossRefGoogle Scholar
  15. 15.
    Friston KJ (2009) Modalities, modes, and models in functional neuroimaging. Science 326(5951):399–403PubMedCrossRefGoogle Scholar
  16. 16.
    De Martino F, Valente G, de Borst AW et al (2010) Multimodal imaging: an evaluation of univariate and multivariate methods for simultaneous EEG/fMRI. Magn Reson Imaging 28(8):1104–1112PubMedCrossRefGoogle Scholar
  17. 17.
    Eichele T, Specht K, Moosmann M et al (2005) Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proc Natl Acad Sci U S A 102(49):17798–17803PubMedPubMedCentralCrossRefGoogle Scholar
  18. 18.
    Astolfi L, Cincotti F, Mattia D et al (2004) Estimation of the effective and functional human cortical connectivity with structural equation modeling and directed transfer function applied to high-resolution EEG. Magn Reson Imaging 22(10):1457–1470PubMedCrossRefGoogle Scholar
  19. 19.
    Rykhlevskaia E, Gratton G, Fabiani M (2008) Combining structural and functional neuroimaging data for studying brain connectivity: a review. Psychophysiology 45(2):173–187PubMedCrossRefGoogle Scholar
  20. 20.
    Oakes TR, Fox AS, Johnstone T et al (2007) Integrating VBM into the General Linear Model with voxelwise anatomical covariates. Neuroimage 34(2):500–508PubMedCrossRefGoogle Scholar
  21. 21.
    Schlosser R, Gesierich T, Kaufmann B et al (2003) Altered effective connectivity during working memory performance in schizophrenia: a study with fMRI and structural equation modeling. Neuroimage 19(3):751–763PubMedCrossRefGoogle Scholar
  22. 22.
    Calhoun VD, Adali T, Kiehl KA et al (2006) A method for multitask fMRI data fusion applied to schizophrenia. Hum Brain Mapp 27(7):598–610PubMedPubMedCentralCrossRefGoogle Scholar
  23. 23.
    Correa N, Adali T, Calhoun VD (2007) Performance of blind source separation algorithms for fMRI analysis using a group ICA method. Magn Reson Imaging 25(5):684–694PubMedCrossRefGoogle Scholar
  24. 24.
    Groves AR, Beckmann CF, Smith SM et al (2011) Linked independent component analysis for multimodal data fusion. Neuroimage 54(3):2198–2217PubMedCrossRefGoogle Scholar
  25. 25.
    Sui J, He H, Pearlson GD et al (2012) Three-way (N-way) fusion of brain imaging data based on mCCA + jICA and its application to discriminating schizophrenia. Neuroimage 2(66):119–132Google Scholar
  26. 26.
    Sui J, Pearlson G, Caprihan A et al (2011) Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model. Neuroimage 57(3):839–855PubMedPubMedCentralCrossRefGoogle Scholar
  27. 27.
    Chen K, Reiman EM, Huan Z et al (2009) Linking functional and structural brain images with multivariate network analyses: a novel application of the partial least square method. Neuroimage 47(2):602–610PubMedPubMedCentralCrossRefGoogle Scholar
  28. 28.
    Martinez-Montes E, Valdes-Sosa PA, Miwakeichi F et al (2004) Concurrent EEG/fMRI analysis by multiway Partial Least Squares. Neuroimage 22(3):1023–1034PubMedCrossRefGoogle Scholar
  29. 29.
    Correa NM, Eichele T, Adali T et al (2010) Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI. Neuroimage 50(4):1438–1445PubMedPubMedCentralCrossRefGoogle Scholar
  30. 30.
    Liu J, Pearlson G, Windemuth A et al (2009) Combining fMRI and SNP data to investigate connections between brain function and genetics using parallel ICA. Hum Brain Mapp 30(1):241–255PubMedPubMedCentralCrossRefGoogle Scholar
  31. 31.
    Sui J, Adali T, Pearlson GD et al (2009) A method for accurate group difference detection by constraining the mixing coefficients in an ICA framework. Hum Brain Mapp 30(9):2953–2970PubMedPubMedCentralCrossRefGoogle Scholar
  32. 32.
    Sui J, Adali T, Pearlson GD et al (2009) An ICA-based method for the identification of optimal FMRI features and components using combined group-discriminative techniques. Neuroimage 46(1):73–86PubMedPubMedCentralCrossRefGoogle Scholar
  33. 33.
    Caprihan A, Pearlson GD, Calhoun VD (2008) Application of principal component analysis to distinguish patients with schizophrenia from healthy controls based on fractional anisotropy measurements. Neuroimage 42(2):675–682PubMedPubMedCentralCrossRefGoogle Scholar
  34. 34.
    Liu J, Xu L, Calhoun VD (2008). Extracting principle components for discriminant analysis of FMRI images. ICASSP 449–452Google Scholar
  35. 35.
    Ulloa AE, Chen J, Vergara VM et al (2014) Association between copy number variation losses and alcohol dependence across African American and European American ethnic groups. Alcohol Clin Exp Res 38(5):1266–1274PubMedPubMedCentralCrossRefGoogle Scholar
  36. 36.
    Calhoun VD, Adali T, Giuliani NR et al (2006) Method for multimodal analysis of independent source differences in schizophrenia: combining gray matter structural and auditory oddball functional data. Hum Brain Mapp 27(1):47–62PubMedCrossRefGoogle Scholar
  37. 37.
    Xu L, Pearlson G, Calhoun VD (2009) Joint source based morphometry identifies linked gray and white matter group differences. Neuroimage 44(3):777–789PubMedCrossRefGoogle Scholar
  38. 38.
    Franco AR, Ling J, Caprihan A et al (2008) Multimodal and multi-tissue measures of connectivity revealed by joint independent component analysis. IEEE J Sel Top Signal Process 2(6):986–997PubMedPubMedCentralCrossRefGoogle Scholar
  39. 39.
    Calhoun VD, Adali T, Liu J (2006) A feature-based approach to combine functional MRI, structural MRI and EEG brain imaging data. Conf Proc IEEE Eng Med Biol Soc 1:3672–3675PubMedGoogle Scholar
  40. 40.
    Teipel SJ, Bokde AL, Meindl T et al (2010) White matter microstructure underlying default mode network connectivity in the human brain. Neuroimage 49(3):2021–2032PubMedCrossRefGoogle Scholar
  41. 41.
    Eichele T, Calhoun VD, Debener S (2009) Mining EEG-fMRI using independent component analysis. Int J Psychophysiol 73(1):53–61PubMedPubMedCentralCrossRefGoogle Scholar
  42. 42.
    Xu L, Groth KM, Pearlson G et al (2009) Source-based morphometry: the use of independent component analysis to identify gray matter differences with application to schizophrenia. Hum Brain Mapp 30(3):711–724PubMedPubMedCentralCrossRefGoogle Scholar
  43. 43.
    Shenton ME, Dickey CC, Frumin M et al (2001) A review of MRI findings in schizophrenia. Schizophr Res 49(1–2):1–52PubMedPubMedCentralCrossRefGoogle Scholar
  44. 44.
    Correa NM, Li YO, Adali T et al (2008) Canonical correlation analysis for feature-based fusion of biomedical imaging modalities and its application to detection of associative networks in schizophrenia. IEEE J Sel Top Signal Process 2(6):998–1007PubMedPubMedCentralCrossRefGoogle Scholar
  45. 45.
    Li Y-O, Adali T, Wang W et al (2009) Joint blind source separation by multiset canonical correlation analysis. IEEE Trans Signal Process 57(10):3918–3929PubMedPubMedCentralCrossRefGoogle Scholar
  46. 46.
    Lin FH, McIntosh AR, Agnew JA et al (2003) Multivariate analysis of neuronal interactions in the generalized partial least squares framework: simulations and empirical studies. Neuroimage 20(2):625–642PubMedCrossRefGoogle Scholar
  47. 47.
    Krishnan A, Williams LJ, McIntosh AR et al (2010) Partial Least Squares (PLS) methods for neuroimaging: a tutorial and review. Neuroimage 56(2):455–475PubMedCrossRefGoogle Scholar
  48. 48.
    Correa NM, Adali T, Li YO et al (2010) Canonical correlation analysis for data fusion and group inferences: examining applications of medical imaging data. IEEE Signal Process Mag 27(4):39–50PubMedPubMedCentralCrossRefGoogle Scholar
  49. 49.
    McIntosh AR, Lobaugh NJ (2004) Partial least squares analysis of neuroimaging data: applications and advances. Neuroimage 23(Suppl 1):S250–S263PubMedCrossRefGoogle Scholar
  50. 50.
    Camara E, Rodriguez-Fornells A, Munte TF (2010) Microstructural brain differences predict functional hemodynamic responses in a reward processing task. J Neurosci 30(34):11398–11402PubMedCrossRefGoogle Scholar
  51. 51.
    Sui J, He H, Yu Q et al (2013) Combination of resting state fMRI, DTI, and sMRI data to discriminate schizophrenia by N-way MCCA + jICA. Front Hum Neurosci 7:235PubMedPubMedCentralCrossRefGoogle Scholar
  52. 52.
    Beckmann CF, Smith SM (2005) Tensorial extensions of independent component analysis for multisubject FMRI analysis. Neuroimage 25(1):294–311PubMedCrossRefGoogle Scholar
  53. 53.
    Hyvarinen A, Oja E (2000) Independent component analysis: algorithms and applications. Neural Netw 13(4–5):411–430PubMedCrossRefGoogle Scholar
  54. 54.
    Bell AJ, Sejnowski TJ (1995) An information-maximization approach to blind separation and blind deconvolution. Neural Comput 7(6):1129–1159PubMedCrossRefGoogle Scholar
  55. 55.
    Bishop CM (1999) Variational principal components. Artif Neural Netw 7:509, Conference Publication No. 470Google Scholar
  56. 56.
    Liu J, Chen J, Ehrlich S et al (2014) Methylation patterns in whole blood correlate with symptoms in schizophrenia patients. Schizophr Bull 40(4):769–776PubMedCrossRefGoogle Scholar
  57. 57.
    Segall JM, Allen EA, Jung RE et al (2012) Correspondence between structure and function in the human brain at rest. Front Neuroinform 6:10PubMedPubMedCentralCrossRefGoogle Scholar
  58. 58.
    Khullar S, Michael AM, Cahill ND et al (2011) ICA-fNORM: spatial normalization of fMRI data using intrinsic group-ICA networks. Front Syst Neurosci 5:93PubMedPubMedCentralCrossRefGoogle Scholar
  59. 59.
    Salgado-Pineda P, Junque C, Vendrell P et al (2004) Decreased cerebral activation during CPT performance: structural and functional deficits in schizophrenic patients. Neuroimage 21(3):840–847PubMedCrossRefGoogle Scholar
  60. 60.
    Salgado-Pineda P, Fakra E, Delaveau P et al (2011) Correlated structural and functional brain abnormalities in the default mode network in schizophrenia patients. Schizophr Res 125(2–3):101–109PubMedCrossRefGoogle Scholar
  61. 61.
    Skudlarski P, Jagannathan K, Anderson K et al (2010) Brain connectivity is not only lower but different in schizophrenia: a combined anatomical and functional approach. Biol Psychiatry 68(1):61–69PubMedPubMedCentralCrossRefGoogle Scholar
  62. 62.
    Schlosser RG, Nenadic I, Wagner G et al (2007) White matter abnormalities and brain activation in schizophrenia: a combined DTI and fMRI study. Schizophr Res 89(1–3):1–11PubMedCrossRefGoogle Scholar
  63. 63.
    Staempfli P, Reischauer C, Jaermann T et al (2008) Combining fMRI and DTI: a framework for exploring the limits of fMRI-guided DTI fiber tracking and for verifying DTI-based fiber tractography results. Neuroimage 39(1):119–126PubMedCrossRefGoogle Scholar
  64. 64.
    Koch K, Wagner G, Schachtzabel C et al (2011) Neural activation and radial diffusivity in schizophrenia: combined fMRI and diffusion tensor imaging study. Br J Psychiatry 198(3):223–229PubMedCrossRefGoogle Scholar
  65. 65.
    Kim J, Lee JH (2012) Integration of structural and functional magnetic resonance imaging improves mild cognitive impairment detection. Magn Reson Imaging 31:718–732PubMedCrossRefGoogle Scholar
  66. 66.
    Fan Y, Resnick SM, Wu X et al (2008) Structural and functional biomarkers of prodromal Alzheimer’s disease: a high-dimensional pattern classification study. Neuroimage 41(2):277–285PubMedPubMedCentralCrossRefGoogle Scholar
  67. 67.
    Wee CY, Yap PT, Zhang D et al (2011) Identification of MCI individuals using structural and functional connectivity networks. Neuroimage 59(3):2045–2056PubMedPubMedCentralCrossRefGoogle Scholar
  68. 68.
    Westlye LT, Walhovd KB, Bjornerud A et al (2009) Error-related negativity is mediated by fractional anisotropy in the posterior cingulate gyrus—a study combining diffusion tensor imaging and electrophysiology in healthy adults. Cereb Cortex 19(2):293–304PubMedCrossRefGoogle Scholar
  69. 69.
    Sui J, Yu Q, He H et al (2012) A selective review of multimodal fusion methods in schizophrenia. Front Hum Neurosci 6:27PubMedPubMedCentralCrossRefGoogle Scholar
  70. 70.
    Zhang H, Liu L, Wu H, Fan Y (2012) Feature selection and SVM classification of multiple modality images for predicting MCI, in OHBM, Beijing, China.Google Scholar
  71. 71.
    Kim DI, Sui J, Rachakonda S et al (2010) Identification of imaging biomarkers in schizophrenia: a coefficient-constrained independent component analysis of the mind multi-site schizophrenia study. Neuroinformatics 8(4):213–229PubMedPubMedCentralCrossRefGoogle Scholar
  72. 72.
    Tian L, Meng C, Yan H et al (2011) Convergent evidence from multimodal imaging reveals amygdala abnormalities in schizophrenic patients and their first-degree relatives. PLoS One 6(12):e28794PubMedPubMedCentralCrossRefGoogle Scholar
  73. 73.
    Casey BJ, Tottenham N, Liston C et al (2005) Imaging the developing brain: what have we learned about cognitive development? Trends Cogn Sci 9(3):104–110PubMedCrossRefGoogle Scholar
  74. 74.
    Smieskova R, Allen P, Simon A et al (2012) Different duration of at-risk mental state associated with neurofunctional abnormalities. A multimodal imaging study. Hum Brain Mapp 33(10):2281–2294PubMedCrossRefGoogle Scholar
  75. 75.
    Rasser PE, Johnston P, Lagopoulos J et al (2005) Functional MRI BOLD response to Tower of London performance of first-episode schizophrenia patients using cortical pattern matching. Neuroimage 26(3):941–951PubMedCrossRefGoogle Scholar
  76. 76.
    Fusar-Poli P, Broome MR, Woolley JB et al (2011) Altered brain function directly related to structural abnormalities in people at ultra high risk of psychosis: longitudinal VBM-fMRI study. J Psychiatr Res 45(2):190–198PubMedCrossRefGoogle Scholar
  77. 77.
    Michael AM, Baum SA, White T et al (2010) Does function follow form?: methods to fuse structural and functional brain images show decreased linkage in schizophrenia. Neuroimage 49(3):2626–2637PubMedCrossRefGoogle Scholar
  78. 78.
    Michael AM, King MD, Ehrlich S et al (2011) A data-driven investigation of gray matter-function correlations in schizophrenia during a working memory task. Front Hum Neurosci 5:71PubMedPubMedCentralGoogle Scholar
  79. 79.
    Rektorova I, Mikl M, Barrett J et al (2012) Functional neuroanatomy of vocalization in patients with Parkinson’s disease. J Neurol Sci 313(1–2):7–12PubMedCrossRefGoogle Scholar
  80. 80.
    Harms MP, Wang L, Csernansky JG et al (2012) Structure-function relationship of working memory activity with hippocampal and prefrontal cortex volumes. Brain Struct Funct 218(1):173–186PubMedCrossRefGoogle Scholar
  81. 81.
    Choi K, Yang Z, Hu X et al (2008) A combined functional-structural connectivity analysis of major depression using joint independent components analysis. Psychiatric MRI/MRS 3555Google Scholar
  82. 82.
    Camchong J, MacDonald AW III, Bell C et al (2011) Altered functional and anatomical connectivity in schizophrenia. Schizophr Bull 37(3):640–650PubMedCrossRefGoogle Scholar
  83. 83.
    Kim DJ, Park B, Park HJ (2012) Functional connectivity-based identification of subdivisions of the basal ganglia and thalamus using multilevel independent component analysis of resting state fMRI. Hum Brain Mapp 34:1371–1385PubMedCrossRefGoogle Scholar
  84. 84.
    Olesen PJ, Nagy Z, Westerberg H et al (2003) Combined analysis of DTI and fMRI data reveals a joint maturation of white and grey matter in a fronto-parietal network. Brain Res Cogn Brain Res 18(1):48–57PubMedCrossRefGoogle Scholar
  85. 85.
    Matthews SC, Strigo IA, Simmons AN et al (2011) A multimodal imaging study in U.S. veterans of Operations Iraqi and Enduring Freedom with and without major depression after blast-related concussion. Neuroimage 54(Suppl 1):S69–S75PubMedCrossRefGoogle Scholar
  86. 86.
    Zhou Y, Shu N, Liu Y et al (2008) Altered resting-state functional connectivity and anatomical connectivity of hippocampus in schizophrenia. Schizophr Res 100(1–3):120–132PubMedCrossRefGoogle Scholar
  87. 87.
    Yan H, Tian L, Yan J et al (2012) Functional and anatomical connectivity abnormalities in cognitive division of anterior cingulate cortex in schizophrenia. PLoS One 7(9):e45659PubMedPubMedCentralCrossRefGoogle Scholar
  88. 88.
    Soldner J, Meindl T, Koch W et al (2011) Structural and functional neuronal connectivity in Alzheimer’s disease: a combined DTI and fMRI study. Nervenarzt 83(7):878–887CrossRefGoogle Scholar
  89. 89.
    Wang F, Kalmar JH, He Y et al (2009) Functional and structural connectivity between the perigenual anterior cingulate and amygdala in bipolar disorder. Biol Psychiatry 66(5):516–521PubMedPubMedCentralCrossRefGoogle Scholar
  90. 90.
    Schonberg T, Pianka P, Hendler T et al (2006) Characterization of displaced white matter by brain tumors using combined DTI and fMRI. Neuroimage 30(4):1100–1111PubMedCrossRefGoogle Scholar
  91. 91.
    Voss HU, Schiff ND (2009) MRI of neuronal network structure, function, and plasticity. Prog Brain Res 175:483–496PubMedCrossRefGoogle Scholar
  92. 92.
    Palacios EM, Sala-Llonch R, Junque C et al (2012) White matter integrity related to functional working memory networks in traumatic brain injury. Neurology 78(12):852–860PubMedCrossRefGoogle Scholar
  93. 93.
    Jacobson S, Kelleher I, Harley M et al (2009) Structural and functional brain correlates of subclinical psychotic symptoms in 11–13 year old schoolchildren. Neuroimage 49(2):1875–1885PubMedCrossRefGoogle Scholar
  94. 94.
    Supekar K, Uddin LQ, Prater K et al (2010) Development of functional and structural connectivity within the default mode network in young children. Neuroimage 52(1):290–301PubMedPubMedCentralCrossRefGoogle Scholar
  95. 95.
    Pomarol-Clotet E, Canales-Rodriguez EJ, Salvador R et al (2010) Medial prefrontal cortex pathology in schizophrenia as revealed by convergent findings from multimodal imaging. Mol Psychiatry 15(8):823–830PubMedPubMedCentralCrossRefGoogle Scholar
  96. 96.
    Sexton CE, Allan CL, Le Masurier M et al (2012) Magnetic resonance imaging in late-life depression: multimodal examination of network disruption. Arch Gen Psychiatry 69(7):680–689PubMedCrossRefGoogle Scholar
  97. 97.
    Qiu MG, Ye Z, Li QY et al (2011) Changes of brain structure and function in ADHD children. Brain Topogr 24(3–4):243–252PubMedCrossRefGoogle Scholar
  98. 98.
    Sui J, He H, Liu J et al (2012) Three-way FMRI-DTI-methylation data fusion based on mCCA + jICA and its application to schizophrenia. Conf Proc IEEE Eng Med Biol Soc 2012:2692–2695PubMedGoogle Scholar
  99. 99.
    Zhang D, Wang Y, Zhou L et al (2011) Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage 55(3):856–867PubMedPubMedCentralCrossRefGoogle Scholar
  100. 100.
    Groves AR, Smith SM, Fjell AM et al (2012) Benefits of multi-modal fusion analysis on a large-scale dataset: life-span patterns of inter-subject variability in cortical morphometry and white matter microstructure. Neuroimage 63(1):365–380PubMedCrossRefGoogle Scholar
  101. 101.
    Sui J, Castro E, Hao H et al (2014) Combination of FMRI-SMRI-EEG data improves discrimination of schizophrenia patients by ensemble feature selection. The 36th Annual International Conference of the IEEE engineering in medicine and biology society (EMBC‘14) Chicago, Illinois, USA, no. August 26–30Google Scholar
  102. 102.
    Eichele T, Calhoun VD, Moosmann M et al (2008) Unmixing concurrent EEG-fMRI with parallel independent component analysis. Int J Psychophysiol 67(3):222–234PubMedCrossRefGoogle Scholar
  103. 103.
    Haller S, Xekardaki A, Delaloye C et al (2011) Combined analysis of grey matter voxel-based morphometry and white matter tract-based spatial statistics in late-life bipolar disorder. J Psychiatry Neurosci 36(6):391–401PubMedPubMedCentralCrossRefGoogle Scholar
  104. 104.
    Chen Z, Cui L, Li M et al (2011) Voxel based morphometric and diffusion tensor imaging analysis in male bipolar patients with first-episode mania. Prog Neuropsychopharmacol Biol Psychiatry 36(2):231–238PubMedCrossRefGoogle Scholar
  105. 105.
    Meda SA, Jagannathan K, Gelernter J et al (2010) A pilot multivariate parallel ICA study to investigate differential linkage between neural networks and genetic profiles in schizophrenia. Neuroimage 53(3):1007–1015PubMedCrossRefGoogle Scholar
  106. 106.
    Jagannathan K, Calhoun VD, Gelernter J et al (2010) Genetic associations of brain structural networks in schizophrenia: a preliminary study. Biol Psychiatry 68(7):657–666PubMedPubMedCentralCrossRefGoogle Scholar
  107. 107.
    Jamadar S, Powers NR, Meda SA et al (2010) Genetic influences of cortical gray matter in language-related regions in healthy controls and schizophrenia. Schizophr Res 129:141–148CrossRefGoogle Scholar
  108. 108.
    Hao X, Xu D, Bansal R et al (2011) Multimodal magnetic resonance imaging: the coordinated use of multiple, mutually informative probes to understand brain structure and function. Hum Brain Mapp 34(2):253–271PubMedPubMedCentralCrossRefGoogle Scholar
  109. 109.
    Fusar-Poli P, McGuire P, Borgwardt S (2011) Mapping prodromal psychosis: a critical review of neuroimaging studies. Eur Psychiatry 27(3):181–191PubMedCrossRefGoogle Scholar
  110. 110.
    Meda SA, Gill A, Stevens MC et al (2012) Differences in resting-state functional magnetic resonance imaging functional network connectivity between schizophrenia and psychotic bipolar probands and their unaffected first-degree relatives. Biol Psychiatry 71(10):881–889PubMedPubMedCentralCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Brainnetome Center and National Laboratory of Pattern Recognition, Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.The Mind Research Network and Lovelace Biomedical and Environmental Research InstituteAlbuquerqueUSA
  3. 3.Electrical and ComputerUniversity of New MexicoAlbuquerqueUSA
  4. 4.Departments and NeurosciencesUniversity of New MexicoAlbuquerqueUSA

Personalised recommendations