Abstract
Mapping brain-behaviour associations is paramount to understand and treat psychiatric disorders. Standard approaches involve investigating the association between one brain and one behavioural variable (univariate) or multiple variables against one brain/behaviour feature (‘single’ multivariate). Recently, large multimodal datasets have propelled a new wave of studies that leverage on ‘doubly’ multivariate approaches capable of parsing the multifaceted nature of both brain and behaviour simultaneously. Within this movement, canonical correlation analysis (CCA) and partial least squares (PLS) emerge as the most popular techniques. Both seek to capture shared information between brain and behaviour in the form of latent variables. We provide an overview of these methods, review the literature in psychiatric disorders, and discuss the main challenges from a predictive modelling perspective. We identified 39 studies across four diagnostic groups: attention deficit and hyperactive disorder (ADHD, k = 4, N = 569), autism spectrum disorders (ASD, k = 6, N = 1731), major depressive disorder (MDD, k = 5, N = 938), psychosis spectrum disorders (PSD, k = 13, N = 1150) and one transdiagnostic group (TD, k = 11, N = 5731). Most studies (67%) used CCA and focused on the association between either brain morphology, resting-state functional connectivity or fractional anisotropy against symptoms and/or cognition. There were three main findings. First, most diagnoses shared a link between clinical/cognitive symptoms and two brain measures, namely frontal morphology/brain activity and white matter association fibres (tracts between cortical areas in the same hemisphere). Second, typically less investigated behavioural variables in multivariate models such as physical health (e.g., BMI, drug use) and clinical history (e.g., childhood trauma) were identified as important features. Finally, most studies were at risk of bias due to low sample size/feature ratio and/or in-sample testing only. We highlight the importance of carefully mitigating these sources of bias with an exemplar application of CCA.
Similar content being viewed by others
Introduction
Mapping the association between brain and behaviour has been a focus of psychiatric research for many decades. However, the analytical methods to achieve this have changed substantially over time. Historically, mass-univariate analysis, where many single brain features are linked to a single behavioural phenotype, have dominated the literature [1,2,3,4]. However, there is growing concern about the reproducibility of these findings, especially when carried out in typically small samples [5, 6]. This approach also does not take into account the mutual dependencies between different brain regions, nor is it consistent with the current view that behaviour is best explained by distributed neural networks, rather than localised regions [7,8,9,10]. Therefore, multivariate methods are better suited to capture brain–behaviour relationships. Within this context, two main families of methods are becoming increasingly popular: 1) mapping multiple brain features to one behavioural feature (i.e., ‘many-to-one’ associations) and 2) discovering latent brain-behaviour associations from multiple brain and multiple behaviour features (i.e., ‘many-to-many’ associations or ‘doubly multivariate’ methods). The former has enjoyed considerable interest in the last two decades, with a wealth of studies investigating how neural features can predict a range of univariate psychiatric outcomes such as functioning [11, 12], diagnosis [13,14,15] and response to treatment [16, 17] using popular approaches such as support vector machine (SVM) [18,19,20]. Within the second group, there are many approaches that could be used in principle, such as independent component analysis (ICA) and its variants (e.g., parallel ICA, joint ICA or linked ICA) [21], multilevel clustering [22], canonical correlation analysis (CCA) [23] and partial least squares [24] (PLS). The latter two emerge as the most established and popular techniques in brain-behaviour studies, as evidenced by several recent studies in the general population [25] and tutorials tailored to brain-behaviour investigations [26, 27]. There are many variations of CCA and PLS [27]. The two most used types of PLS in neuroimaging are Partial Least Squares Correlation (PLSC) and Partial Least Squares Regression (PLSR). PLSC is a correlational technique that estimates associations between two sets of data (e.g., behaviour and brain morphology), while PLSR is a regression technique that predicts one set of data from another (e.g., predicts behaviour from brain activity). Multiset CCA (mCCA) and mCCA+ICA are variations of CCA and have mostly been used to investigate the association between different imaging modalities or incorporate a behavioural constraint (use behavioural data to guide the fusion between imaging modalities) [28,29,30,31]. Here we focus on standard CCA and PLSC (henceforth PLS) (and their regularised variations) as they are the most commonly used approaches to investigate brain-behaviour associations [25, 32]. Both seek to capture shared information, in the form of latent variables, between two sets of multivariate data. Early applications in neuroimaging aimed at combining different imaging modalities, a process also known as multimodal fusion [33]. More recently, the same approach is being used to map brain and behaviour associations. This is achieved by attributing weights to the brain and behavioural variables such that their linear combination maximises the correlation (CCA) or covariance (PLS) between the resulting latent variables (see Box 1 for an overview; for a more in-depth explanation see Tabachnick et al., [34]). The main advantage of this approach is that the multivariate nature of both brain and behaviour can be modelled simultaneously. Indeed, although the standard use of ‘many-to-one’ approaches in psychiatric imaging acknowledges the multifaceted nature of brain data, they do not take into account that different aspects of behaviour also interact with each other [35]. Therefore, approaches such as CCA and PLS are better suited to honour the complex brain-behaviour associations by allowing to uncover joint multivariate relationships.
Although CCA/PLS were introduced many decades ago, they are ´data-hungry’ and can be computationally demanding. The release of large publicly available imaging datasets with comprehensive behavioural assessments, such as the Adolescent Brain Cognitive Development [36], Human Connectome Project [37] and UK BioBank [38], has propelled a renewed interest in the use of these methods to investigate brain-behaviour associations [39,40,41,42,43]. For example, specific brain patterns have been linked to perinatal and early life events, sociocognition, and urbanicity [44,45,46], different types of psychopathology [47], as well as physical features, lifestyle and cognition [48]. Notably, these methods may be particularly useful in psychiatry. Mental health disorders are best understood as a complex pattern of both neural and behavioural changes. For example, psychosis is characterised by widespread changes in brain morphology, structure and connectivity [49,50,51], but also by a pattern of positive, negative, mood and cognitive symptoms, insight and functioning, all relevant for diagnosis and treatment [52]. However, despite the consensus of the multivariate brain-behaviour interaction in psychiatric disorders, these studies remain rare; multimodal data is expensive and requires expertise in several data modalities (e.g., neuroimaging, cognition, psychopathology) as well as in the analytical methods to integrate them. Parallel to this, there is also a growing interest in investigating the generalisability and stability of brain-behaviour associations in psychiatry [27, 53, 54]. This is partly in response to the replication crisis in psychiatry/psychology [25] combined with the pursuit of translational clinical tools [55]. In practice, this means that the typical descriptive analytical framework, where the aim is to find above-chance associations in-sample, is being challenged and more emphasis is being given to predictive approaches, where the goal is to test whether a brain-behaviour association is stable and can be generalised to new data [56, 57]. The aim of this systematic review is to summarise the applications of commonly used ‘doubly’ multivariate methods, namely CCA and PLS, to investigate brain-behaviour associations in psychiatric disorders within this context, as well as discuss main challenges and future directions.
Brain-behaviour associations in psychiatric disorders
Our systematic review identified 39 studies that investigated brain-behaviour associations in mental health disorders using CCA/PLS (see supplementary methods for search strategy and data extraction) across four diagnostic groups: four in attention deficit and hyperactivity disorder (ADHD, N = 569), six in autism spectrum disorders (ASD, N = 1731), five in major depressive disorder (MDD; N = 938) and thirteen in psychosis, including three in individuals at clinical high-risk (CHR-P, N = 278), five in first episode (FEP, N = 455) and five in established schizophrenia (SZ, N = 417). An additional eleven studies were carried out in large samples including several diagnoses (Transdiagnostic; N = 5731) including SZ, bipolar disorder, ADHD and ASD. Twenty-six studies (67%) used CCA (or regularised CCA). The main characteristics of the included studies are summarised in Tables S2–S6. Figure 1 shows the brain-behaviour associations that were investigated for each diagnostic group and total sample, while Fig. 2 displays, for the same groups, the frequency of statistically significant associations. Overall, studies focused mostly on the association between either brain morphology, resting-state functional connectivity or fractional anisotropy against symptoms and/or cognition. A wide range of additional behavioural domains were also investigated, namely demographics, clinical information and history as well as physical health (Fig. 1).
Behaviour was associated with widespread regions, networks and white matter tracts across the brain for all diagnostic groups. In ADHD, cognition and symptoms were mostly linked with the default mode and frontoparietal networks and white matter association fibres, whereas in ASD, symptoms were mostly associated with the morphology of frontal and subcortical regions, as well as white matter association fibres. Similarly, depressive symptoms in MDD were also associated mostly with frontal regions but also with limbic regions and the cingulate cortex. In psychosis, all types of behavioural features were more visibly associated with frontal regions, followed by temporal and subcortical regions. Finally, the prevalence of frontal regions was also clear in the transdiagnostic group, especially in the association with cognition; white matter association fibres were also linked with cognition and symptoms. Notably, there was no clear association between specific patterns of functional connectivity and behaviour (Fig. 2). In the next sections, the brain-behaviour associations for each diagnostic group are presented in more detail.
Attention deficit and hyperactivity disorder
Four studies investigated the association between ADHD symptoms/ general cognition and either cortical gyrification [58], resting-state functional connectivity [59, 60] or fractional anisotropy [61]. The former also included measurements of facial dysmorphology typically associated with pre-natal alcohol exposure alongside full scale IQ, behavioural, emotional and cognitive regulation, and inattention and hyperactivity/impulsivity scores. Results showed one significant latent variable, in which lower gyrification in prefrontal, insular, cingulate, temporal, and parietal cortices was associated with greater facial dysmorphia and hyperactivity/impulsivity, as well as poorer IQ and behavioural regulation [58]. Lin et al. [59] found one significant canonical mode where higher hyperactivity/impulsivity and lower cognitive function were associated with higher connectivity predominantly between the default mode, frontoparietal, cingulo-opercular (e.g., insula and thalamus) and subcortical (e.g., pallidum and putamen) networks. Luo et al. [60] also found several canonical links between several cognitive domains, ADHD and other symptoms (e.g., depression, anxiety, social difficulties) with the default mode and frontoparietal networks. Finally, Tsai et al. [61] reported one significant canonical mode, where higher white matter microstructural integrity in multiple brain systems, mostly in association fibres (e.g., cingulum, inferior fronto-occipital, uncinate and longitudinal fasciculi), was negatively associated with attention dysregulation, aggression, anxiety, depressive and ADHD symptoms, and positively associated with cognitive function.
Autism spectrum disorders
The only three studies mostly investigated the association between different ASD symptoms including difficulty in social interaction and communication, stereotyped behaviour and sensory processing against brain morphology [62, 63], functional connectivity [64, 65] and white matter integrity [66, 67]. In a large sample of 325 participants, Mei et al. [62] found one significant canonical mode, in which stronger negative weights on social communication difficulties and stereotyped behaviour were associated with higher weights on grey matter volume in the lateral occipital and superior parietal lobes, right precentral gyrus, amygdala, hippocampus and parahippocampal gyrus and positively with grey matter volume in the thalamus and putamen. In a separate CCA, the same study also found that higher frequency of sensory symptoms was associated with less volume in the cerebellum, lateral occipital and parietal lobes, precentral and inferior frontal gyri and middle frontal lobe. Using the same sample, Mei et al. [66] expanded on these findings by combining grey matter volume and white matter microstructure as input features along with core ASD symptoms. The only significant canonical mode was dominated by white matter features in the inferior and superior longitudinal fasciculus, inferior fronto-occipital fasciculus, corticospinal tract and thalamic radiation, which was associated with repetitive and stereotyped behaviour. Similar tracts, including the inferior longitudinal and uncinate fasciculi, corpus callosum, corticospinal and frontal aslant tracts, thalamic radiation and cingulum were also associated with affect, behavioural, and cognitive dysregulation in Ni et al. [67]. Another study attempted to parse brain-behaviour heterogeneity and found four subtypes in which social difficulties correlated mostly with either fronto-parietal regions, brainstem, temporal-occipital or fronto-subcortical regions [63]. As for functional connectivity, repetitive and stereotyped behaviour as well as social difficulties were associated mainly with the visual network [64, 65].
Major depressive disorder
Overall, four studies covered a wide range of behavioural features, namely depressive and anxiety symptoms, personality traits and childhood trauma, in association with either brain morphology or resting-state functional connectivity. Drysdale et al. [68] found two clinical profiles – anhedonia & psychomotor retardation and anxiety & insomnia – associated with different functional connectivity networks. The former profile was mostly associated with subcortical structures, as well as the default mode and salience networks, whereas the latter was associated almost exclusively with subcortical regions, especially limbic structures. A similar study with a more comprehensive set of behavioural features (anxiety, depression, personality traits, childhood trauma) found a significant association between functional connectivity in similar networks, especially between subcortical structures, the dorsal attention, sensorimotor, ventral attention networks and childhood trauma [69]. When investigating the association of the same behavioural features with brain morphology, anxious misery was negatively associated with grey matter volume in several cortical and subcortical structures including the middle cingulate and frontal gyri, frontal pole, bilateral amygdala, right hippocampal and superior frontal gyri [70]. Conversely, positive personality traits were positively associated with grey matter volume in the central operculum, posterior and middle cingulate gyri, hippocampus, and amygdala, whereas childhood trauma was negatively associated with grey matter volume in the entorhinal cortex and left hippocampal gyrus. Finally, somatisation was associated with functional connectivity in a widespread group of brain regions including the occipital, temporal and frontal lobes as well as the cerebellum.
Psychosis
Thirteen studies investigated the association between either brain morphology, resting-state connectivity or FA against a wide range of cognitive and clinical measures including, amongst others, symptoms, functioning, duration of untreated illness and antipsychotic medication. Three studies did not find a statistically significant brain-behaviour association [71,72,73]. Of the remaining studies, frontal brain regions were more prominently involved in all behavioural domains investigated, followed by temporal and subcortical regions. Symptoms were associated with a wide range of connectivity networks, but mostly with the default mode and sensorimotor networks. As for FA, links between association fibres and cognition, symptoms and clinical information were also observed. In CHR-P, prodromal symptoms and poor functioning were associated with white matter microstructure in association fibres (sagittal stratum, uncinate fasciculus and cingulum) and in the anterior corona radiata [74]. Conversely, cognitive abilities including processing speed, working memory and executive functioning were mostly associated with association fibres (fornix and uncinate fasciculus) and brainstem tracts (superior cerebellar peduncle and medial lemniscus) [75]. In FEP, the association between several behavioural features (e.g., cognition, symptom severity, medication, education) and cortical thickness and hippocampal white matter revealed two sex-specific latent variables. In males, greater verbal memory impairments, fewer years of education, greater antipsychotic medication and positive/negative symptomatology were associated with reduced volumes in limbic structures, reduced thickness in mostly parietal and temporal regions, and higher thickness in several frontal regions and in the precuneus. In females, fewer depressive/negative symptoms and an earlier age of onset were associated with reduced volumes in limbic structures, low thickness in temporal and frontal regions and in the cingulate cortex, as well as higher thickness in parietal and occipital regions. In established SZ, cognitive [76, 77] and negative [76] symptoms were associated with grey matter volume in a widespread network of predominately prefrontal, temporal, parietal and (para)limbic regions. A significant association was also found between psychotic symptoms and more time in states characterised by inactive default mode and executive networks, and with heightened activity in the sensorimotor network [78].
Transdiagnostic groups
Several studies opted for combining diagnostic groups to identify transdiagnostic biomarkers. Half of the studies focused on psychosis spectrum disorders, namely schizophrenia, schizoaffective disorder and bipolar disorder (with and without psychotic features). The remaining studies investigated mostly a combination of ADHD, ASD as well as mood, anxiety, and borderline personality disorder. The majority of studies investigated cognition, which was mainly associated with the morphology of frontal regions. For example, the superior, inferior, caudal and middle frontal cortices were amongst the features with highest loadings across several brain morphology features, namely grey matter volume, cortical thickness, cortical surface area and local gyrification index in the psychosis spectrum group, ADHD, mood and anxiety disorders [79, 80]. As for white matter microstructure, cognition was mostly associated with fractional anisotropy in the superior longitudinal fasciculus, sagittal stratum and fornix [81, 82]. Interestingly, in the only two studies with physical measures amongst a wide range of behavioural features [41, 80], variables such as BMI, drug use and physical activity had the highest loadings across brain modalities, namely morphology, white matter microstructure and functional activity, as well as across brain regions. Overall, psychopathology (e.g., psychosis, anxiety) was associated with a wide range of brain modalities and regions, as expected due to the different diagnostic groups. However, only five studies investigated the association between brain and symptoms, all with different diagnoses and/or brain features, making it difficult to ascertain which associations were more prevalent in the different diagnostic groups/symptoms. Notably, three out of the four functional connectivity studies did not find any statistically significant brain-behaviour association [41, 83, 84]; however, they were also amongst the most robust ones due to their rigorous out-of-sample testing.
Risk of bias
To our knowledge, there are no established tools to systematically assess the risk of bias in ‘doubly’ multivariate brain-behaviour studies. A subjective judgment was made for each study based on three domains that have been consistently highlighted in brain-behaviour studies using these methods: 1) Data dimensionality, 2) Validation strategy, 3) Stability of feature weights. The former was operationalised as the ratio between sample size (N) and total number of brain and behaviour features entered into the CCA/PLS model (i.e., effective number of variables, f); an N/f ratio lower than 10 was considered high risk bias [34]. In terms of validation strategy, in-sample testing only and out-of-sample testing (through resampling or an independent dataset) were considered high and low risk of bias, respectively [85]. Finally, studies that did not investigate the stability of feature loadings were rated as high bias for the last domain [85]. If any of the information needed to rate the domains was unclear or missing, the study was rated as ‘unclear’. Most studies (80%) were at high risk of bias for Data dimensionality and 68% were also at high risk due to an in-sample Validation strategy (Fig. 3, supplementary information).
Generalisability and stability of brain-behaviour associations
The majority of included studies used a descriptive approach to investigate brain-behaviour associations, that is, CCA/PLS models were trained and tested in the same sample. It is not possible to know whether the same effect size would have held (generalisability) and/or whether the weights would have remained similar (stability) when tested in new data. This is further exacerbated by the fact that most studies had small sample sizes relative to the total number of brain-behaviour features. Although many studies used feature selection/dimensionality reduction to address this, it may not have been enough to mitigate the risk of overfitting as most of them still did not meet the recommended guidelines of 10 participants/feature minimum [34]. More stringent thresholds have been suggested including 42 [86] and 50 [85], indicating that the risk of bias may be even greater. Additionally, very few studies used an independent sample to test their findings. This is a more demanding generalisation than resampling, further highlighting the chances of overfitting. The association between small sample sizes and optimistic performance in predictive modelling is well-established [6, 87]; it can also be observed here (Fig. 4A) and in similar studies in healthy controls [85]. The two limitations, especially when combined, are problematic because CCA and PLS are already prone to overfitting; i.e., brain-behaviour associations tend to be much stronger in the sample where the model is trained/fitted (i.e., in-sample testing) compared to the associations in a new and unseen sample where the same trained model is applied (i.e., out-of-sample testing) [54]. To illustrate how this can lead to biased latent brain-behaviour associations, we present an exemplar standard CCA between cortical thickness (68 original features; 5 after dimensionality reduction) and cognition (11 features) in the Human Connectome Project (HCP) dataset (N = 842, N/f = 52.6) and compare the i) in- and out-of-sample first canonical correlation and ii) weights across models with increasing (i.e., more favourable) N/f ratio (Fig. 4B; see supplementary methods). The former investigates how the first (i.e., strongest) canonical correlation of a CCA model trained on a subsample of the HCP generalises to another subsample of the same dataset; the latter assesses the stability of model weights; i.e., to what extent feature importance varies across multiple models. Ideally, in- and out-of-sample effect sizes should be similar and weights stable across models. However, as expected, for small N/f ratios, there was a clear discrepancy between the means of the in- and out-of-sample effect sizes, and the latter varied substantially, a strong indicator of poor generalizability. As the N/f ratio increased, the effect sizes began to converge, suggesting a more generalizable model. In the total sample, the mean canonical correlation in- and out-of-sample nearly converged at 0.28 (sd=0.02) and 0.21 (sd=0.06), respectively (Fig. 4C). This is in line with more realistic effect sizes for brain-behaviour associations [6]. The stability of the weights (average cosine similarity between feature weights of all 100 iterations for a given N/f ratio; see supplementary methods) follows a similar pattern (Fig. 4D). The weights from models trained in small ratios varied substantially between models (i.e., low cosine similarly) suggesting poor stability of feature importance. For the total sample, the cosine similarity was 0.79, indicating a good weight stability. However, the plot suggests that this could be further improved with a larger sample, which is consistent with more recent suggestions that a minimum N/f ratio of 50 is needed for a stable CCA [85]. All combined, these results raise important questions about the validity and reliability of most of the published studies and highlight the need for careful testing to avoid inflated/unstable results. Although some studies used feature selection, dimensionality reduction and/or regularised CCA/PLS, it may not have been enough to mitigate the risk of overfitting. Future studies should follow guidelines for out-of-sample testing, investigating and reporting weights stability and mitigating the effects of high-dimensionality [27, 85, 88].
Interpretation of brain-behaviour associations in psychiatry
Interpretability of multivariate models is paramount to understand the underlying mechanisms of psychiatric disorders. A strong and robust association which provides no information about its underlying brain and behavioural features would be of limited clinical utility. The interpretation of CCA/PLS models is typically done by inspecting the model’s weights or loadings; the larger the absolute weight/loading, the more a variable contributed to the overall association (Box 1). Importantly, a reliable interpretation of brain-behaviour association requires valid model weights. As shown in Fig. 4D, this may be difficult to secure in small samples. Several studies investigated the stability of weights via bootstrapping and quantified the variability of the weights for each individual variable across iterations. This is a useful strategy as it allows to estimate a confidence interval for each variable weight; however, a more appropriate approach for ’doubly’ multivariate methods would be to report the variability of the entire set of variables’ weights across iterations, using for example cosine similarity. This gives a better estimate of the stability of the entire model’s solution, rather than individual variables. Notably, studies varied in how the most contributing features were identified (e.g., top 25% or 5 features, statistically significant features, features with weights above a certain threshold). This is an important limitation of the literature (and by extension, of this review) as it hinders comparisons between studies. In addition, the term ‘brain-behaviour’ should not be confused with any directionality whereby neural features predict behaviour. CCA and PLS-correlation are symmetrical in nature; i.e., there is no directionality in covariance and correlation. Although sometimes the terms independent/predictor variables and dependent/criterion variables may be used to describe the brain and behaviour datasets, the order of datasets is ultimately arbitrary [34]. Therefore, the interpretation of associations using these methods should be limited to inferring shared information between brain and behavioural features. Finally, key characteristics of behavioural data and how they can impact the strength and interpretability of associations are less discussed than imaging in brain-behaviour studies [89]. Notably however, CCA/PLS are sensitive to associations both between and within datasets. In practice, this means that small changes in the behavioural data (either due to the choice and/or quality of variables) may markedly alter the composition of the latent variables and ultimately the strength and interpretation of brain-behaviour associations [34]. Some of the included studies focused on symptoms or cognition, while others used different combinations of either/both plus demographics, clinical information and history, and/or physical health. However, the choice of behavioural variable matters; for example, cognition has been shown to have more reliable associations with neural features compared to other behavioural variables such as parent-reported behavioural measures [90]. Similarly, a wider range with less inter-correlated behavioural variables may result in a more diluted brain-behaviour association. Additionally, testing less investigated psychometric properties in psychiatric neuroimaging such as the ability to measure high/low levels and unique/shared variance of the behavioural phenotype (phenotypic resolution and complexity) as well as ability to measure the same construct across different groups (measurement invariance) may help increase the effect size and interpretability of current reliable brain-behaviour associations [89]. Going forward, studies should report key psychometric properties of behavioural measurements (e.g., via item-response theory) in their sample in order to increase transparency of the quality of behavioural data. Very few studies have directly investigated the impact of different imaging modalities on brain-behaviour associations [27, 54]. This limited evidence suggests that the behaviour of CCA/PLS is similar between grey matter volume and structural/functional connectivity. However, it is reasonable to expect that modalities/pre-processing pipelines that result in more cohesive (i.e., inter-correlated) data across participants will contribute to stronger brain-behaviour associations, much like in behavioural data.
Prediction of clinical outcomes based on brain-behaviour associations
Most studies used either CCA or PLS as the main approach, that is, the aim of the study was to investigate brain-behaviour associations. Although an interesting aim in itself, these methods can also be used to extract low-dimensional brain-behaviour patterns to be inputted into a machine learning model to predict clinical outcomes, such as relapse or response to treatment; this initial step is known as early fusion [91]. As large brain-behaviour datasets become increasingly accessible, the use CCA/PLS as an early fusion method will likely become more popular. For example, functional connectivity signatures of cognitive performance or symptoms have been used as input for clustering analysis to find subgroups in psychosis [92], ASD [64] and depression [68]. Notably, in a rare effort to replicate findings, the results from the latter study were not reproduced [84]. In the original study, a subset of brain-behaviour features was selected as input for CCA if they were correlated in the entire sample. The replication study addressed this (and other similar limitations) by including the feature selection step and the CCA in the same cross-validation scheme to ensure that both steps were tested for generalisability. Failure to do this is a common source of bias in machine learning studies that leads to optimistic findings [15, 93, 94]. In line with our exemplar CCA application, these two studies highlight the need and the complexity of implementing rigorous out-of-sample testing in brain-behaviour studies. Early fusion could also be used in conjunction with more popular supervised techniques for classification (e.g., SVM) or regression (e.g., support vector regression) to predict clinical outcomes of interest such as conversion to illness and severity of symptoms. Another possible application is to model brain-behaviour associations in a reference cohort (e.g., healthy controls), map patients against this reference model and quantify the deviation from the expected brain-behaviour interaction [95]. Given the known heterogeneity in both neural and behavioural presentations in psychiatric disorders, their interaction is also likely to be heterogeneous [63]. This approach has the advantage of allowing patients to deviate from the expected brain-behaviour interaction in their own unique way, and as such may be a promising approach to model complex and heterogeneous brain-behaviour associations in psychiatry. It would also be interesting to expand on brain-behaviour predictors to other relevant domains such as brain-behaviour-omics associations. This could be achieved via multiset CCA (mCCA), an extension of CCA capable of modelling latent associations between three or more sets of multivariate data [96,97,98]. Multi-level clustering [99] is another promising approach whereby patients are grouped based on multiple domains, e.g., brain and behaviour, simultaneously. For example, Dwyer et al. found a subset of early psychosis patients with low functioning and reduced brain volume [22]. Future studies could investigate how membership of different brain-behaviour subtypes is associated with prognosis. Finally, multi-level networks, where brain and behaviour associations are explored by modelling the links between brain and behaviour networks, are also a promising way forward [100] For example, Hilland et al. developed a joint network of depression-related brain structures and individual depression symptoms [101].
What to use when
A natural question is which approach should be used: CCA or PLS? Theoretically speaking, the difference lies in the quantity that is being optimized (covariance for PLS versus correlation for CCA). In CCA, cross-modality relationships have been normalized by within-block variance; in other words, dependencies between input variables, on either the brain or the behavior side, have been factored out. For this reason, the optimization tends to emphasize unique contributions that variables from the first set make to the prediction of variables within the second. In contrast, collective contributions across different input variables (that is, redundant relationships) are emphasized with PLS [102]. In this review most studies (67%) used CCA; this may be because CCA is an older method that derived from the well-established and popular general linear model, whereas PLS is more recent. Experiments investigating the differences between the two have found that CCA and PLS results are most similar when the correlations within each dataset are low; however, the reliability of CCA worsens with high correlations within either dataset [102]. Notably, PLS has more stable weights in smaller samples [90], however they are more biased and larger samples are needed to have unbiased results compared to CCA [54]. Beyond these findings, knowledge is limited regarding whether other characteristics of the data influence the choice of method. Moreover, given that the stability and reliability of both methods varies with type of input data (e.g., parent-reported child behaviour versus cognitive data) [90], each method should be thoroughly investigated for each problem in order to make an informed decision. This should include the strength of within- and between-datasets correlation, testing different numbers of input features (i.e., does dimensionality reduction/feature selection improve reliability and stability), in- and out-of-sample associations (i.e., quantify the degree of overfitting) and stability of model weights/loadings (i.e., using resampling techniques).
Limitations
This review focused on two popular methods – CCA and PLS – to investigate brain-behaviour associations. Other methods can also be used such as multi-level clustering or independent component analysis. However, application of these methods in brain-behaviour studies in psychiatric disorders is still sparse. In our exemplar CCA, we discuss the importance of ‘out-of-sample’ testing using one large sample. Although this is a significant improvement from in-sample testing, the sub-samples used to find the brain-behaviour association and test it for generalizability follow the same recruitment and assessment procedures, which may mask the true strength of the association. Ideally, testing the CCA model in an independent sample would have yielded a result closer to the true association. Finally, this systematic review was not pre-registered; however, the search strategy is shown in the supplementary information.
Conclusion
‘Doubly multivariate’ methods, namely CCA and PLS, are becoming a popular approach to investigate brain-behaviour associations in psychiatry. These methods are attractive because they honour the multivariate nature of both brain and behaviour in health and disease. We found 37 studies that used either CCA or PLS to investigate brain-behaviour associations across several diagnostic groups. Overall, our findings suggest that frontal regions may be a transdiagnostic marker for psychopathology and cognition in psychiatric disorders. Although only a small number of studies investigated physical measures such as BMI, and developmental history such as childhood trauma, evidence suggests that these factors may be associated with specific brain signatures. This highlights the importance of expanding the investigation of brain-behaviour studies to include transdiagnostic factors of psychiatric illness in addition to symptoms and cognition. In addition, the choice and quality of behavioural measures should be scrutinized to similar standards as neuroimaging data; this could be achieved, for example, by reporting key psychometric properties of the behavioural data, such as validity, reliability and measurement invariance. Finally, we speculate that the overall effect sizes of the ‘doubly multivariate’ brain-behaviour associations reported so far may be optimistic. This is mainly due to the lack of rigorous out-of-sample testing and the suboptimal ratio between sample size and number of features, which combined may lead to overfitting. In conclusion, the capacity of ‘doubly multivariate’ models to learn complex associations, makes this a promising approach for investigating how brain and behaviour interact in psychiatric disorders. While there are still important challenges to overcome, the findings reviewed here provide preliminary evidence for the potential role of these methods in the investigation of mental health disorders.
References
Fusar-Poli P, Borgwardt S, Crescini A, Deste G, Kempton MJ, Lawrie S, et al. Neuroanatomy of vulnerability to psychosis: A voxel-based meta-analysis. Neurosci Biobehav Rev. 2011;35:1175–85.
Vissink CE, Winter-van Rossum I, Cannon TD, Fusar-Poli P, Kahn RS, Bossong MG. Structural Brain Volumes of Individuals at Clinical High Risk for Psychosis: A Meta-analysis. Biol Psychiatry Glob Open Sci. 2022;2:147–52.
Luna LP, Radua J, Fortea L, Sugranyes G, Fortea A, Fusar-Poli P, et al. A systematic review and meta-analysis of structural and functional brain alterations in individuals with genetic and clinical high-risk for psychosis and bipolar disorder. Prog Neuro-Psychopharmacology Biol Psychiatry. 2022;117:110540.
Sacher J, Neumann J, Fünfstück T, Soliman A, Villringer A, Schroeter ML. Mapping the depressed brain: A meta-analysis of structural and functional alterations in major depressive disorder. J Affect Disord. 2012;140:142–8.
Kharabian Masouleh S, Eickhoff SB, Hoffstaedter F, Genon S, Initiative ADN. Empirical examination of the replicability of associations between brain structure and psychological variables. Elife. 2019;8:e43464.
Marek S, Tervo-Clemmens B, Calabro FJ, Montez DF, Kay BP, Hatoum AS, et al. Reproducible brain-wide association studies require thousands of individuals. Nature. 2022;603:654–60.
Biswal BB, Mennes M, Zuo X-N, Gohel S, Kelly C, Smith SM, et al. Toward discovery science of human brain function. Proc Natl Acad Sci. 2010;107:4734–9.
Mulders PC, van Eijndhoven PF, Schene AH, Beckmann CF, Tendolkar I. Resting-state functional connectivity in major depressive disorder: a review. Neurosci Biobehav Rev. 2015;56:330–44.
Kennedy DP, Courchesne E. The intrinsic functional organization of the brain is altered in autism. Neuroimage. 2008;39:1877–85.
Sheffield JM, Barch DM. Cognition and resting-state functional connectivity in schizophrenia. Neurosci Bio Behav Rev. 2016;61:108–20.
Koutsouleris N, Kambeitz-Ilankovic L, Ruhrmann S, Rosen M, Ruef A, Dwyer DB, et al. Prediction Models of Functional Outcomes for Individuals in the Clinical High-Risk State for Psychosis or With Recent-Onset Depression: A Multimodal, Multisite Machine Learning Analysis. JAMA Psychiatry. 2018;75:1156–72.
Sartori JM, Reckziegel R, Passos IC, Czepielewski LS, Fijtman A, Sodré LA, et al. Volumetric brain magnetic resonance imaging predicts functioning in bipolar disorder: A machine learning approach. J Psychiatr Res. 2018;103:237–43.
Vieira S, Pinaya WHL, Mechelli A. Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications. Neurosci Bio Behav Rev. 2017;74:58–75.
Quaak M, van de Mortel L, Thomas RM, van Wingen G. Deep learning applications for the classification of psychiatric disorders using neuroimaging data: Systematic review and meta-analysis. NeuroImage Clin. 2021;30:102584.
Wolfers T, Buitelaar JK, Beckmann CF, Franke B, Marquand AF. From estimating activation locality to predicting disorder: A review of pattern recognition for neuroimaging-based psychiatric diagnostics. Neurosci Bio Behav Rev. 2015;57:328–49.
Del Fabro L, Bondi E, Serio F, Maggioni E, D’Agostino A, Brambilla P. Machine learning methods to predict outcomes of pharmacological treatment in psychosis. Transl Psychiatry. 2023;13:75.
Vieira S, Liang X, Guiomar R, Mechelli A. Can we predict who will benefit from cognitive-behavioural therapy? A systematic review and meta-analysis of machine learning studies. Clin Psychol Rev. 2022;97:102193.
Orrù G, Pettersson-Yeo W, Marquand AF, Sartori G, Mechelli A. Using Support Vector Machine to identify imaging biomarkers of neurological and psychiatric disease: A critical review. Neurosci Biobehav Rev. 2012;36:1140–52.
Pereira F, Mitchell T, Botvinick M. Machine learning classifiers and fMRI: A tutorial overview. Neuroimage. 2009;45:S199–S209.
Vapnik V. The Nature of Statistical Learning Theory. Springer, 1995.
Meda SA, Jagannathan K, Gelernter J, Calhoun VD, Liu J, Stevens MC, et al. A pilot multivariate parallel ICA study to investigate differential linkage between neural networks and genetic profiles in schizophrenia. Neuroimage. 2010;53:1007–15.
Dwyer DB, Buciuman M-O, Ruef A, Kambeitz J, Sen Dong M, Stinson C, et al. Clinical, Brain, and Multilevel Clustering in Early Psychosis and Affective Stages. JAMA Psychiatry. 2022;79:677–89.
Hotelling H. Relations between two sets of variates. In: Breakthroughs in statistics: methodology and distribution. Springer, 1992, 162–90.
Wold H. Partial least squares. In Kotz S, Johnson NL (Eds.) Encyclopedia of statistical sciences, Vol. 6 (Wiley, New York, 1985).
Genon S, Eickhoff SB, Kharabian S. Linking interindividual variability in brain structure to behaviour. Nat Rev Neurosci. 2022;23:307–18.
Wang H-T, Smallwood J, Mourao-Miranda J, Xia CH, Satterthwaite TD, Bassett DS, et al. Finding the needle in a high-dimensional haystack: Canonical correlation analysis for neuroscientists. Neuroimage. 2020;216:116745.
Mihalik A, Chapman J, Adams RA, Winter NR, Ferreira FS. Canonical Correlation Analysis and Partial Least Squares for Identifying Brain – Behavior Associations: A Tutorial and a Comparative Study. Biol Psychiatry Cogn Neurosci Neuroimaging. 2022;7:1055–1067.
Liang C, Pearlson G, Bustillo J, Kochunov P, Turner JA, Wen X, et al. Psychotic Symptom, Mood, and Cognition-associated Multimodal MRI Reveal Shared Links to the Salience Network Within the Psychosis Spectrum Disorders. Schizophr Bull. 2022;49:172–84.
Sui J, Adali T, Pearlson G, Yang H, Sponheim SR, White T, et al. A CCA+ICA based model for multi-task brain imaging data fusion and its application to schizophrenia. Neuroimage. 2010;51:123–34.
Sui J, He H, Pearlson GD, Adali T, Kiehl KA, Yu Q, et al. Three-way (N-way) fusion of brain imaging data based on mCCA + jICA and its application to discriminating schizophrenia. Neuroimage. 2013;66:119–32.
Sui J, Yu Q, He H, Pearlson GD, Calhoun VD. A Selective Review of Multimodal Fusion Methods in Schizophrenia. Front Hum Neurosci. 2012;6:27.
Krishnan A, Williams LJ, Randal A, Abdi H. Partial Least Squares (PLS) methods for neuroimaging: A tutorial and review. Neuroimage. 2011;56:455–75.
Calhoun VD, Sui J. Multimodal fusion of brain imaging data: A key to finding the missing link(s) in complex mental illness. Biol Psychiatry Cogn Neurosci neuroimaging. 2016;1:230–44.
Tabachnick BG, Fidell LS, Ullman JB. (eds) Using multivariate statistics (Boston, MA: pearson, 2013).
Schöttner M, Bolton TAW, Patel J, Nahálka AT, Vieira S, Hagmann P. Exploring the latent structure of behavior using the Human Connectome Project’s data. Sci Rep. 2023;13:713.
Casey BJ, Cannonier T, Conley MI, Cohen AO, Barch DM, Heitzeg MM, et al. The Adolescent Brain Cognitive Development (ABCD) study: Imaging acquisition across 21 sites. Dev Cogn Neurosci. 2018;32:43–54.
Van Essen DC, Smith SM, Barch DM, Behrens TEJ, Yacoub E, Ugurbil K. The WU-Minn Human Connectome Project: An overview. Neuroimage. 2013;80:62–79.
Littlejohns TJ, Holliday J, Gibson LM, Garratt S, Oesingmann N, Alfaro-Almagro F, et al. The UK Biobank imaging enhancement of 100,000 participants: rationale, data collection, management and future directions. Nat Commun. 2020;11:2624.
Modabbernia A, Reichenberg A, Ing A, Moser DA, Doucet GE, Artiges E, et al. Linked patterns of biological and environmental covariation with brain structure in adolescence: a population-based longitudinal study. 2021;26:4905–4918.
Miller KL, Alfaro-almagro F, Bangerter NK, Thomas DL, Yacoub E, Xu J, et al. Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nat Neurosci. 2016;19:1523–36. https://doi.org/10.1038/nn.4393
Moser DA, Doucet GE, Lee WH, Rasgon A. Multivariate associations among behavioral, clinical, and multimodal imaging phenotypes in patients with psychosis. JAMA. 2018.https://jamanetwork.com/journals/jamapsychiatry/article-abstract/2673930.
Nicolaisen-Sobesky E, Mihalik A, Kharabian-Masouleh S, Ferreira FS, Hoffstaedter F, Schwender H, et.al. A cross-cohort replicable and heritable latent dimension linking behaviour to multi-featured brain structure. Commun Biol. 2022;26:1297. https://doi.org/10.1038/s42003-022-04244-5.
Smith SM, Nichols TE, Vidaurre D, Winkler AM, Behrens TEJ, Glasser MF, et al. A positive-negative mode of population covariation links brain connectivity, demographics and behavior. Nat Publ Gr. 2015; 18. https://doi.org/10.1038/nn.4125.
Alnæs D, Kaufmann T, Marquand AF, Smith SM, Westlye LT. Patterns of sociocognitive stratification and perinatal risk in the child brain. 2020. https://doi.org/10.1073/pnas.2001517117.
Xu X, Hilal S, Collinson SL, Chong EJY, Ikram MK. Association of magnetic resonance imaging markers of cerebrovascular disease burden and cognition. Stroke 2015. https://doi.org/10.1161/strokeaha.115.010700.
Xu J, Liu N, Polemiti E, Garcia-Mondragon L, Tang J, Liu X, et al. Effects of urban living environments on mental health in adults. Nat Med. 2023;29:1456–67.
Xia CH, Ma Z, Ciric R, Gu S, Betzel RF, Kaczkurkin AN, et al. Linked dimensions of psychopathology and connectivity in functional brain networks. Nat Commun. 2018;9:3003.
Moser DA, Doucet GE, Ing A, Dima D, Schumann G, Bilder RM, et al. An integrated brain–behavior model for working memory. 2018;23:1974–1980.
Pettersson-Yeo W, Allen P, Benetti S, McGuire P, Mechelli A. Dysconnectivity in schizophrenia: Where are we now? Neurosci Biobehav Rev. 2011;35:1110–24.
Wheeler AL, Voineskos AN. A review of structural neuroimaging in schizophrenia: from connectivity to connectomics. Front Hum Neurosci. 2014;8:653.
Li S, Hu N, Zhang W, Tao B, Dai J, Gong Y, et al. Dysconnectivity of Multiple Brain Networks in Schizophrenia: A Meta-Analysis of Resting-State Functional Connectivity. Front Psychiatry 2019; 10. https://doi.org/10.3389/fpsyt.2019.00482.
McCutcheon RA, Reis Marques T, Howes OD. Schizophrenia—An Overview. JAMA Psychiatry. 2020;77:201–10.
Mihalik A, Ferreira FS, Moutoussis M, Ziegler G, Adams RA, Rosa MJ, et al. Multiple Holdouts With Stability: Improving the Generalizability of Machine Learning Analyses of Brain – Behavior Relationships. 2020;87:368–376.
Helmer M, Warrington S, Mohammadi-Nejad A-R, Ji JL, Howell A, Rosand B, et al. On the stability of canonical correlation analysis and partial least squares with application to brain-behavior associations. Commun Biol. 2024;7:217.
Mechelli A, Vieira S. From models to tools: clinical translation of machine learning studies in psychosis. npj Schizophr. 2020;6:4.
Shmueli G. To Explain or to Predict? Stat Sci. 2010;25:289–310.
Yarkoni T, Westfall J. Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning. Perspect Psychol Sci. 2017;12:1100–22.
Kilpatrick LA, Joshi SH, O’Neill J, Kalender G, Dillon A, Best KM, et al. Cortical gyrification in children with attention deficit-hyperactivity disorder and prenatal alcohol exposure. Drug Alcohol Depend. 2021;225:108817.
Lin HY, Cocchi L, Zalesky A, Lv J, Perry A, Tseng WYI, et al. Brain-behavior patterns define a dimensional biotype in medication-naïve adults with attention-deficit hyperactivity disorder. Psychol Med. 2018;48:2399–408.
Luo L, Chen L, Wang Y, Li Q, He N, Yuanyuan Li, et al. Patterns of brain dynamic functional connectivity are linked with attention-deficit/hyperactivity disorder-related behavioral and cognitive dimensions. Psychol Med. 2023;53:6666–77.
Tsai CJ, Lin HY, Tseng IWY, Gau SSF. White matter microstructural integrity correlates of emotion dysregulation in children with ADHD: A diffusion imaging tractography study. Prog Neuro-Psychopharmacology Biol Psychiatry. 2021;110:110325.
Mei T, Llera A, Floris DL, Forde NJ, Tillmann J, Durston S, et al. Gray matter covariations and core symptoms of autism: the EU-AIMS Longitudinal European Autism Project. Mol Autism. 2020;11:1–13.
Zhang J, Fang S, Yao Y, Li F, Luo Q. Parsing the heterogeneity of brain-symptom associations in autism spectrum disorder via random forest with homogeneous canonical correlation. J Affect Disord. 2023;335:36–43.
Buch AM, Vértes PE, Seidlitz J, Kim SH, Grosenick L, Liston C. Molecular and network-level mechanisms explaining individual differences in autism spectrum disorder. 2023. https://doi.org/10.1038/s41593-023-01259-x.
Ilioska I, Oldehinkel M, Llera A, Chopra S, Looden T, Chauvin R, et al. Robust Patterns of Atypical Functional Connectivity in Autism. Biol Psychiatry 2023;94:29–39.
Mei T, Forde NJ, Floris DL, Dell’Acqua F, Stones R, Ilioska I, et al. Autism Is Associated With Interindividual Variations of Gray and White Matter Morphology. Biol Psychiatry Cogn Neurosci Neuroimaging 2022. https://doi.org/10.1016/j.bpsc.2022.08.011.
Ni HC, Lin HY, Tseng WYI, Gau SSF. Association of self-regulation with white matter correlates in boys with and without autism spectrum disorder. Sci Rep. 2020;10:1–12.
Drysdale AT, Grosenick L, Downar J, Dunlop K, Mansouri F, Meng Y, et al. Resting-state connectivity biomarkers define neurophysiological subtypes of depression. Nat Med. 2017;23:28–38.
Yu M, Linn KA, Shinohara RT, Oathes DJ, Cook PA, Duprat R. Childhood trauma history is linked to abnormal brain connectivity in major depression. 2019. https://doi.org/10.1073/pnas.1900801116.
Yu M, Cullen N, Linn KA, Oathes DJ, Seok D, Cook PA, et al. Structural brain measures linked to clinical phenotypes in major depression replicate across clinical centres. Mol Psychiatry 2021;26:2764–75.
Dean AM, Goodby E, Ooi C, Nathan PJ, Lennox BR, Scoriels L, et al. Speed of facial affect intensity recognition as an endophenotype of first-episode psychosis and associated limbic-cortical grey matter systems. 2013;43:591–602.
Raghava JM, Mandl RCW, Nielsen MØ, Multimodal assessment of white matter microstructure in antipsychotic-naïve schizophrenia patients and confounding effects of recreational drug use. 2021;15:36–48.
Thomas MB, Raghava JM, Pantelis C, Rostrup E, Nielsen MØ, Jensen MH, et al. ScienceDirect Associations between cognition and white matter microstructure in first-episode antipsychotic-naı patients with schizophrenia and healthy controls: A multivariate pattern analysis. 2021; 9. https://doi.org/10.1016/j.cortex.2021.03.003.
Krakauer K, Ebdrup BH, Glenthøj BY, Raghava JM, Nordholm D, Randers L. Patterns of white matter microstructure in individuals at ultra-high-risk for psychosis: associations to level of functioning and clinical symptoms. 2017;47:2689–707.
Kristensen TD, Mandl RCW, Raghava JM, Jessen K, Jepsen JRM, Fagerlund B, et al. Widespread higher fractional anisotropy associates to better cognitive functions in individuals at ultra-high risk for psychosis. Hum Brain Mapp. 2019;40:5185–201.
Kirschner M, Shafiei G, Markello RD, Makowski C, Talpalaru A, Hodzic-santor B, et al. Latent Clinical-Anatomical Dimensions of Schizophrenia. Schizophr Bull. 2020;46:1426–38.
Syeda WT, Wannan CMJ, Merritt AH, Raghava JM, Jayaram M, Velakoulis D, et al. Cortico-cognition coupling in treatment resistant schizophrenia. Neuroimage Clin. 2022;35:103064 https://doi.org/10.1016/j.nicl.2022.103064.
Kottaram A, Zalesky A, Johnston LA, Cocchi L, Ganella EP. Brain network dynamics in schizophrenia: Reduced dynamism of the default mode network. 2019;40:2212–28.
Rodrigue AL, Mcdowell JE, Tandon N, Keshavan MS, Tamminga CA, Pearlson GD, et al. Archival Report Multivariate Relationships Between Cognition and Brain Anatomy Across the Psychosis Spectrum. Biol Psychiatry Cogn Neurosci Neuroimaging. 2018;3:992–1002.
Voldsbekk I, Kjelkenes R, Wolfers T, Dahl A, Lund MJ, Kaufmann T, et al. Developmental Cognitive Neuroscience Shared pattern of impaired social communication and cognitive ability in the youth brain across diagnostic boundaries. Dev Cogn Neurosci. 2023;60:101219.
Tung Y, Lin H, Chen C, Sc M, Shang C, Ph D, et al. Whole Brain White Matter Tract Deviation and Idiosyncrasy From Normative Development in Autism and ADHD and Unaffected Siblings Link With Dimensions Psychopathol Cogn. 2021. https://doi.org/10.1176/appi.ajp.2020.20070999.
Chien Y, Lin H, Tung Y, Hwang T, Chen C, Wu C Neurodevelopmental model of schizophrenia revisited: similarity in individual deviation and idiosyncrasy from the normative model of whole-brain white matter tracts and shared brain-cognition covariation with ADHD and ASD. 2022. https://doi.org/10.1038/s41380-022-01636-1.
Ji JL, Helmer M, Fonteneau C, Burt JB, Tamayo Z, Adkinson BD, et al. Mapping brain-behavior space relationships along the psychosis spectrum. 2021;10:e66968.
Dinga R, Schmaal L, Penninx BWJH, Jose M, Tol V, Veltman DJ, et al. Evaluating the evidence for biotypes of depression: Methodological replication and extension of. NeuroImage Clin. 2019;22:101796.
Helmer M, Warrington S, Mohammadi-Nejad AR, Ji JL, Howell A, Rosand B, et al. On the stability of canonical correlation analysis and partial least squares with application to brain-behavior associations. Commun Biol 2024;7:217.
Pituch KA, Stevens JP. Applied multivariate statistics for the social sciences: Analyses with SAS and IBM’s SPSS. Routledge, 2015.
Varoquaux G. Cross-validation failure: Small sample sizes lead to large error bars. Neuroimage. 2017;180:68–77.
Monteiro JM, Rao A, Shawe-taylor J. A multiple hold-out framework for Sparse Partial Least Squares. J Neurosci Methods. 2016;271:182–94.
Tiego J, Martin EA, Deyoung CG, Hagan K, Cooper SE, Pasion R, et al. Precision behavioral phenotyping as a strategy for uncovering the biological correlates of psychopathology. Nat Ment Health. 2023;1:304–15. https://doi.org/10.1038/s44220-023-00057-5
Nakua H, Yu J, Abdi H, Hawco C, Voineskos A, Hill S. Comparing the stability and reproducibility of brain-behaviour relationships found using Canonical Correlation Analysis and Partial Least Squares within the ABCD Sample. 2023.
Vieira S, Pinaya WHL, Garcia-Dias R, Mechelli A. Multimodal integration. In: Machine Learning. Academic Press, 2020, 283–305.
Viviano JD, Buchanan RW, Calarco N, Gold JM, Foussias G, Bhagwat N, et al. Resting-State Connectivity Biomarkers of Cognitive Performance and Social Function in Individuals With Schizophrenia Spectrum Disorder and Healthy Control Subjects. 2018;84:665–74.
Arbabshirani MR, Plis S, Sui J, Calhoun VD. Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls. Neuroimage. 2017;145:137–65.
Scheinost D, Noble S, Horien C, Greene AS, Lake EM, Salehi M, et al. Ten simple rules for predictive modeling of individual differences in neuroimaging. Neuroimage. 2019;193:35–45.
Marquand AF, Rezek I, Buitelaar J, Beckmann CF. Understanding Heterogeneity in Clinical Cohorts Using Normative Models: Beyond Case-Control Studies. Biol Psychiatry. 2016;80:552–61.
Parra LC Multiset Canonical Correlation Analysis simply explained. https://arxiv.org/pdf/1802.03759.pdf (accessed 25 May2020).
Lerman-Sinkoff DB, Kandala S, Calhoun VD, Barch DM, Mamah DT. Transdiagnostic Multimodal Neuroimaging in Psychosis: Structural, Resting-State, and Task Magnetic Resonance Imaging Correlates of Cognitive Control. Biol Psychiatry Cogn Neurosci Neuroimaging. 2019;4:870–80.
Liu S, Wang H, Song M, Lv L, Cui Y, Liu Y, et al. Linked 4-Way Multimodal Brain Differences in Schizophrenia in a Large Chinese Han Population. Schizophr Bull. 2018;45:436–49.
Deshpande G, Jia H. Multi-Level Clustering of Dynamic Directional Brain Network Patterns and Their Behavioral Relevance. Front Neurosci. 2020; 13. https://doi.org/10.3389/fnins.2019.01448.
Blanken TF, Bathelt J, Deserno MK, Voge L, Borsboom D, Douw L. Connecting brain and behavior in clinical neuroscience: A network approach. Neurosci Biobehav Rev. 2021;130:81–90.
Hilland E, Landrø NI, Kraft B, Tamnes CK, Fried EI, Maglanoc LA, et al. Exploring the links between specific depression symptoms and brain structure: A network study. Psychiatry Clin Neurosci. 2020;74:220–1.
McIntosh AR. Comparison of Canonical Correlation and Partial Least Squares analyses of simulated and empirical data. arXiv preprint arXiv:2107.06867. 2021 Jul 14.
Acknowledgements
This work was supported by a Sir Henry Wellcome postdoctoral fellowship from the Wellcome Trust to SV (grant number 221638/Z/20/Z). TB, MS, and PH were supported by Swiss National Science Foundation grant #320030_197787. Data were provided by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.
Author information
Authors and Affiliations
Contributions
SV: Conceptualization, Data Curation, Analysis, Writing – Original Draft, Writing – Review and Editing, Funding Acquisition. TB: Analysis, Writing – Review and Editing. MS: Data Curation, Writing – Review and Editing. LB: Data Curation, Writing – Review and Editing. AMa, AMe, and PH: Writing – Review and Editing.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Vieira, S., Bolton, T.A.W., Schöttner, M. et al. Multivariate brain-behaviour associations in psychiatric disorders. Transl Psychiatry 14, 231 (2024). https://doi.org/10.1038/s41398-024-02954-4
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41398-024-02954-4
- Springer Nature Limited