Introduction

Since its launch in 2010, the Research Domain Criteria (RDoC) framework by the National Institute of Mental Health (NIMH) [1] gained traction, in an effort to transgress established symptom based diagnostic systems [Diagnostic and Statistical Manual of Mental Disorders (DSM) [2]/International classification of diseases [ICD] [3]], implementing new categories representing fundamental principles underlying these taxonomies [4]. This transnosological approach aims to understand the full spectrum of mental health and illness through incorporating biological, physiological and behavioral knowledge, while it seeks to overcome existing problems of symptom based heterogeneity, comorbidity, and research limitations induced by diagnostic categories [5,6,7].

The RDoC approach represents a framework based on behavioral dimensions and neurobiological measures with the final goal of improving classification systems for mental diseases and treatment approaches [1, 8]. This goal arose from a fundamental critique on the DSM-5 and its lack of validity, which is caused by symptom-based diagnosis that do not categorize by etiology and fail to match mechanisms or markers identified in biological psychiatry [9]. These shortcomings have been linked to low response and remission rates in psychopharmacology, furthermore to potential harming of patients with needless treatments and diagnoses [10, 11].

The RDoC matrix [6] offers a systematic overview of currently six core domains forming basic dimensions of human functioning: positive (PVS) [12] and negative valence systems (NVS) [13], cognitive systems (CS) [14], social processes (SP) [15], arousal and regulatory systems (ARS) [16] and sensorimotor systems (SMS) [17]. For each domain a hierarchical system of constructs and subconstructs is defined to cover specific facets of this domain. Each (sub-)construct has eight “units of analysis” representing methodological aspects to integrate the following levels of information covering genes, molecules, cells, circuits, physiology, behavior, self-report, and paradigms. In this current study, only four domains are being investigated since ARS and SMS were subsequently added to the matrix after data collection started. The domains, (sub-)constructs and units of analysis investigated in the current work, are described in detail in the methods section.

Little work has been done to validate these RDoC defined core domains using the units self-report and behavioral investigation, while the existing literature on this subject shows large variations of methodological approaches and definitions on these functional core domains. Recent studies using self-report measures explored or confirmed the multi-factorial structure of the PVS domain and showed connection to common constructs of personality [18, 19]. In addition, a specific PVS-scale and a subscale for empathy were implemented and validated [18, 20]. In a purely psychometric approach, Tsanas et al. (2017) found valid subconstructs assessing the latent structure of mood symptoms that further validate the RDoC construct PVS [21]. Paulus et al. [22] used an RDoC framework with self-report and behavioral measures to define the domains NVS and PVS and provided evidence suggesting that both domains should be treated independently and not as two sides of the same coin [22]. Their findings supported the assumption of an independent reward neural circuit [23].

Regarding the NVS domain, a review of self-report measures concluded that more exploratory work needs to be conducted to develop valid instruments to measure this domain and its subconstructs [24]. Nonetheless, a confirmatory study in children with internalizing disorder symptoms using self-report measures revealed an “higher order NVS” with a multifactorial internal structure, supporting the idea of a latent NVS domain in developing children and suggesting an underlying set of biological mechanisms with construct specific elements [25].

Until this date, no validation studies regarding the CS domain exist, that use an explicit RDoC framework for self-report or behavioral measures. However, the sub-constructs integrating the domain have been investigated fairly well and have strong neurobiological support [26]. Furthermore, a latent cognitive multifactorial structure common in subjects diagnosed with schizophrenia, bipolar disorder, and healthy adults has been validated which supports the idea of a cognitive multifactorial system congruent to the propositions of the RDoC framework [27]. Recent research on the SP domain showed promising results on capturing dimensional SP constructs with an already existing self-report measure in children with normative development and autism spectrum disorder [28].

In summary, single domains forming the latent structure of the RDoC framework regarding self-report measures have partially been substantiated. Following this research and the current recommendations by the NIMH [29, 30] our study sought to investigate the RDoC framework spanning across four of the core domains for the first time.

Specifically, the goal of the current study was to establish a first look at the latent constructs of PVS, NVS, CS, and SP and their relationship using already existing self-report and behavioral assessments in a transnosological mixed population which cuts across DSM-V/ICD-10 disorder criteria categories. Moreover, we aimed to improve our understanding about the characteristics of these latent variables and their intercorrelations.

Methods

Participants and procedures

Overall, 1912 participants were recruited for the Phenotypic, Diagnostic and Clinical Domain Assessment Network Germany (PD-CAN) within the German research network for mental disorders [31]. All patients were initially recruited for specific intervention and observation studies of a given disease entity within each of the nine network consortia. Specifically, in this study, we report data from eight research consortia (PROTECT-AD, ESCAlife, ASD-Net, BipoLife, OptiMD, GCBS, APIC, ESPRIT). Since the main cohort of the AERIAL project focusing on the development of substance use disorders represents a primarily adolescent at risk sample with comparably low prevalence of mental disorders, measuring similar domains using partially different assessment methods, we excluded data from this consortium from the present analyses given the focus on confirmatory factor analysis. Due to the heterogeneity of the studies within the network, each of the projects implemented specific in- and exclusion criteria as well as the Mini-RDoC assessment in toto or partially depending on the individual assessment fit in the respective study (see SI1 or below for details). An overview on the main aims, sampling including in-/exclusion criteria and number of participants included for each specific study is available in supplementary material SI2. All subjects gave additional written informed consent to participate in and contribute their data to the PD-CAN network in an anonymized fashion at the local sites. Data were transmitted from the partner site via secure servers and data carriers, or an anonymized electronic research file implemented in secuTrial® (interActive Systems GmbH, Berlin).

A consented test-battery with 16 psychological tests was administered after recruitment in addition or embedded into the usual testing of each study, with the principal aim to measure behavioral and self-report constructs of the RDoC matrix [1, 4, 6]. The battery comprises a shell model (Fig. SI1) with two layers and a core. Baseline implementation of the core variables was obligatory and shell variables were optional depending on their fit to the specific assessment process (f.e. questionnaire processing time) to accommodate the individual study designs. The consent process on the battery was managed through a Delphi process including experts from each of the nine consortia and resulted in a selection of tasks for shell and core assessments in 2014. The consent group also determined and assigned a priori the domains assessed within the RDoC matrix by using the information given by the NIMH and publications on self-report/behavioral measures within the RDOC framework at that time [12,13,14,15,16, 22, 25, 32]. Supplementary Table SI3 shows a detailed description of this battery and the tests. The derived 29 variables/scales included in the model were implemented to assess PVS, NVS, CS und SP as latent factors.

Measurements

Positive and negative valence systems: the Positive and Negative Affect Schedule (PANAS) [33] is a 10-item self-report scale on positive and negative affect. The Behavioral Inhibition and Activation Scale (BIS/BAS) [34] includes 24-items assessing motivation towards goal-motivated or avoidance of aversive outcomes. The BSI-53 (Brief Symptom Inventory) [35] is a self-report psychometric instrument to assess a broad range of psychological problems and symptoms of psychopathology. It consists of 53 items yielding 9 scores for primary symptom dimensions and three global distress indices. Abuse and neglect during childhood and adolescence was measured using the 5-item Childhood Trauma Screener (CTS) [36].

Cognitive systems: the two-tiered TMT A/B [37] is a widely used neuropsychological instrument that measures speed of scanning, visuomotor tracking, divided attention and cognitive flexibility. Also included were two subtests of the Wechsler adult intelligence scale (WAIS-IV) [38]. First, Digit-span Forward task (DF) assesses working memory capacity by asking participants to recall an increasing sequence of spoken digits. Second, the Digit Symbol Substitution Test (DSST) measures cognitive processing speed, short term memory, learning ability, visual perception, visuomotor coordination, ability for visual scanning and attention. The short form of the Barratt Impulsiveness Scale (BIS-15) [39] consists of 15 items assessing the behavioral or personality construct of impulsiveness represented in three subscales of non-planning, attentional and motor impulsivity. Lastly, the Multiple-choice Word Test (MWT-B) [40] with 37 items offers an estimate for premorbid cognitive ability.

Social processes: the WHO disability assessment schedule 2.0 (WHO-DAS 2.0) [41] is a 12-item instrument developed by the WHO for assessing health status and disability. Specifically, the single item reflecting social integration was used. Similarly, additional subscales from the BSI-53 [35] assessing social relationships and social anhedonia were used. The emotion regulation questionnaire (ERQ) [42] is a 10-item scale assessing two emotion regulation strategies, cognitive reappraisal and expressive suppression, in relation to SP. In addition, three sociodemographic variables were implemented to include indirect measures of existing social relations, affiliation and attachment. Used subscales and their relation to the RDoC matrix are shown in Table 1.

Table 1 A priori allocation of the PD-CAN assessment to RDoC
Full size table

The structure of data showed heterogeneous missingness (36.68% throughout the whole raw dataset). To deal with missingness, we applied the following strategy: first, we excluded all participants (N = 481) lacking all the indicator variables/scales for at least one of the four RDoC from our analyses. Overall missingness was thus reduced by 12.69%. Missing values within the observed variables (see Table 2) that were considered (a priori) for the factor analysis by the consent group amounted to 11.12%. Even though the overall missing rate can be considered as typical [43], in a second step, we considered individual items exhibiting missingness in more than 35% of data. Four specific variables retained missingness at approximately 39%: BIS/BAS subscales Behavioral Inhibition, BAS-Drive, BAS-Reward Responsiveness and PANAS Positive Affect (see also Table 2). Given that three of the indicators had been selected for PVS, we decided to use three of the six single items from the BSI-53 Obsessive–compulsive scale (focusing on inhibition and habituation as parts of PVS [12]) instead of the whole scale as indicators in order to strengthen the database informing the latent factor PVS. This benefited the full information maximum likelihood method (FIML) [44] used to handle missing data, because more detailed information was available for missing variable estimation. The final sample to evaluate the structure of our four-factor model consisted of n = 1431 participants. A descriptive overview on the demographic and diagnostic information of the final sample can be found in Table 3.

Table 2 Descriptive Statistics of Observed Variables (untransformed)
Full size table
Table 3 Total sample characteristics and consortia sample details
Full size table

Statistics

The underlying latent RDoC factor structure of the PD-CAN assessment was tested using confirmatory factor analysis (CFA) with Maximum likelihood (ML) estimation. Specifically, we fit the confirmatory four-factor model using lavaan version 0.6–6 in R version 4.0.2 with RStudio 1.3.1073 with FIML [44] handling missing data which was considered missing at random (MAR). Latent factors were standardized, i.e., variance was restricted to 1, allowing free estimation of all factor loadings. In addition, the four-factor model was compared to a one-factor solution using the same variables, which sets the correlation between the latent factors to 1 and another model which doesn’t allow covariances between the latent factors treating them as independent.

Given that exploratory data analysis with the Shapiro–Wilk Test [45] for multivariate normal distribution revealed that none of the indicator variables were normally distributed. We used natural logarithm (ln) for right skewed variables or Johnson transformation [46] for variables with a high range to adjust the distribution. Variables/scales that were already z-transformed (DSST score) or were dichotomous or categorical (e.g., residential status) did not undergo transformation. To ensure a congruent polarity for SP we reversed the WHO-DAS 2.0 single item ‘friendships’ since all utilized indicators for SP had negative polarity. Raw scores for all observed variables as well as item-specific missingness are provided in Table 2.

Results

Confirming the primary hypothesis, all indicators showed significant (p < 0.001, except BIS-15 Non-planning Impulsivity scale with p < 0.05) factor loadings on the considered domains with standardized coefficients ranging from − 0.76 to 0.49 for PVS, − 0.53 to 0.85 for NVS, − 0.79 to 0.71 for CS and − 0.92 to 0.54 for SP. However, the four-factor model fit (model 1) was poor with a comparative fit index (CFI) of 0.77, a Tucker–Lewis index (TLI) of 0.75 and a root mean square error of approximation (RMSEA) of 0.078 with a 90% confidence interval (CI) (0.076–0.081). However, compared to a single factor solution (model 2: χ2(6) 1158.1, < 0.001) or a solution assuming the four factors as independent (model 3: χ2(6) 2571.2, p < 0.001), fit was significantly better. For more information on all analyzed CFA-models see supplementary Table SI5 for details.

To address poor fit, we examined the amount of variance explained by each variable/scale using R-Squared estimates for each indicator. Twelve indicators explained less than 20% of the variance in the respective domain and were excluded from further analyses (see Table 4).

Table 4 Dropped indicators with low R-square
Full size table

In addition, using modification indices (mi) as a starting point, we reconfigured the model for two indicators. Modification indices reflect a test for covariance across the four factors (RDoC) under study in CFA when covariances are fixed a priori. Specifically, we defined that all variables with a mi > 200, that showed larger covariance of an indicator to another factorial domain were changed to the other domain. In a first step, we relocated the BSI53 Anhedonia item (mi = 270.30) to PVS (instead of NVS). This change was informed by research that suggests anhedonia to be strongly associated with general approach behavior [19], a decrease in positive affect [47] and the reward system as a core component of PVS [48, 49]. In a second step, the BSI53 Hostility subscale (mi = 203.40) was moved to indicate SP since examining the scale items revealed proximity to interpersonal hostility and significant distance to frustrative nonreward to which it was assigned a priori. Subsequently, all modification indices ranged below 200.

Finally, we re-examined the scales on item level that were removed for low R-squares earlier to test for an increased shared variance in the latent structure altered based on modification indices. To maximize explained variance, we formed mean scores based on these items. Shared variance significantly improved when we combined items from the BIS/BAS Drive and Reward Responsiveness subscales. Additionally, we also restricted items in the PANAS Positive Affect subscale to reflect hedonic items, items from the BSI Obsessive–Compulsive subscale to items reflecting habituation, and items from the BIS Behavioral Inhibition subscale to reflect anxiety and threat more closely. All changes can be found in Table SI4.

Applying these changes created a significantly improved model fit CFI of 0.93, TLI of 0.92. and RMSEA of 0.077 with 90% CI (0.072–0.082). The overall fit as estimated with the CFI now indicated good fit [50]. This full four factor model (model 4) again fitted the data significantly better than a single factor solution (model 5: χ2(6) = 1656.3, p < 0.001) or the solution with four independent factors (model 6: χ2(6) = 2327.8, p < 0.001).

Regarding the relationship between the indicators and their latent factors, highly significant factor loadings suggest that participants with higher scores in PVS tended to have more hedonic affect (β = 0.545, p < 0.001), better habituation (β = 0.810 p < 0.001) and less anhedonia (β =  − 0.758, p < 0.001).

Participants with high scores in NVS tended to have higher levels of general (β = 0.909, p < 0.001) and phobic anxiety (β = 0.812, p < 0.001), more somatization (β = 0.746, p < 0.001), and more anxiety-based behavioral inhibition (β = 0.491, p < 0.001).

As expected, participants with higher cognitive abilities exhibited better premorbid intelligence (β = 0.688, p < 0.001), better cognitive speed processing (β =  − 0.796, p < 0.001) and executive functioning (β =  − 0.796, p < 0.001).

Higher scores in the SP domain aligned with better skills in keeping friendships (β = 0.532, p < 0.001) and less paranoid ideation (β =  − 0.787, p < 0.001), less social anhedonia (β =  − 0.756, p < 0.001), as well as less interpersonal sensitivity (β =  − 0.909, p < 0.001) and hostility (β =  − 0.764, p < 0.001).

There were also significant relations between the four latent factors in that participants with positive affectivity had higher social (β = 0.891, p < 0.001) and cognitive skills (β = 0.221, p < 0.001) and less negative affect or aversion against events, objects, and situations (β =  − 0.757, p < 0.001). At the same time, participants with high levels of negative affect exhibited decreased cognitive (β =  − 0.232, p < 0.001) and social skills (β =  − 0.818, p < 0.001). Finally, higher cognitive skills were related to better social skills (β = 0.175, p < 0.001, see Fig. 1 for details).

Fig. 1
figure 1

Factorial loadings of the final model on the four Research Domain Criteria. Standardized latent variables: PVS Positive valence systems, NVS Negative valence systems, CS Cognitive systems, SP Social processes, Manifest variables: BIS/BAS Behavioral Inhibition Anxiety Anxiety based inhibition aspects of BIS/BAS subscale Behavioral Inhibition, BSI-53 Anhedonia BSI-53 single item Anhedonia, BIS-53 Anxiety BIS-53 subscale Anxiety, BSI-53 Habituation habituational aspects of BSI-53 single items Obsessive–compulsive, BSI-53 Hostility BSI-53 subscale Hostility, BSI-53 Interpersonal Sensitivity BSI-53 subscale Interpersonal Sensitivity, BSI-53 Phobic Anxiety BSI-53 subscale Phobic Anxiety, BSI-53 Paranoid Ideas BSI-53 subscale Paranoid Ideation, BSI-53 Social Anhedonia BSI-53 single Item Social Anhedonia, BSI-53 Somatization BSI-53 subscale Somatization, DSST Work Memory Digit Symbol Substitution Test raw score, PANAS Hedonic Hedonic aspects of PANAS subscale Positive Affect, TMT-A Time Attention Trail Making Test—Version A completion time, TMT-B Time Cognitive Control Trail Making Test—Version B completion time, WHO-DAS Friendships WHO-DAS 2.0 Single item Friendships reversed

Full size image

Discussion

The present study used CFA to delineate four core domains of the RDoC framework using behavioral and self-report assessments in a heterogeneous sample of patients suffering from mental disorders and controls. Following the implementation of a short and efficient Mini-RDoC-Assessment approach for this task in multiple studies from within the German Research Network for mental disorders, it was expected to identify latent constructs shared by multiple disorders that may eventually generate a better understanding of the transnosological structure formed by the RDoC framework.

The four-factor model reflecting the core domains PVS and NVS as well as CS and SP showed good fit across a sample of clinical and nonclinical participants spanning across major mental disorder diagnoses supporting the potential transnosological validity of the RDoC framework as implemented using behavioral assessments only. Compared to a one factor solution and a version treating all factors as independent, it also showed significantly better fit.

Specifically, regarding PVS, hedonic (PANAS) and anhedonic aspects of reward responsiveness as well as habituation (BSI-53) connected with reward learning as part of this domain could be confirmed. However, items reflecting reward valuation and reward responsiveness had to be excluded because of high levels of error variance indicating poor fit with the overall construct of PVS. At the same time, anhedonia showed to be a valid indicator of the dimension forming PVS as compared to NVS. These results correspond to previous findings for this construct [24].

For NVS, especially potential threat indicators (BSI-53, BIS/BAS) remained valid within the overall factorial structure. Interestingly, hostility shared more variance with SP than with NVS as a measure for frustrative nonreward.

For CS, behavioral measurements for attention (TMT-A), cognitive control (TMT-B) and working memory (DSST) confirm these constructs as informative for this latent factor. Presumably because of measurement invariance, the self-report measures for subconstructs of cognitive control failed to contribute to the model.

As for SP, this domain could be best represented by the a priori set variables. Despite the observation that almost half of the measures had high levels of error variance (sociodemographic and emotion regulation) and in result had to be removed as indicators from the model (see [22] for similar results), the final model represents a clear representation of SP including social anhedonia (BSI-53), the ability to maintain friendships (WHO-DAS), as well as interpersonal sensitivity and paranoid ideation (BSI-53).

Across domains, a strong connection between the domains PVS, NVS and SP could be substantiated, indicating a universal latent structure spanning across known nosological entities. CS showed smaller but meaningful correlations with the other domains, suggesting that the associations of cognitive abilities with key aspects of affectivity and SP are small and may be moderated by specific disease mechanisms in e.g. schizophrenia [51], autism spectrum disorders [52, 53] and affective disorders [54].

Concerning all removed subconstructs and their measurements, further research needs to be conducted examining existing self-report measures and their allocation inside the RDoC framework as well as to conceptualize new comprehensive measurement tools improving valid measurement of its dimensional latent structure for better adoption in clinical assessment and research.

Finally, our findings suggest that the Mini-RDoC test battery, specifically subsets of the self-report questionnaires BSI-53, PANAS, BIS/BAS, WHO-DAS and TMT A/B and DSST as cognitive tests, successfully resemble aspects of the four core domains measured. Revealing a latent factor structure common to all mental disorders included in this study, as anticipated by the RDoC framework, and gives space for a definition improvement on the latent (sub-)constructs and their relations in-between.

Several limitations need to be addressed considering our findings from this study. The implementation of the Mini-RDoC assessment as a core assessment inside the German research network for mental disorders enabled us to build a considerable amount of data providing a transnosological view cutting across known disorder-based categories. However, some diagnostic categories were over- or underrepresented. Thus, while findings are generalizable throughout a large variety of mental diseases future research could validate our findings using a more balanced distribution.

The initial poor model-fit and the need to use variable reduction and modification indices to guide and reshape our a priori assumptions introduced a bias resulting in reduction of robustness and generalizability of our final model. However, we used these methods very carefully and gave detailed information on the changes made. We would like to point out that all changes applied refer to a theoretical basis in our procedures and that model fit was superior to other factor solutions. Nevertheless, future research should replicate our findings to confirm the formed latent structure. Though this procedure may have been a little exploratory, the significantly better fit in comparison to the one factor solution, supports the assumption of a four factor latent structure.

In contrast to the basic assumption of the RDoC approach that latent variables would become apparent across units of analyses (i.e., considering molecules, cells, physiology, circuits, behavior, and self-report) within domains, our approach is mainly symptom-oriented and focuses on the behavioral and self-report units. Therefore, our study did not evaluate cross-unit validity of the RDoC but investigated latent variables within the self-report unit.

Furthermore, our findings on the assignment changes implicated by modification indices and the removal of several constructs due to high error variance should be re-examined and cross-validated in further, preferentially larger, datasets.

Also, there is some ambiguity with respect to the self-report measurements within the RDoC framework [24], suggesting that more research on embedding already validated and reliable self-report measures into the RDoC framework needs to be done, as well as validating new measurements for specific domains that emerged after our initial consensus on the used measurements (e.g., the sensorimotor domain [17]) and their integration with other units of analysis as suggested by f.e. MacNamara and Phan (2016) [55].

To conclude, this study gives a first impression on the latent structure and intercorrelations between four core Research Domain Criteria in a transnosological sample cutting across symptom-based diagnostics. We emphasize the possibility of using already existing and well validated self-report and behavioral measurements to capture aspects of the latent structure formed by the RDoC matrix. This will enable future research connecting the RDoC matrix and its core domains PVS, NVS, CS and SP to outcome measures like disease severity to better characterize domain-specific effects across mental disorders, which may help inform the development of stratified treatment strategies.