Tests and scales measuring psychological disorders should provide information about how scores relate to other constructs such as quality of life or functional impairment. Such information is necessary to allow that their scores contribute to clinical decision making. The current study analyzes the clinical utility of the Spanish version of the Inventory for Depression and Anxiety Symptoms (IDAS-II) to discriminate between different levels of functional impairment and identify the IDAS-II scales that contribute most to explaining impairment. The total sample (N = 1390) consists of two subsamples: a community sample of the general population (n = 1072) selected by random sampling; and a sample of patients (n = 318) from public and private mental health services. The Spanish IDAS-II for measuring internalizing symptoms and WHODAS 2.0 for measuring impairment were administered to all participants. All scales show statistically significant higher scores in the patient sample, with Cohen's d effect sizes values greater than 0.30, except for well-being (d = 0.19). The cutoff values and their confidence intervals do not overlap with the means of either the community or patient sample. AUC values for most of the scales are above .70, except for appetite gain, ordering, euphoria, cleaning, and well-being. Multiple linear regression model using IDAS-II scales explain 57.1% of the variance of the WHODAS 2.0 (F 12.1377 = 155.305; p < .001). Cutoff values provided allow us to reliably differentiate between the patients and community samples. Spanish IDAS-II scores show greater sensitivity and specificity in detecting those with greater impairment. General Depression, Lassitude, Panic and Claustrophobia contribute to impairment in a greater extent. Knowledge of which symptoms are most related with impairment, allows healthcare providers to improve treatment planning based on empirical evidence.
Mental disorders are among the leading causes of years lived with disability worldwide and a major cause of health burden (Global Burden of Diseases – GBD Mental Disorders Collaborators, 2022). Among them, depression and anxiety disorders contributed most to the proportion of mental disorder disability-adjusted life years in 2019 (37.3% and 22.9%, respectively) (Whiteford et al., 2013). Further, as a consequence of the impact of COVID-19, the prevalence of these disorders remained high or even increased (Kumar & Nayar, 2020), with studies indicating rates in the general population of up to 34.31% and 38.12% for depression and anxiety, respectively (Necho et al., 2021). These emotional disorders can present with different degrees of severity, ranging from mild to severe and with different symptomatology. More severe symptomatology is generally associated with significant impairment (Hasin et al., 2018; Hammer-Helimich et al., 2018; Morin et al., 2020), requiring specialized attention from psychiatrists and psychologists who will evaluate the therapeutic needs.
The diagnosis of presence and severity of these disorders has generally been made following the Diagnostic and Statistical Manual of Mental Disorders (DSM) and International Classification of Diseases (ICD) classification systems. Tests based on these nosotaxies determine the severity and absence/presence of these mental disorders, based on a count of diagnostic criteria and a categorical cutoff for disorder presence. These tests are thus essential for diagnosing and monitoring patients (Jablensky, 2016), although some authors question basing clinical decisions on the count of diagnostic criteria (Lane & Sher, 2015; Markon et al., 2011; Østergaard et al., 2011; Zimmerman et al., 2015).
As a complement to the above instruments, other scales and tests assessing emotional disorders use dimensional approaches, that ease the correspondence between scale and test scores and indicators of severity (Krueger et al., 2018; Stanton et al., 2020) and allow to infer the severity of disorders by comparing to test norms (i.e. standard of percentile norms). For example, scales such as the Depression Anxiety Stress Scales (DASS; Crawford & Henry, 2003), the Hospital Anxiety and Depression Scale (Crawford et al., 2001; Hinz & Brähler, 2011) or the Inventory of Depression and Anxiety Symptoms (IDAS-II; Nelson et al., 2018; Sánchez-García et al., 2021) have available reports of the score percentiles obtained in random samples extracted from general population. These allow clinicians to infer the severity of a patient’s disorder locating his or her score on the patient’s reference group percentiles (i.e. norms). Also, these instruments give the opportunity to detect greater heterogeneity between individuals (compared to categorical diagnostic approaches) and are thus more sensitive to measurement of patients’ change during the therapeutic process (Kraemer et al., 2004).
The interpretation of scores based on diagnostic criteria count and normative cutoffs are complementary approaches, and both provide useful information that enrich clinical judgment (Trivedi, 2009). However, several authors argue that tests and scales measuring psychological disorders should also provide information about how scores relate to other constructs such as quality of life or functional impairment (Fried et al., 2022; McKnight & Kashdan, 2009). This is especially important in the measurement of depression and anxiety, due to the strong influence of these disorders on impairment and social daily life (Rapaport et al., 2005). Symptoms most strongly related to functional impairment in psychiatric patients are fatigue, concentration problems and negative alterations in mood (Tanner et al., 2019). Previous evidence suggests that symptoms most differentiating clinical and non-clinical samples include lassitude (related with fatigue) and dysphoria for depression (Stasik-O’Brien et al., 2019; Watson et al., 2012; Watson & O’Hara, 2017) or panic attacks and claustrophobia in the case of anxiety symptoms (Irak & Albayrak, 2020). In the case of depression, it is reported that only 41.9% of patients respond to treatment, indicating that most of patients still have substantial functional impairment even after treatment (McKnight & Kashdan, 2009). Concerning anxiety disorders, reductions in symptom severity and restoration of function, while related, appear to be disorder-specific (McKnight et al., 2016). Therefore, disentangling which depression and anxiety symptoms are most related with functioning and providing guiding lines for clinicians is a priority.
In this sense, patient-report outcomes measures (PROM) are crucial methods to evaluate the impact of mental disorders/physical illnesses and their treatment on daily life (Lloyd et al., 2014; Yorkston & Baylor, 2019). These instruments are generally easy to administer and have proven useful for monitoring patients in mental health services (Knaup et al., 2009; Øvretveit, et al., 2017; Shimokawa et al., 2010). For example, clinician experts in insurance medicine might take the empirical evidence for PROMs into account in the decision making for recommending medical leaves or disability benefits (Tanner et al, 2019). Among the PROMs, both the DSM-5 (American Psychiatric Association, 2013) and some authors (Obbarius et al., 2017) highlight the usefulness of the WHODAS 2.0 (World Health Organization, 2000). The WHODAS 2.0 is an instrument that differentiates between different levels of impairment according to the International Classification of Functioning, Disability and Health (ICF). In addition, the WHODAS is adapted to numerous languages and has shown adequate psychometric properties in diverse populations (e.g. Ćwirlej-Sozańska et al., 2020; Federici et al., 2022; Koumpouros et al., 2018; Saltychev et al., 2021).
However, evidence from daily clinical practice reflects that use of PROM is implemented by less than 20% of clinicians (Lewis et al., 2020), despite the existing recommendations (Knaup et al., 2009; Shimokawa et al., 2010). The lack of clinically meaningful information or the effort required to administer these tests might hinder regularly use in clinical sessions (Campbell et al., 2021; Gelkopf et al., 2022). In addition, for tests and scales to be routinely administered in clinical sessions, it is necessary that their scores contribute to clinical decision making, such as drug administration, hospitalization, sick leave, etc. (Kraemer et al., 2004; Sharma, 2021; Widiger & Samuel, 2005). Therefore, it might be useful to provide clinicians information on the impairment of patients through the scales that assess mental disorders such as anxiety and depression, given these scales are more frequently used than PROM instruments. Research studies are consequently needed that establish associations between the scores of tests measuring mental disorders and the ICF levels. Thus, and consistent with the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN; Mokkink et al., 2016), test scores assessing symptoms of mental disorders such as anxiety and depression would also allow a qualitative interpretation in terms of impairment.
Bearing in mind the above, the general objective of the current study is to analyze the clinical utility of the Spanish version of IDAS-II (De la Rosa-Cáceres et al., 2020), an instrument that assesses internalizing symptoms (including anxiety and depression symptoms), by evaluating its ability to discriminate between different levels of functional impairment (as measured by the WHODAS 2.0). Given more impairment can be expected in treatment-seeking samples (Irak & Albayrak, 2020; Watson et al., 2012; Watson & O’Hara, 2017), the Spanish version of IDAS-II should also be able in this study to discriminate patient from community samples in terms of scores obtained (with patient samples having higher mean scores compared to community samples in analogy to earlier studies). In addition, this study examines which emotional disorders symptoms contribute most to explain the total degree of impairment being found. In order to address the general objective, three specific objectives have been established: 1) examine the ability of the Spanish version of IDAS-II (De la Rosa-Cáceres et al., 2020) to discriminate between community and patient samples and providing clinical cutoffs for each IDAS-II scale; 2) identify the cut-off for the Spanish IDAS-II scales associated with moderate and severe impairment according to the ICF on the whole sample (as measured with the WHODAS 2.0); 3) identify the Spanish IDAS-II scales that contribute most to explaining impairment, as indexed by the WHODAS 2.0 total scale score.
According to previous research, it is hypothesized that scores in the clinical sample will be higher than those in the non-treatment seeking community sample, especially on the General depression and Dysphoria scales, which will present the largest effect sizes (Irak & Albayrak, 2020; Watson & O'Hara, 2017). Cutoffs for moderate and severe impairment levels have not been determined by previous studies, yet are expected to be discriminative (AUC values > .7) for all scales except for Well-Being, as higher well-being scores should not be associated with greater impairment. Finally, similar to Tanner et al. (2019), the scales most associated with functional impairment are expected to be Lassitude, General depression and Social anxiety.
The total sample (N = 1390) consists of two subsamples: a community sample of the general population (n = 1072) and a sample of patients (n = 318). The 1,072 participants in the community sample were selected by random sampling, divided into strata representative of the Spanish population for gender, age and geographical region of Spain. The patient sample consisted of 318 patients from public and private mental health services in the province of Huelva (Spain). Inclusion criteria for both samples (community and patients) were as follows: 1) be at least 18 years of age; 2) sign the informed consent; 3) not have any medical or psychological diagnosis that would preclude the administration of the tests. The clinical sample also met the following inclusion criteria: 1) being under treatment in a mental health service during the data collection; 2) have been diagnosed with a mental disorder according to DSM diagnostic criteria (at time of data collection, DSM-IV was used in clinical practice). These diagnoses were only used to determine whether patients were eligible for the clinical group, the specific diagnoses were not used in the analyses in present research.
Table 1 shows the sociodemographic distribution of the total, clinical and community sample. 53.3% of the participants of the total sample were women and aged between 18 and 80 years (M = 43.12; SD = 14.76). The clinical sample had a significant greater proportion of women (64.9% compared to 49.9% on community sample) (χ2 = 21.76; p < .001) and lower mean age (M = 39.09; DT = 14.33 compared to the community sample M = 44.32; DT = 14.68) (t = 21.79; p < .001). In the total sample, 1.2% had not completed primary education, 4.5% had completed primary education, 54.4% had completed secondary education and 39.8% had completed university studies. With regard to employment status, 56.2% were working. 36.2% of the sample has been diagnosed with more than one mental disorders. Differences between the community and clinical samples were observed on education level (χ2 = 181.58; p < .001) and occupational status (χ2 = 163.53; p < .001) (see Table 1). Table 2 shows the diagnoses present in the clinical sample. The most frequent diagnostic categories among patients were Depressive Disorders (38.99%) and Anxiety Disorders (35.53%).
Spanish version of the Inventory of Depression and Anxiety Symptoms-II (IDAS-II; De la Rosa-Cáceres et al., 2020; Sanchez-Garcia et al., 2021; Watson et al., 2012). The IDAS-II is an instrument that assesses the severity of symptoms of depression, anxiety and bipolar disorder during the last two weeks. It is composed of 99 items with a 5-point Likert scale (from 1 = “not at all” to 5 = “extremely”). The items are organized in 18 non overlapping scales (Dysphoria, Lassitude, Insomnia, Suicidality, Appetite Loss, Appetite Gain, Well-Being, Ill Temper, Mania, Euphoria, Panic, Social Anxiety, Claustrophobia, Traumatic Intrusions, Traumatic Avoidance, Checking, Ordering and Cleaning) and an overlapping scale (General Depression). Higher scores are indicative of greater symptom severity for all scales except well-being (higher scores for this scale indicate higher well-being).
In this study, the reliability estimated by Cronbach's alpha coefficient provided values between .71 and .91. These values are similar to previous studies (De la Rosa-Cáceres et al., 2020; Irak & Albayrak, 2020; Watson & O’Hara, 2017; Watson et al., 2012).
12 items Spanish version of the WHO Disability Assessment Schedule II (WHODAS 2.0; Vázquez-Barquero et al., 2000; WHO, 2000). This instrument was developed from a set of ICF items to measure functional impairment. Each item is scored on a 5-point Likert scale (from 0 = “none” to 4 = “extreme or cannot do”) which grades the difficulty experienced by the participant in performing a given activity. This instrument provides an overall score ranging from 0 to 100, differentiating between no impairment (0–4 points), mild impairment (5–24), moderate impairment (25–49), severe impairment (50–95), and complete impairment (96–100) according to the ICF classification (WHO, 2013). The estimated reliability through the Cronbach's alpha coefficient was .91 for community sample and .88 for the patient sample.
In addition to the above-mentioned instruments, a questionnaire was administered that included sociodemographic information on sex, age, educational level and employment.
The administration of the instruments in the community sample was carried out through a company specialized in online surveys, accredited with ISO-26362 quality standards (quality standard for the management of online research). Prior to the administration of the instruments, each participant completed a pre-test that assessed his or her reading and comprehension skills and to verified that no automatic responses were made. Before starting test administration, participants were informed of the objectives of the study and were informed of their right to withdraw from participation during the test administration process. After receiving this information, the patients signed the informed consent form. Participants received a reward for their participation in the study consisting of a voucher redeemable for gifts.
Data collection from the patient sample was performed by a psychologist trained in the administration of the instruments. Tests were administered in individual sessions in the mental health centers where they were recruited. Patients received the same prior information as community participants and were informed about the anonymous and voluntary nature of their participation in the study before signing informed consent. They also received a voucher redeemable for gifts.
In order to evaluate the ability of IDAS-II scores to discriminate between patients and community samples, means and standard deviations were calculated for the IDAS-II scales and the WHODAS 2.0 total scale score in each sample separately. T-tests for independent samples were used to check the differences between two samples. Cohen’s d was applied to calculate effect size. According to Cohen (1992), d-values greater than |0.20|, |0.50|, and |0.80| represent small, medium, and large effect sizes, respectively.
The following formula of Jacobson and Truax (1991) was applied to identify the clinical cutoff that differentiates between the community and patient samples:
where MD is the mean of patient group, SDD is standard deviation of patient group (which is expected to be more dysfunctional), MF is the mean of community group, and SDF is the standard deviation of community group (which is expected to be more functional). The 95% of confidence intervals for each cutoff were also estimated. IDAS-II scales values above the clinical cutoff thus indicate more dysfunctional scores for all scales, except for Well-Being, where values above the clinical cutoff indicate more functional scores.
As to assess the ability of IDAS-II scores to explain functional impairment, both samples were grouped to increase variability. Values higher than 25 in WHODAS 2.0 were used to classify persons with moderate impairment. Values higher than 50 in WHODAS 2.0 were used classify persons with severe impairment and ROC analyses were executed to identified cutoffs in IDAS-II scores according to these WHODAS 2.0 defined ICF scores. IDAS-II values with best balance between sensitivity and specificity were used as cutoff, with minimum specificity set at .70 (Power et al., 2013).
Finally, a regression model analysis, was executed to identify scales of IDAS-II which explained total impairment (measured by the WHODAS 2.0). Gender and age were included as controlled variables, as according to previous literature they are relevant variables related with the level of internalizing symptoms (Jalnapurkar et al., 2018; Sánchez-García et al., 2021; Nelson et al., 2018). A stepwise procedure was used to identify the predictive scales.
All analyses were executed using SPSS version 27.0.
IDAS-II Clinical Cutoffs to Differentiate Community From Patient Samples
Table 3 shows the means of the IDAS-II scales of the community and patient samples. All scales, except well-being, show higher scores in the patient sample. Effect sizes for all scales show Cohen's d values greater than 0.30, except well-being (d = 0.19), and are statistically significant. Medium effect sizes are observed for six scales and large effect sizes for eight scales. The largest effect sizes correspond to the scales of dysphoria (d = 1.31), general depression (d = 1.22), panic (d = 1.18), mania (d = 1.14), and traumatic intrusions (d = 1.10). Higher total score on patient sample are observed on the WHODAS 2.0 (d = 1.24).
The cutoff values and their confidence intervals do not overlap with the means of either the community or patient sample. Thus, these cutoff values allow us to reliably differentiate between the two types of samples.
IDAS-II Estimated Cutoff Point Related to Moderate and Severe Impairment According to ICF
According to WHODAS-2.0, 21.15% of the sample had moderate impairment and 3.53% had severe impairment.
Table 4 shows AUC, cutoffs and sensitivity/specificity values of the IDAS-II scales, using the WHODAS 2.0 values corresponding to moderate impairment (score above 25) and the severe impairment (scores above 50) as criteria. AUC values for most of the scales are adequate, except for five scales that are not able to discriminate between persons without impairment and persons with moderate/severe impairment (i.e. appetite gain, ordering, euphoria, cleaning, and well-being). It is also observed that the sensitivity/specificity shows higher values for detecting severe impairment versus moderate/severe impairment. That is, IDAS-II scores show greater sensitivity and specificity in detecting those with greater impairment.
IDAS-II Scales Explaining Impairment
Table 5 shows the results of the regression model of the WHODAS 2.0 scores, adjusting for age and sex. The model is statistically significant, explaining 57.1% of the variance of the WHODAS 2.0 (F 12.1377 = 155.305; p < .001). The scales with the highest values of the standardized regression coefficients are general depression (β = .236, p = < .001), panic (β = .170, p = < .001), claustrophobia (β = .148, p = < .001) and lassitude (β = .136, p = < .001).
The current study presents novel evidence on the clinical utility of IDAS-II scores by providing information about the associated degree of impairment. The IDAS-II has demostrated to be a reliable (De la Rosa-Cáceres et al., 2020; Irak & Albayrak, 2020; Sanchez-Garcia et al., 2021; Watson et al., 2012) and useful assessment measure of internalizing symptoms in research using a transdiagnostic approach to emotional disorders (Kotov et al., 2017). Present results complement previous evidence, by bringing forth reliable cutoffs that differentiate patient and community samples and by estimating cutoffs of IDAS-II scales associated with different levels of impairment that can help clinicians with therapeutic planning. Furthermore, our results suggest that General Depression, Lassitude, Panic and Claustrophobia contribute to impairment in a greater extent.
In relation to the first objective, and in congruence with previous studies (Irak & Albayrak, 2020; Watson & O’Hara, 2017; Watson et al., 2012), the patient sample in the current study is more dysfunctional in terms of presence of internalizing symptoms as captured by the IDAS-II scales and higher impairment as measured by the WHODAS 2.0 total score. Most of the IDAS-II scales show moderate to high effect sizes when comparing the community and patient sample, with largest effect sizes for the IDAS-II scales of General Depression and Dysphoria, as hypothesized. These two subscales also best differentiated patients and healthy controls in previous research (Irak & Albayrak, 2020; Stasik-O’Brien et al., 2019). The newly provided cutoff points, that differentiate between patient and non-patient samples with non-overlapping confidence intervals, may allow to use this instrument to detect emotional disorders in primary care patients based of a composite profile, as has been recommended (Ferenchick et al., 2019; Park & Zarate, 2019).
However, it should be noted that the IDAS-II Cleaning scale did not show the ability to provide a reliable cutoff in the current study. This result may be due to the impact of COVID-19 during data collection. A recent systematic review suggested that both patients and healthy individuals have experienced contamination obsessive–compulsive-like symptoms related with COVID-19 (Guzick et al., 2021). Considering that cleaning was a protective factor against COVID-19, and thus ‘healthy’ behavior, it is reasonable that effect sizes on this scale are among the lowest and that the scores observed in the community sample are even higher than those observed in the patient sample. In this sense, it is likely that the assessment of this specific symptom of obsessive–compulsive disorder through the IDAS-II needs to be taken with caution when administered during COVID-19 or similar conditions.
Concerning the second aim, this study provides for the first time cutoffs for different levels of impairment according to the ICF, as measured by WHODAS 2.0. Therefore, present results contribute to the interpretablity of the IDAS-II, assigning qualitative meaning to quantitative scores (Terwee et al., 2007). Up to our knowledge, clinical utility of IDAS-II was limited to the study of its discriminative abitlity to diferentiate between distinct disorders (Stasik-O’Brien et al., 2019). However, emotional disorders generate high disability in different areas of people’s live even beyond patients samples (Guilera et al., 2020). Our study could support this finding, our values of the IDAS-II scales are also differentially associated with moderate and severe impairment in the community sample. Our findings more specifically show that both General Depression and Dysphoria are the scales that most diferentiate among level of impairment. Previous research suggested that the General Depression IDAS-II scale can be used to screen for the presence of internalizing psychopathology (Stasik-O’Brien et al., 2019; Watson et al., 2007). Similarly, Dysphoria has been noted as a nonspecific assessment of core emotional symptoms of depression and anxiety, representing the general distress dimension of psychopathology (Watson et al., 2007). Overall, our findings support to use these scales, for both screening purposes on one hand, as well a to evaluate the impact of internalizing symptoms on impairment on the other hand.
On the other hand, it is necessary to emphasize that some specific scales of the IDAS-II show a limited ability to discriminate between degree of impairment severity in the current study (e.g. euphoria, well-being, traumatic intrusions, checking) and show lower effect sizes. To maximize the proportion of patients correctly classified among the most severe impairment group, the present study applied a .70 as the minimum value for specificity.. Although it might be very interesting to extend the evidence for other sensitivity and specificity parameters in further research, the current results provide useful guidance for clinical decision making (Pintea & Moldovan, 2009).
Finally, IDAS-II scales explained 57% of the variability of the WHODAS 2.0 scores. Consistent with previous studies (Löwe et al., 2008; Tanner et al., 2019), we observed that two of the scales with the greatest explanatory capacity for impairment are Lassitude and General depression. This finding is in congruence with studies that point to depression as the leading cause of disability worldwide and the fact that the disorder is responsible for the highest proportion of disability-adjusted life-years (DALYs) (GBD Mental Disorders Collaborators, 2022; WHO, 2017). In addition, the scales of Claustrophobia, Panic indicate these two anxiety- related symptoms are most related with impairment. Although the relationship between panic and impairment has been widely reported in the literature (Batelaan et al., 2012; Cha et al., 2022), studies associating specific phobias with impairment are scarce and showed mixed results (Burstein et al., 2012; Emmelkamp & Ehring, 2014; Essau et al., 2000; Stinson et al., 2007). In this sense, the present study adds a new piece of evidence to the possible contribution of phobic symptoms (i.e. claustrophobia) to impairment.
From a clinical standpoint, the present study contributes in a novel and useful way to the identification of impairment by providing cutoffs for IDAS-II scales. Despite the benefits that the administration of PROM can bring in clinical routine, several studies indicated scarce use of specific PROM tests (Lewis et al., 2020). For tests and scales to be regularly administered in clinical sessions, it is necessary that their scores contribute clearly to clinical decision making (Kraemer et al., 2004; Widiger & Samuel, 2005). The cutoffs provided in present paper, allow to describe each patient profile in detail and to identify those emotional symptoms on which psychological interventions should target. Moreover, knowledge of which symptoms are most related with impairment, allows healthcare providers to improve treatment planning based on empirical evidence. Finally, the information provided can be useful to assess the impact of treatment by using outcome monitoring. A patient is expected to present with a score above the cutoff at the start of treatment. After several sessions, clinicians can observe whether the patient's score has changed towards the range of functional values (below the cutoff), or whether the observed changes are attributable to measurement error (scores in the range of the cutoff confidence interval).
Despite the promising results, we must also report some limitations. First, the sample of mental health service patients was not drawn by a randomized procedure. Probably, the difference between the patients who participated and those who did not participate lies exclusively in the availability of the time needed to administer the instruments. Therefore, we consider that the impact of this issue on results is limited. Furthermore, although the socio-demographic characteristics of the two samples appeared to be different, these differences are a reflection of different population characteristics, with, for example, the female gender and younger people having higher rates of internalizing disorders in previous research (Jalnapurkar et al, 2018; Sánchez-García et al., 2021).
Second, data from the community sample was collected online, a procedure that, according to Arditte et al. (2016), could lead to higher psychopathological scores compared to other traditional data collection procedures. However, the study by Sanchez-Garcia et al. (2021) using the same data set showed that these do not differ from data from other Spanish samples collected by traditional methods (De la Rosa-Cáceres et al., 2020) nor from the American normative sample used by Nelson et al. (2018). Further, a comparative study by Weigold et al. (2013) indicated equivalence across paper-and-pencil and internet-based data collection methods.
Third, it should be noted that unexpected results have been obtained on scales such as "cleaning". This is probably due to the impact of the COVID-19 pandemic and it is unknown to what extent other scales may be affected.
Finally, we would like to point out that the patient's functional impairment has been assessed using the 12-item version of the WHODAS 2.0 (WHO, 2000). This version has shown psychometric properties equivalent to the 36-item version (Saltychev et al., 2021; Üstün et al., 2010). However, it should be noted that future studies may complement the results of this study with other functional impairment instruments and scales.
The datasets analysed during the current study are available in Arias Montano, the Institutional Repository of the University of Huelva: http://hdl.handle.net/10272/20953
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Association.
Arditte, K. A., Çek, D., Shaw, A. M., & Timpano, K. R. (2016). The importance of assessing clinical phenomena in Mechanical Turk research. Psychological assessment, 28(6), 684. https://doi.org/10.1037/pas0000217
Batelaan, N. M., Van Balkom, A. J., & Stein, D. J. (2012). Evidence-based pharmacotherapy of panic disorder: an update. International Journal of Neuropsychopharmacology, 15(3), 403–415. https://doi.org/10.1017/S1461145711000800
Burstein, M., Georgiades, K., He, J. P., Schmitz, A., Feig, E., Khazanov, G. K., & Merikangas, K. (2012). Specific phobia among US adolescents: phenomenology and typology. Depression and anxiety, 29(12), 1072–1082. https://doi.org/10.1002/da.22008
Campbell, R., Ju, A., King, M. T., & Rutherford, C. (2021). Perceived benefits and limitations of using patient-reported outcome measures in clinical practice with individual patients: A systematic review of qualitative studies. Quality of Life Research, 31, 1587–1620. https://doi.org/10.1007/s11136-021-03003-z
Cha, E. J., Hong, S., Park, D. H., Ryu, S. H., Ha, J. H., & Jeon, H. J. (2022). A network analysis of panic symptoms in relation to depression and anxiety sensitivity in patients with panic disorder. Journal of affective disorders, 308, 134–140. https://doi.org/10.1016/j.jad.2022.04.062
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
Crawford, J. R., & Henry, J. D. (2003). The Depression Anxiety Stress Scales (DASS): Normative data and latent structure in a large non‐clinical sample. British journal of clinical psychology, 42(2), 111–131. https://doi.org/10.1348/014466503321903544
Crawford, J. R., Henry, J. D., Crombie, C., & Taylor, E. P. (2001). Normative data for the HADS from a large non‐clinical sample. British Journal of Clinical Psychology, 40(4), 429–434. https://doi.org/10.1348/014466501163904
Ćwirlej-Sozańska, A., Sozański, B., Kotarski, H., Wilmowska-Pietruszyńska, A., & Wiśniowska-Szurlej, A. (2020). Psychometric properties and validation of the polish version of the 12-item WHODAS 2.0. BMC Public Health, 20, 1203. https://doi.org/10.1186/s12889-020-09305-0
De la Rosa-Cáceres, A., Stasik-O’Brien, S. M., Rojas, A. J., Sanchez-García, M., Lozano, O. M., & Díaz-Batanero, C. (2020). Spanish adaptation of the Inventory of Depression and Anxiety Symptoms (IDAS-II) and a study of its psychometric properties. Journal of Affective Disorders, 271, 81–90. https://doi.org/10.1016/j.jad.2020.03.187
Emmelkamp, P., & Ehring, T. (Eds.). (2014). The Wiley handbook of anxiety disorders. John Wiley & Sons.
Essau, C. A., Conradt, J., & Petermann, F. (2000). Frequency, comorbidity, and psychosocial impairment of anxiety disorders in German adolescents. Journal of anxiety disorders, 14(3), 263–279. https://doi.org/10.1016/S0887-6185(99)00039-0
Federici, S., Balboni, G., Buracchi, A., Barbanera, F., & Pierini, A. (2022). WHODAS-Child: psychometric properties of the WHODAS 2.0 for children and youth among Italian children with autism spectrum disorder. Disability and Rehabilitation, 1–7. https://doi.org/10.1080/09638288.2022.2071481
Ferenchick, E. K., Ramanuj, P., & Pincus, H. A. (2019). Depression in primary care: part 1—screening and diagnosis. BMJ, 365. https://doi.org/10.1136/bmj.l794
Fried, E. I., Flake, J. K., & Robinaugh, D. J. (2022). Revisiting the theoretical and methodological foundations of depression measurement. Nature Reviews Psychology. https://doi.org/10.1038/s44159-022-00050-2
GBD Mental Disorders Collaborators. (2022). Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: A systematic analysis for the Global Burden of Disease Study 2019. The Lancet Psychiatry, 9(2), 137–150. https://doi.org/10.1016/s2215-0366(21)00395-3
Gelkopf, M., Mazor, Y., & Roe, D. (2022). A systematic review of patient-reported outcome measurement (PROM) and provider assessment in mental health: goals, implementation, setting, measurement characteristics and barriers. International Journal for Quality in Health Care, 34(Supplement_1), ii13–ii27. https://doi.org/10.1093/intqhc/mzz133
Guilera Ferré, G., Pino López, Ó., Barrios Cerrejón, M. T., Rojo Rodés, J. E., Vieta Pascual, E., & Gómez Benito, J. (2020). Towards an ICF Core Set for functioning assessment in severe mental disorders: Commonalities in bipolar disorder, depression and schizophrenia. Psicothema. 32(1), 7–15. https://doi.org/10.7334/psicothema2019.186
Guzick, A. G., Candelari, A., Wiese, A. D., Schneider, S. C., Goodman, W. K., & Storch, E. A. (2021). Obsessive-Compulsive Disorder During the COVID-19 Pandemic: A Systematic Review. Current Psychiatry Reports, 23(11), 1–10. https://doi.org/10.1007/s11920-021-01284-2
Hammer-Helmich, L., Haro, J. M., Jönsson, B., Melac, A. T., Di Nicola, S., Chollet, J., ... & Saragoussi, D. (2018). Functional impairment in patients with major depressive disorder: The 2-year PER FORM study. Neuropsychiatric Disease and Treatment. 14, Article 239–249. https://doi.org/10.2147/NDT.S146098
Hasin, D. S., Sarvet, A. L., Meyers, J. L., Saha, T. D., Ruan, W. J., Stohl, M., & Grant, B. F. (2018). Epidemiology of adult DSM-5 major depressive disorder and its specifiers in the United States. JAMA Psychiatry, 75(4), 336–346. https://doi.org/10.1001/jamapsychiatry.2017.4602
Hinz, A., & Brähler, E. (2011). Normative values for the hospital anxiety and depression scale (HADS) in the general German population. Journal of psychosomatic research, 71(2), 74–78. https://doi.org/10.1016/j.jpsychores.2011.01.005
Irak, M., & Albayrak, E. O. (2020). Psychometric properties of the expanded version of the Inventory of Depression and Anxiety Symptoms in a Turkish population. Psychological Reports, 13(2), 517–545. https://doi.org/10.1177/0033294118813844
Jablensky, A. (2016). Psychiatric classifications: validity and utility. World Psychiatry, 15(1), 26–31. https://doi.org/10.1002/wps.20284
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19.
Jalnapurkar, I., Allen, M., & Pigott, T. (2018). Sex differences in anxiety disorders: A review. HSOA Journal of Psychiatry, Depression & Anxiety, 4(12), 3–16. https://doi.org/10.24966/PDA-0150/100011
Knaup, C., Koesters, M., Schoefer, D., Becker, T., & Puschner, B. (2009). Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis. The British Journal of Psychiatry, 195(1), 15–22. https://doi.org/10.1192/bjp.bp.108.053967
Kotov, R., Waszczuk, M. A., Krueger, R. F., Forbes, M. K., Watson, D., Clark, L. A., Achenbach, T. M., Althoff, R. R., Ivanova, M. Y., Michael Bagby, R., Brown, T. A., Carpenter, W. T., Caspi, A., Moffitt, T. E., Eaton, N. R., Forbush, K. T., Goldberg, D., Hasin, D., Hyman, S. E., … Zimmerman, M. (2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454–477. https://doi.org/10.1037/abn0000258
Koumpouros, Y., Papageorgiou, E., Sakellari, E., Prapas, X., Perifanou, D., & Lagiou, A. (2018). Adaptation and psychometric properties evaluation of the Greek version of WHODAS 2.0. pilot application in Greek elderly population. Health Services and Outcomes Research Methodology, 18(1), 63–74. https://doi.org/10.1007/s10742-017-0176-x
Kraemer, H. C., Noda, A., & O’Hara, R. (2004). Categorical versus dimensional approaches to diagnosis: Methodological challenges. Journal of Psychiatric Research, 38(1), 17–25. https://doi.org/10.1016/S0022-3956(03)00097-9
Krueger, R. F., Kotov, R., Watson, D., Forbes, M. K., Eaton, N. R., Ruggero, C. J., Simms, L. J., Widiger, T. A., Achenbach, T.M., Bach, B., … & Zimmermann, J. (2018). Progress in achieving quantitative classification of psychopathology. World Psychiatry, 17(3), 282–293. https://doi.org/10.1002/wps.20566
Kumar, A., & Nayar, K. R. (2020). COVID 19 and its mental health consequences. Journal of Mental Health, 30(1), 1–2. https://doi.org/10.1080/09638237.2020.1757052
Lane, S. P., & Sher, K. J. (2015). Limits of current approaches to diagnosis severity based on criterion counts: An example with DSM-5 alcohol use disorder. Clinical Psychological Science, 3(6), 819–835. https://doi.org/10.1177/2167702614553026
Lewis, C. C., Washington, K. P., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., Hoffman, M., Scott, K., Lyon, A., Douglas, S., & Simon, G. (2020). Implementing Measurement-Based Care in Behavioral Health: A Review. JAMA Psychiatry, 76(3), 324–335. Clinical Psychological Science, 3(6), 819–835. https://doi.org/10.1001/jamapsychiatry.2018.3329
Lloyd, H., Jenkinson, C., Hadi, M., Gibbons, E., & Fitzpatrick, R. (2014). Patient reports of the outcomes of treatment: A structured review of approaches. Health and Quality of Life Outcomes, 12(5), 1–9. https://doi.org/10.1186/1477-7525-12-5
Löwe, B., Spitzer, R. L., Williams, J. B., Mussell, M., Schellberg, D., & Kroenke, K. (2008). Depression, anxiety and somatization in primary care: Syndrome overlap and functional impairment. General Hospital Psychiatry, 30(3), 191–199. https://doi.org/10.1016/j.genhosppsych.2008.01.001
Markon, K. E., Chmielewski, M., & Miller, C. J. (2011). The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psychological bulletin, 137(5), 856. https://doi.org/10.1037/a0023678
Mokkink, L. B., Prinsen, C. A., Bouter, L. M., de Vet, H. C., & Terwee, C. B. (2016). The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Brazilian journal of physical therapy, 20, 105–113. https://doi.org/10.1590/bjpt-rbf.2014.0143
Morin, R. T., Nelson, C., Bickford, D., Insel, P. S., & Mackin, R. S. (2020). Somatic and anxiety symptoms of depression are associated with disability in late life depression. Aging & mental health, 24(8), 1225–1228. https://doi.org/10.1080/13607863.2019.1597013
McKnight, P. E., & Kashdan, T. B. (2009). The importance of functional impairment to mental health outcomes: A case for reassessing our goals in depression treatment research. Clinical Psychology Review, 29(3), 243–259. https://doi.org/10.1016/j.cpr.2009.01.005
McKnight, P. E., Monfort, S. S., Kashdan, T. B., Blalock, D. V., & Calton, J. M. (2016). Anxiety symptoms and functional impairment: A systematic review of the correlation between the two measures. Clinical Psychology Review, 45, 115–130. https://doi.org/10.1016/j.cpr.2015.10.005
Necho, M., Tsehay, M., Birkie, M., Biset, G., & Tadesse, E. (2021). Prevalence of anxiety, depression, and psychological distress among the general population during the COVID-19 pandemic: A systematic review and meta-analysis. International Journal of Social Psychiatry, 67(7), 892–906. https://doi.org/10.1177/00207640211003121
Nelson, G. H., O’Hara, M. W., & Watson, D. (2018). National norms for the expanded version of the Inventory of Depression and Anxiety Symptoms (IDAS-II). Journal of Clinical Psychology, 74(6), 953–968. https://doi.org/10.1002/jclp.22560
Obbarius, A., van Maasakkers, L., Baer, L., Clark, D. M., Crocker, A. G., de Beurs, E., ... & Rose, M. (2017). Standardization of health outcomes assessment for depression and anxiety: recommendations from the ICHOM Depression and Anxiety Working Group. Quality of Life Research, 26(12), 3211–3225. https://doi.org/10.1007/s11136-017-1659-5
Østergaard, S. D., Jensen, S. O. W., & Bech, P. (2011). The heterogeneity of the depressive syndrome: when numbers get serious. Acta Psychiatrica Scandinavica. https://doi.org/10.1111/j.1600-0447.2011.01744.x
Øvretveit, J., Zubkoff, L., Nelson, E. C., Frampton, S., Knudsen, J. L., & Zimlichman, E. (2017). Using patient-reported outcome measurement to improve patient care. International Journal for Quality in Health Care, 29(6), 874–879. https://doi.org/10.1093/intqhc/mzx108
Park, L. T., & Zarate, C. A., Jr. (2019). Depression in the primary care setting. New England Journal of Medicine, 380(6), 559–568. https://doi.org/10.1056/NEJMcp1712493
Pintea, S., & Moldovan, R. (2009). The receiver-operating characteristic (ROC) analysis: Fundamentals and applications in clinical psychology. Journal of Evidence-Based Psychotherapies, 9(1), 49.
Power, M., Fell, G., & Wright, M. (2013). Principles for high-quality, high-value testing. Evidence-Based Medicine, 18(1), 5–10. https://doi.org/10.1136/eb-2012-100645
Rapaport, M. H., Clary, C., Fayyad, R., & Endicott, J. (2005). Quality-of-life impairment in depressive and anxiety disorders. American Journal of Psychiatry, 162(6), 1171–1178. https://doi.org/10.1176/appi.ajp.162.6.1171
Saltychev, M., Katajapuu, N., Bärlund, E., & Laimi, K. (2021). Psychometric properties of 12-item self-administered World Health Organization disability assessment schedule 2.0 (WHODAS 2.0) among general population and people with non-acute physical causes of disability–systematic review. Disability and Rehabilitation, 43(6), 789–794. https://doi.org/10.1080/09638288.2019.1643416
Sanchez-Garcia, M., De la Rosa-Cáceres, A., Stasik-O’Brien, S., Mancheño-Barba, J.J., Lozano, O. M., & Díaz-Batanero, C. (2021). Norms according to age and gender for the Spanish version of the Inventory of Depression and Anxiety Symptoms (IDAS-II). Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.748025
Sharma, H. (2021). Statistical significance or clinical significance? A researcher’s dilemma for appropriate interpretation of research results. Saudi Journal of Anaesthesia, 15(4), 431–434. https://doi.org/10.4103/sja.sja_158_21
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of consulting and clinical psychology, 78(3), 298. https://doi.org/10.1037/a0019247
Stanton, K., McDonnell, C. G., Hayden, E. P., & Watson, D. (2020). Transdiagnostic approaches to psychopathology measurement: Recommendations for measure selection, data analysis, and participant recruitment. Journal of Abnormal Psychology, 129(1), 21. https://doi.org/10.1037/abn0000464
Stasik-O’Brien, S. M., Brock, R. L., Chmielewski, M., Naragon-Gainey, K., Koffel, E., McDade-Montez, E. A., O’Hara, M. W., & Watson, D. (2019). Clinical utility of the Inventory of Depression and Anxiety Symptoms (IDAS). Assessment, 26(5), 944–960. https://doi.org/10.1177/1073191118790036
Stinson, F. S., Dawson, D. A., Chou, S. P., Smith, S., Goldstein, R. B., Ruan, W. J., & Grant, B. F. (2007). The epidemiology of DSM-IV specific phobia in the USA: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Psychological medicine, 37(7), 1047–1059. https://doi.org/10.1017/s0033291707000086
Tanner, J., Zeffiro, T., Wyss, D., Perron, N., Rufer, M., & Mueller-Pfeiffer, C. (2019). Psychiatric symptom profiles predict functional impairment. Frontiers in Psychiatry, 10(FEB). https://doi.org/10.3389/fpsyt.2019.00037
Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., ... & de Vet, H. C. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. https://doi.org/10.1016/j.jclinepi.2006.03.012
Trivedi, M. H. (2009). Tools and strategies for ongoing assessment of depression: A measurement-based approach to remission. Journal of Clinical Psychiatry, 70(SUPPL. 6), 26–31. https://doi.org/10.4088/JCP.8133su1c.04
Üstün, T. B., Chatterji, S., Kostanjsek, N., Rehm, J., Kennedy, C., Epping-Jordan, J., Saxena, S., von Korff, M., & Pull, C. (2010). Developing the World Health Organization Disability Assessment Schedule 2.0. Bulletin of the World Health Organization, 88(11), 815–823. https://doi.org/10.2471/BLT.09.067231
Vázquez-Barquero, J. L., Vázquez Bourgón, E., Herrera Castanedo, S., Saiz, J., Uriarte, M., Morales, F., Gaite, L., Herrán, A., & Ustün, T. B. (2000). Spanish version of the new World Health Organization Disability Assessment Schedule II (WHO-DAS-II): initial phase of development and pilot study. Actas Espanolas de Psiquiatria, 28(2), 77–87. PMID: 10937388
Watson, D., & O’Hara, M. W. (2017). Understanding the emotional disorders: A symptom level approach based on the IDAS-II. Oxford University Press.
Watson, D., O’Hara, M. W., Simms, L. J., Kotov, R., Chmielewski, M., McDade-Montez, E. A., Gamez, W., & Stuart, S. (2007). Development and validation of the Inventory of Depression and Anxiety Symptoms (IDAS). Psychological Assessment, 19(3), 253–268. https://doi.org/10.1037/1040-35126.96.36.199
Watson, D., O’Hara, M. W., Naragon-Gainey, K., Koffel, E., Chmielewski, M., Kotov, R., Stasik-O’Brien, S. M., & Ruggero, C. J. (2012). Development and validation of new anxiety and bipolar symptom scales for an expanded version of the IDAS (the IDAS-II). Assessment, 19(4), 399–420. https://doi.org/10.1177/1073191112449857
Weigold, A., Weigold, I. K., & Russell, E. J. (2013). Examination of the equivalence of self-report survey-based paper-and-pencil and internet data collection methods. Psychological Methods, 18(1), 53–70. https://doi.org/10.1037/a0031607
Whiteford, H. A., Degenhardt, L., Rehm, J., Baxter, A. J., Ferrari, A. J., Erskine, H. E., Charlson, F. J., Norman, R. E., Flaxman, A. D., Johns, N., Burstein, R., Murray, C. J. L., & Vos, T. (2013). Global burden of disease attributable to mental and substance use disorders: Findings from the Global Burden of Disease Study 2010. The Lancet, 382(9904), 1575–1586. https://doi.org/10.1016/S0140-6736(13)61611-6
Widiger, T. A., & Samuel, D. B. (2005). Diagnostic categories or dimensions? A question for the diagnostic and statistical manual of mental disorders — Fifth edition. Journal of Abnormal Psychology, 114(4), 494–504. https://doi.org/10.1037/0021-843X.114.4.494
World Health Organization. (2000). Disability Assessment Schedule II (WHO- DAS II). WHO.
World Health Organization. (2013). How to use the ICF: A practical manual for using the International Classification of Functioning. WHO.
World Health Organization. (2017). Depression and other common mental disorders. Global Health estimates. World Health Organization.
Yorkston, K., & Baylor, C. (2019). Patient-reported outcomes measures: An introduction for clinicians. Perspectives of the ASHA Special Interest Groups, 4(1), 8–15. https://doi.org/10.1044/2018_PERS-ST-2018-0001
Zimmerman, M., Ellison, W., Young, D., Chelminski, I., & Dalrymple, K. (2015). How many different ways do patients meet the diagnostic criteria for major depressive disorder?. Comprehensive psychiatry, 56, 29–34. https://doi.org/10.1016/j.comppsych.2014.09.007
Funding for open access publishing: Universidad de Huelva/CBUA. This work was supported by the grant “Reliable and clinical relevant change of Inventory of Depression and Anxiety Symptoms II – IDAS-II: a longitudinal clinical utility study (RELY-IDAS-II)”, project PID2020-116187RB-I00 on Proyectos I + D + i 2020 “Retos del Conocimiento” provided by Ministerio de Ciencia e Innovación (Spain); grant number FPU19/00144 provided by Ministerio de Universidades (Spain) and grant number PRX21/00319 provided by Ministerio de Ciencia e Innovación (Spain).
The study was approved by the Bioethics Committee of the Province of Huelva (Junta de Andalucía, Spain) (No. PY18-4584).
Informed consent was obtained from all individual participants involved in the study.
Conflicts of Interest
A. De la Rosa-Cáceres, O. M. Lozano, M. Sanchez-Garcia, F. Fernandez-Calderon, G. Rossi and C. Diaz-Batanero declare that there are no conflicts of interest to report.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
De la Rosa-Cáceres, A., Lozano, O.M., Sanchez-Garcia, M. et al. Assessing Internalizing Symptoms and Their Relation with Levels of Impairment: Evidence-Based Cutoffs for Interpreting Inventory of Depression and Anxiety Symptoms (IDAS-II) Scores. J Psychopathol Behav Assess 45, 170–180 (2023). https://doi.org/10.1007/s10862-022-10008-6
- Emotional Disorders