Introduction

Clinical decision-making is a hallmark of clinical psychology, psychiatry, and psychotherapy, being a core skill in mental health responsiveness (Norcross & Goldfried, 2019; Norcross & Wampold, 2019). Recent empirical research suggests that variance in psychotherapy outcomes can be attributed to patient variables (30%), therapeutic relationship (15%), intervention methods/techniques (10%), individual therapist variables (7%) and other factors (3%) such as therapeutic setting. The other 35% is attributable to unexplained variance (Norcross & Wampold, 2019). These generic research factors at a higher level of abstraction encompass several concrete elements that, at a lower level of abstraction, can be differentiated into basic therapeutic elements (e.g., strategies, goals, and tasks) related to interactions between patient and therapist (Goldfried, 2019). Within patient variables, several meta-analytic studies revealed a cluster of patient characteristics and behaviors that showed to be consistently associated with psychotherapy outcomes (Beutler et al., 2005, 2018; Flückiger et al., 2018; Krebs et al., 2018; Levy et al., 2018). In this sense, these patient variables might be candidates for the development of a new assessment measure focused on clinical decision-making, which is the aim of this study.

Previous instruments have focused on clients’ preferences that were thought to be related to psychotherapy outcomes. The Cooper-Norcross Inventory of Preferences (CNIP, Cooper & Norcross, 2016) is a self-report instrument designed to assess patients’ preferences regarding therapist directiveness, emotionality, time orientation and support/challenge. The Therapy Personalisation Form (TPF, Bowen & Cooper, 2012) is an assessment instrument with two forms, one for the assessment phase and the other for the therapy sessions with 20 semantic differential items focused on the client’s preferences for therapist activities. The Psychotherapy Preferences and Experiences Questionnaire (PEX, Sandell et al., 2011) is a 29-item measure focused on the assessment of how individuals evaluate therapist activities, therapist characteristics, and client activities as beneficial to them. The Counseling Preference Form (CPF, Goates-Jones & Hill, 2008) is a self-report measure that asks patients to rate 10 therapists which activities they prefer that counselors use in counseling sessions. Similarly, The Preference for College Counselling Inventory (PCCI, Hatchett, 2015) is a 90-item instrument that asks patients for therapist characteristics, therapist activities, and client activities. These assessment instruments were almost focused on patients’ preferences rather than patient variables, and this aspect may support the need for a new measure focused on patients’ variables associated with psychotherapy outcomes.

Furthermore, metanalytic studies refer to such patient variables as the specific traits, behaviors, and therapeutic timings that research showed to be associated with psychotherapy outcomes (Norcross & Wampold, 2019). The variables are emotional stability (severity and complexity), therapeutic alliance, motivational stage, reactance level, coping style and attachment style. However, the stages of psychotherapy are not derived from metanalytic studies and are referred to as one core domain for clinical decision-making (Vasco et al., 2018). Emotional stability depends on the severity and complexity of the problem and is directly associated with emotional suffering and long-lasting dysfunctional patterns (Beutler et al., 2005). The therapeutic alliance is defined as the emotional bond and articulation between tasks and goals between the patient and psychotherapist (Flückiger et al., 2018). The motivational stage is the readiness for change that people experience at each moment in time to pursue transformational actions (Krebs et al., 2018). Reactance level is the specific way that individuals react to psychotherapeutic proposals (Beutler et al., 2018). Coping styles are individuals’ internalization or externalization tendencies in dealing with distress, emotional pain, and suffering (Beutler et al., 2011). Attachment style is the characteristic and recurring pattern of thoughts, feelings, and behaviors that individuals exhibit when forming emotional bonds with significant others (Levy et al., 2018). Finally, stages of psychotherapy are the sequential stages that individuals enter during the psychotherapy process (e.g., stage one– alliance formation and bonding, stage two-consciousness awareness, stage three—meaning-making, stage four –regulation of responsibility, stage five—repairing actions, stage six—consolidation of change and stage seven—anticipation of the future and relapse prevention), which are significant to specific patient acquisitions (Vasco et al., 2018). Despite this empirical evidence, no single assessment instrument is specifically designed to gauge these constructs within a unified measure, even though research has consistently demonstrated their strong correlation with psychotherapy outcomes (Norcross & Wampold, 2019).

The main idea of the CDMI is to have a measure that can rapidly assess several clinical constructs related to psychotherapy outcomes. For instance, if a patient has an internalizing coping style, clinicians should focus on insight/restructuring strategies, while clinicians should focus more on behavioral tasks if a patient has an externalizing coping style (Beutler et al., 2005, 2018). Another example is, if a patient has difficulties in forming secure attachments, the clinician should focus more on the relational domains of therapy. Therapeutic strategies are expected to match the patient’s styles and needs by correctly assessing these constructs. Developing a new measure encompassing several clinical decision-making constructs may not only be a cost-effective reduction in time, but it may also function as a relevant unified instrument based on various empirical evidence.

Several constructs were selected to start the CDMI validation process. Previous research has emphasized several significant constructs associated with symptomatology and psychological distress, that may relate to clinical decision-making.

Metacognition defined as a set of higher-order mental abilities related to identification, decentration, self-reflection, and mastery was previously associated with symptomatology (Faustino et al., 2021a; Semerari et al., 2003). Nevertheless, another metacognitive model (Wells & Matthews, 1994; Wells, 2009), can be used to explore if these results are stable. However, the Metacognitions Questionnaire 30 (MCQ-30, Wells & Cartwright-Hatton, 2004) is focused on metacognitive beliefs, which is a specific form of metacognitive variables and in the present study, we believe that a broader notion of metacognition can be an asset to perform a preliminary analysis of the CDMI. Cognitive fusion (as a measure of psychological inflexibility), cognitive reappraisal, and expressive suppression have been associated with symptomatology (Faustino, 2021), attachment styles (Gardner et al., 2020), and emotional instability (Carlo et al., 2012). Interpersonal problems have been associated with difficulties in forming alliances and secure attachment (Wong & Pos, 2014). Finally, coping states of mind which may be regarded as state-like manifestations of coping mechanisms were previously associated with symptomatology (Faustino et al., 2021b) and therapeutic alliance (Black et al., 2013). Therefore, based on this previously documented evidence, these constructs were selected to explore the convergent and divergent validity in the present study. The core idea behind the development of this measure is that if the therapist adopts his intervention according to the patient's characteristics, this will increase responsiveness. Decisions can be taken as a function (but not adhered to) of the degree of emotional suffering, of the therapeutic relational posture due to the attachment style, of the strategies centered on the coping style and of the actions centered on the motivational level. In this sense, from a transtheoretical perspective different variables of symptomatology, cognition, emotion regulation, coping, and interpersonal behaviors were selected based on previous research related to the variables measured by the CDMI.

Study Aims

This study focuses on the development and preliminary psychometric study of the CDMI through an Exploratory factor Analysis, convergent, divergent, and predictive validity with theoretically related constructs. To our knowledge, this is the first attempt to develop a clinical decision-making instrument based on empirical data. Therefore, we expected that the CDMI has a unidimensional structure (H1). Expressive suppression, symptomatology, interpersonal problems, coping mechanisms, and cognitive fusion are expected to correlate negatively with all CDMI constructs showing convergent validity (H2). Metacognition is a higher-order intrapsychic cognitive construct different from some CDMI constructs, namely, stages in psychotherapy, therapeutic alliance, and emotional stability. Therefore, metacognition is expected to not correlate with these variables, showing evidence of divergent validity (H3). Finally, CDMI variables are expected to have predictive value on symptomatology, interpersonal problems, and cognitive fusion (variables related to psychotherapy outcome).

Methods

Participants

The sample consisted of 123 participants who were undergraduate students in the Faculty of Psychology of the University of Lisbon, 15 males (12.2%) and 108 females (87.7%), with an age range between 18 and 57 years old (M = 20.80, SD = 5.28). The majority of participants completed the 12th grade (90.2%), while 7 (5.7%) held bachelor's degrees, and 5 (4.1%) possessed master's degrees. Most of the sample was Portuguese (90.2%). Regarding marital status, 119 participants (96.7%) reported being single, 1 (0.8%) indicated being married, 1 (0.8%) reported being in common-law union, and 2 (1.6%) were divorced. All participants actively engaged in psychotherapy participated in the psychotherapy processes, and various self-reported diagnostic criteria were identified among them. Generalized anxiety disorder was reported by 11 participants (2.2%), major depression by 8 (1.6%), and depression with anxiety by 7 (1.4%). A detailed breakdown of demographic variables is provided in Table 1.

Table 1 Descriptive statistics of the sample under study (N = 123)
Full size table

Measures

Preliminary Scale Development of the Clinical Decision-Making Inventory

The Clinical Decision-Making Inventory (CDMI) is an assessment instrument based on previous meta-analytic findings regarding patient characteristics and variables that predict psychotherapy outcomes. Based on the previous literature reviewed in several metanalytic studies, seven constructs were selected: therapeutic relationship, motivational level, reactance, emotional stability, coping style, attachment style and stages in psychotherapy.

There were seven constructs; the initial pool of items was composed of 10 items per dimension. The authors wrote the items after a careful reading of other instruments, such as the Brief Revised Working Alliance Inventory (BR-WAI, Horvath & Greenberg, 1989) and Ways of Coping Questionnaire (WCQ, Folkman & Lazarus, 1988) and Adult Attachment Scale-R (AASR, Collins & Read, 1990). A similar method was used in the CPF development where individuals must rate 10 therapeutic activities based on a priori assumptions regarding core dimensions of patient’s preferences (Goates-Jones & Hill, 2008). All items were written to assess the construct from the patient’s perception. Our aim was to develop a short decision-making scale with a single representative item of each construct. Thus, evidence of single-item measures may be found in the literature (e.g., Pollack and Alexandrov, 2013; Sauro and Dumas, 2009). Moreover, according to Freed (2013), a single-item measure may suffice to construct a measure if it is sufficiently narrow or is unambiguous to the respondent (e.g., the measurement of subjective probabilities, such as future behaviors), but for more complex behaviors a multiple item system may be better fitted. In this sense, a new reading of the items was performed and most objective items were selected narrowing the chances of interpretation for the respondents. Thus, we tried to make the items as straightforward as possible, leaving an unambiguous notion about what was asked of the participant. By doing this, we matched the notion of unambiguity and concreteness to respondents suggested by Freed (2013). Some examples include Attachment style: “In terms of your ability to create emotionally satisfying attachments, how do you rate yourself?” Emotional Stability: “Regarding your stability/emotional comfort, how do you rate yourself?” Then, 2 independent judges assessed the quality of the items regarding the objective and the association with the belonging construct. Independent judges were professional psychotherapists with more than 5 years of clinical experience. They gave their qualitative assessment in terms of adequate or inadequate assessment of several aspects: item wording, item comprehensives, conceptual clarity, and construct representativeness. Their assessment was satisfactory for all items.

Based on the theoretical underpinnings of each dimension, a 5-point Likert scale was attributed. The instructions ask participants to rate themselves in the described seven domains of psychotherapy, namely, emotional stability (severity and complexity), stages of psychotherapy, therapeutic alliance, motivational stage, reactance level, coping style, and attachment style. Despite the stages of psychotherapy based on the Paradigmatic Complementarity Model (PCM, Vasco et al., 2018) having 7 stages, here we fused stage 3 with stage 4 (awareness and meaning-making) and stage 6 with stage 7 (consolidation of change and relapse prevention), to match with the 5-point Likert scale of the previous items. It briefly describes the different therapeutic stages, and individuals are asked to rate themselves in which phase they are engaged. Finally, before administering the inventory to this sample, the CDMI was given to 10 randomly assigned individuals to test conceptual clarity and comprehensiveness. Their feedback was satisfactory for all items.

Brief Symptom Inventory (BSI-53)

The BSI (Derogatis, 1993, European Portuguese version by Canavarro, 1999), is a self-report instrument that assesses psychopathological symptoms (e.g., depression, somatization). In the present study, the general index showed an adequate internal consistency (α = .96).

Inventory of Interpersonal Problems (IIP-32)

IIP-32 (Barkham et al., 1996; European Portuguese version by Faustino & Vasco, 2020a), is a self-report instrument focused on the assessment of eight interpersonal domains divided into two dimensions (affiliation and dominance). In the present study, the total index was used, showing adequate reliability (α = .87).

Emotion Regulation Questionnaire (ERQ)

Emotion regulation was assessed by ERQ (Gross & John, 2003; European Portuguese version by Vaz & Martins, 2009). The ERQ is a self-reporting measure focused on assessing two regulatory strategies, which showed adequate internal consistency (cognitive reappraisal, α = .78; emotional suppression, α = .75).

Metacognition Self-Assessment Scale (MSAS)

The MSAS (Pedone et al., 2017; European Portuguese version by Faustino et al., 2021a) is a self-report measure with 18 items intended to assess MMFM sub-functions. It is scored on a five-point Likert scale with a raw score that ranges from 18 to 90. High scores indicate better metacognitive self-evaluation on metacognitive abilities. The total index of Cronbach alfas was considered acceptable (α = .72).

States of Mind Questionnaire (SMQ)

The SMQ (Faustino et al., 2021b) is a self-report instrument focused on the assessment of twenty-four different states of mind, grouped into four higher-order domains, namely: vulnerable states (e.g., Abandonment/Non-lovability, item 1—I feel abandoned, alone and without value), coping states (e.g., Subjugation/Constriction, item 55—I subjugate myself to desires of certain people to avoid confrontation) egosintonic states (e.g., Pleasure-seeking, item 62—I’m always looking to have fun and enjoy), and self-care states (e.g., Acceptance/Mindfulness, item 84—I can understand and satisfy my needs as a person). It has 84 items and is scored on a six-point Likert scale. In the present study, only the sub-scale of coping states was used, which showed adequate internal consistency (α = .94).

Cognitive Fusion Questionnaire (CFQ)

The CFQ (Gillanders et al., 2014; European Portuguese version by Pinto-Gouveia, et al., 2017) is a self-report instrument to evaluate the degree of cognitive fusion. In the present study, the CFQ showed adequate reliability (α = .92).

Procedures and Statistical Analysis

Participants were evaluated individually within a 5-day maximum period to complete the research protocol. Exclusion criteria were being less than 18 years old and having a major neurocognitive disorder diagnosis. All participants gave informed consent, and there was no compensation for participating in the study. This research was approved by the ethics committee of the Faculty of Psychology of the University of Lisbon. Descriptive statistics were used to explore sociodemographic description, using frequencies, percentages, averages, standard deviations, and confidence intervals. All multicollinearity values showed to be adequate |VIF < 2; T < 7|, normal distribution was assumed (N > 30) and a 95% confidence interval was assumed with a p-value of .05 (Pallant, 2007). To explore the association between constructs, Pearson correlations were used. The predictive value was explored with Stepwise regression analysis. All statistical analysis was performed in IBM SPSS Statistics version 25.

Results

Factor Structure

To explore the factor structure of the CDMI, an Exploratory Factor Analysis (EFA) was used. Kaiser-Mayer-Olkin showed a satisfactory value (.733), and the Bartlett sphericity test was [216,564, p < .001], showing adequate associations between variables (Field, 2009). Eigenvalues higher than 1 were extracted with an oblimin rotation procedure, which converged in 5 iterations. EFA showed two factors that accounted for 57.4% of the variance. Items below .40 were removed. Factor 1 explained 42.6% of the variance being named Therapeutic Engagement; factor 2 accounted for 14.7% of the variance and was named Clinical Styles—see Table 2.

Table 2 Exploratory factor analysis of the clinical decision-making inventory (N = 123)
Full size table

The internal consistency of the Therapeutic Engagement (factor 1) was .65 and the internal consistency of the Clinical Styles (factor 2) was .59. Based on these lower alpha values, a one-factor solution was forced and explained 57,4% of the variance. It shows a Kaiser-Mayer-Olkin value of .733 and a significant Bartlett sphericity [216,564, p < .001], which were considered satisfactory. These values are the same as the two-solution factor. The total scale alfa was considered acceptable and was used in the present study (α = .72).

Table 3 shows the Pearson correlations matrix between all CDMI subscales. All correlations are statistically significant, ranging from low to high. Stage of psychotherapy correlated positively with therapeutic relationship (r = .48, p < .05), motivational level (r = .40, p < .05), reactance (r = .29, p < .05), emotional stability (r = .39, p < .05), coping style (r = .18, p < .05), and attachment style (r = .22, p < .05). Only emotional stability did not correlate with coping style.

Table 3 Pearson correlations matrix between all CDMI subscales (N = 123)
Full size table

Table 4 shows Pearson correlations between all CDMI subscales with the other clinical variables used to explore convergent and divergent validities. Most correlations were statistically significant, ranging from low to high. Adaptive variables correlated positively, and dysfunctional variables correlated negatively with the CDMI subscales. Metacognition correlated positively with motivational level (r = .22, p < .05), reactance (r = .22, p < .05), coping style (r = .22, p < .05) and attachment style (r = .22, p < .05).

Table 4 Pearson correlations matrix between all CDMI subscales and Clinical Variables (N = 123)
Full size table

Similarly, cognitive reappraisal correlated positively with stages of psychotherapy (r = .26, p < .05), therapeutic relationship (r = .18, p < .01), motivational level (r = .23, p < .05), reactance (r = .24, p < .05), emotional stability (r = .22, p < .01), coping style (r = .26, p < .05) and attachment style (r = .23, p < .05). Symptomatology correlated negatively with stage of psychotherapy (r = − .18, p < .05), therapeutic relationship (r = − .17, p < .05), motivational level (r = − .24, p < .05), reactance (r = − .29, p < .05), emotional stability (r = − .49, p < .05), coping style (r = − .30, p < .05) and attachment style (r = − .53, p < .05). Also, interpersonal problems correlated negatively with stages of psychotherapy (r = − .23, p < .05), therapeutic relationship (r = − .29, p < .05), motivational level (r = − .41, p < .05), reactance (r = − .32, p < .05), emotional stability (r = − .47, p < .05), coping style (r = − .52, p < .05) and attachment style (r = − .35, p < .05)—see Table 4.

Table 5 describes the three Stepwise Regression analyses with CDM subscales on symptomatology, interpersonal problems, and cognitive fusion. Only these variables were selected because they tend to be sensitive to changes resulting from psychotherapy outcomes. Symptomatology was predicted by a model with two variables (β = − .32, t = − 4.16, p < .01). Interpersonal problems were predicted by a model with three variables (β = − .21, t = -2.82, p < .01). Cognitive fusion was predicted by the model with three variables (β = − .18, t = -2.62, p < .01).

Table 5 Stepwise regression analysis of the CDMI subscales on symptomatology, interpersonal problems, and cognitive fusion (N = 123)
Full size table

Discussion

Study aims were achieved, which was to start a preliminary study of the CDMI. This is just the initial study, and much work must be done. Developing an empirically based instrument to help clinicians make decisions is not an easy task. We regard this effort as the beginning of a line of research focused on the CDMI. Hypothesis one was not confirmed. The CDMI showed a two-factor solution, however, both factors had a low internal consistency, which result from the confluence of different constructs. In this sense, a unidimensional scale was computed to explore further validation. Nevertheless, to some extent, the two-factor solution (therapeutic engagement and clinical styles) matches the underlying variables. Clinical decision-making takes into consideration several psychological variables that are very different, and this is reflected in the study instrument (Beutler et al., 2005). For instance, stages in psychotherapy or motivational level differ from attachment and reactance. However, they are still correlated, so a unidimensional factor structure may be adequate to represent the theoretical assumptions of the CDMI. As detailed before, single-item measurements seem to be sufficient if the construct is clearly described (Freed, 2013). Despite our effort to match this principle, exploring the constructs contained in the CDMI through multi-item measurement is also plausible. Both versions need to be studied simultaneously to explore validity criteria.

Hypothesis two was partially confirmed; however, the results were mostly aligned with theoretical predictions. Only expressive suppression did not correlate with stages in psychotherapy, therapeutic relationship, motivational level, and emotional stability. A previous study showed that expressive suppression does not differentiate between non-clinical and clinical samples (Faustino, 2021), which is why it did not correlate with the specific variables associated with the psychotherapy process (e.g., therapeutic relationship). However, all other variables were significantly correlated. For instance, it was expected that symptomatology and interpersonal problems were correlated with emotional stability, therapeutic relationships, and attachment styles because these variables were associated previously (Gardner et al., 2020; Wong & Pos, 2014). Also, it was expected that coping mechanisms and cognitive fusion were correlated with emotional stability and coping styles, first because of the previous studies (Faustino, 2021; Faustino et al., 2021b) and the similarities between coping mechanisms and coping styles. Therefore, matching theoretical assumptions implies, to some extent, some preliminary evidence of convergent validity.

Divergent validity was explored through correlations between constructs that were somewhat different from the CDMI variables. Metacognition did not correlate with stages in psychotherapy, therapeutic alliance, and emotional stability, confirming the third hypothesis. Metacognition as defined by Semerari et al. (2003) and measured by the MSAS (Pedone et al., 2007) seems to be very different from these CDMI variables, such as stage in psychotherapy, therapeutic relationship, and emotional stability. In this sense, these results match previous theoretical assumptions and may be considered as preliminary evidence of divergent validity.

Finally, it was expected that some of the CDMI variables showed predictive value on symptomatology, interpersonal problems, and cognitive fusion. Only a few variables showed predictive value, non-confirming the fourth hypothesis. However, these results may imply that some CDMI variables (e.g., emotional stability and attachment style) have predictive value on other clinical variables, which makes them relevant for clinical decision-making and differentiated targets for psychological interventions (Faustino & Vasco, 2020b, 2020c).

Limitations and Future Directions

Some limitations may be described. The one-item measure may be attractive to clinicians and researchers. However, these instruments may lack the dimensionality required to capture the measured construct. The selection of CDMI items was based on assessments by independent clinical experts, allowing the development of a single-item per domain. However, item selection would also benefit from applying the 10 items per construct that were elaborated on and then explored through a correlational analysis. Item narrowing would be performed by analyzing the items that best correlated with the others or a 10-item total score. This process should be conducted in the future. Data was gathered with self-reported instruments, which are limited to participants' self-awareness of the given constructs. This study was conducted online, without the supervision of the main researcher. The sample size (N = 123) is small, and this study was conducted with university students engaged in psychotherapy, which limits generalizations and extrapolations of the results. The sample had more female responders than males, which could have introduced biased results. In the future, CDMI should be studied in community samples. The items must be correlated similar constructs to deepen the convergent and divergent validity. Also, a Rach analysis may be used to increase scale reliability as the exploration of associations with other clinically related constructs, such as maladaptive core schemas (Faustino, 2023). Moreover, other variables from different theoretical orientations should be used to explore the convergent and divergent validity of the CDMI, along with similar assessment instruments such as the CNIP (Cooper &  Norcross, 2016).

Conclusions

Preliminary results suggest that the CDMI showed the potential to become a reliable instrument to help clinicians in tailor interventions to patients’ specifications. This assessment instrument may offer a cost-effective choice in the decision-making process while being a coherent empirical-based instrument. Nevertheless, more research is required to explore the psychometric properties of the CMDI before it can be used as a reliable measure.