Introduction

The prediction of outcome is an important issue in psychotherapy research. The endeavor of identifying contributors and predictors is motivated by the hope of optimizing the results of psychotherapy and of making the results more sustainable for a greater number of patients. Especially during the last decade, the necessity for optimizing psychotherapy, linking it more to the clinical practice, has become evident (Lambert, 2013; Cuipers et al., 2018; Shedler, 2018). In the tradition of common factors research (“contextual model”, Wampold & Imel, 2015), the importance of factors such as the alliance and cooperation between patient and therapist in explaining the variance of the outcome has been proven (e.g., goal consensus and collaboration, empathy, alliance, positive regard/affirmation, congruence/genuineness). In addition to this, factors such as expectations, rituals, or the cultural fit of the treatment to the patient, and many patient-related aspects like comorbidity, chronification, axis-II diagnoses (personality disorders), competencies, resources, motivation for change, self-efficacy, social support and social networks play an important role in influencing the course of psychotherapy, promoting or impeding change (Duncan et al., 2010; Sparks & Duncan, 2010).

Commonly, the literature tries to confirm that common factors, such as alliance, are real independent variables, not just co-evolving variables dependent on the micro- and meso-outcomes of the process (Flückiger et al., 2018). However, based on a systematic literature review, Felice et al. (2019a, b) showed that common (relational and unspecific) and specific (technique) factors are mutually interdependent, thereby pointing to the limitations of all the conceptual and statistical approaches that assume the independence of the included variables (e.g., ANOVA based models), and promoting the shift towards multiplicative models (e.g. Schiepek et al., 2017; Malkina-Pykh, 2018). Consequently, any intervention “…only becomes real when it unfolds during the course of time” and “…the most constrained and manualized treatments unfold differently in each instance, due to characteristics of the therapist and the client” (Wampold et al., 2017, pp. 24).

Following this research path, Schiepek and colleagues (Schiepek et al., 2017) proposed a nonlinear mathematical model on the interactions of patient-related factors implied in fostering change and the personality development of patients (Schöller et al., 2018). According to this approach, the explanandum is not the outcome, as is normally the case, but the process itself. It concerns the ingredients of the psychotherapeutic process fostering or impeding the dynamics of change.

In classical terms, the word “prediction” points to the statistical “explanation” of the variance of one variable (“dependent”) by other variables (so called “predictors” or “regressors”). Other than this, in the nonlinear dynamic systems approach the term “prediction” is used to point out the capacity of a model to simulate a given dynamic pattern. Predictors, following this research line, are usually represented by the oscillation between critical instabilities of the process (destabilization) and the eventual reaching of stable states (attractors). The prediction of reaching one stable state results from the specific dynamics of the process.

The predictive power of the alternation between destabilization and stable states for good-outcome psychotherapies has been empirically shown in different studies (e.g., Schiepek et al., 2014; de Felice & Andressi, 2014; Haken & Schiepek, 2006; Halfon et al., 2019; Olthof et al., 2019). Furthermore, the dynamic system approach (e.g. Gelo & Salvatore, 2016; Halfon et al., 2016; de Felice et al., 2019a, 2019b, 2020) has been proven effective in the detection of early warning signals of critical instabilities (e.g. Schiepek et al., 2014; Fartacek et al., 2016; Olthof et al., 2019), hence, in finding specific patterns of process variables promoting or impeding the patients’ change.

The present study refers to a variety of process variables and moderators of psychotherapeutic change which were assessed by the Therapy Process Questionnaire (TPQ) (Schiepek et al., 2019). The Therapy Process Questionnaire factors refer to important ingredients and mediators of therapeutic change. They give evidence that routine monitoring of psychotherapy is not restricted to outcome but can mirror the multidimensional features of human change (Schiepek et al., 2017, 2019). “Therapeutic alliance and clinical setting” (TAS) refers to the therapeutic alliance, and is one of the most important and most intensely investigated factors in psychotherapy (e.g., Norcross & Lambert, 2011; Wampold & Imel, 2015; Flückiger et al., 2018). “Relationship with fellow patients” (RFP) assesses the patient’s relationship with other patients and the way he/she is experiencing living within the therapeutic community. There is a robust body of literature highlighting the importance of the therapeutic community for making new social experiences (Jörgensen et al., 2009) and enabling social learning (Adler & Stead, 2015). “Well-being and positive emotions” (WPE) represent an important secondary outcome criterion of good-outcome psychotherapies (Wampold et al., 2017). In a theoretical model of therapeutic change (Schiepek et al., 2017; Schöller et al., 2018), experiences of well-being and positive emotions are related to motivation for change, insight, and problem reduction, as well as to the long-term evolution of personality traits (reduced hopelessness or increased self-efficacy, mindfulness, and competences in emotion regulation). “Insight/confidence/therapeutic progress” (ICP) is a factor linked to the patient’s trust in his/her personal development, experiences of self-efficacy and progress (e.g., Catty, 2004; Maddux, 2013). “Motivation for change” (MOT) is an intensely investigated factor contributing to therapeutic progress (e.g., Grawe, 2004). “Mindfulness/self-care” (MSC) has been recognized, during the last two decades, as an important therapeutic mechanism (e.g. Bateman & Fonagy, 2013, 2015) and it is linked to the patient’s capacity to recognize his/her own feelings and those of significant others. Finally, “Emotional and problem intensity” (EPI) represents a primary outcome criterion for mental diseases which are related to worrying, stressful and negative emotions.

The processes of twenty psychotherapies have been monitored by a high frequency assessment, i.e., once per day. A Principal Component Analysis (PCA) and a Linear Discriminant Analysis (LDA) have been applied to the time series of the Therapy Process Questionnaire factors with the aim of investigating: (a) the main clinical dimensions of the 20 psychotherapeutic processes (PCA), and (b) the core difference across good and poor-outcome cases (LDA). In so doing, the mental health institutions we have worked with are able to have the opportunity to offer better inpatient treatments.

Methods and Materials

Sample

The 20 patients of this study were treated at two psychotherapy centers, the Department of In-patient Psychotherapy at the University Hospital of Psychiatry, Psychotherapy, and Psychosomatics (Paracelsus Medical University Salzburg, Austria and Department of Psychotraumatology at the Clinic St. Irmingard, Prien am Chiemsee, Germany). The diagnostics were done by experienced psychiatrists, based on the ICD-10 F- categories. The first order diagnosis of most of the patients was Adjustment to Severe Stress and Adjustment Disorder (F43: 11 cases). The characteristics of the patients are shown in Table 1. The descriptive statistics show a clear difference between the ‘good’ and the ‘poor’ cases in terms of the mean effect sizes of the outcome criterion, the ICD-10 Symptom Rating (ISR-10: Tritt, 2015; Tritt et al., 2008) (1.96 [SD: 0.19] vs. − 1.09 [SD: 0.49]). The difference is statistically highly significant (Mann–Whitney U test, p < 0.0001). All the other differences in the descriptive statistics were not significant.

Table 1 Patients’ characteristics
Full size table

The ten ‘good’ and ten ‘poor’ cases were included based on a criterion of less than 10% missing data in the time series of the process measure (Therapy Process Questionnaire, see below). The mean number of missing data in the full sample was 2.3 days which corresponds to a compliance rate of 96.6%. The mean time series length was 68.4 days (SD: 22.6). The inclusion criterion of less than 10% missing data is due to the necessity of having time series with high variability (missing data produce straight lines in the process) to get a realistic picture of the dynamics and to get valid inter-item and inter-factor correlations.

Written informed consent was obtained from every patient. Due to the retrospective nature of our investigation, a formal consent of the local ethics committee was not required. A general approval for using the Synergetic Navigation System (SNS) in clinical settings was stated by the ethics commission of the Salzburg government (No. 415-E/1068/3-2009). All procedures were in accordance with the Helsinki Declaration as revised in 2013.

Outcome and Process Measures

The outcome of the inpatient treatment was assessed by the ICD-based Symptom Rating (ISR; Tritt et al., 2008; Fischer et al., 2009, 2010, 2011; Tritt, 2015). The ISR is a first-order outcome measure which assesses symptom severity and corresponds to the criteria of the diagnostic F-clusters of the ICD-10. The subscales of the ISR are “depression”, “anxiety”, “obsessive–compulsive disorder”, “somatoform disorder”, “eating disorder”, and an additional scale with problems not related to the other subscales. The total score of the ISR averages all subscales by a weight of 1, except for the additional scale which is weighted by 2. For all patients, ISR-based assessments at the beginning of the hospital stay (pre) and at the release (post) were available.

The process was assessed by the Therapy Process Questionnaire (TPQ) which was developed for routine process monitoring with an equidistant time sampling rate of once per day (Schiepek et al., 2016a). This questionnaire is a multidimensional self-rating scale for the high-frequency monitoring of psychotherapeutic processes. The factor structure and the statistic properties were published in Schiepek et al. (2019). The seven factors are “well-being and positive emotions” (WPE), “relationship with fellow patients” (RFP), “therapeutic alliance and clinical setting” (TAS), “emotional and problem intensity” (EPI), “insight/confidence/therapeutic progress” (ICP), “motivation for change” (MOT), and “mindfulness/self-care” (MSC). All 43 items are rated on Visual Analog Scales. Both questionnaires, the TPQ and the ISR, were administered by an internet- and app-based monitoring system, the Synergetic Navigation System (SNS), which was developed for the assessment and analysis of processes and outcomes in naturalistic settings (Schiepek et al., 2016a, b, 2018).

Statistical Procedures

Principal Component Analysis (PCA) was used in order to extract the orthogonal (linearly independent) components of the dataset, listing in rows the daily assessments in temporal order (1244 observations, concatenated time series from all patients) and in columns the Therapeutic Process Questionnaire factors’ scores (seven variables). Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to project a data set spanned by n descriptors into a space spanned by linearly uncorrelated variables (linear combination of original descriptors), called ‘principal components’. This transformation is defined in such a way that the first principal component has the largest possible variance (that is, it accounts for as much of the variability in the data as possible), and each succeeding component in turn has the highest possible variance, under the constraint that it is orthogonal to the preceding components. The resulting vectors are an uncorrelated orthogonal basis set. In presence of a correlation among original descriptors, p principal components (with p <  < n) are sufficient to explain the most relevant part of original variance (Wold et al., 1987; Abdi & Williams, 2010; Giuliani, 2017). PCA is used to reduce the dimensionality of the dataset, collapsing the initial information into a small number of independent dimensions characterizing the evolution of poor and good-outcome cases.

PCA is an unsupervised method and allows for a well-conditioned representation of the data set. In order to check for the predictability of the representation in terms of good/poor-outcomes, we need to shift to a supervised approach. This task is achieved by Linear Discriminant Analysis (LDA). LDA is a generalization of Fisher's linear discriminant, a method used to find a linear combination of features that characterizes, or separates, two or more classes of objects or events. In our case, if for a statistical unit (patient i) the value of the good-outcome function is greater than that of the poor-outcome function (fgood—fpoor > 0), the patient is classified as ‘good’, otherwise, as ‘poor’. To ensure greater classification strength, the function is estimated at each iteration excluding the patient to be classified, thus obtaining the Confusion Matrix, which only takes the categorical yes/no aspect of the classification into account. In order to get a continuous measure of the accuracy of the classification of each patient, the probability of each patient i belonging to the class ‘good’ (pgood) is estimated by the Generalized Square Distance, weighted by the a-priori probability (which in our case is 0.5, therefore identical for the two classes). The probability of being a good-outcome patient, which here is conditional to the relative values of the principal components scores (PC1 and PC2), is given by:

$$p\left( {{\text{good}}|PC1, \, PC2} \right) = \exp \left( { - 0.5 \, D^{2} {\text{good}}} \right)/\exp \, \left( { - 0.5 \, D^{2} {\text{good}}} \right) + \exp \, \left( { - 0.5 \, D^{2} {\text{poor}}} \right).$$

In the formula, the value D is the weighted distance of the covariance between the patient descriptors of the observation i and the centre of the ‘good-outcome’ class D2 (good), or the ‘poor-outcome’ class D2 (poor). Given the independence of the descriptors, and given that there are only two classes, p(poor) = 1–p(good) is the Euclidean distance.

The LDA procedure is closely related to regression analysis, which also attempts to express one dependent variable as a linear combination of other features or measurements. However, as a general rule, regression analysis uses categorical independent variables and a continuous dependent variable, whereas discriminant analysis has continuous independent variables and a categorical dependent variable (i.e., the class label). LDA is also closely related to principal component analysis (PCA) in that both look for linear combinations of variables which best explain the data variance. LDA explicitly attempts to model the difference between the classes of data (in this study, ‘poor’ and ‘good’ outcome). PCA, in contrast, does not take into account any difference in terms of classes. Hence, the joint application of PCA and LDA allows for a careful investigation of both the analogies and differences in the two outcome groups.

Results

The results of the correlation matrix of the seven factors’ scores were analogous to those obtained with a sample of 150 patients in the Therapy Process Questionnaire validation study (Schiepek et al., 2019). Sixteen out of the twenty-one correlations indicated a similar magnitude and direction (sign). Interestingly, all correlations with larger deviations and inverted signs concerned the relationship with fellow patients (RFP): RFP and Well-being and positive emotions: − 0.229 (validation study: 0.40); RFP and Emotional and problem intensity: 0.372 (validation study: − 0.44); RFP and Insight/confidence/therapeutic progress: − 0.127 (validation study: 0.26); RFP and Motivation for change: − 0.158 (validation study: 0.27); RFP and Mindfulness/self-care: − 0.216 (validation study: 0.30). The inter-correlation matrix of our sample of 20 patients is shown in Table 2.

Table 2 Factor inter-correlations
Full size table

The result of the PCA is shown in Table 3. By the criterion to consider a component as a bona fide signal only if its eigenvalue is greater than 1, two relevant principal components (PCs) could be identified, with a cumulated explained variance of 68.24%. The first PC can be interpreted as an individual component which is characterized by well-being, positive emotions, high motivation for change, perception of progress, insight and mindfulness/self-care (see Table 3; the variables most meaningful for the components interpretation are those with higher loadings, in bold). PC1 can be considered as a cognitive component in which the patients highlight their willingness and optimism of following the therapeutic path of the inpatient treatment. Despite that, the burdensome emotional and interpersonal experiences, and the symptom-related problems give rise to an orthogonal component (PC2) so underlying the split between cognitive and emotional/relational dimensions. On the other hand, PC2 highlights the patient’s emotional problems emerging within interpersonal relations with fellow patients and with professionals. Briefly stated, the main difference between PC1 and PC2 lies in the loading “emotional and problem intensity” (EPI), which is negatively associated with PC1 and positively associated with PC2. Emotions like sadness, anger, anxiety, guilt or shame, and problem/symptom severity are items of the EPI factor and characterize the second component. Thus, PC1 can be regarded as an individual and cognitive component (named, “Cognitive Processing”), whereas PC2 can be regarded as a relational and emotional component (named, “Emotional Processing”). PCs have a mean of 0 (unit standard deviation) and are linearly independent.

Table 3 Loadings of the TPQ factors on the two PCs
Full size table

In order to confirm the above results, based on the correlation of the variables computed over the entire dataset of 1244 concatenated observations (68.24% of variance explained with two principal components), PCA was also performed over the aggregate data matrix with observations based on mean values of each Therapeutic Process Questionnaire factor for each patient (20 observations, 69.37% of variance explained with two principal components). The result confirmed the independence of the two principal components (r = 0.002) as well as their loading patterns (Table 3) and legitimates the use of the principal components for further analysis.

The discrimination between poor and good-outcome cases based on the mean values of the Therapeutic Process Questionnaire factors for each patient shows a significant difference in terms of the total scores (effect sizes) of the ISR-10 (effect size, good-outcome cases: m. 1.96, SD 0.19; poor-outcome cases: m. − 1.09, SD 0.49; Mann–Whitney U test p < 0.0001). In terms of TPQ factors, the “Relationship with fellow patients” (RFP) and “Well-being and positive emotions” (WPE) show a significant difference. RFP is higher within the poor-outcome cases whereas WPE is higher in the good-outcome cases. Additionally, the variabilities of these factors are higher within the poor-outcome cases. In summary, the poor-outcome cases present a closer relationship towards their fellow patients during the treatment process than the good-outcome cases (57.08 vs. 50.10, p = 0.021), but a lower feeling of well-being and positive emotions (31.32 vs. 45.52, p = 0.040). In terms of Principal Components, the results show that PC1 is not significant in distinguishing the two outcome groups, whereas PC2 (positively associated with emotional and problem intensity, EPI) is significantly higher within the poor-outcome group (− 0.374 vs. 0.263, p = 0.040) (Table 4).

Table 4 Differences across good and poor-outcome cases of the mean levels of the following values: (a) “Relationship with fellow patients”, factor of the therapeutic process questionnaire; (b) “Well-being and positive emotions”, factor of the therapeutic process questionnaire; (c) scores of the first principal component; (d) scores of the second principal component; (e) probability of being a good-outcome case in accordance with linear discriminant analysis (this probability includes PC1 and PC2 scores with different weights)
Full size table

The probability of being a ‘good case’ estimated by the Linear Discriminant Analysis (LDA) operated in the bivariate PC1-PC2 space is the descriptor that reveals the maximum separation of the two groups (last row in Table 4). P(good) is derived from the LDA formula which was introduced in the Statistical Procedures section. On the one side, the highly significant classification is an effect of the supervised nature of the discriminant analysis, but on the other side, it also confirms that the combination of the two independent components PC1 and PC2 effectively discriminates the two groups. The significance level (p = 0.006) shows that the LDA-based probability estimate discriminates the groups better than any single Principal Component alone. From the results of the Linear Discriminant Analysis it is possible to evince the different contributions of PC1 and PC2 for the discrimination of the two groups. In order to predict the classification of good-outcome cases and poor-outcome cases, the Linear Discriminant Analysis assigned the following values: PC1 = 0.720, PC2 = − 0.944 (good-outcome); PC1 = − 0.588, PC2 = 0.671 (poor-outcome). The good-outcome cases have a positive first component and a negative second component, while the opposite is true for the poor-outcome cases. This confirms the main difference between the groups: the second component (PC2), peculiar of poor-outcome cases, is associated with high scores of emotional and problem intensity (EPI), relationship with fellow patients (RFP), and positive perception of therapeutic alliance and the clinical setting (TAS). The LDA model ensures a significant discrimination power (p = 0.030 Fisher's exact test; p = 0.025, Chi-Square). Eight good cases and seven poor cases, each out of ten, were correctly classified by the Linear Discriminant Analysis (LDA), while on closer consideration of the misjudged classifications, one good-outcome and one poor-outcome case lay very close to the distribution of the correctly classified cases. The correlation between the LDA classification function and the a-priori outcome differentiation based on the effect sizes of the ISR-10 is r = 0.637 (p < 0.003). This result constitutes a mutual validation of both the a-priori (ISR-10) and the a-posteriori (LDA) outcome classification. Additionally, the different contributions of PC1 and PC2 towards the prediction of belonging to a specific outcome group allows us to identify a clear path to elucidate the main difference between the poor and good-outcome cases and the way in which it might be possible to help ‘poor’ patients belong to the ‘good’ category. This could be done by giving them the possibility of experiencing burdensome emotions (problem actualization) in the clinical setting. In fact, looking at the correlations between PC1 and PC2 within poor and good outcomes, we can extract their core difference. Note that, at the general level, PC1 and PC2 are orthogonal to each other, that is, linearly independent (r = 0). This independency is completely maintained within the poor-outcome group (r = − 0.255; p = 0.478). On the other hand, within the good-outcome group, they are strongly correlated (r = 0.702; p = 0.024), demonstrating that the integration of PC1 and PC2 is specific for good-outcome cases only.

Discussion

In this study the processes of twenty psychotherapies have been monitored by a high frequency assessment, i.e., once per day. A Principal Component Analysis (PCA) and a Linear Discriminant Analysis (LDA) have been applied to the time series of the Therapeutic Process Questionnaire factors with the aim of investigating: (a) the main clinical dimensions of the 20 psychotherapeutic processes (PCA), and (b) the core difference across good and poor-outcome cases (LDA).

Through Principal Component Analysis it was possible to extract two main clinical dimensions on which the 20 psychotherapeutic processes were based. They also represented two complementary ways through which the patients processed their psychotherapies within their respective Austrian and German inpatient centres. In fact, the first Principal Component was associated with the patients’ cognitive processing and perception of well-being, while the second Principal Component was associated with the patients’ emotional processing and emotional problems. It is particularly interesting that the relational aspects of the inpatient treatments (“Relationship with fellow patients” and “Therapeutic alliance and clinical setting”) were significantly more correlated with the second Principal Component (linked to emotional problems), highlighting the patients’ main difficulty, particularly within poor-outcome cases. These two ways of experiencing the inpatient treatment represented by PC1 and PC2 were considerably different across good and poor-outcome groups. Through Linear Discriminant Analysis it was possible to underline their crucial divergence: within good-outcome cases the cognitive processing (PC1) and the emotional processing (PC2) were highly correlated (r = 0.702; p = 0.024), while they were independent within the poor-outcome cases (r = − 0.255; p = 0.478). In other words, in the poor-outcome inpatient treatments the emotional processing was completely split, excluding the emotional problems and the interpersonal involvement from the therapeutic process, despite the patients’ positive cognitive participation towards the treatment program. These results constitute an opportunity to improve the clinical services offered: it is of the utmost importance to promote meticulous attention to a patient’s relational life (both with the fellow patients and with the psychotherapist) as well as to “negative” or painful emotions, rather than to a patient’s conscious affirmations of well-being. The more we promote the integration of these difficult emotional and relational aspects, the more a patient can have the opportunity to get better, resulting in a “good-outcome case”. In fact, the integration of cognitive and emotional processing is at the basis of how the mind learns from experience, working through painful emotions. Without this integration, the mind is forced to enact one of the many defence mechanisms able to protect the patient’s health (Bateman & Fonagy, 2013, 2015; de Felice et al., 2020b).

Finally, although the PCA-LDA approach was successful in showing both the main clinical dimensions and their differences across poor and good-outcome cases, this study is not devoid of limitations. The sample size was certainly the main constraint of this study. Generalisation of the results is not possible unless these analyses are repeated. Another limit is the length of treatments: on average 61 days for poor-outcomes and 75 for good-outcomes. Although we must consider that they are inpatient treatments, it is not possible to consider them as long-term psychotherapies. However, this study is part of a larger project also investigating the complex temporal dynamics of poor and good-outcome psychotherapies (Schiepek et al., 2020). We plan to extend the whole methodology to a bigger sample of about 1,700 cases treated in different Austrian and German inpatient centres.