Background

Since the introduction of therapeutic communities in the 1940s, group treatments have become a common part of care for patients with severe mental illness [1–7]. Group treatments are widely provided in psychiatric settings and generally considered as a routine therapeutic intervention [8–11]. It has been suggested that group experiences can be a powerful agent of change and efficacy has been demonstrated across a range of approaches including cognitive-behavioural therapy [8], integrated psychological therapy [9], social skills training [10], cognitive remediation [11], and others [12–17].

Several processes have been proposed to explain the mechanisms of group therapy [1, 4, 5, 18, 19]. Yalom’s studies of the essential mechanisms for change common to group treatments [20, 21] identified eleven factors consisting of: 1) instillation of hope, 2) universality, 3) imparting of information, 4) altruism, 5) the corrective recapitulation of the primary family group, 6) development of socializing techniques, 7) imitative behaviour, 8) interpersonal learning, 9) group cohesiveness, 10) catharsis, and 11) existential factors. Bloch and Crouch [22] later formulated a model that proposed self-disclosure and self-understanding as predictors of change. A recent study suggests that Yalom’s factors do not have equal importance to patients and therapists. Cohesiveness, universality and self-understanding were indicated as most important in the group experience [23].

A number of scales have been developed to explore different aspects of group treatment. Many have been designed to specifically evaluate the presence of Yalom’s 11 factors [6, 7, 23–27]. More recently, Dierick and Lietaer [28], developed a questionnaire to investigate therapeutic factors across an extensive range of group therapies from the client’s perspective. Participants were diverse and represented inpatients and outpatients, as well as students and trainee psychotherapists. Therapeutic approaches included client centred/experiential, Gestalt, psychoanalytic, behavioural, drama and body therapies. The questionnaire comprised of 133 items and 28 basic scales. Questions were written from the perspective of the patient’s experience, and focused primarily on helpful or instructive experiences. The method used a series of multivariate analyses to define the links between therapeutic factors and their degree of interconnectedness. Two different dimensions were proposed to account for the therapeutic factors intervening in groups. These two dimensions were interpreted as “Relational Climate” and “Psychological Work”.

Despite the comprehensiveness of this questionnaire, its use may be limited in a community mental health setting as the number of items makes the scale difficult to implement in routine clinical evaluation and items were derived from a mostly non-clinical sample. The questionnaire focuses only on items experienced as helpful or instructive and therefore does not explore experiences and feelings that were perceived as less therapeutic or detrimental.

Despite the body of research regarding therapeutic mechanisms in group treatment, a valid, brief and robust scale that explores the core experiences of patients (therapeutic or not) during a single session or course of group therapy does not currently exist in community psychiatry. The purpose of this study was to develop and perform a primary validation of a questionnaire to assess the main subjective experiences of patients with severe mental illness attending group therapy. The questionnaire was designed to be specific to community mental health services, short and easy to administer and to capture different but theoretically meaningful dimensions of experiences across a range of group interventions and community settings.

Methods

Setting

The study took place in community mental health services of three provinces in North-east Italy. Settings included a residential unit in Ferrara, providing short-to medium-term care for patients with acute and subacute psychiatric conditions, and two psychiatric day-hospitals in Bologna and Modena. A range of group activities are regularly offered within these settings and patients are invited and encouraged to participate in all groups.

Group intervention

Group interventions were scheduled on weekly basis. All groups lasted for 90 minutes, except the psychodynamic group (60 minutes) and had the following characteristics:

  • Group psychodynamic psychotherapy provided opportunities for each member to learn about themselves and their interpersonal and intra-psychic functioning, by creating a safe atmosphere that encouraged interaction and invited reflection.

  • Psychosocial rehabilitation groups (e.g. newspaper reading groups) had a more practical focus, whilst pursuing the improvement of patients’ cognitive and social skills.

  • Psycho-educational groups (such as a “wellness group”) provided patients with information about their psychiatric disorders, pharmacological treatments and healthy living.

  • Expressive groups (such as arts and music therapy groups) gave patients the opportunity to be creative and encouraged self-expression.

  • Body oriented groups included a relaxation group and psycho-motor group.

  • “Other” groups represented groups where patients could discuss issues and raise concerns.

Generation of the item pool

The questionnaire was developed in several steps. At each stage, decisions were discussed and made by the project team in Italy and then further checked with an expert in scale development (SP) to reduce bias.

An initial item pool was generated by focus groups held in three community mental health services between September-October 2009 (Ferrara, coordinating centre; Modena and Bologna). Each group comprised of clinicians from different professional backgrounds that commonly participate in the management of groups (Ferrara: 2 psychiatrists, 5 psychiatric nurses, 2 social workers, 1 psychologist; Modena: 2 psychiatrists, 2 psychiatric nurses, 2 rehabilitation therapists, 1 psychologist; Bologna: 1 psychiatrist, 5 psychiatric nurses, 2 rehabilitation therapists and 1 psychologist). Each focus group met four times. The task of each meeting was to generate a complete list of common therapeutic group experiences which resulted in a list of 25 group experiences from the Ferrara group, 21 from Modena and 26 from Bologna.

The items were then examined by the coordinating centre in Ferrara and grouped into common dimensions. Four conceptual dimensions resulted: relational aspects, expression of emotions and mentalization, group learning, and cognitive experiences. Four statements that most clearly represented each dimension were selected from the lists, to form a total of 16 experiences.

These were then transformed into specific questions with multiple questions for each statement to ensure comprehensive coverage (7 questions for relational aspects, 6 for expressions of emotions and mentalization, 6 for group learning, and 6 for cognitive improvement). This resulted in a pool of 25 questions which was circulated and approved by all the participating centres. Inverse items were then included (3 for relational aspects, 3 for expression of emotions, 4 for group learning and 3 for cognitive improvement) expanding the pool to 38 questions.

The questionnaire was read in a meeting with 12 psychiatric patients (5 with a diagnosis of schizophrenia, 3 with bipolar disorder, 3 with unipolar depression and 1 with borderline personality disorder) to verify that all items were clearly understandable and made sense. Any words that were unclear were exchanged with words suggested by the group. On the final reading, all questions were deemed understandable by the patients.

The resulting self-report instrument (Ferrara Group experiences Scale – FE-GES) assessed the main subjective experiences of participants in group therapy and the intensity this was felt. It consisted of 38 items evaluating the four hypothesised areas of relationships (e.g. “I was helped by others”, “I socialised with others”), emotional expression and mentalization (e.g. “I shared my personal experiences and my life problems”, “I met people who were experiencing the same problems as me”), group learning (e.g. “I understood the reasons for my behaviour”, “I understood better how I usually deal with my problems”) and cognitive improvement (e.g. “I paid attention to what others said”, “I could remember what was said”).

Items were rated on a 5-point Likert scale, from not at all (0), a little (1), enough (2), very (3) to a great deal (4). Items 3, 7, 11, 13, 16, 18, 22, 24, 26, 28, 32, 34 and 38 were inversely scored.

A further item was included, which asked the participant to rate the overall usefulness of the group. The questionnaire was written in Italian.

Data collection

Psychiatric patients admitted to the participating services between January 1, 2010 and April 30, 2011 were screened for inclusion. Eligible patients were those with any psychiatric diagnosis according to ICD-10 criteria except for mental retardation (ICD-10 codes F70 to F79), who chose to take part in at least 5 group activity sessions of any modality and consented to participate in the study. Patients completed a psychiatric diagnostic interview according to ICD-10 criteria on admission. All patients were informed of the aim of the study and gave written informed consent. The study was conducted in line with the Code of Ethics of the World Medical Association (Declaration of Helsinki, 1964) and approved by the institutional review board of the University of Ferrara. Socio-demographic and clinical data (e.g. length of illness and number of hospitalizations) were collected. Prior to discharge from services, patients were asked to complete the questionnaire to rate their overall group experiences during the period of admission.

Statistical analysis

Principle Components Analysis (PCA) of the 39 items was used to identify any underlying dimensions of this questionnaire. Both orthogonal (varimax) and oblique (direct-oblimin) rotations were examined using the FACTOR program [29]. Descriptive statistics were used to check the distribution of the data and to identify items with high values of skewness or kurtosis. Polychoric correlations were employed to account for the Likert scale response [30]. Inter-correlations between items were examined for particularly high or low correlations and the Kaiser-Meyer-Olkin measure was used to verify sampling adequacy. The strength of relationships between variables was assessed using Bartlett’s test of sphericity. The number of components to retain was first estimated by parallel analysis [31]. Rotations were examined for a range of 2–6 specified components around this estimate, and a criterion level of 0.4 was set to determine the importance of factor loadings [32]. Reliability of the rotated factor scores [33, 34], Bentler’s [35] simplicity and Lorenzo-Seva’s [36] loading simplicity indices were then examined. Pattern and structure matrices were inspected in oblique rotations for items loading highly onto more than one component and the correlation matrix analysed to determine whether there were relationships between components. Low correlating and highly skewed items were systematically removed and the analysis re-run. The final solution was selected based upon criteria of explanation of a good proportion of the variance, balance of loading onto components and conceptual similarity. Items with the highest loadings onto components were selected and analysed for internal consistency using Cronbach’s α in PASW Statistics (v.18). A criteria of α > .7 was set as a minimum [37].

Results

Sociodemographic and clinical characteristics of patients

During the study period, 191 patients were admitted across the 3 participating centres, of which 166 were eligible and consented to participate. Table 1 displays rates of participation in each group.

Table 1 Number of patients participating in groups by diagnosis
Full size table

The sample consisted of 58 men (35%) and 108 women (65%), with a mean age of 46.4 (± 12.2) years. Half of the participants had never married (83; 50%); 44 were married (27%), 31 divorced (19%) and 8 widowed (5%). Most were unemployed (120; 72%), while 46 had a job (28%). Most of the participants had spent 8 years or more in education (153; 92%).

According to ICD-10 criteria, 57 patients were diagnosed with schizophrenia (34%), 4 with delusional disorder (2%); 10 with schizoaffective disorder (6%), 29 with bipolar disorder (18%), 27 with major depressive disorder (16%), 26 with personality disorder (16%), 12 with eating disorder (7%), and one with an anxiety disorder (1%). Mean age at illness onset was 29.6 (± 13.1) years. The mean number of previous hospitalizations was 6.4 (± 7.7).

Principal components analysis

Item responses used the full range of scores (0–4) for each item, but many were highly skewed indicating that polychoric correlation coefficients were required [30, 37–39]. The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis, KMO = 0.744. Nine items had correlations lower than .3, but none had high correlations (>.8). Diagonals of the anti-image correlation matrix were all above .5. Bartlett’s test of sphericity (χ 2 (166) = 3024.3, p < .001), indicated that relations between items were sufficiently large for PCA.

Parallel analysis using 500 random correlation matrices permuted from the raw data [40] suggested three components should be retained (Table 2). After rotation, the items appeared to split into positively and negatively phrased (inversely scored) components. Cognitive improvement, plus an additional group learning item (item 23) appeared as a consistent component in all 3–6 component solutions. Of these, the 5-component solution was most promising as it explained 50.5% of the variance and had a better conceptual fit. Removal of low correlating items did not significantly improve the explained variance of the model whilst removal of seven inversely scored, highly skewed items resulted in a 5 component solution which explained 50.8% of the variance. This solution was selected as it provided better conceptual distinction between the previously defined components of sharing of emotions and experiences, cognitive improvement, group learning, difficulties in open expression and relationships.

Table 2 Parallel analysis results: eigenvalues greater than 1 and proportions of variance explained (N = 166)
Full size table

Four items loaded highly (>.5) onto each component. PCA of these 20 items explained 60.8% of the variance. Orthogonal and oblique rotations were similar. As components were moderately correlated, the oblique rotation was preferred (Tables 3 and 4). Reliability analysis using Cronbach’s α on standardised values [37] demonstrated good reliability for all sub-scales and item totals (Table 5).

Table 3 Oblique rotation of 5 components with final 20 items and item reliability
Full size table
Table 4 Correlation matrix for oblique rotation of 5 component analysis
Full size table
Table 5 Proposed 5 subscales and item reliability
Full size table

Discussion

Group interventions are widely provided in psychiatric settings, yet there is a paucity of research that focuses upon patients’ experiences of group sessions. In this study, a scale was developed that assessed group experiences of 166 patients with severe mental illness from three Italian community mental health centres. The Ferrara Group Experiences Scale (FE-GES) is relatively brief tool, consisting of 5 subscales with a total of 20 items which capture important and distinct aspects of patient experiences. It resulted from a systematic and conceptually driven development, and shows good psychometric qualities.

The five subscale components confirm factors identified in previous research and identify new aspects that invite further reflection. Below, the five subscale factors are described and their relationship to existing literature is briefly discussed.

Sharing of emotions and experience

The experience of “sharing” with the therapist and other group members is regarded as one of the core elements to promote change in group treatments. In this subscale several existing factors are reflected: 1) sense of universality, 2) development of socializing abilities, 3) self understanding and 4) imitative behaviour [19]. As mentioned previously, universality, self-understanding, and cohesiveness have recently been indicated as the most important therapeutic factors [23]. Sharing of emotions and experiences among group members appears to be the core element of self-disclosure, suggested by Bloch and Crouch [22] as a driver of change. Sharing emotions and experiences may also contribute to the quality of the relational climate that Dierick and Lietaer described as a fundamental dimension of group therapies [28, 41–43].

Cognitive improvement

This subscale corresponds to Yalom’s concept of “imparting of information” [6, 7]. It captures how cognitive abilities can be elicited and strengthened within the group process. Participants can improve their ability to focus, to remember what was said, to express thoughts in a clear way, and to listen. The items suggest that this improvement is connected to the relational aspect of listening and paying attention to statements of other patients, an intrinsic characteristic of group treatments.

Group learning

The “Group learning” factor corresponds to research on group learning processes and reflects Yalom’s concepts of interpersonal learning and self understanding [6, 7]. These concepts refer to developing a greater capacity to understand the rules and codes of relationships, achieving a higher level of self-awareness (self-understanding) and insight into the origins of problems and the unconscious motivations that underlie behaviours (“genetic insight”) [7]. Similar concepts are found in the work of MacKenzie [44] and Dierick and Lietaer [28], who by rearranging Yalom’s therapeutic factors, indicated the elements of “interpersonal learning” and “self-understanding” as an underlying dimension of psychological work.

Difficulties in open expression

The conceptual development of the FE-GES has identified an aspect of group participation which, so far, has not been considered in group experience scales. The items of this subscale contrast with the therapeutic experiences described so far, indicating patients’ discomfort in talking about problems and communicating opinions and feelings. These obstacles to self- expression during group sessions may originate from therapists’ failure to build a safe and non-judgemental atmosphere, from the existence of strong cultural or social diversities within the group, or from the subjective characteristics of one or more members. In their study of inpatient psychotherapy groups, Leszcz, Yalom and Norden [24] suggested that the heterogeneous needs and capacities of patients may require a range of differing forms and structures in groups. When these difficulties arise, it is crucial that the therapist is aware of them so that these problems can be addressed. Such difficulties may undermine participation within the group and the success of intervention, by hampering the development of cohesion and self-disclosure [22]. This subscale may provide a means of identifying such difficulties during the therapy process and enable clinicians to adjust therapeutic programs to meet individual needs. It is important to note that these ‘negative’ experiences do not imply the absence of positive experiences, which would be captured by low scores on the other subscales. It is a distinct factor which may assist in improving group experiences of psychiatric patients.

Relationships

This subscale has strong links to many of the proposed therapeutic factors in group psychotherapy research. Several of Yalom’s therapeutic factors are reflected: 1) Altruism, 2) Instillation of hope, 4) Development of socializing abilities, 5) Catharsis, and 6) “Corrective recapitulation of the primary family group”. The last factor supports reintegration and helps patients to accept insecure or maladaptive patterns of attachment which may have developed in infancy. Hence, it relates to attachment theory and can be particularly relevant to interventions utilising that theory. The items “I met new and positive people”, “I built relationships of trust with others” and “I was helped by others” may also be representative of the dimension that Dierick and Lieater referred to as “relational climate” [28].

Limitations

The study has a number of limitations which should be taken into account. The limited sample size, from only one region of Italy means that results are not generalisable and will require further studies in different countries with larger samples. The measure was validated using a limited range of group treatments, each with differing approaches and levels of evidence to support their use in a clinical setting. It may therefore miss factors specific to a single modality of treatment. However, the application of the FE-GES across different modalities will enable comparisons to be made regarding the presence of common group factors.

As a consequence, the FE-GES should be cross-validated in other countries and in other services with different group treatments.

Finally, patients with a psychiatric diagnosis were not involved in the focus groups to generate items for the questionnaire but were involved subsequently to modify the content so that it was clear and easy to understand.

As a consequence, the FE-GES should be cross-validated in other countries and in other services with different group treatments.

Conclusions

The subscales of the FE-GES are conceptually similar to other studies of group therapeutic factors. Relational and Cognitive dimensions are particularly linked to those originally identified by Yalom and subsequently described by other authors as key therapeutic factors in group therapy [6, 7, 22]. These results suggest a fundamental connection between the emotional and cognitive components of group experiences. Through relational and emotive group experiences cognitive learning is further advanced [24–27, 45]. The questionnaire has good face and content validity. Further studies with existing scales are now needed to confirm concurrent validity.

In summary, our preliminary validation indicates that the FE-GES appears a useful tool in the assessment of group treatment experiences of patients with severe mental illnesses. It is designed to detect distinct and typical factors of group experiences, and can be used to assess mediating processes and outcomes in research. Most importantly, it is very brief which is essential for a scale to be administered in routine care. The FE-GES can therefore be used in both research and in the evaluation and quality management of group treatments in routine care. It may aid not only clinical decision making and supervision, but also contribute to the further development of group therapy theory.