Background

Self-management programs are increasingly used as a means to empower, educate and inform patients with chronic rheumatic diseases. Such interventions are designed to encourage patients to be more active and take responsibility for their own health care with aims of increased self-efficacy, coping with stress, problem solving and interactions with healthcare professionals [1, 2]. Self-management programs have some evidence for effectiveness [3–9]. Evaluation has, however, been hindered by a lack of appropriate outcome measures [10–12].

One-week self-management programs addressing the needs of patients with different rheumatic diseases have been developed at the Hospital for Rheumatic Diseases in Lillehammer, Norway. Several patient-reported outcome measures (PROMs) have been used to evaluate these programs, including measures of different aspects of health status and quality of life [1, 4, 13, 14]. Such outcomes are important for assessing long term benefits but may not be responsive to important changes in the shorter to medium term relating to patient skills in managing and taking an active role in healthcare. Moreover, a systematic review found that over 16 different outcome measures had been used to evaluate self-management interventions [4]. This has important implications for generalisability [15]. Another overview of evaluation of psychoeducational/self-management interventions presented more than 30 different outcomes from 24 studies [1, 16], which indicate the complexity of the self-management education. Different interventions, often delivered in variety of environments and with different outcomes make it difficult to compare results from one study/intervention with another.

The OMERACT (Outcome Measures in Rheumatology) Effective Musculoskeletal Consumer Project was designed to address the need for appropriate outcome measures for evaluating and comparing self-management programs [15, 17]. Previous research demonstrated that existing outcome measures failed to assess a number of important skills, including the ability to find and assess information, decision making and implementation, and to take part in the health care system and society [15, 17]. The Effective Musculoskeletal Consumer Scale (EC-17) was developed to address the need for an instrument that assesses skills and attributes of patients as effective consumers who manage their healthcare, which is an important part of self-management. It is intended for use both to discriminate between patients with different levels of skills and for assessing the outcomes of interventions designed to improve skills. The English language version of the EC-17 was found to be acceptable to patients and have high internal consistency. The aim of this study was to translate the English language version of EC-17 [18] into Norwegian and assesses the data quality, internal consistency and test-retest reliability, construct validity and responsiveness to change of the measure.

Methods

Data collection

Two groups of patients with rheumatic diagnoses, aged 25-85, participated in a one-week in- patient self-management program at the Hospital for Rheumatic Diseases in Lillehammer, Norway. The recruitment of patients took place from January to April 2007. Patients in both samples were diagnosed and referred by rheumatologists and general practitioners from across Norway. The patients, mostly females and with a disease duration of over one year, received the invitation to participate in the study together with a letter with the date of their hospital stay. For purposes of assessing internal consistency, validity and responsiveness to change, 103 patients were asked to complete the EC-17 before and at the end of the one-week self-management program. For purposes of assessing test-retest reliability, 96 patients were sent a postal version of the EC-17 two weeks before attending the program and also completed the questionnaire on arrival and at the end of the one-week self-management program.

Intervention

The one-week self-management program summarised in Table 1, is designed to help people manage their rheumatic disease and challenges in daily life. The program has the same core concepts as outpatient programmes and comprises information, discussions on how to cope with the disease and daily life, cognitive management skills, exercise, engagement in self-care and interactions with healthcare professionals [2, 4, 19, 20]. This program is unique in that patients are hospitalized for the one week program rather than participating in a series of short modules that run over a long period of time. During the program, the inpatient education unit takes up to 16 patients and 5 relatives per week within one diagnostic group. All patients receive a core program but because patients with different diagnoses have different needs, some programs have supplements. For example, patients with ankylosing spondylitis have more physical exercises, whereas fibromyalgia patients have more process-orientated therapy related to stress management and communication. Each educational session lasts for 1.5-2.5 hours and the physical activity sessions for 0.5-1 hour. In the evening group sessions, the patients are divided into small groups of approximately five participants. Relatives have their own group. They talk together for one hour under leadership of one health professional who is educated in coaching. The focus is on coping with the disease and daily life. Exercise facilities including a swimming pool, are available in the evening for individual training.

Table 1 The Self-Management Program
Full size table

Patients gave informed consent after receiving written information about the project. The regional committee for medical research ethics in Health Region East, Norway and The Norwegian Data Inspectorate approved the study.

Instruments

The EC-17 (Table 2) is a self-administered instrument that was developed following a literature review, interviews with patients and physicians, feedback from OMERACT members and pretesting with patients [15, 17]. The resulting questionnaire comprised 64 items which was reduced to 17 items on the basis of expert opinion and item testing [15]. The 17 items relate to knowledge, attitudes and behaviours about self-management skills with a five-point adjectival scale from 'never' to 'always'. The results of principal component analysis support the unidimensionality of the EC-17 [15]. The items are summed and converted to produce a score from 0 to 100 where 100 is the best possible score.

Table 2 EC-17 item a means (sd), frequencies, component loadings and item-total correlation (n = 116)
Full size table

In the present study, the EC-17 translation followed the forward and backwards procedure [21]. Two independent bilingual translators including a health professional whose first language was Norwegian, translated the original version of the EC-17 into Norwegian and had a consensus meeting. Two independent bilingual translators including a health professional whose first language was English, then back translated the Norwegian version and had a consensus meeting. The translators, two researchers and BH discussed the forward and back translation, resolving discrepancies through consensus to achieve conceptual equivalence between the Norwegian and original English version of the EC-17.

The Norwegian version of the EC-17 was pre-tested with ten inpatients with rheumatic diseases recruited from patients in the hospitals rheumatology unit. All patients were Norwegian speakers and had been diagnosed with rheumatoid arthritis, psoriatic arthritis or ankylosing spondylitis. BH interviewed the patients following self-completion, asking them to provide feedback on the questionnaire, including difficulty understanding the questions.

All patients said that the questionnaire was comprehensible and no changes were necessary.

As part of testing for construct validity, the EC-17 was compared to three instruments that assess coping and psychological status. The 12-item Brief Approach/Avoidance Coping Questionnaire (BACQ) is designed to assess approach versus avoidance coping and comprises two scales of approach and avoidance that cover cognitive, emotional and action-related domains [22]. The Norwegian version of the instrument has good evidence for reliability and validity [22]. Items use a five-point scale from disagree completely to agree completely and produce two sum scores that range from 6 to 30 where 30 is the best possible score.

The 16-item Emotional Approach Coping Scale (EAC) is designed to assess coping through emotional approach. It comprises two scales: emotional processing, which assesses active attempts to know and understand one's emotions, and emotional expression, which assesses active verbal and nonverbal attempts to communicate or symbolize one's emotional experience [23, 24]. The Norwegian version of EAC is an acceptable and valid instrument for measuring emotional processing and expression in patients with rheumatic diseases [25]. Items use a four-point scale from "I usually don't do this at all" to "I usually do this a lot" and produce two mean scores from 8 to 32 where 32 is the best possible score.

The General Health Questionnaire (GHQ-20) is a 20-item screening instrument for detecting psychiatric disorders, but is also used for measuring changes in psychosocial status and psychological distress in chronic diseases [26–30]. The instrument has evidence for reliability and validity [26, 31–33] and evidence for high sensitivity and specificity in Norwegian trauma patients [26, 32]. Items use a four-point scale from no distress to severe distress and sum to produce a total score from 0 to 60 where 0 is the best possible score indicating no distress.

Statistical analysis

The EC-17 was assessed for missing data at the item and scale level. Following the instrument developers, principal component analysis (PCA) [34] was used to assess the unidimensionality of the instrument after assessing the ratio of the first to second eigenvalues [15, 34]. The ratios of 3:1 or higher were considered evidence for unidimensionality [34]. Internal consistency was assessed by item-total correlation and Cronbach's Alpha at the item and scale level respectively. Following the findings of the instrument developers it was expected that the 17 items would have levels of correlation of over 0.4 with the remainder of the instrument. The developers reported a Cronbach's Alpha of 0.96 for the longer-form scale and Alpha and test-retest intra-class correlation coefficients of 0.9 and above meet the more stringent reliability criterion for an outcome measure [35].

The construct validity of the EC-17 was evaluated through comparisons with scores for the other three instruments and patient characteristics. Pearson's and Spearman's rank correlation were used for continuous and categorical scales respectively. The association was interpreted as being high, moderate and weak when the correlation was over 0.60, between 0.30 and 0.60 and 0.30 or less, respectively [36]. It was hypothesised that EC-17 scores would be moderately positively correlated with BACQ Approach and negatively correlated with BACQ Avoidance, moderately negatively correlated with the GHQ-20 and moderately positively correlated with EAC Expression and Processing. It was also hypothesised that younger and more highly educated patients would have higher EC-17 scores than older less well educated patients resulting in weak positive correlations with these two variables [37, 38].

The responsiveness of the EC-17 was assessed by calculating change scores for patients who had undertaken the self-management program. The standardised response mean (SRM) was calculated by dividing the mean change in EC-17 scores over the one week period by the standard deviation of the change scores. SRMs of over 0.80, 0.40-0.80 and less than 0.40 were considered high, moderate and small respectively [39, 40].

SPSS for Windows (version 15.0) was used for statistical analysis.

Results

A total of 66 (64%) patients took part in the responsiveness testing and completed both questionnaires while 52 (54%) patients took part in the test-retest. There was no drop-out among the 118 patients who accepted to participate in the study, but two were excluded in the analyses because of too many missing items. Demographic data and characteristics of the patients who underwent the self-management program are shown in Table 3. The majority of patients were females and age and disease duration were similar for both samples. Patients with fibromyalgia and Sjogrens syndrome were represented in both samples. There were no significant differences between responders and non-responders regarding age, sex and diagnosis for the two groups.

Table 3 Mean (sd) patient characteristics
Full size table

EC-17 mean item scores at the start of the self-management program ranged from 2.02 for item 12 "I can be assertive to get what I need to meet my health needs" to 2.85 for item 2 "I understand the information I receive about my disease" (Table 2). The levels of missing data were low for all items.

PCA of item responses for all patients gave a two component solution that explained 62.16% of the total variation. The first component had an eigenvalue 9.21, that was 6.76 times larger than the second component of 1.36, which is adequate evidence for unidimensionality [34]. Table 2 shows that the item-total correlations ranged from 0.59 to 0.82 for the items "I feel a sense of control over my disease" and "I know how to ask good questions about my health and my disease", respectively. Cronbach's Alpha for the 17-item scale was 0.95. The test-retest intraclass correlation coefficient was 0.90.

The results of testing for construct validity are shown in Table 4. The correlation with the BACQ Approach scale was significant and of a small to moderate level, while there was no correlation with BACQ Avoidance. The correlations with the EAC were of a lower level but statistically significant. The correlation with the GHQ-20 were also low but significant. There were no statistically significant correlations between EC-17 and age or education.

Table 4 Pearson correlation coefficients between EC-17 scores and BACQ, EAC, GHQ-20 (n = 66), age and education (n = 116)
Full size table

There was a significant improvement in EC-17 scores following the self-management program of 4.39 (sd = 9.25) on the 0-100 scale where 100 is the best possible scale (Table 5). The SRM was 0.48 which was larger than SRMs for all but the GHQ-20 which produced an SRM of 0.75.

Table 5 Mean (SD) scores and responsiveness of the EC-17, BACQ, EAC and GHQ-20 (n = 66)
Full size table

Discussion

In this, study, we assessed the data quality, reliability, validity and responsiveness of the Norwegian version of the EC-17. Following forward backwards translation, the Norwegian version was clearly understood by patients involved in pre-testing the questionnaire. The 17-items of the Norwegian EC-17 questionnaire had low levels of missing data for the two groups of patients who took part in the assessment of responsiveness and test-retest, which is further evidence for the acceptability of the instrument.

The results of PCA supported the unidimensionality of the EC-17. Both internal consistency and test-retest reliability estimates met widely accepted standards for the use of such instruments. The coefficients also met the more stringent criterion for use in individual patients. The results of validity testing generally followed the a priori hypotheses with EC-17 scores having low to moderate correlations with the other questionnaires and variables. However, there were no statistically significant correlations with BACQ Avoidance, age and education.

The EC-17 was found to have evidence for responsiveness in this group of patients, however, responsiveness should be further assessed at longer-term follow up and in relation to other interventions that are designed to improve self-management. It is also recommended that other outcome measures are included alongside the EC-17 including disease specific and general measures of quality of life. This would promote further understanding of the relationship between the skills and attributes of patients as effective consumers and health outcomes more generally.

The results of responsiveness testing for the EC-17 were, however, encouraging given the short one-week interval of the study. From a self-management perspective, learning is a process, and to change attitudes and put new skills in to practice usually takes more than a week. There are three components of educational objectives which form a hierarchy - affective, cognitive and psychomotor - meaning that learning at the higher levels is dependent on having attained prerequisite knowledge and skills at lower levels [41]. Several items within the EC-17 require high levels of knowledge, attitudes and skills. The items 5 and 16 relating to weighing up the cost and benefits of decisions relating to ones disease and to negotiate with the health care system about what to do to manage ones disease, are examples of skills at high levels. It may also be that the self-management program was not as effective as intended and hence large changes in the EC-17 could not be expected.

The response rate for the patients responding before and after the management program was satisfactory at 64%, however the response rate of 54% for the test-retest group was rather low. The comparison of respondents and non-respondents showed, however, that there were no significant differences in age, sex and diagnoses. However, non-respondents were somewhat older in both groups, indicating that selection bias can not be ruled out.

The results of testing for construct validity were overall supportive of the hypotheses, however, the low correlations between EC-17 scores and age and education were insignificant. The correlation with age was even positive, which was contrary to expectations. The inclusion of very few younger patients who are expected to be more consumer-minded, may have contributed to this finding. Items within the EC-17 may not wholly reflect consumer skills in practice but rather the subjective perceptions of patients. Patient responses may be dependent on their expectations as consumers and users of information. Finally, the development of the EC-17 which involved collaboration between people with chronic rheumatic disease and health professionals [17], where the participants gave feedback regarding relevance, form and language, may have ensured that items were of equal meaning and relevance to all groups of patients thereby negating differential item functioning across groups. These issues should be considered alongside future evaluation and testing of the EC-17.

The original EC-17 which was developed and tested with patients from Australia and Canada [15], is currently being used to evaluate a program for RA in Ireland and will in the near future be used in New Zealand [15]. Our study provides the first evidence for a translated version of the EC-17 and we have demonstrated that the instrument performs satisfactorily and comparably with the original version. Furthermore, our study provides additional evidence for construct validity. Further translations of the EC-17 with accompanying evaluations will serve to broaden the outcome measures available for evaluating interventions and programs designed to enhance patient skills. Moreover, there will be greater scope for international collaboration and meta-analyses if a common primary outcome measure such as the EC-17 is agreed upon.

Patient education is described as one out of four main responsibilities of the specialist health care within Norway [42]. Most of the hospitals have established patient education units where self-management programs are widely implemented for groups of patients with chronic diseases. Today, few instruments are available to measure effectiveness of such programs. The EC-17 is especially relevant for measuring effects of the self-management programme described here, with instrument content mirroring the goals of the programme. As such the EC-17 provides important information for both clinicians and policymakers within health care.

Conclusions

The EC-17 shows great promise as a measure of the effectiveness of self-management programmes in patients with rheumatic diseases. The questionnaire was well understood by the patients and easy to complete. The instrument has evidence for data quality, internal consistency and test-retest reliability and validity. Preliminary evidence was also found for the responsiveness of the EC-17, the instrument showing significant change in patients completing a one week self-management programme. However, further evaluative studies of the EC-17 in different clinical settings are needed.