Abstract
Purpose
The Effective Consumer Scale (EC-17) measures the skills of musculoskeletal patients in managing their own healthcare. The objectives of this study were to translate the EC-17 into Dutch and to further evaluate its psychometric properties.
Methods
The EC-17 was translated and cognitively pretested following cross-cultural adaptation guidelines. Two hundred and thirty-eight outpatients (52 % response rate) with osteoarthritis or fibromyalgia completed the EC-17 along with other validated measures. Three weeks later, 101 patients completed the EC-17 again.
Results
Confirmatory factor analysis supported the unidimensional structure of the scale. The items adequately fit the Rasch model and only one item demonstrated differential item functioning. Person reliability was high (0.92), but item difficulty levels tended to cluster around the middle of the scale, and measurement precision was highest for moderate and lower levels of skills. The scale demonstrated adequate test–retest reliability (ICC = 0.71), and correlations with other measures were largely as expected.
Conclusion
The results supported the validity and reliability of the Dutch version of the EC-17, but suggest that the scale is best targeted at patients with relatively low levels of skills. Future studies should further examine its sensitivity to change in a clinical trial specifically aimed at improving effective consumer skills.
Avoid common mistakes on your manuscript.
Introduction
Self-management interventions are aimed at providing patients with the necessary knowledge, skills, and confidence to effectively manage their condition themselves. The effectiveness of such interventions is typically evaluated by a wide range of clinical severity measures, self-reported symptoms, and presumed psychological mediators such as self-efficacy [1–3]. To date, however, there is no agreement on the actual set of attributes that are important to managing and participating in healthcare and on how to measure these [4]. This makes it difficult to evaluate direct effects on patient skills and to compare the results of various interventions.
To address this issue, the 17-item Effective Consumer Scale (EC-17) was recently developed based on extensive literature reviews, expert and patient interviews and pilot testing [5]. A follow-up study explored its construct validity and responsiveness in participants in the arthritis self-management program (ASMP) [6]. Results showed that the EC-17 addressed skills and behaviours not covered by other relevant scales such as the Health Education Impact Questionnaire [7], Patient Activation Measure [8], and Arthritis Self-Efficacy Scale [9], including identifying quality information and negotiation with health professionals [6]. Moreover, although the ASMP was not tailored to all behaviours measured by the EC-17, the scale was modestly sensitive to change [6]. A similar study examining the Norwegian EC-17 also showed that the scale was easy to complete, internally consistent, reproducible, valid, and moderately responsive to change [10]. The aim of this study was to cross-culturally adapt the EC-17 for use in Dutch patients with musculoskeletal conditions and to evaluate its psychometric properties.
Materials and methods
Cross-cultural translation
Cross-cultural adaptation followed established forward–backward translation procedures [11]. The prefinal EC-17 was cognitively pretested in five patient research partners (four female, age range 29–74 years) with different rheumatic conditions. Pretests were carried out using the three-step test interview method [12]. Based on the results, small wording changes were made in six items (e.g., ‘arrange’ instead of ‘organise’), one response option (‘generally’ instead of ‘usually’), and the instructions (expanded with an explanation of the term ‘management’).
Psychometric evaluation
Participants
A survey was sent in October 2010 to a random sample of 404 patients with osteoarthritis (OA) and 58 patients with fibromyalgia (FM) that had visited the outpatient rheumatology clinic in the preceding year. Two hundred and fifty-three (54.8 %) patients returned a completed survey. The first 120 patients willing to complete the scale a second time were sent a follow-up questionnaire, which was completed by 101 (84.2 %) patients after a median (IQR) time of 20 (18–24) days.
Measures
The EC-17 measures knowledge, attitudes, and behaviours about self-management skills using 17 items with 5-point Likert-type scales (“never” to “always”) [5]. Item scores are summed when ≥14 items are completed and converted to range from 0 to 100, where 100 is the best possible score.
Additionally, patients completed the 5-item Perceived Efficacy in Patient–Physician Interactions scale (PEPPI-5; α = 0.90) [13, 14], the 12-item Dutch General Self-Efficacy Scale (GSES; α = 0.80) [15, 16], the 4-item support from family and friends subscale from the Arthritis Impact Measurement Scales 2 (AIMS2; α = 0.91) [17, 18], and the 36-Item Short Form Health Survey (SF-36v2) [19, 20], from which the physical and mental component summary (PCS and MCS) scores were calculated [21]. Pain in the last week was measured on an 11-point numerical rating scale (NRS) from 0 (‘no pain’) to 10 (‘unbearable pain’).
Data analysis
Fifteen patients had >3 missing values on the EC-17 and were excluded from further analyses (final response rate 51.5 %). Remaining missing values were low, with a maximum of five (2.1 %) for items 10 and 16, and were imputed with their median values.
Unidimensionality of the EC-17 was tested using robust maximum likelihood confirmatory factor analysis (CFA) [22]. Non-normed (NNFI) and comparative fit (CFI) indexes ≥0.95 and standardized root mean square residual (SRMR) and root mean square error of approximation (RMSEA) ≤0.08 and 0.06, respectively, were considered indicative of good fit [23, 24].
Additionally, Rasch partial credit model analyses were performed [25]. Conservative infit values between 0.87 and 1.13 and outfit values between 0.61 and 1.39 were considered to indicate acceptable item fit [26]. Items with residual correlations >0.30 were considered locally dependent [27, 28]. Differential item functioning (DIF) was evaluated across sex, age, and disease duration and considered present when the difference between the item calibrations was statistically significant and >0.5 logits [25, 29]. Person reliability ≥0.70 and ≥0.85 was considered adequate for group-level and individual comparisons, respectively [28]. The person-item map and test information function were examined for mistargeting and local measurement precision [30].
Reproducibility was assessed by intraclass correlation (ICC, type A,1) [31] and considered adequate for group-level and individual measurements over time when ≥0.70 and ≥0.90, respectively [32].
For convergent and discriminant validity, it was hypothesized that an adequate measure of perceived health-management skills should be strongly correlated with perceived effectiveness in patient–physician interaction, which is an important aspect of general health-management skills, and moderately correlated with the conceptually related construct of general self-efficacy and social support [33–35]. Finally, a moderate correlation with psychosocial health (SF-36 MCS) and weak correlations with physical health (SF-36 PCS) and pain were expected [36].
Results
Patient characteristics
Patient characteristics are reported in Table 1. There were no significant differences with respect to age or sex between the respondents and non-respondents. FM patients differed significantly from OA patients on several socio-demographic variables and scored worse on all scales, except the GSES and SF-36 PCS.
Distributional properties
Total scores on the EC-17 showed a near-normal distribution (Kolmogorov–Smirnov, P = 0.058) with skewness and kurtosis values of −0.72 and 0.74, respectively (Fig. 1). Floor and ceiling effects were absent, with no patients scoring zero and only three patients (1.3 %) scoring 100.
Unidimensionality
With the exception of RMSEA, the one-factor model showed a good fit (SBχ2(119) = 488.70, NNFI = 0.96, CFI = 0.96, SRMR = 0.08, RMSEA (90 % CI) = 0.11 (0.10–0.12). Standardized factor loadings were high for all items (Table 2).
Rasch measurement properties
The EC-17 adequately fit the Rasch model. Five items showed infit values slightly outside the range of 0.87–1.13, and no items showed poor outfit (Table 2). Residual correlations revealed some redundancy or multidimensionality, as demonstrated by 4 pairs of items showing positive (r’s between 0.33 and 0.40) and 4 showing negative local dependence (r’s between −0.31 and −0.32). No items showed DIF across sex and disease duration and only one item across age.
Internal consistency was sufficiently high (person reliability = 0.92) for individual-level comparisons. Item difficulty estimates ranged from 0.67 to −0.68 logits and tended to cluster around the middle of the scale, with a large proportion of patients with relatively high skills not being covered by any individual item (Fig. 2). The mean logit score for patients was 1.25, indicating that the sample as a whole was located at a higher ability level than the mean item difficulty.
The information curve (Fig. 3) was peaked at lower levels of the underlying trait, indicating that patients with skills below the mean are measured with more precision than individuals with better skills. Measurement precision was sufficient for group-level analyses across a wide range of the underlying trait, but adequate for individual-level comparisons in persons with moderate and lower levels of self-management skills only.
Test–retest reliability
With an ICC of 0.71 (95 % CI: 0.60–0.80), test–retest reliability of the scale was adequate for group-level comparisons.
External construct validity
As expected, the EC-17 correlated strongly with perceived efficacy in patient–physician interactions, moderately with social support and psychosocial aspects of health, and weakly with physical aspects of health and pain (Table 3). The association with general self-efficacy was just below the cut-off value for moderate correlation.
Discussion
EC-17 scores were normally distributed, and the results of both CFA and Rasch analysis supported the unidimensionality of the scale, indicating that item scores can be summed to create a single total score. The latter is in accordance with previous studies that used principal component analyses [6, 10].
The high internal consistency of the EC-17 corresponds to the ability to discriminate between 3 and 4 distinct levels of skills [25]. However, measurement precision was not equally high across the underlying trait. On a group level, the EC-17 had sufficient precision across a wide range of scores. However, it was adequate for individual-level comparisons only in persons with moderate and lower levels of skills. Although it may be desirable to have a measure that specifically targets patients with lower skills, this also suggests that the EC-17 may lack discriminatory power in patients with relatively high levels of skills. Since a sample size of approximately 240 persons has been shown to provide accurate estimates of item and person locations in Rasch analyses, even for measures with poor targeting, these results are likely to be quite robust [37]. However, no other studies have used Rasch analyses for the current 17 items, and this finding should be further investigated in other populations.
Test–retest reliability was adequate, but lower than previously found [10]. It is possible that we used a more strict ICC model [38] or that the time interval was too long to assure that no inter-individual variation occurred.
Finally, with the exception of general self-efficacy, all hypothesized correlations were confirmed, supporting the convergent and discriminant validity of the scale.
Given the relatively low response rate and the differences in both demographic and clinical characteristics between the OA and FM patients, the current findings should be interpreted with some caution and be cross-validated in other samples of musculoskeletal patients.
In conclusion, this study suggests that Dutch EC-17 is a valid and reliable measure of effective health consumer skills in patients with musculoskeletal conditions. Future studies should further examine its sensitivity to change in a clinical trial specifically aimed at improving the skills and behaviours deemed necessary for effective consumers, before the scale can be fully endorsed as an outcome measure for evaluating self-management interventions.
Abbreviations
- CFA:
-
Confirmatory factor analysis
- DIF:
-
Differential item functioning
- EC-17:
-
17-item Effective Consumer Scale
- FM:
-
Fibromyalgia
- IRT:
-
Item response theory
- OA:
-
Osteoarthritis
- RA:
-
Rheumatoid arthritis
References
Barlow, J., Wright, C., Sheasby, J., Turner, A., & Hainsworth, J. (2002). Self-management approaches for people with chronic conditions: A review. Patient Education and Counseling, 48(2), 177–187.
Mulligan, K., Newman, S. P., Taal, E., Hazes, M., & Rasker, J. J. (2005). The design and evaluation of psychoeducational/self-management interventions. Journal of Rheumatology, 32(12), 2470–2474.
Newman, S., Steed, L., & Mulligan, K. (2004). Self-management interventions for chronic illness. Lancet, 364(9444), 1523–1537.
Tugwell, P. S., Wilson, A. J., Brooks, P. M., Driedger, S. M., Gallois, C., O’Connor, A. M., et al. (2005). Attributes and skills of an effective musculoskeletal consumer. Journal of Rheumatology, 32(11), 2257–2261.
Kristjansson, E., Tugwell, P. S., Wilson, A. J., Brooks, P. M., Driedger, S. M., Gallois, C., et al. (2007). Development of the effective musculoskeletal consumer scale. Journal of Rheumatology, 34(6), 1392–1400.
Santesso, N., Rader, T., Wells, G. A., O’Connor, A. M., Brooks, P. M., Driedger, M., et al. (2009). Responsiveness of the Effective Consumer Scale (EC-17). Journal of Rheumatology, 36(9), 2087–2091.
Osborne, R. H., Elsworth, G. R., & Whitfield, K. (2007). The health education impact questionnaire (heiQ): An outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Education and Counseling, 66(2), 192–201.
Hibbard, J. H., Mahoney, E. R., Stockard, J., & Tusler, M. (2005). Development and testing of a short form of the patient activation measure. Health Services Research, 40(6 Pt 1), 1918–1930.
Lorig, K., Chastain, R. L., Ung, E., Shoor, S., & Holman, H. R. (1989). Development and evaluation of a scale to measure perceived self-efficacy in people with arthritis. Arthritis and Rheumatism, 32(1), 37–44.
Hamnes, B., Garratt, A., Kjeken, I., Kristjansson, E., & Hagen, K. B. (2010). Translation, data quality, reliability, validity and responsiveness of the Norwegian version of the Effective Musculoskeletal Consumer Scale (EC-17). BMC Musculoskeletal Disorders, 11(1), 21.
Beaton, D. E. (2000). Understanding the relevance of measured change through studies of responsiveness. Spine, 25(24), 3192–3199.
Hak, T., van der Veer, K., & Jansen, H. (2008). The three-step test-interview (TSTI): An observation-based method for pretesting self-completion questionnaires. Survey Research Methods, 2(3), 143–150.
Maly, R. C., Frank, J. C., Marshall, G. N., DiMatteo, M. R., & Reuben, D. B. (1998). Perceived efficacy in patient-physician interactions (PEPPI): Validation of an instrument in older persons. Journal of the American Geriatrics Society, 46(7), 889–894.
ten Klooster, P. M., Oostveen, J. C. M., Zandbelt, L. C., Taal, E., Drossaert, C. H. C., Harmsen, E. J., et al. (2011). Further validation of the 5-item Perceived Efficacy in Patient-Physician Interactions (PEPPI-5) scale in patients with osteoarthritis. Patient Education and Counseling. doi:10.1016/j.pec.2011.07.017
Bosscher, R. J., & Smit, J. H. (1998). Confirmatory factor analysis of the general self-efficacy scale. Behaviour Research and Therapy, 36(3), 339–343.
Sherer, M., Maddox, J. E., Mercandante, B., Prentice-Dunn, S., Jacobs, B., & Robers, R. W. (1982). The self-efficacy scale: Construction and validation. Psychological Reports, 51(2), 663–671.
Meenan, R. F., Mason, J. H., Anderson, J. J., Guccione, A. A., & Kazis, L. E. (1992). AIMS2. The content and properties of a revised and expanded arthritis impact measurement scales health status questionnaire. Arthritis and Rheumatism, 35(1), 1–10.
Riemsma, R. P., Taal, E., Rasker, J. J., Houtman, P. M., van Paassen, H. C., & Wiegman, O. (1996). Evaluation of a Dutch version of the AIMS2 for patients with rheumatoid arthritis. British Journal of Rheumatology, 35(8), 755–760.
Aaronson, N. K., Muller, M., Cohen, P. D., Essink-Bot, M. L., Fekkes, M., Sanderman, R., et al. (1998). Translation, validation, and norming of the Dutch language version of the SF-36 health survey in community and chronic disease populations. Journal of Clinical Epidemiology, 51(11), 1055–1068.
Ware, J. E, Jr, & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30(6), 473–483.
Ware, J. E., Kosinski, M., & Dewey, J. E. (2000). How to score version 2 of the SF-36 health survey. Lincoln, RI: QualityMetric Incorporated.
Jöreskog, K. G., Sörbom, D., Du Toit, S., & Du Toit, M. (2001). LISREL 8: New statistical features. Lincolnwood, IL: Scientific Software International.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park CA: Sage Publications.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.
Linacre, J. M. (2006). A user’s guide to WINSTEPS MINISTEP Rasch-model computer programs. Chicago IL: Winsteps.com.
Smith, R. M., Schumacker, R. E., & Bush, M. J. (1998). Using item mean squares to evaluate fit to the Rasch model. Journal of Outcome Measurement, 2(1), 66–78.
Davidson, M. (2009). Rasch analysis of 24-, 18- and 11-item versions of the Roland-Morris disability questionnaire. Quality of Life Research, 18(4), 473–481.
Tennant, A., & Conaghan, P. G. (2007). The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis and Rheumatism, 57(8), 1358–1362.
Lai, J. S., Teresi, J., & Gershon, R. (2005). Procedures for the analysis of differential item functioning (DIF) for small sample sizes. Evaluation and the Health Professions, 28(3), 283–294.
Reeve, B. B., & Fayers, P. (2005). Applying item response theory modelling for evaluating questionnaire item and scale properties. In P. M. Fayers & R. D. Hays (Eds.), Assessing quality of life in clinical trials: Methods and practice (pp. 55–73). Oxford: Oxford University Press.
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.
Lohr, K. N. (2002). Assessing health status and quality-of-life instruments: Attributes and review criteria. Quality of Life Research, 11(3), 193.
Clark, N. M., & Dodge, J. A. (1999). Exploring self-efficacy as a predictor of disease management. Health Education & Behavior, 26(1), 72–89.
Gallant, M. P. (2003). The influence of social support on chronic illness self-management: A review and directions for research. Health Education & Behavior, 30(2), 170–195.
Han, W. T., Collie, K., Koopman, C., Azarow, J., Classen, C., Morrow, G. R., et al. (2005). Breast cancer and problems with medical interactions: Relationships with traumatic stress, emotional self-efficacy, and social support. Psycho-Oncology, 14(4), 318–330.
Smith, M. Y., Winkel, G., Egert, J., Diaz-Wionczek, M., & DuHamel, K. N. (2006). Patient-physician communication in the context of persistent pain: Validation of a modified version of the patients’ perceived involvement in care scale. Journal of Pain and Symptom Management, 32(1), 71–81.
Linacre, M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328.
de Vet, H. C., Terwee, C. B., Knol, D. L., & Bouter, L. M. (2006). When to use agreement versus reliability measures. Journal of Clinical Epidemiology, 59(10), 1033–1039.
Acknowledgments
The authors thank the respondents who participated in this study, the rheumatologists and secretariat staff of Ziekenhuisgroep Twente for their help in organizing the study, Jason Doppenberg and Harald Vonkeman for their help with translating the EC-17, and Stans Drossaert and Roos van der Vaart for their participation in the translation expert committee.
Open Access
This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
ten Klooster, P.M., Taal, E., Siemons, L. et al. Translation and validation of the Dutch version of the Effective Consumer Scale (EC-17). Qual Life Res 22, 423–429 (2013). https://doi.org/10.1007/s11136-012-0162-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-012-0162-2