Background

The Chronic Care Model is a conceptual framework that supports the evidence-based proactive and planned care for chronic diseases [1, 2]. It has received widespread acceptance as a framework for improving the care of chronically ill patients. Measures of chronic care delivery are required to target efforts to improve chronic care and to monitor change of chronic care delivery over time. The Patient Assessment of Chronic Illness Care (PACIC) is a 20-item questionnaire for patients, which intends to measure chronic care delivery and which has been validated in USA for diabetes [3] and in Germany for osteoarthritis [4]. A version for chronic care in general practice in The Netherlands was not yet available, although many chronic patients receive most of their health care in general practice. Therefore the aim of our study was to develop a Dutch version of the PACIC instrument and test it on patients with diabetes or COPD in general practice.

Methods

Design and setting

An observational study was performed in randomly sampled patients from four general practices, which were situated in a rural area in the south-eastern part of The Netherlands. Two practices were single-handed and two practices were group practices. All practices were involved in a program to enhance structured diabetes care, while no such program existed for COPD care. Ethical approval was received for this study from the Arnhem-Nijmegen ethical committee.

Study population

In each practice patients with diabetes mellitus and with COPD were sampled from the medical record system. An alphabetically ordered list of patient names was made, from which every second patients was included up to 30, except for one practice, which could provide only 20 COPD patients. A total of 230 patients was approached (120 diabetes patients and 110 COPD patients). Written questionnaires were sent by the practices, followed by postcard reminders three weeks later. Patients were invited to complete the questionnaire and return it anonymously in a prepaid envelope to the research institute. Informed consent was not explicitly asked, but implied when a patient returned the questionnaire.

Measures

The questionnaire included the following measures. Patient assessment of chronic illness care (PACIC) was measured with a 20 item questionnaire, which used a five point response scale (ranging from 1 = 'almost never' to 5 = 'almost always') [3]. Higher scores mean more frequent presence of the aspect of structured chronic care. This instrument has five pre-defined domains: patient activation (3 items), delivery system/practice design (3 items), goal setting/tailoring (5 items), problem solving/contextual (4 items), follow-up/coordination (5 items). The English version was translated and culturally adapted in a structured procedure, including forward and backward translations, each by two independent researchers and then established in a consensus meeting with the four individuals involved. Next, face to face interviews were done with 15 elderly patients with chronic illness from one general practice. This led to substantial adaptations, mainly to simplify and clarify the questions, which were also discussed with the authors of the PACIC. Finally, we made two slightly different versions, one for patients with diabetes and one for patients with COPD (see Additional file 1). These versions different in two ways: the heading referred to diabetes or lung disease, and item 19 referred to lung physicians (for COPD patients) or internist, surgeon or ophmatologist (for diabetes patients). Aggregated mean scores for five domains and for the total instrument were calculated as described in previous research [3]. No scores per domain were determined for patients with missing values on on more than one third of the items in the domain.

Patient enablement (PEI) was measured with a six-item questionnaire (with three response categories: 0 = 'same or worse', 1 = 'better', 2 = 'much better') [5]. The range of the aggregated sum score was 0 to 12, with a higher score indicating a higher level of enablement. Respondents giving two or more missing values were excluded. Patient evaluations of general practice were measured with the Europep instrument, a 23-item internationally standardised and validated questionnaire (with a five point answering scale, ranging from 'poor' to 'excellent') [6]. For this study, we determined the overall mean value on the 17 dichotomized (excellent versus other values) items focused on clinical performance (Cronbach's alpha = 0.97). Respondents with more than 5 missing items were excluded. Finally the questionnaire contained questions on patient age, gender, highest education, and overall health status (single item with five point response scale, 'excellent to 'poor).

Data-analysis

The analysis of the measurement properties of the PACIC was based on published quality criteria for questionnaires [7]. The content validity of the PACIC is based on the Chronic Care Model [1, 2]. The interpretability of the instrument (the extent that qualitative meaning can be assigned to the qualitative scores) was based on the percentage of chronic patients who provided valid responses on each of the items. In addition, we checked for floor and ceiling effects in terms of percentage of patients using the most extreme (upper or lower) response categories.

Principal factor analysis (PCA, factors with eigenvalue > 1, varimax rotation) was applied to examine the number and type of domains in the instrument [8]. We determined the Kaiser-Meyer-Olkin Measure of sampling adequacy and the Bartlett's test of sphericity. Internal consistency (the extent to which items measure the same concept) was expressed in terms of Cronbach's alpha for each of five domains in PACIC and for the total PACIC instrument. Reliability was expressed as an intra class coefficient (ICC, absolute agreement), which was based on variation between patients divided by total variation (taking patients random and items fixed). Values > 0.70 for alpha and ICC were considered acceptable [7].

The analysis of construct validity was based on the following hypotheses. We expected that higher PACIC scores, reflecting patient perceived presence of structured chronic care, would be positively related to both patients' perceived enablement after the latest visit to the GP and to patients' overall evaluations of general practice. To verify this expectation, we used linear regression analysis [9] with PACIC scores as dependent factor, enablement or evaluation as independent factor, and patient age and gender also included in the model. All data-analysis was done with SPSS 14.

Results

In total, we received completed questionnaires from 165 patients: 88 diabetes patients (response rate 73%) and 77 COPD patients (70%). Table 1 provides descriptive information on the patient samples. The patients' mean age was 68 years); only a minority had medium or high education (36%); and just over half of them (55%) reported a good or excellent health status. Diabetes patients were, compared to COPD patients, more frequently female (57 versus 35%). More diabetes patients than COPD patients (66 versus 41%) reported good to excellent health status.

Table 1 Patient characteristics (n = 165)
Full size table

Table 2 provides descriptive information on the PACIC items. Not all responders had completed all items of the PACIC questionnaire. The percentage of non-responders of all patients varied between 22 and 35%. Three items (numbers 15, 17 and 20) had 30% or more non-responders. The percentage of responders who used the lowest answering category (indicating complete absence of structured chronic care) varied from 7 to 76%, and was in 11 items 30% or higher. The percentages of responders who used the highest answering category (indicated complete presence of the aspect) varied from 10 to 54%, and was in 6 items 30% or higher.

Table 2 Descriptive information on PACIC items
Full size table

The factor analysis identified five factors (explaining 70% of the variation; KMO = 0.844; Bartlett's test of spherity p = 0.000), which mostly confirmed the internal consistency for three of the five pre-defined domains (Table 3). The items for the remaining two domains, delivery system/practice design and follow-up/coordination, were scattered across domains.

Table 3 Factor loadings in rotated factor solution
Full size table

Despite this partial support for the pre-defined factor structure in the PACIC instrument. Cronbach's alpha's and ICCs were above our threshold of 0.70 for the overall measure and for most pre-defined domains (Table 4). Lower than threshold values were identified for the ICCs in the domains delivery system/decision support and follow-up/coordination. The association of the aggregated Europep score and PACIC domains and overall score were all positive, as expected. However, higher enablement in patients was associated with lower scores on PACIC domains and overall score, as opposed to our expectation.

Table 4 Information on the PACIC domains and overall PACIC score
Full size table

Discussion

This study showed that the Dutch version of the PACIC instrument had mixed measurement properties when applied for assessing diabetes care and COPD care in general practice in a rural setting. The five previously defined domains were confirmed and their internal consistency was good. The correlation with patient evaluations of general practice was positive, and diabetes patients reported higher presence of structured chronic care than COPD patients as expected. However, substantial numbers of patients did not provide answers to the PACIC questions, although they returned the questionnaires and completed other parts of the questionnaire reasonably well. Also, we found that a number of items might have floor or ceiling effects. A surprising finding was that better scores for chronic care were linked to lower patient reported enablement after the latest consultation in general practice.

The mean scores on the PACIC domains and total instrument were similar to those found in diabetes patients in the USA [3], but higher than those found in patients with osteoarthritis in Germany [4]. The PACIC scores for diabetes patients in The Netherlands may be explained by the attention for enhancing structured diabetes care in recent years. For instance, there is no such attention for osteoarthritis, so we would expect similar scores for this condition compared to scores found in Germany. Despite the differences, our findings regarding measurement properties were similar to those found in Germany [4].

Obviously, the study had a number of limitations. The patient sample was relatively small, and only four general practices from a rural setting were involved, but it was not our aim to generalize the descriptive figures. It is difficult to speculate on how the validation results could be affected by the rural setting. Criterion validity, test-retest reproducibility and responsiveness to change could not be analysed. A substantial proportion of the patients used the lowest answering category, which may indicate a floor effect of the measure (inability to discriminate between patients). We suggest, however, that the scores might perfectly reflect reality – a complete absence of specific aspects of structured chronic care. The high number of non-responders was worrying. An explanation for the non-response may be a perceived lack of relevance of the aspects covered by the items. Some of the aspects covered in the PACIC instrument may be unknown or not relevant to many chronic patients in general practice in The Netherlands. A second explanation may be that the non-response actually implies absence of the aspects mentioned in the PACIC questionnaire – but we think we cannot be certain about such inferences. A final explanation for this is translation problems. A direct translation of the English questions into the Dutch language did not result in understandable language, so we had to rephrase the items quite substantially. Despite this, the final questionnaire might have remained too difficult for many patients.

We can only speculate about the (weak, but 5 out of 6 times highly significant) negative association between patient enablement with the latest visit in the practice and PACIC scores. Perhaps patients with a stronger internal health locus of control and better self-management 'ask' less for enablement, so that they do not need help in general practice regarding the aspects covered by PACIC. Enablement and structured chronic care (including patient activation) may be fundamentally different concepts, as opposed to our expectations beforehand. The finding might also suggest that structured chronic care could have some negative consequences, despite its intention to enhance self-management in patients.

Conclusion

A validated Dutch version of the PACIC instrument is now available. Further research into its validity is recommended, particularly with respect to the high number of non-responders and the counterintuitive finding regarding patient enablement. Also, the questionnaire needs to be tested in other settings than primary care, before using it in those settings.