Background

Patient satisfaction and experiences are an important part in the evaluation of health care quality [1, 2]. The purpose of patient satisfaction surveys is frequently related to quality improvement, but reports of general satisfaction have limited value in quality improvement processes [3, 4]. Rather than simply asking about patient satisfaction, patient experiences studies identify concrete health care aspects that are important as measures of service quality from the perspective of the patients and hence contribute to their overall satisfaction. This requires an extensive development phase to secure the content validity of the questionnaire and the comparison of the domains of care included in patient experiences questionnaires with more general measures of patient satisfaction as part of validity testing. To inform quality improvement it is common to benchmark results against results for other units or changes over time. This requires case-mix analysis and potential adjustments for case-mix differences. Therefore, quality improvement based on patient satisfaction surveys requires both an assessment of the association between patient reported experiences and patient satisfaction, and between background variables and patient reported experiences of concrete aspects of health care.

A systematic review of the patient satisfaction literature showed that among socio-demographic factors, age and health status are consistently related to patient satisfaction [5]; older respondents and healthier respondents generally have higher satisfaction, while the evidence related to gender, ethnicity and socio-economic status is equivocal. The importance of age and self-perceived health for patient experiences was also found in a recent US study among community health centers, but the authors also identified other important factors like education and race/ethnicity [6]. The generalizability of these findings to the primary care out-of-hours field is uncertain. We found a few studies about the association between demographic factors and patient satisfaction with primary care out-of-hours services [7–10]. One study found that health status and socio-economic status are significantly related to patient satisfaction, but not age and gender [8], while another found that age and ethnicity are related to patient satisfaction [10]. All in all, findings are equivocal and the only pattern that emerges across these studies is that most socio-demographic factors seem to be only weakly related to patient satisfaction.

The systematic review on patient satisfaction also found consistent evidence that the most important health service factor affecting patient satisfaction is the patient-practitioner relationship [5]. Health service factors might be measured by means of patient reporting or evaluation on items and scales for patient experiences, or by other variables concerning organization and type of services. One study about the association between patient experiences and patient satisfaction relating to primary care out-of-hours services found that the doctor's assistant's attitude on the phone, opinion of GP's treatment and waiting time were strongly related to overall satisfaction [7]. Another study confirmed the importance of waiting time for patients' satisfaction [10]. The association between organization and service variables has been assessed in several primary care out-of-hours studies. Two studies found that patients receiving telephone advice were less satisfied than other patients [7, 10], but other studies did not find such an association [8, 9].

The studies above show that the association between socio-demographic factors and patient satisfaction are equivocal within the primary care out-of-hours field. To some degree the existing evidence contradicts findings from the general patient satisfaction literature. The literature also identifies a need for more research regarding the importance of health service factors for patients' satisfaction including factors related to patient reported experiences. Therefore, our study had two primary aims: i) to assess the association between socio-demographic factors and user reported experiences with primary care out-of-hours services; ii) to assess the association between user reported experiences and user satisfaction, controlling for socio-demographic factors.

The analyses were based on a study of three casualty clinics in Norway in 2008. Norway has a two-level public health care system with a small private sector. Four Regional Health Authorities (RHA) under the Ministry of Health and Care services have responsibility for the hospital sector. The 431 municipalities are responsible for organizing primary health care including out-of-hours services [11]. There were 262 out-of-hours districts in Norway in 2006 covering single or several municipalities through inter-municipality co-operatives. Emergency medical services are usually managed at GP offices during office hours, and by municipality maintained out-of-hours duties by GPs during evenings, nights and weekends [11]. Most of the out-of-hours services are located in a casualty clinic in the host municipality, but some use GPs' surgeries. In this paper we use the term casualty clinic when referring to the physical clinics, while primary care out-of-hours services refers to all services offered by the clinics including office visits, telephone advice and home visits.

Methods

Three casualty clinics were chosen to represent different organization types and size of such clinics. The clinics were recruited through the Watchtower project [11]. The study population included patients that had been in contact with one of the three casualty clinics from 15 April to 13 May 2008. For patients younger than 16 years and patients not able to answer themselves their carers and guardians were asked to fill out the questionnaire. The survey included a random sample of 200 patients from each clinic having telephone contact, consultation visit or home visits by a doctor.

The casualty clinics distributed the questionnaires by mail to the patient's home address. We aimed to send out the reminders at two weeks, but due to practical circumstances relating to clinical administration and postal service delays reminders were sent after six weeks. The questionnaires were returned to the Norwegian Knowledge Centre for the Health Services.

The Norwegian Social Science Data Services approved the survey which was in accordance with the Helsinki Declaration of 1975, as revised in 1983. The leader of each casualty clinic had the opportunity to exclude individuals from the survey based on ethical considerations.

Questionnaire and variables

The questionnaire used in this study has undergone a rigorous process of development and evaluation, including a literature review, qualitative interviews with patients, guardians and carers, and input from an expert group of out-of-hours staff [12, 13]. These activities ensured the content validity of the questionnaire. Most of the questionnaire's core items had low levels of missing data, indicating the acceptability and relevance of the questions to patients and guardians. The psychometric tests showed that the questionnaire has satisfactory internal consistency and construct validity [13].

The questionnaire comprises five parts: parts A and E are completed by all patients; part B concerned telephone contact with the clinic; part C concerned consultation at the casualty clinic; part D concerned home visit from a doctor. The questionnaire comprised a total of 42 items including background questions about patients and guardians. The reference period used for survey questions was "during the last visit/encounter".

Based on factor analysis four scales were identified [13]: telephone contact (4 items), nursing services (4 items), doctor services (4 items), and organization (3 items). The questions are described in table 1. Item-scale correlations were above 0.5 for all scales. Cronbachs alpha was 0.91 for telephone contact, 0.90 for doctor services at the casualty clinic, 0.93 for nursing services at the casualty clinic, and 0.82 for organizing the casualty clinic, all of which are regarded as satisfactory [13]. Items relating to experiences of care have a five-point scale of Not at all, To a small extent, To some extent, To a large extent and To a very large extent. Items were transformed to scores ranging from 0 to 100 where 100 is the best possible. Items comprising scales are summed and transformed into percentage scores. Patients with missing values on more than half of the items in a scale were excluded.

Table 1 Mean scale and item scores for experiences with the casualty clinics on a 0-100 scale
Full size table

Analysis

Descriptive statistics for respondents, the four scales and single items are presented (n, mean, SD). Multivariate regression analysis was used to assess the association between background variables and five dependent variables including four user reported experiences scales and one item about global satisfaction. Age and health have been found to be consistently related to patient satisfaction [5] and were included in the regression. We also included other background variables with some empirical evidence of an association with patient satisfaction, including education [6], gender [14, 15], extent of urgency [15] and marital status [16]. Length of stay has been found to be associated with inpatient satisfaction [17], and in our study we used the number of times in contact with the out-of-hours clinic the last two years as an equivalent to this in the outpatient setting. We pooled the guardian and patient sample and included a respondent variable in the regression.

Finally, we assessed the association between user reported experiences and global satisfaction by means of a multivariate regression analysis, controlling for background variables. We controlled for the same background variables as in the first regression. The telephone and nursing scales, which were relevant to a fewer number of respondents, were excluded from this analysis due to small sample sizes. SPSS (15.1) was used to analyze the data.

Results

Data collection

Of the 600 patients included in the study, 58 were excluded due to cancellations and factors such as unknown addresses. Of the 542 patients who were sent a questionnaire, 225 (41.5%) responded. Table 2 shows the respondents characteristics; 148 (68.2%) were patients, 53 (24.4%) were guardians of children under the age of 16, and 16 (7.4%) were carers of patients above 16 years of age. Approximately two thirds of the respondents were women (71.7%), the average age was 46 years, and more than half of the respondents (58.1%) had first contacted the casualty clinic by telephone and then had a consultation at one of the three clinics.

Table 2 Respondent characteristics (n = 225)a
Full size table

Statistical analysis

Table 1 shows mean scale and item scores for experiences with the casualty clinics. The users reported most positive experiences with the doctor services and nursing services at the casualty clinic. The former scale had an average score on 81.9, the latter 81.4. The telephone contact scale had a mean score of 80.6, while organization at the casualty clinic had a mean score of 64.7. The items about health care personnel being understandable and self perceived incorrect treatment had the highest scores for the individual items. The two questions about waiting time at the casualty clinic had the lowest scores.

Table 3 shows the results of multivariate regression analysis with background variables as independent variables and the four patient experiences scales and one item about global satisfaction as dependent variables. Explained variance for the regression models ranged from 5.6% (doctor services) to 11.4% (organization). Only a few background variables had a significant association with the dependent variables. Self perceived health was significantly associated with global satisfaction (p = .03), while age (p = .02) and self perceived health (p = .04) were significantly related to the scale about organization of the casualty clinic.

Table 3 Multivariate linear regression models: association between background variables and the five dependent variables
Full size table

Table 4 shows the results of multivariate regression analysis with user satisfaction as the dependent variable and user reported experiences as predictors, controlled for background variables. Explained variance for the regression model was 73.2%. Most patient reported experiences were significantly associated with user satisfaction, while two were close to significant (information on tests and unanswered questions). The most important predictors were doctor services (p < .00) and incorrect treatment (p < .00).

Table 4 Multivariate linear regression model: association between background variables, user reported experiences and global satisfaction (n = 154)
Full size table

Discussion

In general, age and self perceived health are the most consistent socio-demographic factors related to patient satisfaction [5]. However, within the primary care out-of-hours field the few identified studies revealed inconsistencies [7–10]. One study found the importance of health status and socio-economic status, but not age and gender [8], another found the importance of age and ethnicity [10]. The only clear finding was that most socio-demographic factors seem to be only weakly related to patient satisfaction. Our study identified self-perceived health as a significant predictor for patient satisfaction and one of four experiences scales, and age as significant associated with user experiences with organization at the clinics. This follows the general patient satisfaction literature [5], and these variables have partial empirical support in the primary care out-of-hours literature. However, the small number of studies and lack of consistent findings means that associations should be assessed in future studies. Furthermore, to use this information in case-mix adjustments further work is necessary including assessing the variation of these variables across the unit of analysis [6]. Future studies in the primary care out-of-hours field might use these findings to test hypotheses about associations between health, age and patient satisfaction, but the effects of case-mix will depend on both the strength of association and the variation between the units in question.

The majority of the user reported experiences domains had a significant association with user satisfaction, after controlling for user characteristics. The most important predictor was the experiences the users had with the doctors at the casualty clinic. This concurs with findings from the systematic review of the patient satisfaction literature [5] and shows that the most important user experience domain for user satisfaction is the relationship between the user and the caregiver. The final regression model found that more than 70% of the variation in global satisfaction was explained, which also gives strong support to the validity of the user experiences questions as an indirect measure of user satisfaction. Since ratings of general satisfaction have limited value in quality improvement processes [3, 4], the approach of asking about experiences with health care providers is used as a means to identify concrete improvement areas.

The primary aims of this paper were to assess the associations between socio-demographic variables and user satisfaction/experiences, and between user-reported experiences and global satisfaction. Naturally, the sample of three clinics is inadequate to represent the population of clinics in Norway. Also, the number of clinics is too small for multilevel regression, making it difficult to separate individual and clinic level effects. This means that individual level effects might be overestimated, especially if the intraclass correlation coefficient (ICC) is substantial. Future studies in Norway should include a representative sample of casualty clinics, and enough clinics to allow empirical assessment of effects at different levels.

The questionnaire is developed specifically for primary care out-of-hours users and includes items of relevance for measurement of user reported experiences with out-of-hours care. It is based on a literature review, interviews with users of out-of-hours services and consultation with an expert group that was designed to ensure the content validity of the questionnaire [12, 13]. Compared to questionnaires such as the Clinician & Group CAHPS survey or the EUROPEP questionnaire, this questionnaire can be used with different types of contact with the out-of-ours services (telephone contact, and/or at the casualty clinic, and/or home visit from the doctor). This questionnaire is also designed for patients that have had one contact with the doctor which is in contrast to questionnaires specific to general practice care that relate to consultations with the patient's usual general practitioner.

The consideration of any differences found through the comparison of user experiences and satisfaction of out-of-hours care with those for general practice care more generally might inform quality improvement initiatives. The EUROPEP questionnaire has been used in large scale surveys of general practice in Norway [18]. However, the differences in the content of items, including items scaling, and scales composition rules out any comparison. Moreover, the study findings relate to three clinics and hence are not representative of all out-of-hours clinics.

Low response rates are a common problem in patient experience surveys in general [5], and also in the primary care out-of-hours field [7, 19–22]. The response rate in our study was 41.5% which is similar to other surveys in this field [7]. A low response rate may cause non-response bias if non-respondents differ systematically from respondents [5]. Some studies have found differences on socio-demographic variables between respondents and non-respondents [5, 20–22], but a Dutch study on patient satisfaction with out-of-hours primary care found that overall satisfaction did not differ much between respondents and non-respondents [7]. This corresponds to findings in studies conducted by the Norwegian Knowledge Centre for the Health Services that shows small differences between respondents and non-respondents in user experiences surveys [23–26]. Therefore, we expect small effects related to non-response in our study.

Conclusions

Users report positive experiences with the three casualty clinics, with organization as the aspect with largest improvement potential. The importance of age and health status for users' experiences and satisfaction with casualty clinics was shown, but a range of user reported experiences with the clinics were the most important predictors for user satisfaction.