Introduction

In many western countries, workers can claim compensation when they are losing (part of) their income due to disability. To judge these disability benefit claims, assessments are carried out by specialized physicians. These physicians have to make judgments regarding the claimants’ medical status and his or her functional capacities concerning vocational rehabilitation [1, 2]. In the Netherlands, disability assessments are performed by social insurance physicians, who work for the Dutch Workers Insurance Authority (UWV). Worldwide, physicians are involved in similar assessments, even though national practices may vary considerably under social insurance or disability legislation [3].

The way of assessing workers’ disability by (insurance) physicians remains a topic of interest and discussion [4, 5]. One of the main problems in adequately evaluating disability claims lies in the complicated physician-patient interaction within this specific setting [6, 7]. Physicians frequently report feeling uncomfortable with engaging in disability evaluations [8] and have little confidence in their ability to determine workers disability [9], based on, in most cases, only one meeting with the claimant.

On the claimant side, a common idea is that a passive and defensive attitude among claimants causes strain in the physician-claimant relationship. This frequently observed attitude among claimants can be the result of:

  1. (1)

    Social security arrangements, which causes claimants having to prove that they are ill (in order to receive a disability pension) [10]. This results in problems in the assessment of disability and discourages claimants to return to work [11, 12],

  2. (2)

    Claimants perceived poor health status which frequently is not congruent with the severity of disability [13, 14] and which results in discrepancies between the physicians’ and claimants’ views on the claimants’ functional capacity,

  3. (3)

    Complicated and not fully transparent disability legislation procedures, which causes a lack of knowledge and understanding about this topic, and frequently results in claimants having unrealistic expectations about assessment outcomes.

As a consequence of the complicated relationship between physicians and claimants, many claimants experience disability assessments as injustice [15, 16] and patient satisfaction with insurance physicians seems lower than, for example, with occupational physicians [17].

In an attempt to enhance the physician-patient relationship in the specific context of the disability assessment, an intervention was developed [18]. This intervention joins the latest developments in physician-patient communication research, in which the trend is to put more emphasis on the patients’ role in order to improve the physician-patient relationship [19, 20]. One way to achieve this is by patient empowerment. Empowerment, a term often described as a process by which patients gain control over situations and things that are important to them [21], is thought to influence the physician-claimant relationship in a beneficial way by strengthening the sense of control among the claimant, so that more directed information concerning his or her disability can be shared with the physician. Furthermore, empowering claimants by giving them more information about social security legislation and procedures will enhance the transparency of disability assessments, which, at its turn, can result in more satisfaction among claimants with and acceptance of the outcome of the assessment [22].

Since the Internet has the possibility to easily reach a large audience at a low cost, and some evidence exists that web-based interventions can be effective in increasing patient empowerment [23], it was chosen to deliver the intervention web-based.

It is hypothesized that this web-based intervention will increase empowerment among disability claimants and, by adapting claimants expectations, will increase claimant satisfaction and perceived justice. The aim of this article is to describe the results of a pragmatic, randomized controlled trial designed to evaluate the effectiveness of an interactive website aimed at empowerment of disability claimants, prior to the assessment of disability by an insurance physician.

Methods

Design

A two-armed randomized controlled trial (RCT) was conducted among persons claiming a disability pension. A detailed description of the design of the study has been published elsewhere [18] and will only be presented here briefly. The Medical Ethics Committee of the VU University Medical Center approved the study protocol (under number 08/194).

Participants

Participants were claimants for a disability pension according to the Dutch Work and Income Act (WIA). According to the WIA, this disability pension can be claimed after being sick-listed for 104 weeks. All disability claimants were recruited approximately 1–2 weeks prior to their appointment for disability assessment by an insurance physician. Recruitment took place through three different offices (Leiden, The Hague, Rotterdam) of the Dutch Workers Insurance Authority, UWV. UWV is the organization in the Netherlands responsible for evaluating disability claims. Together with a standard invitational letter and brochure from UWV, claimants received a study information brochure, which directed them to an online application form. This application form included questions concerning the study’s in- and exclusion criteria and an informed consent. Claimants were considered eligible to participate in the study if they had an email address. Recruitment took place over a 9-month period (January 2009–September 2009).

All insurance physicians from the three participating UWV offices, and responsible for disability assessments concerning the Dutch Work and Income Act (WIA), were asked to participate in the study.

Randomization and Blinding

Randomization took place at the individual claimant level. After baseline measurement, disability claimants were randomized into either the intervention or control group. Randomization to these two groups was done by block randomization. To prevent unequal groups, three blocks were created (three participating UWV offices). A computerized random number generator drew up an allocation schedule for each block.

The use of a ‘sham’ website for participants of the control group (see below), caused claimants to be blinded for study design. Insurance physicians were aware of the study’s design, but were not informed about the group allocation of disability claimants.

Intervention Group

Participants randomized in the intervention group were able to logon to the web-based intervention www.wiagesprek.nl with an obtained username and password. The development and exact content of this intervention has been described elsewhere [18]. Briefly, the web-based intervention consisted of several components:

  1. (1)

    Five interactive lessons or ‘modules’. Each module prepared participants step-by-step for their meeting with the insurance physician. Participants were able to finish the modules in their own pace in a period of approximately 1 week prior to their disability assessment. In module 1 (~20 min), Dutch legislation procedures were explained in order to increase subjects’ knowledge about WIA procedures and the exact content of a disability assessment. An interactive quiz tested subjects’ knowledge at the end of the module. Module 2 (~20 min) focused on the meeting with the insurance physician of UWV. Subjects were asked to fill out their medical record and keep up an online diary that prepared them for the actual disability assessment. In module 3 (~15 min), videos of patient-physician interaction were shown to subjects in order to teach them how to actively participate during their meeting with the insurance physician. In module 4 (~15 min), expectations of subjects’ disability claim outcomes were discussed. Also, an interactive tool (the “WIA meter”) helped subjects to increase their self-awareness and tested their motivation to return to work. Module 5 (~5 min) summarized all previous modules and discussed the six most important tips concerning preparation for the upcoming disability assessment.

  2. (2)

    General information and features concerning absenteeism from work, such as social security law arrangements, explanation of disability assessment procedures, return to work information, personal experiences of people who underwent disability assessment procedures, information as to how to cope with disease and work disability, and links to other related websites.

  3. (3)

    A forum that gave participants the ability to interact with other claimants on issues such as coping with disease or exchanging experiences concerning disability assessments.

Control Group

Participants from the control group also received a username and password, which directed them to a ‘sham’ website with very brief, commonly available and UWV provided information only, and some links to other related websites.

Outcome Measures

Outcome measures were mainly extracted from online questionnaires. After baseline measurement (T0), participants were sent an email with a link to the questionnaires 2 days before their disability assessment (T1), as well as 1 day after their disability assessment (T2), 6 weeks (T3), and 4 months after their disability assessment (T4). Reminders were sent to decrease loss to follow up. The following outcomes were assessed:

Empowerment

Empowerment was measured with the ‘VrijBaan’ questionnaire, an instrument designed to measure empowerment among people with a work disability [24]. The VrijBaan questionnaire consists of 60 items divided over six subscales: Competence (13 items), Self-determination (11 items), Meaning (9 items), Impact (8 items), Positive Identity (10 items), Group Orientation (9 items). Internal consistency of this questionnaire has shown to be good (all subscales had Cronbach’s alphas higher than 0.80). The subscales Competence and Impact were assessed at T0, T1, and T4. All other subscales were assessed at T0 and T4 only.

In addition to the ‘VrijBaan’ questionnaire, mastery [25] and general self-efficacy [26], frequently mentioned as important components of empowerment [27], were assessed at T0, T1, and T4.

Coping Strategy

We measured coping strategy with the Dutch adaptation of the Ways of Coping Questionnaire (WCQ, [28]). This questionnaire is based on Lazarus’ Theory of Stress and Coping [29], which states that coping is situation-specific rather than a trait or disposition. Three dimensions of the WCQ were included: Problem Solving (8 items), Seeking Social Support (6 items) and Avoidance (7 items). Questions from these scales were adapted to the context of the disability assessment and were asked at T0 and T1.

Subjective Knowledge

With a 10-point Visual Analogue Scale (VAS), we measured claimants subjective knowledge about social security law arrangements and disability assessment procedures. At T0 we asked claimants: “How much do you know about social security law arrangements and disability assessment procedures?” (0 = I know nothing, 10 = I know everything). At T1 we asked: “To what extent did the intervention increase your knowledge about social security law arrangements and disability assessment procedures?” (0 = my knowledge did not increase, 10 = I gained maximum knowledge).

Claimant Satisfaction

The satisfaction of claimants with their insurance physicians was measured (at T2) with the AStri questionnaire [30]. This questionnaire is specially designed to measure patient satisfaction in the field of insurance medicine and contains 29 items divided over six subscales, each representing a different component of patient-insurance physician interaction (Listening, Empathizing, Correctness, Clearness, Rigorousness, and Professionalism). The AStri questionnaire showed good internal consistency (all subscales had a Cronbach’s alpha greater than 0.78) [25].

Claimant Perceived Justice

To measure claimant perceived justice with the final verdict on their disability pension, a Dutch translation [31] of Moorman’s justice questionnaire [32] was used. This questionnaire consists of 30 items measuring three dimensions of justice perceptions: distributive justice (the perception of fairness of the outcomes a claimant receives, 7 items), procedural justice (the perception of fairness of the procedures used to determine these outcomes, 12 items), and interactional justice (the perception of fairness of contact with the organization that determines the outcomes, 11 items). Each item can be scored on a 7-point scale ranging from “I totally do not agree” (1 point) to “I totally agree” (7 points). Average scores were calculated for each separate dimension. In the present study we assessed the subscales distributive justice and procedural justice at T3 only. Cronbach’s alpha for these dimension of the questionnaire has proven to be high (distributive justice: α = 0.91, procedural justice: α = 0.82) [33].

Physician Satisfaction

Insurance physician satisfaction with the disability assessment and claimants’ attitude during the assessment was assessed directly after the assessment with a questionnaire specially designed for this study. In this 10-item questionnaire physicians could react to specific statements on a 5-point Likert scale ranging from “I totally disagree” (1 point) to “I totally agree” (5 points).

Other Variables

Additional data was obtained from UWV on: (1) claimants’ official complaints with their insurance physician, (2) claimants’ objections to the outcome of the assessment, and (3) the outcome of the assessment.

Statistical Analyses

Baseline Differences and Attrition

Baseline differences in demographic characteristics were investigated using Chi-square tests and independent sample t-tests. Drop-out attrition was defined as the phenomenon of losing participants to follow-up (e.g., participants who did not fill out follow-up questionnaires). Non-usage attrition was defined as not using the intervention or not complying with the intervention [34].

Effectiveness of the Intervention

A priori, effect modification and confounding were checked for gender, age, level of education, country of birth, disease type, internet use, work status, and perceived work ability for all outcome measures. Analyses to determine effectiveness were then performed using multiple linear regression (continuous outcomes) and logistic regression (dichotomous outcomes), with the follow-up outcome measure as the dependent variable. Assumptions of linear regression were verified. All analyses were adjusted for baseline values (if applicable) and possible confounding, thus creating an adjusted follow-up score [35]. The parameters of interest were the regression coefficients (B), indicating the effect of the intervention compared to the control group. Additional longitudinal analysis were performed on the outcomes that were assessed at three measurements, i.e.: empowerment (subscales Competence and Impact), general self-efficacy and mastery. On these outcomes, generalized estimation equations (GEE) [36] were used to investigate the effect of the intervention on the development of empowerment over time, correcting for regression to the mean by only adjusting the first follow-up measurement of the outcome variable for its baseline value (as is described as method four by Twisk and colleagues [36]). All analyses were performed according to the intention-to-treat principle and were performed using SPSS version 15.0.

Sample size and Missing Values

We aimed to recruit 115 claimants per study group, to retain 86 per group after allowing for some loss to follow-up (25%). This sample size was sufficient to detect a difference of 10% in empowerment between the two study groups, assuming that power is 0.90 and alpha is 0.05. Missing data were not imputed.

Results

Participants

Figure 1 shows the flow of participants throughout the trial. From the 2,780 disability claimants who were approached, 2,329 (84%) disability claimants did not respond to the study’s invitation and 209 (7.5%) were excluded because they did not meet the inclusion criteria (n = 95), were unwilling to participate (n = 84), or responded too late (i.e. their application was received after their disability assessment: n = 30). The remaining 242 participants (8.7%) were randomized into either the intervention group (n = 123) or the control group (n = 119). Baseline characteristics for participants are shown in Table 1. Despite adequate randomization procedures, gender was found to be unevenly distributed between the two study groups (χ2 = 4.65, P = .03), and appeared to be a confounder. No other differences between study groups were found at baseline.

Fig. 1
figure 1

Flow of participants throughout the phases of the trial

Full size image
Table 1 Baseline characteristics
Full size table

Drop-Out Attrition

Drop-out attrition rates were comparable for both study groups (no statistically significant differences at all follow-up measurements) and satisfactory for T2, T3, and T4 (20% or less). An exception was the non-response percentage at T1 (34%). After analyzing T1 drop-outs, it appeared that the number of drop-outs was significantly higher (χ2 = 42.26, P < .001) among claimants who enrolled within 2 days of the disability assessment and thus had to fill out the T1 questionnaire shortly after the baseline measurement. This characteristic was, however, evenly distributed between both study groups. Figure 1 shows the exact number of drop-outs per measurement and study group. Reasons for not filling out the follow-up questionnaires remained unknown.

Out of the 194 questionnaires that were sent to physicians directly after meeting with a claimant who participated in the trial, 141 were returned (73%). The number of questionnaires that were sent to physicians was not equal to the number of participants included in the trial, because some claimants did not show up at the disability assessment meeting, and in some cases it was not possible to match a claimant with his or her assessing physician.

Non-Usage Attrition

Non-usage of the web-based intervention was high: 41 participants (33%) from the intervention group did not log on even once, so did 45 participants (37%) from the control group. On the other hand, 63 (51%) completed at least 1 module, 46 (37%) completed more than three modules, and 27 participants (22%) finished all five modules.

Effects of the Intervention

Table 2 shows the short-term effects of the intervention. The mean time between enrolment and the disability assessment meeting was 6.48 days (SD 6.27). The intervention www.wiagesprek.nl had no significant effects on empowerment, general self-efficacy, mastery, and coping. The intervention did significantly increase knowledge compared to the control group (1.38, 95% CI 0.59–2.17). However, satisfaction with the disability assessment interview and perceived justice on the outcome of the assessment was lower in the intervention group (not statistically significant). The intervention had a significant negative effect on perceived procedural justice (−0.50, 95% CI −0.94 to −0.05).

Table 2 Short-term effects of the intervention
Full size table

No effects of the intervention were found on physician satisfaction and duration of the meeting. When examining data retrieved from UWV, no significant differences between the intervention and control group were found on the outcome of the disability assessment, the proportion of official complaints against the physician, and objections to the outcome (data not shown).

With regard to the long-term effects of the intervention, Table 3 shows the 4-month effects of the intervention on empowerment, general self-efficacy, and mastery. Although it seemed as if that the intervention has a slight adverse effect for all subscales, none of these differences were statistically significant, with the exception of the subscale Meaning. Additional longitudinal GEE analysis furthermore pointed out no statistically significant differences in the development of empowerment over time, between the two study groups (data not shown).

Table 3 Long-term (4 month) effects of the intervention
Full size table

Since non-usage attrition was high, we performed additional sub-group analyses, in which only participants from the intervention group who logged on at least once (n = 82) were included. These analyses did not result in significant changes compared to the effects found from the ITT analyses, for all reported outcomes.

Discussion

This study aimed to increase empowerment among disability claimants prior to meeting an insurance physician for assessment of disability. Before conducting this trail, it was hypothesized that the intervention www.wiagesprek.nl would increase levels of empowerment among disability claimants. Empowerment, at its turn, would have beneficial effects on claimants’ satisfaction with the assessment, as well as on claimants’ perceived justice with the process of disability assessment and the outcome of the assessment. Another hypothesized beneficial effect of the intervention was increased satisfaction perceived by physicians.

Main Findings

Almost none of the, beforehand, formulated hypotheses were verified. Compared to the control group, the web-based intervention www.wiagesprek.nl did not increase levels of empowerment and coping, although knowledge about social security arrangements and disability assessment procedures was significantly increased. The intervention did not have a beneficial effect on claimant satisfaction and perceived justice on the outcome of the assessment. An adverse negative effect of the intervention was found on perceived procedural justice: claimants from the intervention group significantly experienced the procedures within UWV as more unjust than claimants from the control group.

Interpretation of Findings

No Effects on Empowerment

This study was an effectiveness study, which answers the question of whether an intervention does more good than harm, when it is delivered under usual or ‘real-world’ conditions [37], that is, with taking into account a variation in the target audience compliance rate. The rate of non-usage was high in the present study. Among claimants from the intervention group, 33% did not use the intervention. Although, non-usage attrition is a very common feature in online trials [34], in this study this might have contributed to not finding an improvement in empowerment. However, additional sub-group analyses performed among participants who, at least, logged on to the intervention once, showed no significant differences in the estimated effects, compared to the ITT analyses. More detailed information on compliance with the intervention is needed, and should give insight into the exact relationship between the compliance rate and the outcomes measured in this study.

A second explanation for not finding an effect on empowerment could involve the selection of measurement instruments. In psychology, attitude is broadly categorized into states and traits [38]. A trait refers to a relative stable personal characteristic (i.e. personality), while a state is more changeable and situation-specific. The instruments we used in this trial might define empowerment more as a trait than a state. For example, statements like: “I have little control over things that happen to me” (VrijBaan questionnaire, subscale Impact) or “I can usually handle whatever comes in my way” (General Self-Efficacy Scale) are referring to a more stable personal characteristic and might therefore be less responsive to detect changes than would have been the case in situation-specific questionnaires. Finally, a third and most straightforward explanation could be the fact that the intervention was simply not intensive enough and too brief in order to change claimants behavior. The mean time participants had to use the website before their appointment with the physician was 6.48 days. It is plausible that in such a short timeframe it is difficult to change empowerment among claimants. Also, it is possible that the intervention was too much focused on knowledge improvements and less aiming at change of behavior. In that light, improving knowledge alone seems not sufficient in order to improve empowerment.

Perceived Justice

Claimants from the intervention group significantly experienced the procedures within UWV as more unjust than claimants from the control group. A possible explanation for this finding can be the fact that claimants that were motivated by the intervention to, for example, bring along a personal health record and medical reports of their treatment history, were experiencing injustice in the case the physician did not handle these sources of information adequately in the opinion of the claimant.

Comparison with Other Studies

This is the first study available that evaluated an intervention aimed at empowerment of disability claimants and the first study that focused on empowerment in the field of insurance medicine. Comparisons with other studies are therefore difficult to make. However, the present study has some overlap with other research areas.

First, this study joins up with a wealth of literature on patient-physician interaction. In this research area, some work has been done on promoting patient participation in the consultation process. Although, to our knowledge, the outcome empowerment was never measured before in this field, a review [39] found 27 studies that examined the effects of interventions delivered prior to consultations on the outcome self-efficacy. From these studies, in which the intervention of interest varied from a simple checklist to intensive group education sessions, only six studies found significant positive effects on patients’ self-efficacy, indicating that these interventions mostly had a minor impact on this outcome. Another systematic review conducted on the effectiveness of interventions for patients before consultations, examined the outcomes question asking, patient satisfaction, knowledge, and anxiety [40]. In a meta-analysis, Kinnersley and colleagues found statistically significant improvements in question asking and patient satisfaction after patients made use of interventions, such as question prompt sheets or (computerized) coaching sessions. Overall, patient satisfaction, an outcome also measured in our trial, was only slightly improved (standardized mean difference (SMD) = 0.09, 95% CI 0.03–0.16) within 17 studies after using these interventions, with only four studies showing significant improvements. Thus, when summarizing evidence in the field of the patient-physician interaction, one can conclude that, in clinical care, the benefits of interventions aimed at empowerment of patients prior to consultations seems to be limited. As a possible explanation for finding no strong effects, Kinnersley and colleagues [40] suggested that focusing on the patient alone may not produce the best benefits for patients because of the complexity of the dialogue between patients and clinicians. A combined approach, in which patients are encouraged to actively participate in their consultations, and clinicians have the skills to identify and adapt to the needs of their patients, is proposed by these authors.

Another field of research in which comparisons can be made with the current study, is the field of eHealth research. In this field, some web-based interventions were evaluated as to their effectiveness on the outcome patient empowerment or empowerment-related outcomes. Recently, we conducted a systematic review to summarize the evidence in this field [23]. Results from this review showed that web-based interventions had significant, but small, positive effects on empowerment and situation-specific self-efficacy. No positive effects were found on general self-efficacy and self-esteem. Although the levels of evidence were generally limited in this review, it did show that web-based interventions can have positive effects on empowerment. Duration and intensity of successful interventions were, however, much higher (1–6 months) than the brief intervention used in the trial described in this article.

Strengths and Limitations

One of the strengths of our study was the fact that it was highly pragmatic. We kept exclusion criteria to a minimum, did not stimulate participant compliance in any way, changed neither procedures within the participating organizations nor professionals for the sake of the trial, and delivered the intervention to its target population on basis of ‘real world’ conditions. These issues all contributed to a higher generalizibility and external validity of the study’s results, as it is mostly the case in pragmatic trials [41]. Furthermore, blinding of patients and physicians in this trial were unique in the area of web-based research. We used a ‘sham’ website, with commonly available information only, to serve as a control condition. Through the use of this parallel used website, claimants were not aware of the study design and the existence of two separate study conditions. Moreover, physicians were not told which claimant was randomized into what group, and thus, were blinded for the allocation of the claimant who they assessed for disability. For web-based trials in general, blinding is a complicated issue to accomplish [42]. For that reason, many trials use alternate designs, such as a waiting list control group. The context and design of our study, however, made blinding possible. Although the response rate at the T1 measurement was disappointing, internal validity was strengthened by the drop-out attrition rate of 18% at 4-month follow-up.

There are also several limitations of our study to consider. First, only 9% of the invited claimants took part in the trial. As appeared from analysis among non-participants, the major reason for this low initial response rate was the fact that many claimants did not have access to the Internet at home. The low recruitment rate, however, has serious consequences for the external validity of the study’s results [43]. A conducted process evaluation showed that self-selection took place through which relatively more females, more higher educated claimants, and less ethnical minorities were reached. Generalization of the study’s results should, therefore, be made with caution. Secondly, this study did not assess participants’ information seeking behavior outside of the trial. The possibility of participants using information and preparing themselves with the use of other websites or sources other than the intervention www.wiagesprek.nl is present and could have had influence on our results. Especially participants from the control group could have been motivated to seek information from other sources, since they received a website that only contained five unique pages with commonly available information. Non-trial related information seeking behavior could have modified the results of our trial and could account for not finding an effect of the intervention.

Conclusions

The web-based intervention www.wiagesprek.nl was not effective in increasing levels of empowerment and coping among disability claimants, despite showing an increase in claimants’ knowledge about social security arrangements and disability assessment procedures. The intervention had a significant adverse effect on perceived procedural justice and no effect on claimant satisfaction, physician satisfaction, and the duration of the meeting.

Implications

Because of the lack of effectiveness of this trial, serious considerations should be made before thinking about implementing the intervention www.wiagesprek.nl in daily practice. Although the results of this trial suggest that, should the intervention be implemented, it cannot be expected that this would lead to a successful empowerment tool. The intervention, however, could serve as a useful information source for disability claimants.

From a research perspective, more carefully selected or newly developed outcome measures should be considered to measure the possible benefits of empowering claimants in the context of the disability assessment and social insurance.