Background

Major inequalities in health result from differences in socio-economic position between individuals, families and population groups[1]. Whilst the link between resources and health is well established[1–4], there has been little conclusive research evaluating the impact of increasing resources on health[5]. In the UK large amounts of welfare benefits are unclaimed[6], particularly amongst vulnerable groups such as older people. It has been estimated that only around 40–60% of those eligible actually claim the health-related benefits to which they are entitled[7]. Appropriate targeting and active assistance with benefit claims can result in substantial increases in financial and non-financial resources (e.g. parking permits, household aids and adaptations) for eligible non-claimants[8–10]. In the UK welfare rights advice is offered through local government social services departments, Citizens Advice Bureaux (one of the UK's largest voluntary organisations, providing free advice and information on money, legal or other problems) or primary care[11], with clients accessing the services either through self-referral, referral from another agency or both. However, these services are not available to everyone.

Evaluating the health impact of complex social interventions poses ethical and methodological challenges. In particular, the ethical acceptability of withholding welfare advice or benefits from control group participants has been questioned[12]. Pilot studies play an important role in health research and are a necessary prerequisite for definitive randomised controlled trials[13, 14]. Evaluations of the health effects of social interventions are essential to identify effective ways to reduce health inequalities[10, 12].

We report the results of a pilot randomised controlled trial of welfare rights advice in a primary care setting, whilst Moffatt et al, in an accompanying paper[15], report the findings of an embedded qualitative study. We aimed to evaluate the methods, including a range of potential outcome measures, estimate effect sizes and sample size for a definitive trial.

Methods

Study design

We conducted a single blind RCT with individuals randomly allocated to intervention (receipt of immediate welfare rights assessment, advice and active assistance with claims) or control condition (receipt of the intervention after a six month delay).

Recruitment

General practices

Four practices working from five premises in Newcastle upon Tyne participated. Three premises were in the top ten per cent of the most deprived wards in England and two were in the top one per cent, measured using the Index of Multiple Deprivation[16].

Participants

A random sample of 400 patients aged 60 years or over was generated using participating practices' computer systems, and invited to participate. Patients likely to have received full welfare assessments, such as the permanently hospitalised or those in residential or nursing care homes, were excluded. The sample size was pragmatic and aimed to enable us to estimate sample size for a future definitive trial.

Intervention condition

Within 3 weeks of the baseline assessment, a welfare rights officer from Newcastle City Council Social Services undertook a structured assessment of current welfare status and benefits entitlement, including a full assessment of household income and expenditure. Participants were then offered active assistance with making benefit and other welfare claims where appropriate over the following months. Advice was offered either at home or at the GP surgery, but all participants preferred a domiciliary service.

Control condition

Participants in the control group were given an appointment for a follow up interview with the researcher six months after their baseline assessment. Following this second interview, the control group were offered an appointment with the welfare rights officer, who offered a full welfare benefits assessment and active assistance with claims as appropriate, following the same procedure as the intervention group.

Randomisation and blinding

Following the baseline assessment, participants were randomly allocated to the intervention or control group by the project secretary using a sequential allocation table independently generated from random number tables prior to recruitment. The researcher conducting the assessments was blind to the randomisation group.

Baseline assessment and outcome measures

Written informed consent was obtained at baseline assessment. The following outcomes were collected in a structured, face-to-face interview at baseline, 6, 12 and 24 months:

Health

Short Form 36 (SF36)[17], Hospital Anxiety and Depression Scale (HADS)[18], activity limiting long term illness[19], symptoms inventory[20], the Pittsburgh Sleep Quality Index[21], and self-reported height and weight (used to calculate Body Mass Index (weight in kg/height in m2)).

Health related behaviours

The Dietary Inventory for Nutrition Education (DINE),[22] the Physical Activity Scale for the Elderly (PASE)[23], current smoking status and weekly alcohol consumption[24].

Psycho-social factors

Social Support Questionnaire[25], the Self-Esteem Inventory[26], the Personal Mastery Scale[27] and the Life Events Inventory[20].

Socio-economic status

Affordability (financial vulnerability)[28] and Standard of Living Index[29].

In addition data were collected at baseline on age, sex, ethnicity and educational attainment. The salary, travel and administrative costs of providing welfare rights advice were estimated.

Participants were contacted by telephone for repeat welfare assessments at 6, 12 and 24 (intervention) or 6 and 18 months (control) from the initial welfare assessment. When an application for benefits was made, the Welfare Rights Officer asked participants to notify her when a decision letter arrived, so that an accurate record could be kept of benefits received. Participants who had not notified her were followed up with telephone calls.

Intervention cost per case was estimated by recording the number of minutes spent on each participant's case by the Welfare Rights Officer, including advice sessions, telephone calls and letter writing, and recording the number of miles travelled. Travel was costed at £0.404 (€0.595, $0.716) per mile, and staff time at £0.934 (€1.374, $1.656) per minute. Household income and major expenditure (housing costs, council tax, loan repayments and energy costs) were used to estimate household disposable income.

Statistical methods

Initial analyses were conducted on an 'intention-to-treat' basis, without regard to whether participants had received benefits as a result of the intervention. Explanatory analyses then compared outcomes in the intervention at different time points.

The differences between mean changes from baseline in each outcome measure for the intervention and control groups are reported along with 95% confidence intervals for those differences. Bootstrap methods were used when the distributions were markedly non-normal. ANOVA was used to compare outcomes at different time points.

Ethical approval

The protocol for the study was approved by Newcastle upon Tyne joint universities and NHS research ethics committee.

Results

Participant flow and follow-ups

Out of the 400 people invited to participate, 199 (49.8%) did not respond, and 58 (14.5%) declined to participate. Of the latter, 30% gave no reason, 32% said they were not interested or felt they would be ineligible for welfare benefits and 39% cited being too old, sick or frail. Between August and December 2002, 126 people were recruited and randomised (figure 1). Characteristics of the intervention and control groups were similar at baseline (table 1). SF-36 scores showed poorer physical health than expected for their ages, although mental health scores were within normal range[17]. There was little drop out in subsequent assessments (figure 1).

Figure 1
figure 1

Participant flow and follow-up.

Full size image
Table 1 Baseline characteristics of intervention and control groups
Full size table

Material and financial outcomes

Table 2 shows the distribution of initial disposable income and any awards made to participants, along with the period they had to wait for financial benefits. Sixty eight (58%) of all participants received a welfare benefit award (31 financial, 16 non-financial and 21 both). Non-financial awards included the disabled parking permit ("blue badge"), aids and adaptations around the home, "Staywarm"- a national energy scheme, and the Community Care Alarm scheme. Benefit eligibility was greater in general practices with higher levels of deprivation, measured using the Index of Multiple Deprivation[16] (table 3).

Table 2 Distribution of household income, welfare benefit variables and type of award by group allocation
Full size table
Table 3 Eligibility for financial benefits and median amount awarded by general practice
Full size table

Median time to receipt of benefits from initial assessment overall was 14 (range 1 to 78) weeks and median financial award was £55 (€81, $98) per household per week. At 6 months after the initial welfare assessment, eligible participants had been in receipt of benefits for a median of 9 weeks but seven (14%) had not yet received their benefits (table 2). However, all participants had received their benefits by 12 months (figure 2).

Figure 2
figure 2

Time from own first assessment to receiving financial benefits in intervention and control groups combined.

Full size image

The mean intervention cost per case was £120.18 (€172.34, $209.94). This comprised salary of £115.24 (€165.20, $201.21) and travel of £4.94 (€7.08, $8.62). The mean cost was £161.23 (€231.14, $281.46) for those who received benefits and £63.62 (€91.21, $111.07) for those who did not.

Health, behavioural and psychosocial outcomes

There was considerable variability in the change of most outcomes at 6 months (table 4) but the mean change was close to zero for most scales. The distributions of the changes over 6 months were very similar for both groups. The only significant difference in mean change was for the financial vulnerability score; those in the intervention group felt less vulnerable than those in the control group at six months (difference = -1.6 (95% CI: -2.6 to -0.7)). However, the change on this scale was zero for most participants.

Table 4 Changes in continuous outcome measures# from baseline to 6 months for intervention and control groups
Full size table

Since welfare benefits did not always start until late in the initial follow-up period, 6 months may have been too early to detect any health-related improvements. However, the mean outcome measures varied very little across 6, 12 and 24 month time points in the intervention group, apart from sleep quality and social interaction, which both improved between 6 and 12 months and declined between 12 and 24 months (table 5).

Table 5 Comparison of mean outcome measure scores# at 6, 12 & 24 months – Intervention group (n = 57)
Full size table

Discussion

Main findings

This is the first pilot RCT to measure a broad range of health and social outcomes in relation to a welfare rights advice intervention accessed via a health care setting. Both the intervention and research procedures proved feasible and acceptable to participants and professionals involved.

Around 60% of participants were eligible for some benefits and 40% for financial benefits, confirming that offering welfare advice to older people via primary care is an effective way of identifying those entitled to benefits who would otherwise be unlikely to claim[8, 9, 30]. However, there was little evidence of differences in health outcomes between those who did or did not receive the advice at 6 months, or within the intervention group over time. These could reflect a genuine lack of effect, or limitations in the study design; this was a pilot study and not powered to detect small differences. Previous studies examining the impact of welfare rights advice on health have been observational and therefore provide only weak evidence[10, 31]. Our qualitative study[15] suggests that those in receipt of extra benefits were more able to participate in society and experienced greater 'peace of mind', but this was not evident using scales such as Social Interaction[25] or HAD-Anxiety[18] in this pilot trial.

Strengths and weaknesses

The study was conducted in the UK and the findings should therefore be interpreted with caution in relation to the health and welfare systems of other countries. Nevertheless, most of the implications are methodological in nature and thus widely applicable.

The methodological and ethical challenges in evaluating an intervention of this type have been discussed before[12], but the low drop-out rate suggests that participants found our study procedures acceptable, a finding corroborated by the accompanying qualitative study[15]. This was a single blind RCT and the letter arranging the follow up appointment specifically asked the participant not to tell the researcher whether or not they had seen the welfare adviser. Only one participant told the researcher conducting follow-up interviews (JM) that she had received benefits, but this was an isolated incident and we do not believe that it affected the results. Furthermore, the main analyses were conducted by independent statisticians (DH, TC) who were blind to this information.

Although the health and social assessment interview covered a wide range of outcome measures, taking between 30 and 90 minutes to complete, participants did not report feeling this was too onerous a task for them.

Whilst the intervention in this study was the provision of a welfare rights assessment, in 40% of the sample this did not result in any welfare benefits because of ineligibility. This will have diluted any resultant health improvements in the intention to treat analysis. Using practice deprivation scores to target the poorest areas would increase the proportion of participants eligible for welfare benefits in a future trial. Further exploration of individual predictors of eligibility using GP data (e.g. morbidity indicators) in this study was hindered by the small sample size. It may be possible in future work to improve recruitment materials to persuade others likely to gain from a welfare rights assessment to participate. Thirty nine percent of those giving a reason for non-participation said that it was because they were too old, ill or frail. Whilst this is not uncommon in research with the elderly, these are likely to be people who could benefit from a welfare rights assessment. In contrast, the embedded qualitative study[15] indicated that some participants agreed to take part in this research study out of a sense of altruism, rather than because they thought they might qualify for welfare benefits. All participants reported their ethnic group to be white British. Whilst Newcastle overall has a small ethnic population, one of the study practices has a sizeable proportion of patients from the South Asian community. It is not known how many people from this community were invited to participate and either did not respond or declined. However, the invitation to participate was only available in English and this is likely to have affected recruitment of people for whom English was not their first language.

The potential health-related impact of additional benefits is hard to quantify. Some concerns were raised about the standard scales used to assess health related outcomes. Interviewer feedback suggested that some of the questions on the scales we used may have been inappropriate for this population; most were not designed specifically for use with older people and, additionally, some may not be sufficiently responsive to small changes in health which would be of interest. A related issue arose in interviews with the 14 (out of the sample of 25) people in the qualitative study[15] who had received financial benefits. A positive and wide-ranging impact on a number of aspects of quality of life such as 'maintaining independence' or 'peace of mind' was reported. These are all factors relevant to an older population but not specifically measured by the scales used in the RCT.

The optimum time at which to measure any health benefits was unclear at the trial outset. The six month delay prior to the control group's benefit assessment was thought to be a reasonable compromise between impeding the receipt of benefit entitlements and allowing time for any health improvements to appear. This was justified ethically on the grounds that the intervention is presently rationed and recipients would not have been identified within the welfare rights officer's normal caseload. However, 14% of those eligible for financial benefits did not receive them until after the six month follow up assessment, and the average time between receipt of financial benefits and the next assessment was only two months. Health improvements resulting from increased benefits may not be detectable after such a short time. The time lapse needed for additional resources to have any effect on health is unclear, but there is a methodological argument for delaying the welfare rights advice longer in the control group in a future trial. However, this remains ethically contentious, as it is hard to justify withholding an intervention that is known to be beneficial in financial terms[12].

Conclusion

Our trial design was feasible and acceptable[15]. The study found a large proportion of participants in the sample was eligible for welfare benefits but not claiming them. However, there was little evidence of differences in health outcome measures between groups or over time. If there was a real effect, possible reasons for a lack of evidence include: the small sample size; inadequate lengths of time for additional welfare benefits to have health and psycho-social effects; many participants did not qualify for any benefits; and the outcome measures used may not have been the most appropriate. These factors combined would have reduced the observed strength of any possible effects.

Modification of the study design to reduce the dilution effects described above, including selection of study participants, timing of interventions and length of follow up will be necessary for a definitive trial. There is also a need to look at some alternative or additional measures of health outcomes relevant to an older population, particularly those highlighted by the accompanying qualitative study[15], such as maintenance of independent living : however many of the outcome measures used in this pilot RCT would remain relevant. This study provides an example of how a pilot RCT with an embedded qualitative study identified unforeseen problems that will inform the design of a definitive evaluation.