Background

Improving functional independence and health-related quality of life are two common and inter-related goals of health care services. These objectives are particularly important for services that cater for the needs of older adults. Evaluation of these services and ongoing monitoring of their patients requires that participation in functional activities and health-related quality of life be measured with an approach that is amenable to the clinical context. [1–3] A difficulty with evaluating these constructs is that many older adults find it difficult to attend hospitals or other health care settings for appointments. This is often due to the difficulty associated with travelling to, within and from a hospital facility for the purpose of a face-to-face assessment.[4, 5] A viable alternative may be to complete relevant survey instruments via a telephone interview.

Different modes of administration of a range of self-reported outcomes have previously been investigated in a number of clinical populations. [6–15] Some of these investigations have found that equivalent responses were yielded from the different modes of administration [6–10], while others have found that responses were dependent on the mode of administration (such as self-completed postal surveys versus face-to-face administration) [11–15]. However, in order for telephone administration to be employed, it is important to demonstrate that equivalent answers would be elicited from respondents regardless of whether the mode of survey administration was face-to-face or via the telephone. This study aims to investigate equivalence between telephone and face-to-face administration of the Frenchay Activities Index[16] (FAI) and the Euroqol-5D[17] (EQ-5D) amongst sample of older adults who are accessing health care services. There have been no previous investigations of agreement between telephone and face-to-face administration of these instruments amongst older adults.

Methods

Design

Equivalence investigation between telephone and face-to-face administration for the FAI and the EQ-5D.

Participants and setting

Older adults taking part in a larger evaluation of a novel home-based rehabilitation program following discharge from hospital participated in this study. Any person over the age of 65 years of age discharged from the geriatric rehabilitation, medical and surgical units of the Princess Alexandra Hospital during the study period were available for inclusion. Participants were excluded from the study if they had severe cardiac disease (unstable angina), cognitive impairment (Mini-Mental State Examination[18, 19] score <23/30), restricted weight bearing status (non or partial weight bearing), aggressive behaviour, or referral for post-discharge community rehabilitation services.

Measures

The FAI is a 15 item report of participation in functional activities recently undertaken by the respondent. Each of the 15 items require the participant to select one of four possible responses that best describes their recent level of participation in each nominated activity. Although the four possible responses varied between items, they generally ranged between 'never' and a more frequent response such as 'most days' or 'at least once weekly'. The longest time a respondent is required to recall is during item 11 which refers to the frequency of travel for the purpose of pleasure (for example a coach or rail trip) within the past 6 months. Each response was scored between zero (least frequent level of participation) and 3 (most frequent level of participation). An overall score out of 45 is calculated by summing each of the individual item scores. Evidence of sound validity and reliability have previously been reported for this instrument. [20–22]

The EQ-5D is a generic health-related quality of life measurement instrument consisting of 5 multiple choice questions, and a 100 point overall health state visual analogue scale (VAS). [17] The first 5 questions relate to mobility, personal care, usual activities, pain/discomfort and anxiety/depression respectively. The respondent is required to select one of three ordinal statements (e.g. no problems, some problems, unable) which best describes their health state in relation to these 5 domains. The responses from these 5 questions can be converted to a utility score where 0 and 1 represent death and perfect health respectively. To calculate the EQ-5D utility score the application of tariffs previously derived from population based surveys investigating preferences for all possible health state combinations is undertaken. The Dolan formula was used for utility calculation in this investigation[23]. Evidence of sound validity and reliability have previously been reported for this instrument. [17, 24–30]

During telephone administration questions were read directly from the FAI and EQ-5D instruments. The first five EQ-5D questions were read verbatim as per telephone instructions [31], while the EQ-5D VAS component was modified slightly from the original version with a repetition of the question (for wording of EQ-5D VAS script see Additional file 1).

Procedure

Hospital physiotherapists identified potential participants meeting the inclusion criteria who were about to be discharged from the hospital between March and December 2007 and sought verbal consent for the patient to be approached by a project research assistant. Pending this consent, the research assistant then provided a full description of the project to the patient and then sought written informed consent for participation. The research assistant then collected demographic information, including the FAI, EQ-5D and the Activity-specific Balance Confidence Scale[32] (a measure of how confident participants feel they can complete functional tasks without falling) prior to the patient being discharged from hospital.

Participants completed the FAI and EQ-5D via telephone interview with a research assistant (PL) seven days prior to an eight week post discharge outpatient review appointment at a tertiary hospital. The FAI and EQ-5D were then completed again at the subsequent outpatient appointment seven days later where the measures were administered face-to-face by the same research assistant. A seven day period was chosen so that participants would have a lower chance of remembering their response to the telephone-administered survey items when the time came for them to again complete the surveys via the face-to-face administration approach. A longer period was not chosen to minimise the risk that the participant's health would change in a measureable way during the between-assessment period.

This study was approved by the Human Research Ethics Committee of the Princess Alexandra Hospital, and the Medical Research Ethics Committee of The University of Queensland.

Analysis

The Kappa statistic was used to describe the agreement between assessment approaches for individual items within the outcome measures examined. Confidence intervals for kappa statistics were calculated using bootstrap resampling (2000 replications of original sample size)[33, 34]. The number of exact matches was also calculated for each item. For FAI and EQ-5D items the frequency of each possible response was computed to identify whether responses were distributed across the range of possible scores or whether they were at either end of the scales (potentially indicating ceiling or floor effects within an item).

FAI summative scores were calculated and EQ-5D response items were converted to utility scores using the Dolan formula.[23] Limits of agreement and intraclass correlation coefficients (ICC) were calculated to investigate agreement between assessment approaches for the FAI summary score, EQ-5D VAS and EQ-5D utility score. Paired t-tests were also employed to examine whether systematic differences between the two modes of administration existed for the FAI summary score, EQ-5D VAS and EQ-5D utility scores. Bland-Altman plots[35] are also presented for the FAI summary score, and EQ-5D VAS and utility scores.

Results

Sixty-eight patients were screened by the recruiting research assistant after being referred by ward staff for potential participation in the study. All sixty eight met the inclusion criteria; however, fifteen did not provide informed consent. Fifty-three patients consented to participate. Forty subjects' datasets were complete for the FAI and EQ-5D at both assessments and were included in the analysis. Reasons for participants' incomplete sets of data excluded from analysis included unable to attend face-to-face reassessment on scheduled appointment day (8), not able to be contacted via telephone seven days prior to reassessment (3), readmitted to hospital with acute illness at time of assessment (1) and death (1). Participant demographics for patients included in analysis are displayed in Table 1. A high proportion of participants (90%) required a walking aid when outside their home and a substantial level of concern about falling over was present amongst this population.

Table 1 Participant demographics
Full size table

Levels of agreement, and 95% confidence intervals, between telephone and face-to-face administration of the FAI and EQ-5D items are displayed in Table 2. For 11 out of 15 items of the FAI, kappa was >0.80 and was between 0.60 and 0.80 for the remaining 4 items. Similarly kappa for each of the EQ-5D domain items were either between 0.60 and 0.80 (3/5) or >0.80 (2/5). Mean scores, p-values from paired t-tests for differences in group means, limits of agreement and intraclass correlation coefficients between telephone and face-to-face administration for the FAI summary score, and for the EQ-5D VAS and utility scores are displayed in Table 3.

Table 2 Level of agreement (kappa) and frequency of exact matches between face-to-face and telephone administration of the Frenchay Activities Index and the EQ-5D items
Full size table
Table 3 Intraclass correlation coefficient (ICC), mean scores, and limits of agreement (LOA) between telephone and face-to-face administration of the Frenchay Activity Index (FAI) and EQ-5D
Full size table

Two FAI items did not have participant answers distributed across the spectrum of responses. All (40) participants reported they did not undertake paid work while 33 reported that they never drive. The Bland-Altman plots for the FAI summary score, and EQ-5D VAS and utility scores are displayed (Figure 1) and do not indicate a relationship between overall score and level of agreement between telephone and face-to-face administration.

Figure 1
figure 1

Bland Altman plots for Frenchay Activity Index (a), EQ-5D VAS (b) and EQ-5D utility (c).

Full size image

Discussion

The FAI and EQ-5D generally had high levels of agreement between telephone and face-to-face administration of these instruments at both the individual item level and overall score. Within the EQ-5D instrument, the intraclass correlation coefficients and limits of agreement (Table 3) indicated agreement was lower for the VAS, than for the utility score. Greater variability within the VAS score is not surprising given its sensitivity to smaller amounts of change that might occur over a one week period relative to the discrete response items that combine to form the utility score.[36, 37] While it is logical that some differences between modes of administration for the VAS may be, in part, attributable to the absence of visual representation of the scale during phone administration, agreement between responses from the two modes of administration for these instruments is comparable to test-retest reliability investigations for each the EQ-5D [24–27] and FAI [20–22] instruments where the same mode of administration was used at each assessment. This further strengthens the argument that telephone administration of these instruments is valid. In the same way, the mean differences, limits of agreement and paired t-tests indicate that although some variation existed between responses from each assessment for each instrument, no systematic difference was present. The high levels of agreement between the two modes of administration at the individual item level and overall scores indicate that telephone administration of these instruments is a valid method of obtaining this information from elderly patients in the community. The ability to collect this survey based information via the telephone offers viable and potentially more efficient and convenient approach to face-to-face assessment amongst older adults with cognition intact at a basic level.

The findings from this investigation are in line with previous reports of high levels of agreement between modes of administration for other related instruments [6–10]. Previous investigations which have not found high levels of agreement between modes of administration for survey instruments have often compared self-completed to interview administered modalities [11–15]. The high levels of agreement between the two modes of administration found in this investigation may be explained by similar nature of telephone and face-to-face administration of a survey instrument in comparison to self-completion.

A study of this nature will always have two key potential limitations that need to be counterbalanced. First is the risk that a participant may have anticipated the purpose of this study, recalled their original answer and responded in the same way when completing the questionnaires for the second time. Second is the risk that a participant's health may have measurably changed between the two assessment points. We believe that this study was more at risk of the second limitation than the first as we allowed a seven day washout period between assessments. This, combined with the shear number of items that a respondent would have had to remember correctly gave some protection against the memory-recall limitation. By doing so however, our results were likely to be more conservative than what could be expected in real life. Hence, given the nature of our design, we argue that the results of this investigation provide evidence that telephone administration of the FAI and EQ-5D (utility and VAS) instruments could be validly used in research or clinical practice.

The extrapolation of results from this investigation is limited somewhat as we focused our investigation solely upon older adults who are accessing health care services. Notably though, it is this population for whom telephone assessment of the constructs of participation in functional activities and health-related quality of life may be most important. We did however exclude participation by older adults with cognitive impairment as assessed by a Mini-Mental State Examination score of <23 out of 30. It is possible that older adults with cognitive impairment may not respond as consistently between the two administration approaches as our study sample did. Similarly some participants who were unable to attend the appointment to complete the face-to-face assessment may have had poorer health than those participants with complete datasets resulting in a slightly healthier sample. Along these lines, the study did not include many patients with poor to very poor self-rated health-related quality of life as a majority of responses were greater than 40 out of 100 on the VAS (Figure 1). It is possible that people with poor to very poor self-rated health may not respond as consistently between the two administration approaches as our study sample did. It is also noteworthy that some FAI items such as gainful work are likely to have low variance amongst older adult populations (as many are retired). Items like this may have exceptionally high levels of agreement, at least in part, due to a floor effect. Within this investigation this applied to both items with perfect agreement (gainful work and driving). However one could reason that the low variance observed in this investigation may often be present within these items amongst responses from this type of population. Thus the high level of agreement observed between modalities for these items may be reflective of what would occur in clinical, epidemiological and research settings that utilise these instruments amongst older adults.

Future investigations may consider the validity of telephone administration of other survey based instruments for the elderly as a way of reducing the burden of health assessments amongst this population. The ability to complete survey based instruments such as the FAI and EQ-5D via the telephone is likely to increase the feasibility of following up elderly patients in both clinical and research environments.

Conclusion

This study has indicated that telephone and face-to-face administration of the Frenchay Activity Index and EQ-5D yields comparable responses amongst older adults with cognition intact at a basic level.